Method for reducing interference signals in a top-down view showing a motor vehicle and its surrounding area, driver assistance system and motor vehicle

Abstract

Method for reducing interference signals (17) in a top-down image (14) showing a motor vehicle (1) and an area (4) surrounding the motor vehicle (1), wherein the top-down image (14) is determined on the basis of raw images (11a, 11b, 11c, 11d) acquired by at least two vehicle-mounted cameras (5a, 5b, 5c, 5d), and wherein the method includes the following steps a) to c): a) suppressing or attenuating contrast sharpening and / or edge sharpening for the acquired raw images (11a, 11b, 11c, 11d) in the case of cameras equipped with integrated sharpening functions (5a, 5b, 5c, 5d), and determining the top-down image (14) on the basis of the raw images (11a, 11b, 11c, 11d) without the contrast sharpening. and / or edge sharpening, b) Creating focus offsets within the captured raw images (11a, 11b, 11c, 11d) and determining the top view image (14) based on the raw images (11a, 11b, 11c,11d) with the focusing offsets, c) Identifying at least one image area (16) in the top view image (14) containing interference signals (17) and reducing the interference signals (17) by applying a filter to image elements of the top view image (14) and / or the raw images (11a, 11b, 11c, 11d) corresponding to this image area (16).

Description

The invention relates to a method for reducing interference signals in a top-view image showing a motor vehicle and its surrounding area, wherein the top-view image is determined based on raw images captured by at least two vehicle-mounted cameras. The invention also relates to a driver assistance system for a motor vehicle and a motor vehicle. It is already known from the prior art to monitor the area surrounding a motor vehicle by recording the surrounding area with cameras mounted on the vehicle as part of a camera system, for example, a surround-view camera system, and displaying the images captured by the cameras on a display device. Increasingly, three-dimensional representations of the surrounding area are also being displayed on the display device from a so-called third-person perspective. Such a third-person perspective shows the area surrounding the motor vehicle as well as the motor vehicle itself from the viewpoint of an external observer, a so-called virtual camera. Such a third-person view can be a top-down view in which the observer looks down on the motor vehicle.This top-down image can be displayed on the screen and shows the upper side of the vehicle, including the roof, as well as the surrounding area. The top-down image is generated by combining the raw images captured by the cameras of the camera system. The resulting image gives the impression of having been captured by a real camera from the position of the virtual camera. Examples of this can be found in the following prior art patent applications: US 2002 / 0047901 A1, which describes a system for generating panoramic views by merging individual images. EP 2916540 A1 deals with image processing for surveillance scenarios, while EP 2466889 A2 discloses the synthesis of a virtual bird's-eye view from overlapping camera data. Furthermore, DE 102014110516 A1 discloses a method for the geometric alignment and calibration of vehicle cameras. Although these documents address fundamental aspects of image combination and perspective transformation, they do not offer a specific solution for suppressing aliasing effects that arise during the dynamic rendering process of a top-down view from raw vehicle images. It can happen that the combined top-down image contains some interfering signals, such as artificial flickering effects, also known as aliasing. These flickering effects degrade the image quality displayed to the driver and can therefore be distracting. It is generally known from the prior art to use filters to reduce aliasing. For example, US 5,684,293 A describes an anti-aliasing low-pass blur filter for reducing artifacts in an imaging device. US 6,021,005 A discloses an anti-aliasing device and methods for optical imaging. The object of the present invention is to provide a solution for generating high-quality top-view images showing a motor vehicle and its surrounding area so that they can be displayed to the driver of the motor vehicle. According to the invention, this problem is solved by a method, a driver assistance system, and a motor vehicle with the features according to the respective independent claims. Advantageous embodiments of the invention are the subject of the dependent claims, the description, and the figures. In a method according to the invention for reducing interference signals in a top-view image showing a motor vehicle and an area surrounding the motor vehicle, the top-view image is determined on the basis of raw images taken by at least two vehicle-side cameras.The following steps a) to c) are performed within the procedure: a) Suppressing or attenuating contrast sharpening and / or edge sharpening for the captured raw images in the case of cameras equipped with integrated sharpening functions, and determining the top-view image based on the raw images without contrast sharpening and / or edge sharpening, b) Generating focus offsets within the captured raw images and determining the top-view image based on the raw images with the focus offsets, c) Identifying at least one image area containing noise in the top-view image and reducing the noise by applying a filter to image elements of the top-view image and / or the raw images corresponding to this image area. The method serves to generate high-quality top-down images showing the vehicle and its surrounding environment, which can be displayed to the driver of the vehicle in the form of a video sequence, in particular a real-time video. The top-down images are generated from the raw images captured by at least two vehicle-mounted cameras. To generate or render the top-down image, the raw images are combined, for example, by a vehicle-mounted control device. In particular, the raw images are captured by high-resolution, wide-angle, multi-camera systems mounted on the vehicle. The multi-camera system can, for example, be configured as a surround-view camera system with four cameras mounted on the vehicle.The surround-view camera system can include a front camera for capturing raw image data of the area in front of the vehicle, a rear camera for capturing raw image data of the area behind the vehicle, and two side cameras for capturing raw image data of the area beside the vehicle. The raw images, as well as the top-down images, can be displayed on a screen, for example, according to a view selected by the driver. Displaying the top-down images on the vehicle's screen can assist the driver in maneuvering the vehicle. The driver can perceive the surroundings simply by looking at the screen. The surround-view camera system and the screen form a camera-monitor system (CMS) that can replace the vehicle's mirrors. The CMS can therefore enable a mirrorless car. To improve the image quality of the top-down views, interference or aliasing effects in the top-down images are at least reduced. Specifically, the interference consists of high-frequency signals that appear as flickering or sparking in the top-down images. This interference is not present in the raw images, but is introduced during the rendering of the top-down images based on the raw images, which are often captured while the vehicle is in motion. As a first step (step a), built-in sharpening functions or characteristics of the cameras are disabled or reduced. Most cameras or camera systems include integrated image processing units with built-in contrast sharpening and / or edge sharpening. Even if the interference is not generated by the built-in sharpening functions, these functions can exacerbate the severity of the interference.The camera's processing unit can be controlled, for example, by the vehicle's control unit. This prevents the sharpening functions from being applied to the raw images, providing the control unit with unprocessed raw images for rendering the top-down view. By suppressing or at least attenuating these camera sharpening functions and determining the top-down views based on the unprocessed raw images, the introduction of interference signals into the top-down views can be easily avoided or reduced. Additionally, a second step, step b), is performed, which involves an optical process. In step b), a focus offset is intentionally created within the raw images. This focus offset produces a focusing error and is visible as blurred raw images captured by the cameras. The optical blur reduces the high-frequency signals and consequently the aliasing in the top-down images, which are determined based on the blurred raw images. Steps a) and b) are performed specifically before rendering the top-down images and can therefore be seen as preventative measures to avoid introducing interference signals into the top-down image. Furthermore, a third step, step c), is performed, which relates to an image processing procedure. Within this third step, the at least one image area in the top-down image that contains the interference signals is determined or predicted. Image elements or pixels within this image area of ​​the top-down image and / or image elements of the raw images that contribute to the affected image area are filtered. Step c) can be performed before top-down image generation by applying the filter to the raw images, after top-down image rendering by applying the filter to the top-down image, or both. By identifying and filtering the affected image areas in the top-down images and / or the raw images, computation time and computational effort can be reduced. In summary, by performing all steps a) to c), the image quality of the top-down view displayed to the driver is improved, and therefore the driver's visual quality is enhanced. In one embodiment of the invention, a sub-area within the respective detection range of the cameras is defined with a predetermined dimension and / or a predetermined position relative to the respective camera, and the image elements of the top-view image corresponding to this sub-area are defined as image elements of the image area containing interference signals. This embodiment is based on the understanding that aliasing occurs primarily in specific areas surrounding the vehicle, the position and size of which depend on camera parameters. This means that for each camera, there exists a sub-area within that camera's detection range whose corresponding image elements introduce the interference signals into the top-view image.By locating this sub-area, the image elements of the raw images and / or the top-down image elements responsible for the aliasing can be identified. To determine the sub-area of ​​a camera's detection range, a distance from that camera is specified, and the edges of the sub-area are defined at this distance. This method takes into account the fact that aliasing primarily occurs on a road surface close to the vehicle and, consequently, close to the camera. Furthermore, an angular range around a central projection axis of the camera can be specified, and the edges of the sub-area can be adjusted to this angular range. By specifying the dimensions and / or the location of the sub-area for each camera, the sub-area can therefore be defined quickly and easily. Consequently, the image area affected by the interference signal can also be determined quickly and easily. In an advantageous embodiment of the invention, a distribution is determined that describes the number of image elements from the raw images that contribute to generating the top-view image within a specific image area, and the at least one image area containing interference signals is determined based on this distribution. Preferably, the distribution is determined by subdividing the surrounding area, e.g., a ground surface, into sub-areas and determining a measure for each sub-area that describes a ratio between the number of image elements from the raw images and the top-view image that are used to represent the respective sub-area in the top-view image.In other words, the distribution, also known as a "pixel density map," is a metric used to measure the pixel ratio of the raw images to the combined top-down images, and it reveals the severity of the aliasing effect depending on the location of the image elements in the top-down image. Therefore, the surrounding area is subdivided, a specific region on the ground surface is selected, and the number of pixels occupied by this region in the raw and top-down images is determined. Because the distribution is independent of the camera parameters, the image area affected by noise can be advantageously determined in a universally applicable way. In a further development of the invention, only the image elements within the at least one image area and / or the image elements contributing to the at least one image area containing interference signals are filtered by applying a low-pass filter to these image elements. If the image area of ​​the top view is determined by defining the position and / or the size of the partial detection areas of the cameras, the filter is applied locally to the image area of ​​the top view and / or to the image elements of the raw images that contribute to the image area of ​​the top view. Such a filter operation can be downsampling, neighborhood interpolation, and / or pixel averaging and filters out high-frequency components of the image signal. Alternatively, a guided filter is used to filter the image elements of the raw images and / or the top-down image by determining weighting factors based on the distribution of these elements. Here, the pixel density map serves as a guide image for this guided filter, which is applied to the top-down image and / or the raw images. The guide image allows for localized restriction of the image processing operation, ensuring a smooth transition between image areas containing noise and those without. In an advantageous embodiment of the invention, the at least one image area containing interference signals within the top-down image is estimated, and the integrated sharpening functions are suppressed or attenuated for those image elements of the raw images that correspond to the estimated image area of ​​the top-down image. In this embodiment, edge sharpening and / or contrast sharpening are locally suppressed or attenuated and applied only to those image elements of the raw images that are responsible for the aliasing artifacts in the combined top-down image. Therefore, a high-quality, high-contrast top-down image with reduced interference signals can be combined. Alternatively, the integrated sharpening functions for all image elements of the raw images are suppressed or attenuated, and selective edge and / or contrast sharpening is performed on image areas of the top-down view without interference signals after the top-down view has been determined. Here, the top-down view is post-processed, with edge sharpening and / or contrast sharpening applied to those image areas that are free of interference signals. By applying the sharpening functions only to image areas where the aliasing effect is not amplified, the driver can be displayed a high-contrast and low-noise top-down view. Furthermore, the focus misalignment is preferentially generated in the raw images by introducing a lens misalignment into the camera and capturing the raw images with these misaligned lenses. The cameras, in particular, incorporate fisheye lenses to increase their detection range. These lenses can be slightly displaced from their nominal position, resulting in a focus misalignment and thus lens misalignment. This misalignment can occur during camera manufacturing or during the mounting of the cameras on the vehicle. Consequently, the cameras incorporate a built-in lens misalignment and can capture blurry raw images. Within these blurry raw images, high frequencies of the pixel luminance are smoothed, thereby reducing the noise signals within the combined top-down images. In one embodiment of the invention, before performing steps a) to c), it is determined whether interference signals are to be expected when determining the top-view image. Steps a) to c) are only performed if interference signals are expected. This step is based on the understanding that the aliasing effect does not always occur, for example, depending on environmental conditions. In the presence of these environmental conditions, anti-aliasing is not necessary, and therefore steps a) to c) can be omitted. This proves advantageous because the method is efficient. Preferably, at least one environmental condition, in particular a road surface texture for the vehicle and / or a time of day and / or weather conditions, is determined. Based on this at least one environmental condition, it is determined whether interference signals are to be expected. For example, if the road is covered with a film of water, aliasing effects are very low due to the reduced contrast of the road surface and the reflections caused by the water film on the road surface. The water film may also cover the camera lenses, causing the cameras to capture blurred images due to the water-covered lenses. A top-down image determined based on blurred images includes reduced interference signals. A road surface texture that reduces flicker in the top-down images may be formed by very small or very large road gravel. At low light levels, such as...Furthermore, during night driving, interference signals are barely visible in the top-down images. In these cases, the anti-aliasing process can be suppressed. If interference is expected, its severity is preferably determined, and the strength of steps a) to c) is adjusted accordingly. The severity of the interference can vary, for example, depending on the environmental conditions. If the weather changes from heavy rain to light rain, the severity of the interference can also change. Therefore, the severity of the interference is determined, particularly in relation to the current environmental conditions, to adjust the strength of the anti-aliasing algorithm. Edge sharpening and / or contrast sharpening, for example, can be completely disabled or simply reduced. This ensures that the anti-aliasing method works well in every case, such as every environmental condition. In one embodiment of the invention, the severity of interference signals is determined by calculating the statistical dispersion of pixel values ​​as a function of the pixels' positions in the top-down image. Threshold parameters can be calculated as aliasing area indicators based on this statistical dispersion, since aliasing top-down image areas exhibit a higher dispersion of digital pixel values ​​than non-aliasing image areas. Based on the calculated aliasing indicator, aliasing areas can be distinguished from aliasing-free areas. Such aliasing area indicators can be a pixel value standard deviation calculated for defined areas. For example, if the standard deviation calculated for a defined area exceeds a predefined threshold, the anti-aliasing method comprising at least one of steps a) to c) can be performed. Furthermore, the invention relates to a driver assistance system for a motor vehicle comprising at least two cameras for capturing raw images from an area surrounding the motor vehicle, a control device designed to perform a method according to the invention, and a display device for showing the top-view image with reduced interference signals, which is determined by the control device. In particular, the driver assistance system comprises four cameras forming a surround-view camera system. A motor vehicle according to the invention comprises a driver assistance system according to the invention. The motor vehicle is in particular designed as a passenger car. The preferred embodiments and their advantages presented with reference to the method according to the invention apply accordingly to the driver assistance system according to the invention and to the motor vehicle according to the invention. The terms "in front of", "behind", "next to", "above", "left", "right", etc., indicate the positions and orientations of an observer standing in front of the motor vehicle and looking in one direction along a longitudinal axis of the motor vehicle. Further features of the invention are evident from the claims, the figures, and the description of the figures. The features and combinations of features mentioned above in the description, as well as those subsequently mentioned in the description of the figures and / or shown in the figures alone, are not only usable in the combinations specified, but also in other combinations or on their own, without departing from the scope of the invention. Thus, embodiments of the invention are also to be considered as encompassed and disclosed that are not explicitly shown and explained in the figures, but which can be derived and generated from the explained embodiments by separate combinations of features. Embodiments and combinations of features are also to be considered disclosed that do not exhibit all the features of an originally formulated independent claim.Furthermore, embodiments and combinations of features, in particular those set out above, are to be considered disclosed which go beyond or deviate from the combinations of features set out in the cross-references of the claims. The invention will now be explained in more detail on the basis of preferred embodiments and with reference to the accompanying drawings. Figure 1 shows a schematic representation of an embodiment of a motor vehicle according to the invention; Figure 2 shows a schematic representation of a raw image captured by a vehicle-mounted camera; Figure 3 shows a schematic representation of a top-view image created from raw images; Figure 4 shows a schematic representation of a general image pipeline; Figure 5 shows a schematic representation of an image pipeline for generating the top-view image; Figure 6 shows a schematic block diagram of an example of a method according to the invention; Figures 7a and 7b show images processed by built-in sharpening functions of the cameras; Figures 8a and 8b show images without processing by built-in sharpening functions of the cameras; Figure 9 shows an image area captured by a focused camera; Figure 1010 a representation of an image area captured by a defocused camera; Fig. 11a, Fig. 11b, Fig. 11c representations of top-view images with aliasing areas; Fig. 12a, Fig. 12b representations of filtered images; and Fig. 13 diagrams of pixel values ​​of an arbitrarily selected row of pixels from the image areas shown in Fig. 9 and Fig. 10. In the figures, identical and functionally equivalent elements are provided with the same reference symbols. Fig. 1 shows a motor vehicle 1, which in this case is configured as a passenger car. The motor vehicle 1 includes a driver assistance system 2 that can assist the driver of the motor vehicle 1 in driving the motor vehicle 1. The driver assistance system 2 includes a surround-view camera system 3 for monitoring an area 4 around the motor vehicle 1. The driver assistance system 2 comprises four cameras 5a, 5b, 5c, 5d mounted on the vehicle. A first camera 5a is mounted in a front section 6 of the motor vehicle 1 and serves to capture raw images 11a showing the area 4 in front of the motor vehicle 1. A second camera 5b is mounted in a rear section 7 of the motor vehicle 1 and serves to capture raw images 11b showing the area 4 behind the motor vehicle 1.A third camera 5c is mounted on the left side 8 of the vehicle 1 and is used to capture raw images 11c showing the area 4 to the left of the vehicle 1, and a fourth camera 5d is mounted on the right side 9 of the vehicle 1 and is used to capture raw images 11d showing the area 4 to the right of the vehicle 1. The raw images 11a, 11b, 11c, 11d or raw video frames captured by the cameras 5a, 5b, 5c, 5d can be displayed on a display device 10 of the driver assistance system 2 in the form of a video. Fig. 2 shows the raw image 11b, which is captured by the second camera 5b, mounted in the rear section 7 of the motor vehicle 1. Consequently, the raw image 11b is a rear-view image showing the surrounding area 4 behind the motor vehicle 1. In particular, the raw image 11b shows a road surface 12 of a road for the motor vehicle 1, as well as guide lines 13 that visualize the movement of the motor vehicle 1's wheels for the driver. The raw image 11b shown in Fig. 2, together with the raw images 11a, 11c, 11d captured by the remaining cameras 5a, 5c, 5d of the surround-view camera system 3, can be combined to determine a top-view image 14', 14 of the motor vehicle 1 and the surrounding area 4. Fig. 3 shows a top-down view 14', which is affected by interference signals 17. Top-down views 14', 14 or top-down video frames can be displayed on the display device 10. In the top-down view 14, 14', the surrounding area 4 is shown from the perspective of an observer above the vehicle 1, looking down at it. The top-down view 14, 14' gives the impression that it was recorded by a camera, a so-called virtual camera, positioned above the vehicle 1. Since the vehicle 1 itself cannot be captured by the cameras 5a, 5b, 5c, 5d of the surround-view camera system 3, a model 15 of the vehicle 1 is inserted into the top-down view 14, 14'. Due to image fusion, image areas 16, which are affected by interference signals 17 in the form of flickering effects, are present in the top-down view 14'.These flickering effects, also known as aliasing effects or aliasing, occur mainly near the mounting points of cameras 5a, 5b, 5c, 5d, especially when the vehicle 1 is moving. Fig. 4 shows a schematic representation of a general image pipeline 18 or video pipeline, which is represented by a set of components 19, 20, 21, 22, 23. Using the video pipeline 18, a customer view image 24, such as the top view image 14', can be generated based on raw images or raw videos 25, such as the raw images 11a, 11b, 11c, 11d, which are captured by the cameras 5a, 5b, 5c, 5d of the surround-view camera system 3. Light from the ambient area 4 of the cameras 5a, 5b, 5c, 5d is projected via lenses 19 onto an image sensor unit 20 of the cameras 5a, 5b, 5c, 5d, which includes an image converter and a microprocessor, e.g., an accompanying chip. Based on the image quality settings 21 of the cameras 5a, 5b, 5c, 5d, the image sensor unit 20 generates the raw images 25 as output. A computing device 22, which, for example,A virtual view, for example a top view, may be contained in a vehicle-side control device 26 or ECU (electronic control unit). Based on the settings of the virtual view and a calibration output 23, the customer view image 24 for display on the display device 10 can be determined. Fig. 5 shows a representation of a video pipeline 27 for generating the top-view image 14' with a point at which the interference signals 17 are introduced into the top-view image 14'. Here, the video pipeline includes a parallel path for machine vision algorithms. The raw images 11a, 11b, 11c, 11d, which are captured by the cameras 5a, 5b, 5c, 5d, are fed to the control device 26. In particular, interference signals 17 are not present or rather not visible within the raw images 11a, 11b, 11c, 11d. The raw images 11a, 11b, 11c, 11d can be stored in a storage device 28 or in RAM (random access memory). The stored raw images 11a, 11b, 11c, 11d can be provided to a machine vision processing unit 29, which analyzes the raw images 11a, 11b, 11c, 11d.The raw images 11a, 11b, 11c, 11d can, for example, be analyzed with regard to objects present in the surrounding area 4 in order to output object-based information 30. This object-based information 30 can also be displayed to the driver. Furthermore, the raw images 11a, 11b, 11c, 11d can be fed to a digital signal processor 31 with a pre-filter 32 to filter the raw images 11a, 11b, 11c, 11d, and a top-down renderer 33 to render the top-down image 14'. The interference signals 17 are introduced by the top-down renderer 33 as part of the top-down image rendering. The top-down image 14' includes the interference signals 17. Despite the application of a post-filter 34 to the top-down image 14', the interference signals 17 cannot be removed from the top-down image 14'. These top-down images 14', which are affected by the flickering effects, can be perceived as disturbing when displayed to the driver. To generate a top-down image 14 in which the interference signals 17 are at least reduced, the control device 26 of the driver assistance system 2 is designed to perform step S1, S2, S3, S4, S5 of a method shown by way of example in Fig. 6. The purpose of steps S1, S2, S3, S4, S5 of the anti-aliasing method is to improve the visual quality of the surround-view camera system 3 in a motor vehicle application. A first step S1 involves suppressing or attenuating built-in contrast sharpening and / or edge sharpening of at least one of the cameras 5a, 5b, 5c, 5d. Before a final image is delivered, a camera system can usually perform certain preprocessing procedures, such as local edge sharpening and unsharp masking, to improve the visual quality of the final output image. Contrast and edge sharpening are widely used techniques for increasing the apparent sharpness of an image, i.e., image sharpness. Edge sharpening is frequently applied in a camera-image signal processing (ISP) chain. If edge sharpening is applied in the camera ISP, or before the top-view image 14' is generated, in areas where top-view aliasing occurs, the edge sharpening can exacerbate the aliasing effect.It is worth noting, however, that edge sharpening does not create the aliasing effect, but rather intensifies the existing effect. Fig. 7a shows the top view image 14' within the image areas 16 with the interference signals 17, which are also referred to as aliasing areas. The interference signals 17 are present in the top view image 14' due to the contrast and / or edge sharpening applied to the raw images 11a, 11b, 11c, 11d before image fusion. Fig. 7b shows, by way of example, the raw image 11a, which is captured by the first camera 5a. To improve the image quality of the raw image 11a, contrast and / or edge sharpening was applied to the raw image 11a. Due to these built-in sharpening functions of the camera 5a, the raw image 11a includes high-contrast image areas 35. By switching off the built-in sharpening functions of at least one of the cameras 5a, 5b, 5c, 5d, the aliasing effect or the noise signals 17 can be significantly reduced. Preferably, the sharpening is switched off locally for the aliasing areas 16, while the sharpening functions are kept switched on for other non-aliasing image areas. To attenuate the top-view aliasing, the edge and contrast sharpening in the raw images 11a, 11b, 11c, 11d is reduced or deactivated before the top-view image is generated. To prevent areas without aliasing from appearing less sharp than desired, selective edge sharpening can be applied as a post-processing step to the output top-view image 14, for example in the digital signal processor 31. Fig. 8a shows the top-view image 14 with reduced noise signals 17, which is created using the raw images 11a, 11b, 11c, 11d with sharpening functions switched off. In Fig.8b shows the raw image 11a taken with the first camera 5a, with low contrast image areas 36. A second step S2 for reducing the interference signals 17 involves an optical method. The optical lenses 19, for example fisheye lenses, of cameras 5a, 5b, 5c, 5d have the ability to modify frequency components in the final raw images 11a, 11b, 11c, 11d. To reduce the interference signals 17 in the combined top-view image 14, the optical fisheye lenses 19 are slightly displaced from their nominal positions to achieve certain defocused cameras 5a, 5b, 5c, 5d. A certain degree of optical blurring is introduced, and high-frequency aliasing can be reduced. Fig. 9 shows a sharp image area 37 from one of the raw images 11a, 11b, 11c, 11d, which were captured by a focused camera 5a, 5b, 5c, 5d. Within the sharp image area 37, a texture 38 of the road surface 12 is clearly visible. Fig.Figure 10 shows a blurred image area 39 from one of the raw images 11a, 11b, 11c, 11d, which were taken by a defocused camera 5a, 5b, 5c, 5d. From Figure 10, it can be observed that the high frequency of the pixel luminance is smoothed when the camera lens 19 has a focus offset. Therefore, the aliasing effect is reduced. A third step, S3, involves an image processing method for processing the input / output images, namely the raw images 11a, 11b, 11c, 11d and / or the top-down image 14, at the pixel level. This can help to filter out high-frequency aliasing. Applying common image processing methods for high-frequency filters, such as downsampling, neighborhood interpolation, and / or pixel averaging (e.g., luma part for the YUV image format), reduces the aliasing effect. This can be performed on the input raw images 11a, 11b, 11c, 11d, or on the combined output images 14, or on both, both spatially and temporally. Since implementing these image processing techniques in an embedded system environment where real-time speed is a critical requirement is challenging, it is desirable to improve the third step S3 with regard to time efficiency. Since top-view aliasing occurs mainly at specific local locations 40 within the vehicle's surroundings 4, image processing can only be applied to those image areas 16 that correspond to these locations 40. These locations 40, which correspond to sub-areas of viewports or detection areas of cameras 5a, 5b, 5c, 5d within the surroundings 4, are shown within the top-view image 14' in Fig. 11a. Consequently, the sub-areas 40 have relationships with few parameters, such as a distance to the respective real camera 5a, 5b, 5c, 5d and / or an angle of incidence to a central projection axis of the camera 5a, 5b, 5c, 5d. The aliasing effect, for example, progressively weakens with increasing distance of the ground surface or road surface 12 from the real camera 5a, 5b, 5c, 5d. Based on these locations 40, the image areas 16 with interference signals 17 are identified as semicircular areas 41, which are shown in Fig. 11b.Image processing techniques such as pixel downsampling, averaging and interpolation can only be applied locally to the raw images 11a, 11b, 11c, 11d within the image areas of the raw images 11a, 11b, 11c, 11d that contribute to the aliasing areas 16 of the top view image 14. Since a non-smooth transition can occur from a locally processed area 16, e.g., the semicircular areas 41, to areas that are not processed, a local operation can be performed to overcome this problem. To perform the filtering of the raw images 11a, 11b, 11c, 11d and / or the top-down image 14, a pixel density map 42 (see Fig. 11c) can be determined. Pixel density is a quantitative measure of the severity of the aliasing effect. It is a metric for measuring the pixel ratio of the camera raw images 11a, 11b, 11c, 11d to the combined output top-down image 14. Several methods can be used to calculate this pixel ratio, and to make their calculation independent of variable parameters of the cameras 5a, 5b, 5c, 5d and viewports, a specific area on the floor 12 can be selected.It can then be calculated how many pixels or image elements this area can occupy in the raw images 11a, 11b, 11c, 11d or the output images 14. Fig. 11c shows the pixel density map 42 or pixel density distribution in the top-view image 14. Areas 43 with higher pixel density correspond to the aliasing areas 16 and indicate that more pixels from the original pixels in the raw images 11a, 11b, 11c, 11d are used for the top-view image. The density generally decreases away from a certain point depending on the position of the real camera 5a, 5b, 5c, 5d. Using this trend and applying different weights to the pixel processing can be the content of the third step S3. This can help to smooth the boundary of the processed area and also help to selectively filter the high frequency.Using pixel density data as a guide, the image processing operation can therefore be locally restricted and target only the aliasing area 16, which has a certain pixel density range. The rectangular image area 44 in Figs. 11a and 11b corresponds to the image area occupied by the model 15 of the motor vehicle 1. Fig. 12a shows the top view image 14, where the interference signals 17 are at least reduced due to the image processing applied here to the top view image 14 and the raw image 11b shown in Fig. 12b. A fourth step, S4, involves detecting the aliasing image areas 16 and evaluating their severity. The severity can be indicated using an aliasing indicator. Under certain scenarios, the top-view aliasing effect may be weak or even disappear. In other words, this means that the aliasing effect does not always occur, and under certain environmental conditions, it might not be necessary to apply anti-aliasing solutions. Without anti-aliasing, system resources can be saved. For example, the aliasing effect was observed to be low when the road surface 12 was wet, such as after rain, because the film of water on the road surface 12 reduces the contrast of the road surface 12 and also creates some diffuse light reflection. If the camera lens 19 has a film of water on it, the aliasing effect also decreases. This can result from the blurring caused by the lens 19. Furthermore, the interference signals 17 are not present at low light levels, such as during night driving, due to the low light intensity of the road surface 12. When sunlight is present during the day, the aliasing effect is also not visible in the shadow of the vehicle 1 due to the relatively low light level in the shadowed area. Moreover, the aliasing effect is reduced on certain road surfaces with very small or very large gravel.This could be explained by the frequency range of the road gravel, indicating that a rough road surface 12 is a required but not sufficient condition for the aliasing output. One principle of calculating the aliasing indicator is to distinguish aliasing areas from aliasing-free areas. A pixel value standard deviation in defined areas can serve as a simple indicator of aliasing. As mentioned above, the severity of the aliasing effect can vary depending on the smoothness of the road surface and the size of the road gravel. To improve the anti-aliasing process, a metric for evaluating the severity of aliasing can be determined. Based on this metric, which also provides an evaluation of the results of anti-aliasing operations, the strength of the anti-aliasing can be adjusted for varying scenarios, for example, by disabling the anti-aliasing algorithm or reducing its strength. Since the top-view aliasing area 16 exhibits a higher dispersion of digital pixel values ​​than the non-aliasing area, the data variability of the pixel matrix, also called statistical dispersion, can be calculated to detect aliasing areas 16. Statistical data dispersion can be measured using several statistical indices, such as data range, standard deviation, distance standard deviation, mean absolute deviation, coefficient of variation, and relative mean difference, etc. In general, higher indices indicate greater dispersion of the data. The severity of the aliasing could be assessed by the relative values ​​of the statistical indices. Fig. 13 shows two graphical representations of data sets 45, 46 of the digital values ​​47 (ordinate) of the pixels for an arbitrarily selected pixel row 48 (abscissa) of the two images shown in Fig. 9 and Fig. 10. Data set 45 is taken from Fig. 9 and data set 46 is taken from Fig. 10.Standard deviations for the two sets of data 45 and 46 shown in Fig. 13 can be calculated, with the standard deviation of data set 45 being higher than the standard deviation of data set 46. This indicates that the aliasing effect caused by image area 39 in Fig. 10 is less pronounced than the aliasing effect caused by image area 37 in Fig. 9. The standard deviation of another part of the raw image, in which no overall aliasing effect occurs, can also be calculated, and the result can be set as the target of the anti-aliasing operation. A fifth step S5 comprises a hybrid form of steps S1 to S4. This means that at least one of steps S1 to S4 is performed by the control device 26 of the driver assistance system 2 in order to reduce interference signals in the top view image 14.

Claims

Method for reducing interference signals (17) in a top-down image (14) showing a motor vehicle (1) and an area (4) surrounding the motor vehicle (1), wherein the top-down image (14) is determined on the basis of raw images (11a, 11b, 11c, 11d) acquired by at least two vehicle-mounted cameras (5a, 5b, 5c, 5d), and wherein the method includes the following steps a) to c): a) suppressing or attenuating contrast sharpening and / or edge sharpening for the acquired raw images (11a, 11b, 11c, 11d) in the case of cameras equipped with integrated sharpening functions (5a, 5b, 5c, 5d), and determining the top-down image (14) on the basis of the raw images (11a, 11b, 11c, 11d) without the contrast sharpening. and / or edge sharpening, b) Creating focus offsets within the captured raw images (11a, 11b, 11c, 11d) and determining the top view image (14) based on the raw images (11a, 11b, 11c,11d) with the focusing offsets, c) Identifying at least one image area (16) in the top view image (14) containing interference signals (17) and reducing the interference signals (17) by applying a filter to image elements of the top view image (14) and / or the raw images (11a, 11b, 11c, 11d) corresponding to this image area (16). Method according to claim 1, characterized in that a partial area (40) in a respective detection area of ​​the cameras (5a, 5b, 5c, 5d) is determined with a predetermined dimension and / or a predetermined position with respect to the respective cameras (5a, 5b, 5c, 5d) and the image elements of the top view image (14) corresponding to the respective partial area (40) are determined as image elements of the at least one image area (16) containing interference signals (17). Method according to claim 1, characterized in that a distribution (42) is determined which describes a number of image elements of the raw images (11a, 11b, 11c, 11d) which contribute to the generation of the top view image (14), and at least one image area (16) containing interference signals (17) is determined based on the distribution (42). Method according to claim 3, characterized in that the distribution (42) is determined by dividing the surrounding area (4) into sub-areas and determining for each sub-area a measure which describes a ratio between the number of image elements of the raw images (11a, 11b, 11c, 11d) and the top view image (14) that are used to represent the respective sub-area. Method according to one of claims 2 to 4, characterized in that only the image elements within the at least one image area (16) and / or the image elements contributing to the at least one image area (16) containing interference signals (17) are filtered by applying a low-pass filter to these image elements. Method according to claim 3 or 4, characterized in that a guided filter for filtering the image elements of the raw images (11a, 11b, 11c, 11d) and / or the top view image (14) is determined by determining weighting factors depending on the distribution (42) for weighting the image elements of the raw images (11a, 11b, 11c, 11d) and / or the top view image (14). Method according to one of the preceding claims, characterized in that the at least one image area (16) containing interference signals (17) within the top view image (14) is estimated and the integrated sharpening functions for those image elements of the raw images (11a, 11b, 11c, 11d) that correspond to the estimated image area of ​​the top view image (14) are suppressed or attenuated. Method according to one of claims 1 to 6, characterized in that the integrated sharpening functions for all image elements of the raw images (11a, 11b, 11c, 11d) are suppressed or attenuated and selective sharpening is performed for image areas of the top view image (14) without interference signals (17) after the determination of the top view image (14). Method according to one of the preceding claims, characterized in that the focusing offset is generated in the raw images (11a, 11b, 11c, 11d) by providing a misalignment of lenses (19) of the cameras (5a, 5b, 5c, 5d) and by taking the raw images (11a, 11b, 11c, 11d) with the lenses (19) with the misalignment. Method according to one of the preceding claims, characterized in that before carrying out steps a) to c), it is determined whether interference signals (17) are to be expected when determining the top view image (14), wherein steps a) to c) are only carried out if interference signals (17) are to be expected. Method according to claim 10, characterized in that at least one environmental condition, in particular a texture (38) of a road surface (12) for the motor vehicle (1) and / or a time of day and / or weather conditions, is determined and on the basis of the at least one environmental condition it is determined whether interference signals (17) are to be expected. Method according to claim 10 or 11, characterized in that if interference signals (17) are to be expected, the severity of the interference signals (17) is determined and the strength of steps a) to c) is adapted to the severity of the interference signals (17). Method according to claim 12, characterized in that the severity of the interference signals (17) is determined by determining a statistical dispersion of image element values ​​as a function of the position of the image elements in the top view image (14). Driver assistance system (2) for a motor vehicle (1) comprising at least two cameras (5a, 5b, 5c, 5d) for capturing raw images (11a, 11b, 11c, 11d) of an area (4) of the motor vehicle (1), a control device (26) designed to perform a method according to one of the preceding claims, and a display device (10) for displaying the top view image (14) with reduced interference signals (17) determined by the control device (26). Motor vehicle (1) with a driver assistance system (2) according to claim 14 .

Images