Method for reducing color shift of image pixels of motor vehicle image captured by camera

Abstract

A method for reducing color shift of image pixels of an image of a motor vehicle captured by a camera, a computer program product and a control device for the vehicle are disclosed. For a respective image pixel of an image, the method comprises: determining color information describing a color of the image pixel; it is checked whether the determined color information is greater than the minimum color information and less than the maximum color information, where the minimum and maximum color information define a range of color information in which reference image pixel wobbles of the reference object are described. In the case of successful inspection, corrected color information is calculated in consideration of the determined color information and the correction factor, and the calculated corrected color information is provided instead of the determined color information, in which the corrected color information has a reduced color shift compared to the determined color information.

Description

Technical Field

[0001] The invention relates to a method for reducing color shifts of image pixels of an image of a motor vehicle captured by a camera. Furthermore, the invention relates to a computer program product and a control device for a motor vehicle. Background Art

[0002] By means of an optical capture system for a motor vehicle, which typically comprises a plurality of cameras, various representations of the motor vehicle's environment can be realized, each representation being composed of a plurality of camera images captured by the capture system. Firstly, for example by selecting and combining several camera images, which for example describe predetermined sub-areas of the motor vehicle's environment, can be remapped in an overall representation for this purpose. Thus, a mosaic image can be created as a new overall image, which for example represents a top view of the motor vehicle's environment observed from a bird's eye perspective, the motor vehicle being arranged in the center of this top view. Such a top view representation can also be referred to as a top view representation.

[0003] Camera images and / or general representations captured by a camera of a motor vehicle are usually modified by means of at least one image processing algorithm before being displayed to a user of the motor vehicle, for example by a display device inside the motor vehicle.

[0004] For example, US2018 / 0204310 A1 shows a visualization system for a motor vehicle. The noise level in the image data of the visualization system is improved by applying a lens shading algorithm to the image data.

[0005] Processing of the camera image and / or the overall representation may be due, for example, to changing or different lighting conditions and / or the behavior of a camera lens of the camera employed, so that, for example, individual or several image pixels of the camera image or the overall representation, respectively, may exhibit color shifts which should be corrected. Summary of the invention

[0006] The object of the present invention is to provide a solution by which color shifts of image pixels can be reduced.

[0007] A first aspect of the invention relates to a method for reducing color shifts of image pixels of a motor vehicle image captured by a camera. It is assumed that at least one image is captured, for example, by a camera of a motor vehicle. The image describes, for example, a sub-area of ​​the motor vehicle environment. This means that the image, for example, at least partially describes at least a portion of a road surface of a road on which the motor vehicle is arranged, with infrastructure adjacent to the road surface, such as buildings, factories and / or another road user, such as another motor vehicle and / or a person. The image may also be referred to as a camera image. The image may also be an overall image composed of several camera images. The image includes a number of image pixels, which together form an image. The camera that captures the image at least partially is, for example, a front camera, a rear camera and / or a side camera of the motor vehicle. In addition, the captured image may not be captured by the motor vehicle itself, but may be provided to the vehicle, for example, by an external device via a communication link, such as another motor vehicle, a computing device and / or an infrastructure device, such as a traffic observation camera.

[0008] The invention is based on the recognition that an image usually comprises a color shift of the hue of the image pixels there in at least one subregion of the image. The color shift can also be referred to as a hue shift. The color shift can be caused by camera lens-related inhomogeneities in the color representation of the camera lens of the corresponding camera. Depending on the conditions of the camera lens, the hue in the image, for example in a subregion of the image, can have a color shift towards cyan or blue, while in another subregion, a color shift towards magenta or red, so that the hue of, for example, a monochrome area in the image can swing locally. This leads, for example, to a single subregion of the image having a blue color shift visible to the image observer, while other subregions of the image have a red color shift. Such inhomogeneities with regard to the hue often occur in edge regions of the image, which are far from the image center. Particularly noticeable are the color shifts of large-scale monochrome objects in the described image, such as the gray surface of a road. Therefore, the color values ​​of the individual image pixels of the image should be corrected in order to be able to achieve an overall representation of the image that is harmonious in terms of the color representation.

[0009] It is conceivable that by the image signal processor of the camera, the inhomogeneities in the color representation are already counteracted, for example by applying a lens shading algorithm of the image signal processor to the image. However, it may be the case that this is not sufficient and further measures are required to reduce the color shifts of individual image pixels of the image. This may be the case, for example, of severe color decay between individual camera lenses if the color decay has a profile that cannot be modeled by simple methods, for example because it cannot be modeled using polynomial curve fitting. Then an additional correction of the color values ​​of at least individual image pixels may be useful.

[0010] This means that the images should be provided with an additional processing step which is adapted to reduce the color shift of the image pixels. Since the observed color shift may depend on the incident light, so that for example depending on the position of the sun the color shifted image pixels vary, the processing step should provide a correction for each image pixel individually which always takes into account the current color shift affecting the respective image pixel.

[0011] The method according to the invention comprises several method steps, which are performed for respective image pixels of an image captured by a camera. Preferably, as part of the method, each individual image pixel of a captured image or at least one sub-region of a captured image is considered. The method steps envisage determining color information describing the color of an image pixel. The image information for example indicates whether the image pixel is red, orange, yellow, green, blue, purple or magenta. Alternatively, the color information may be referred to as color data, wherein the color data then describes the color of the image pixel, for example by means of a color value. The color value may for example take a numerical value or a coordinate in a color space of a color model.

[0012] In a further method step, it is checked whether the determined color information is greater than the minimum color information and less than the maximum color information. The minimum and maximum color information delimit a color information range, in which the reference image pixels describing the reference object swing, in particular typically swing. The reference object can be, for example, a road surface, i.e. a light grey, dark grey or black object. The minimum and maximum color information delimit a color space region, for example a color space of a color model, according to which the determined color information is described. Within the color information range there are color information of many, in particular all image pixels of the image, which describe the reference object or an object at least similar to the reference object in terms of color. The minimum and maximum color information can be determined, for example, by a calibration method, which is performed, for example, before applying the method described so far or at the beginning thereof. In other words, this means checking whether the color information of the image pixels lies within the color information range delimited by the minimum and maximum color information. Preferably, the minimum and maximum color information delimit a predetermined sub-region of the entire color space, which sub-region, for example, lies only within the typical color information range of the reference object, for example a grey road surface. The delimited color information range includes color information with a color cast or hue shift, for example towards cyan or magenta. This means that when checking, for example, whether the color information of each image pixel indicates that it describes a reference object, such as a gray road surface, because the determined color information is greater than the minimum color information and less than the maximum color information, in the case where the reference object is a gray road surface, an image pixel describing, for example, a blue sky or green grass in the environment of a motor vehicle in this example will not be recognized as color information of an image pixel that lies within the color information range bounded by the minimum and maximum color information.

[0013] The calculation of the corrected color information of the image pixel is only implemented if the determined color information is greater than the minimum color information and less than the maximum color information. The calculation is implemented taking into account the determined color information and the correction factor. The correction factor is predetermined and depends, for example, on the camera lens of the camera that captures the image. Subsequently, it is implemented to provide the calculated corrected color information instead of the determined color information. The corrected color information has a reduced color shift compared to the determined color information. This is achieved by recalculating the color information of the affected image pixels and thus shifting it to the corrected color information according to the correction factor. The corrected color information is different from the calculated color information. In other words, for all image pixels that were successfully checked, the determined color information is replaced by the corrected color information. In the case where the determined color information is not both greater than the minimum color information and less than the maximum color information, it is envisaged that the corrected color information is not calculated. Therefore, the determined image information is provided for these image pixels. This means that only the color information assigned to the image pixels that were successfully checked is modified.

[0014] The method can end with providing an image with all its image pixels, wherein for all image pixels whose check was successful, corrected color information is provided instead of the determined color information. This means that those image pixels whose color information lies outside the color information range bounded by the minimum and maximum color information are accurately corrected. The color information of all other image pixels remains unchanged. Thus, a reduction in the color shift in the selected image pixels is ultimately achieved, since, if necessary, corrected color information with a reduced color shift compared to the determined color information is provided instead of the determined color information.

[0015] An advantageous embodiment envisages that the reference object is a road surface or a light box. The road surface is in particular a grey road surface, for example a light grey or dark grey road surface. The road surface is for example the surface of an asphalt road. The light box is for example an object emitting light of a predetermined wavelength, usually in the range of white light. The color information and / or brightness information of the light emitted by the light box is preferably predetermined and therefore known. Preferably, the color of the reference object is therefore in the color information range from black / dark grey to light grey / white. Since motor vehicles are usually located on grey road surfaces, objects with color information in this color information range are often encountered in the environment of the motor vehicle. Therefore, the top view of the environment directly adjacent to the motor vehicle usually includes at least one grey sub-area, which usually includes a large sub-area of ​​the top view compared to the rest of the top view. Depending on the brightness selected, a grey area in the color space of the color model can correspond to a white area in the color space, so that by referring to the light box as a reference object, a color shift can be identified, which can be transferred to a grey object, for example a road surface. Thus, by selecting a reference object, the method is specifically designed for identifying and correcting color shifts in a color space that is a typical environment for motor vehicles, leading to a wide range of application of the method in motor vehicles.

[0016] Additional embodiments contemplate describing the corresponding determined color information by means of the U color value and / or the V color value of the YUV color model. This means that a common color model suitable for describing the color information of image pixels can be adopted. The YUV color model can describe the brightness of image pixels with a Y value, and describe the color with a U value and a V value, i.e., describe the color with a U color value and a V color value. The U and V values ​​span a color space, which can be referred to as a UV color space, in which all colors of the corresponding Y value can be represented. In the case of considering only the U and V color values, a two-dimensional color space is therefore mapped, which is, for example, from the U to V axis, each from a value of 0 to a value of 255. Therefore, the color information includes two single color values, i.e., the U and V color values. By indicating the two color values ​​together, the color information can be accurately indicated, i.e., the color of the image pixel can ultimately be described.

[0017] It is particularly envisaged that the minimum color information is described by the minimum U color value and / or the minimum V color value, and the maximum color information is described by the maximum U color value and / or the maximum V color value. Therefore, a total of four color extremes are known, thereby ultimately defining a sub-region of the UV color space of the YUV color model.

[0018] In another embodiment, it is envisaged that the minimum and maximum U color value and the minimum and maximum V color value delimit a color space region arranged around the center of the UV color space of the YUV color model. The UV color space at a U color value of 128 and a V color value of 128 comprises a center, wherein around the center, depending on the Y value, colors from black / dark gray to light gray / white are arranged. By placing the delimited color information range in terms of color at this location, it is achieved that only objects in the image whose colors lie between black / dark gray and light gray / white can be corrected in terms of their color information, while, for example, objects in the image with different colors (e.g. blue, green, red or yellow) remain unaffected in terms of their color information. Thus, it is ensured that a reduction in color shifts is achieved only in sub-regions of the image, in which this is meaningful, for example due to large surface representations of gray road surfaces, in which color shifts are particularly noticeable.

[0019] Additional embodiments envisage that the distance between the determined color information and the corrected color information in the UV color space is greater the further the determined color information is from the center of the UV color space. This means that a degree of correction that depends on the color information of a single pixel can be envisaged. This leads to the fact that, for example, an image pixel with a large color deviation, whose color information is therefore far away from the center of the UV color space, is modified more strongly than an image pixel with a small color deviation in contrast, because it is arranged closer to the center of the UV color space. This leads to the difference between the determined and corrected U and V color values ​​of the color information of an image pixel with a large color deviation being greater than for an image pixel with a small color deviation. This means that not every image pixel that is found to require correction during inspection is corrected to the same extent.

[0020] The calculation of the corrected U color value from the determined U color value can be achieved, for example, by subtracting the number 128 from the determined U color value of the image pixel to determine the difference with the center of the UV color space. The corrected U color value is then calculated as a difference, wherein the determined U color value is the minuend and the product of the difference and the correction factor is the subtrahend. The calculation of the corrected V color value is achieved in a similar manner. The correction factor is, for example, 0.75. Alternatively, the correction factor can be between 0.5 and 1, between 0.6 and 0.9, and in particular between 0.7 and 0.8. It is thus recognized that the degree of correction applied depends on the distance of the determined U or V color value from the center of the UV color space, respectively, and therefore on the individual image pixel. Ultimately, it is achieved that the color information of the edge areas closer to the defined color information range is corrected more strongly toward the center, thereby generally giving a color representation that is as homogeneous as possible in the provided image, such as a color representation of a gray road surface. Through the described consideration of the determined color information and the correction factors, it can be achieved that not every image pixel is corrected to the same extent, but rather a progressive correction is possible, or gradients are taken into account during the correction.

[0021] In an embodiment, it is conceivable to determine a plurality of image pixels for which the determined color information is greater than the minimum color information and less than the maximum color information. Then, it is checked whether the determined number is greater than the minimum number. In the case where the determined number is less than or equal to the minimum number, alternative minimum color information and alternative maximum color information are determined by evaluating the image pixels. The color information range bounded by the alternative minimum and maximum color information is arranged to be spaced apart in particular from the center of the UV color space of the YUV color model. Subsequently, for example, the above method can be performed taking into account the alternative minimum value and the alternative maximum value color information. Thus, for example, in the case where the motor vehicle is not arranged on a gray road surface, but, for example, on a grass surface, first, an adjustment of the minimum and maximum color information of the environment of the motor vehicle is performed. This is because all or at least many image pixels of the captured image of the grass surface do not describe a gray or white object, however, for example, the methods to date are configured for this gray or white object. It can thus be conceived that a reduction in the color shift of the image pixels is still possible, because possible deviations in the green range can be reduced by calculating the alternative minimum and maximum color information and taking it into account in the following. It is conceivable that, first, a new calibration is to be performed in order to be able to calculate alternative minimum and maximum color information, for example using a reference object having a different color, in particular green.

[0022] Alternatively or additionally, it is conceivable that corrected color information for the corresponding image pixel is calculated only if the determined number is greater than a minimum number. This means, for example, that a control step can be provided in which, whenever, for example, only a sufficiently small number of grayscale image pixels is found in the image that is less than or equal to the minimum number, first no further method steps are carried out, but rather it is merely determined that the captured image is not suitable for the method, because not enough image pixels are given in the relevant color space region. This means that, first, it is checked whether the captured image is suitable at all for processing by the method, because it, for example, comprises gray and / or white areas in which a correction can be performed. Thus, unnecessary calculation work is avoided, which could otherwise occur if the method were continued without taking into account the number of image pixels affected.

[0023] The minimum number can be expressed as a percentage, for example 5%, 10%, 20%, 30%, 40%, 50%, 60%, 70%, 80%, 90% of the total number of image pixels in the image, or in particular between greater than 90% and less than 100%.

[0024] Furthermore, an embodiment envisages applying a coordination function to the image before checking the determined color information of the corresponding image pixels. The coordination function performs brightness and / or color coordination of the image. Thus, it is envisaged that before checking whether the determined color information is greater than the minimum color information and less than the maximum color information, a coordination with respect to the brightness and color of the image can already be performed. This coordination is particularly meaningful if the image is an overall representation consisting of individual camera images, which images are captured, for example, under different lighting conditions, because they show, for example, different areas of the motor vehicle environment. Then, by applying the coordination function, a brightness and color correction can be performed, wherein in particular in the transition area between the two camera images, a brightness and color adjustment between the two camera images is achieved. However, color shifts of individual image pixels in the coordinated image may also be observed, for example due to different conditions and quality of the camera lens from camera lens to camera lens of a particular camera type. However, such slight color shifts cannot usually be corrected by applying the coordination function, but require the method of the invention to calculate the corrected color information. Thus, it is achieved that by applying the coordination function earlier, a pre-correction of the colors in the image is already achieved, so that the starting image represents a reasonable basis for a fine-tuning correction, which can be achieved by calculating the corrected color information. Thus, an optimal reduction of the color shift can ultimately be achieved.

[0025] In addition, it is conceivable in an embodiment that an image is a partial image of an overall image composed of at least two partial images. The overall image can be composed of, for example, four partial images, one of which is captured by a front camera, one by a rear camera, and two by corresponding side cameras, which are arranged in the side mirrors of a motor vehicle. The composite image can be realized by combining or merging a single partial image. Therefore, each partial image describes a sub-area of ​​the motor vehicle environment. This means that a partial image describing the environment of the area in front of the motor vehicle, a partial image describing the environment of the area behind the motor vehicle, a partial image describing the environment of the motor vehicle on the left side of the motor vehicle longitudinal direction, and a partial image describing the environment of the motor vehicle on the right side of the motor vehicle longitudinal direction are given. These four partial images can be combined with each other, for example, into a 360-degree image of the environment or a top view of the motor vehicle. It is conceivable that, for example, before applying a coordination function, the synthesis of the individual partial images into an overall image is performed. With the help of a coordination function, the specific boundary areas between the individual partial images can be coordinated, that is, they can be adjusted to each other in terms of brightness and / or color values. This means that the best application of images captured by various cameras of a motor vehicle can be provided.

[0026] Another embodiment envisages that after checking the determined color information of each image pixel of at least two images, an overall image is composed. Thus, first, the determined color information of each image pixel of at least two different images is checked and possibly corrected, and only thereafter the overall image is composed of at least two images. Thus, the overall image comprises at least two partial images. Thus, the individual images of the overall image have been checked according to the above-described method, depending on the color information of the image pixels and the corresponding image pixels corrected by providing corrected color information or not by maintaining the determined color information. Here, a coordination function is applied to the overall image, wherein at least two images describe different sub-areas of the environment of the motor vehicle, in particular the front area, the rear area and / or the side area of ​​the motor vehicle. As already described above, the coordination function performs brightness and / or color coordination on the overall image. This means that it may be the case that the above-described method, which envisages checking the determined color information and (if applicable) calculating the corrected color information and providing it to the corresponding image pixels, is applied to the partial images, and these partial images are synthesized and adjusted with each other only on this basis by the coordination function.

[0027] An embodiment envisages that each partial image is a window of a camera image captured by a camera, wherein the overall image shows a top view of the motor vehicle's surroundings. It can therefore be envisaged that, instead of viewing and further considering the complete image captured by the camera, the window is selected from the complete image. A window is the name for an observation window. In order to display a top view of the motor vehicle's surroundings, for example, an area of ​​the environment that is located at the top in the height direction of the motor vehicle and is contained in the camera image, such as a part of the sky, is not required. For this reason, only the window is considered as a partial image that makes up the overall image. This is a particularly preferred representation for a motor vehicle and is useful, for example, for the driver of the motor vehicle when assisting with parking operations.

[0028] Another embodiment envisages identifying at least one object in the image by applying an object recognition algorithm to the image. Only if the identified object is a road surface, corresponding corrected color information for corresponding image pixels of the image is calculated. Therefore, first, it is checked whether the captured image, partial image or overall image is suitable for the described method by performing an object check based on the object recognition algorithm and a check as to whether the identified object is a road surface. The object recognition algorithm is, for example, a computer program, which contains at least one rule or instruction according to which at least one predetermined object in the image can be identified. The predetermined object can be a road surface. In addition, the object recognition algorithm can distinguish between various types of objects, i.e. recognize, for example, various types of objects. Therefore, it is avoided to calculate corrected color information for image pixels of an image that is not suitable for applying the described method at all because it does not, for example, show a road surface.

[0029] In another embodiment, it is conceivable to predetermine the correction factor depending on the minimum and maximum color information. For example, a larger correction factor can be conceived if the distance between the minimum and maximum color information is larger than if the distance between the minimum and maximum color information is smaller, so that in this case a smaller correction factor can be selected by comparison. This means that the smaller the color information space delimited by the minimum and maximum color information, the smaller the correction factor that can be selected, and vice versa. Thus, the correction factor can be adjusted to the behavior of the camera lens so that a reasonably reduced color shift can always be achieved.

[0030] Furthermore, an embodiment is envisaged according to which, when performing the calibration method, reference pixels are captured by several cameras. These several cameras may be assigned to a common camera type. A common camera type may relate to a common camera lens. This means that the several cameras each have substantially the same camera lens. Thus, the several cameras, for example, have a camera lens of a common type and therefore have the same camera lens. Substantially the same means that there is a maximum difference between the several camera lenses, which is independent of the color information range in which the color information of the image pixels may vary. Preferably, the camera used when capturing the image also has the camera lens used when performing the calibration method. Therefore, the camera used when capturing the image is also assigned to a common camera type. In other words, the calibration method is performed for a specific camera lens.

[0031] Preferably, at least ten reference images with reference pixels are captured from the reference object in order to calculate the minimum and maximum color information therefrom. In the reference images, for example, areas of a road surface and / or a light box are shown and selected from them. Subsequently, it is calculated or determined in which color information range, i.e. in which color space region, the image pixels describing the road surface or the light box swing, even if they should actually represent the same color information. This means determining whether the reference object in the reference image is described by at least one reference pixel with a color shift, for example a red shift or a blue shift. The color information with the largest or smallest shift relative to the center of the color space is used here as the maximum or minimum color information, respectively. In this regard, it is relevant that, first of all, for a normal camera lens, several calibration measurements are performed before the minimum and maximum color information can be provided as predetermined values.

[0032] Another aspect of the invention relates to a computer program product. The computer program product may be, for example, a computer program. The computer program product may be stored on a computer, for example a control device in a motor vehicle. The computer program product comprises at least one instruction, preferably several instructions, which, when executed by a computer, for example a control device of a motor vehicle, will perform the above method. The computer is, for example, a computing device. Furthermore, a second computer program product may be provided, according to which a camera captures an image of the surroundings of the motor vehicle and / or a reference image. Furthermore, a third computer program product may be provided, according to which the described calibration method may be performed.

[0033] Another aspect of the invention relates to a control device for a motor vehicle, which is configured to perform the above method. The control device comprises a processor device, which can perform the described method. For this purpose, the processor device can include at least one microprocessor and / or at least one microcontroller and / or at least one FPGA (field programmable gate array) and / or at least one DSP (digital signal processor). In addition, the processor device can include a computer program product. The control device preferably performs the described method.

[0034] The exemplary embodiments described in conjunction with the method according to the invention apply analogously, both individually and in combination with one another, insofar as applicable and insofar as they are not mutually exclusive, to the computer program product according to the invention and to the control device according to the invention.

[0035] Another aspect of the invention may be a motor vehicle comprising a control device according to the invention. The motor vehicle may be, for example, a passenger car, a truck, a bus and / or a motorcycle. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 is a schematic diagram of a motor vehicle including a control device and a number of cameras,

[0037] Figure 2 is a signal flow diagram schematically illustrating a method for reducing color shifts of image pixels of an image of a motor vehicle captured by a camera,

[0038] Figure 3 It is a schematic diagram of the UV color space of the YUV color model.

[0039] Figure 4 is a schematic diagram of camera image correction for a motor vehicle,

[0040] Figure 5 is a schematic diagram showing the Figure 2 A signal flow diagram of additional method steps of the method,

[0041] Figure 6 is a schematic diagram showing the Figure 2 A signal flow diagram of the method steps of the calibration method, and

[0042] Figure 7 is a signal flow graph schematically illustrating the application of an object recognition algorithm to an image. DETAILED DESCRIPTION

[0043] exist Figure 1In FIG. 1 , a motor vehicle 1 is shown, which includes a control device 2. The control device 2 is a computing device, which can include, for example, at least one microprocessor and / or a microcontroller. In other words, the control device 2 of the motor vehicle 1 is a computer, which can, for example, execute a computer program product as a computer program.

[0044] The motor vehicle 1 comprises several cameras, namely a front camera 3, a rear camera 4 and two side cameras 5, each of which is arranged in a side mirror of the motor vehicle 1. In the synthesis of the camera images captured by these four cameras of the motor vehicle 1, for example, a 360° environment of the motor vehicle 1 and / or a top view of the motor vehicle 1 can be presented. The representable environment is defined at least by the capture range of the respective camera, namely the front camera 3, the rear camera 4 or the side camera 5.

[0045] In the motor vehicle 1, a display device 6 is arranged, which is for example a display, in particular a touch-sensitive display. On the display device 6, for example, camera images captured by the front camera 3, the rear camera 4 and / or the corresponding side camera 5 can be presented. For this purpose, the camera images can first be manipulated or processed by the control device 2. The individual cameras, namely the front camera 3, the rear camera 4 and / or the side cameras 5, can for example each include a fisheye lens, which allows 180° capture of the environment. A top view of the environment of the motor vehicle 1 can be provided, for example, by selecting a corresponding part from each camera image, so that, for example, the current environment of the motor vehicle 1 can be at least partially or preferably completely displayed to a user of the motor vehicle 1, i.e., for example, via the display device 6, for assisting parking operations.

[0046] exist Figure 2 In FIG. 4 , a method is shown for reducing color shifts of image pixels 8 of an image 7 of a motor vehicle captured by a camera, in particular a front camera 3, a rear camera 4 and / or a side camera 5. The image 7 may be, for example, a camera image captured by the respective camera.

[0047] In a first method step S0, an image 7 of the environment of the motor vehicle 1 can be captured by the front camera 3, the rear camera 4 and / or the side camera 5, the image 7 consisting of individual image pixels 8. The number of image pixels 8 depends on the respective camera. Preferably, for each image pixel 8 of the image 7 captured by the respective camera (i.e. the front camera 3, the rear camera 4 and / or the side camera 5), ​​the further method steps S1 to S5 described below are performed.

[0048] In method step S1, color information 9 is determined which describes the color of an image pixel 8. The color information 9 preferably describes a U color value and / or a V color value of a YUV color model. Thus, for each image pixel 8, the color information 9 preferably comprises two values, namely a U color value and a V color value.

[0049] In method step S2, it is checked whether the determined color information 9 is greater than the minimum color information 10 and less than the maximum color information 11. The minimum color information 10 can alternatively be referred to as the minimum color information and the maximum color information 11 can alternatively be referred to as the maximum color information. The minimum color information 10 and the maximum color information 11 define a color information range, in which the reference image pixel swing of the reference object is described. To this end, the minimum color information 10 can include a minimum U color value 14 and / or a minimum V color value 15, and the maximum color information 11 includes a maximum U color value 16 and / or a maximum V color value 17.

[0050] Only in the case where the determined color information 9 is greater than the minimum color information 10 and less than the maximum color information 11, in method step S3, the calculation of the corrected color information 13 for the image pixel 8 is implemented. The calculation is implemented taking into account the determined color information 9 and the correction factor 12. The correction factor 12 can be up to 0.75, for example. Therefore, in method step S4, instead of the determined color information 9, the calculated corrected color information 13 is provided. Compared with the determined color information 9, the corrected color information 13 includes a reduced color shift. This means that it has been achieved that the individual color information 9 of the image pixel 8 for which the check step S2 was successful is modified in such a way that a uniform color distribution in the image 7 is achieved. In addition, as a result, in the case where the check step S2 is unsuccessful, i.e., in the case where the determined color information 9 is not both greater than the minimum color information 10 and less than the maximum color information 11, in method step S5, the color information 9 that has been determined is provided for the corresponding image pixel 8, because no corrected color information 13 is calculated. Overall, the image 7 will therefore comprise image pixels 8 which either comprise determined color information 9 or comprise corrected color information 13, depending on the result which the checking step S2 has produced.

[0051] exist Figure 3 , a UV color space 18 of a YUV color model is outlined. A minimum U color value 14, a maximum U color value 16, a minimum V color value 15, and a maximum V color value 17 are exemplarily shown therein. These four extreme values ​​are arranged around a center 19 of the UV color space 18. If the U axis and the V axis of the UV color space 18 each extend between 0 and 255, the center 19 of the UV color space 18 has a U value of 128 and a V value of 128, which is a typical case in the YUV color model. These four extreme values ​​define a color space region 20 around the center 19 of the UV color space 18. The color space region 20 is a range of color information describing the swing of a reference pixel of a reference object.

[0052] The UV color space 18 has areas of different colors, which are shown as different shades depending on the color. The colors range from magenta 50 to red 51, yellow 52, ​​green 53 and blue 54. The mixed color areas between the colors are drawn by overlapping shades. Around the center point 19, the gray tones also vary from one another, ranging from gray with a red cast 55 to pure gray 56 to gray with a blue cast 57.

[0053] exist Figure 3 , two exemplary color information 9 are plotted for the corresponding image pixel 8, namely the first color information 9a and the second color information 9b. Both are located within the plotted color space region 20, that is, when the check is performed in method step S2, the corresponding color information 9a, 9b is evaluated as color information 9, which is greater than the minimum color information 10 and less than the maximum color information 11. For this purpose, in method step S3, for each of the two color information 9a, 9b, corrected color information 13 is calculated, namely the first corrected color information 13a and the second corrected color information 13b. Obviously, the distance 21 between the first determined color information 9a and the first corrected color information 13a in the UV color space is greater than the distance 21 between the second determined color information 9b and the second corrected color information 13b. This is because the further the determined color information 9 is from the center 19 of the UV color space 18, the greater the distance 21 between the determined color information 9 and the corrected color information 13, and here the first determined color information 9a is farther from the center than the second determined color information 9b.

[0054] exist Figure 3 In the example shown, for example, the minimum U color value of 14 may be located at 124 , the maximum U color value of 16 may be located at 130 , the minimum V color value may be located at 123 , and the maximum V color value may be located at 137 .

[0055] In the event that the checking step S2 is successful, firstly, in order to calculate the corrected U color value from the determined U color value, the difference to the center of the UV color space can be determined by subtracting the number 128 from the determined U color value of the image pixel 8. The corrected U color value is then calculated as a difference value, wherein the determined U color value is the minuend and the product of the previously calculated difference value and the correction factor 12 is the subtrahend. The calculation of the corrected V color value is carried out in an analogous manner. In method step S4, the corresponding calculated values ​​are now provided as corrected color information 13 of the image 7, which includes the corrected U color value and the corrected V color value.

[0056] exist Figure 4In the method, for example, four images 7 of four cameras of the motor vehicle 1 can first be predetermined. This means that initially, for method step S0, an image 7 of the front camera 3 is shown in the upper left corner, an image 7 of the rear camera 4 is shown in the upper right corner, an image 7 of the side camera 5 on the left in the longitudinal direction of the motor vehicle 1 is shown in the lower left corner, and an image 7 of the side camera 5 on the right in the longitudinal direction of the motor vehicle 1 is shown in the lower right corner. In each image 7, a viewing window 22 is drawn, which is here alternatively or additionally referred to as a partial image 23. The individual images 7 show the environment of the motor vehicle 1, in which a road surface 24 is drawn over a large area, on which the motor vehicle 1 is arranged. The road surface 24 is gray, in particular between black to dark gray and light gray to white. In addition, in the respective images 7, a parking space marking 25, a shadow 26 of the motor vehicle 1, several background objects 27, such as trees, sky 28 and grass 29 are drawn. Purely by way of example, two areas in the image 7 are highlighted in their color by corresponding hatching, wherein these areas are shown here in gray with a red color cast 55. Alternatively, the image 7 may comprise at least one grey region with a bluish colour cast 57 and / or another grey region with a reddish colour cast 55. Furthermore, further or alternative colour casts are conceivable.

[0057] In method step S6, the four viewing windows 22 can be synthesized as corresponding partial images 23 to form an overall image 30. In other words, the individual images 7 and here in particular the viewing windows 22 can be corresponding partial images 23 of an overall image 30 composed of several partial images 23. Each partial image 23 describes a sub-region of the environment of the motor vehicle 1, in particular the front region, the rear region and / or the side region of the motor vehicle 1. Therefore, the image 7 can alternatively be considered as an overall image 30 composed of several partial images 23. The motor vehicle 1 is located in a central region of the overall image 30, which is not captured by the camera of the motor vehicle 1. By taking into account the four viewing windows 22, the overall image 30 here exemplarily represents a top view of the environment of the motor vehicle 1. In the synthesized overall image 30, various image areas are highlighted, in which, for example, a red shift occurs. For this purpose, a method similar to Figure 3 Same shade in .

[0058] In method step S7, a coordination function 31 can be applied to the image 7, here the overall image 30. The coordination function 31 performs a brightness coordination 32 and / or a color coordination 33 on the image 7 or the overall image 30, respectively. Thus, an example image of a coordination image 34 is shown here, from which it can be clearly seen, for example, that the transition areas between the individual viewing windows 22 are adjusted to each other, so that a continuous transition in color and brightness between the individual partial images 23 is achieved. In addition, various objects from the individual images 7 of the camera, that means, for example, parking space markings 25 or shadows 26, can also be seen in the coordination image 34. The coordination image 34 is therefore the result of the synthesis of the image 7, here the viewing windows 22, in method step S6 and the subsequent application of the coordination function 31 to the overall image 30.

[0059] Subsequently, the method according to method steps S1 to S3 can be applied to the harmonized image 34 as image 7. The harmonized image 34 thus corresponds to the image 7, whose individual image pixels 8 are taken into account in method steps S1 to S5. A resulting image 35 is drawn, which is provided in method steps S4 and S5 by the correspondingly provided image pixels 8. As a result, all red-shifted regions of the image 7 are corrected, which means that the gray regions with a red color cast 55 and thus the color shift there are reduced in such a way that a correspondingly improved image 7 is provided in the form of the resulting image 35. The resulting image 35 preferably shows only the road surface 24 in pure gray 56, without a noticeable color cast.

[0060] As a pair Figure 4 As an alternative to the sequential arrangement of the method steps described in , after method step S0, method steps S1 to S5 can already be performed directly, and only then can the overall image 30 be composed of individual partial images 23 (method step S6). Then, method step S7, i.e. the coordination function, is applied to the composite image, so that the resulting image 35 can be provided.

[0061] exist Figure 5, further possible method steps S8 to S10 are outlined. In method step S8, a number 36 of image pixels 8 is determined for which the determined color information 9 is greater than the minimum color information 10 and less than the maximum color information 11. In method step S9, it is then checked whether the determined number 36 is greater than the minimum number 37. In the case that the determined number 36 is less than or equal to the minimum number 37, in method step S10, a determination of an alternative minimum color information 10' and an alternative maximum color information 11' is performed. The color space area 20 delimited by the alternative minimum color information 10' and the maximum color information 11' is arranged in particular to be spaced apart from a center 19 of the UV color space 18 of the YUV color model. For example, it is conceivable that, in the case where the motor vehicle 1 is located on green grass which does not correspond to the grey tone of, for example, the road surface 24 and is therefore not located in the area of ​​the center 19 of the UV color space 18 in terms of color, alternative minimum color information 10' and alternative maximum color information 11' are determined and that method steps S1 to S5 can then be performed, for example, taking into account the alternative minimum color information 10' and the alternative maximum color information 11'.

[0062] It is also conceivable that corrected color information 13 of the corresponding image pixel 8 is calculated, ie method step S3 is performed, only if determined number 36 is greater than minimum number 37. This means that it can be assumed that a check is first performed to determine whether sufficient image pixels 8 are present in the relevant color region to be corrected.

[0063] exist Figure 6 In the present invention, a calibration method is outlined, wherein firstly, a particularly grey road surface 24 or a light box 39 is captured by means of a front camera 3, a rear camera 4, a side camera 5 and / or a further reference camera 38, for example arranged outside the vehicle, in order to be able to provide an image 7 comprising reference pixels. Preferably, at least ten reference images are captured as images 7 in method step S11 in order to be able to reliably provide the calibration method.

[0064] In method step S12, a calibration algorithm 40 comprising at least one rule for calibrating the camera is applied to the reference pixels captured here with their respective captured information 9. The reference pixels represent a reference object, i.e. the road surface 24 and / or the light box 39. For this purpose, the determined color information 9 is always arranged around the center 19 of the UV color space 18. By applying the calibration algorithm 40, the minimum color information 10 and the maximum color information 11 can be determined.

[0065] The cameras employed in method step S11 essentially all comprise the same camera lenses. This means that they can be assigned to a common camera type. The camera lenses employed are therefore comparable to one another at least with regard to the color shift, which will be observed in the images 7 captured by the respective camera lenses. It can therefore be concluded that the camera lens-specific color space region 20 is bounded by the determined extreme values. Depending on the camera lenses of the front camera 3, the rear camera 4 and / or the side cameras 5 of the motor vehicle 1, the corresponding minimum color information 10 and the maximum color information 11 determined in method step S12 can be provided to the motor vehicle 1. Furthermore, method steps S11 and S12 can be performed in the motor vehicle 1, for example, based on at least ten images 7 captured by the respective cameras of the motor vehicle 1.

[0066] exist Figure 7 , an embodiment is depicted in which, in method step S13, an object recognition algorithm 41 is applied to an image 7 in order to recognize at least one object 42 in the image 7. If, in method step S14, it is found that the recognized object 42 is a road surface 24, method steps S2 to S5 are performed. This is because the motor vehicle 1 is then found to be located on a gray road surface 24, so that the above-described method associated with the center 19 of the UV color space 18 in the gray area can be performed.

[0067] In the event that the check performed in method step S14 is unsuccessful, method step S5 can be implemented, for example, that for each image pixel 8, already determined color information 9 can be provided, and no corrected color information 13 is calculated and provided. Alternatively or additionally, method step S10 can be performed, that is, alternative minimum color information 10' and alternative maximum color information 11' can be determined. Alternatively, in method step S15, the method can be terminated, that is, method steps S1 to S5 can not be performed at all, because image 7 is not recognized as suitable for the method, because no road surface 24 is shown in image 7.

[0068] In general, these examples illustrate algorithms for mitigating color distribution inhomogeneities of camera lenses. It is proposed to apply the described method after applying a coordination function 31 to a synthetic overall image 30, which is a top view of the environment of the motor vehicle 1. Thus, each individual image pixel 8 that is tilted in a certain direction in the UV color space 18 is individually corrected. Each image pixel 8 is considered individually by the method, which checks how far the determined color information 9 of the image pixel 8 is from the center 19 of the UV color space 18 relative to its UV coordinates in the UV color space 18, the center 19 comprising the coordinates (128, 128). The road surface 24 is the most uniform surface observed by the camera of the motor vehicle 1 in the top view and will therefore cover a large area in the overall image 30 composed of the individual viewing windows 22. In addition, the coordination function 31 will correct the individual partial images 23 towards the center 19 of the UV color space 18. In method steps S1 to S5, the corresponding color information 9 is checked with respect to the distance 21 of its U and V color values ​​from the center 19 of the UV color space 18. As described above, the distance 21 is shifted by shifting the U and V values ​​towards the center 19 by multiplying the correction factor 12. The gradient of the color information 13 to which the corresponding color information 9 of the corresponding image pixels 8 is shifted to reach the correction determines the intensity of the non-uniform camera lens performance. This should be measurable by a change in the color value of a homogeneous surface of a reference object, which is here given by the road surface 24 or the light box 39. Thus, it is achieved that the correction is performed only in certain color areas in the image 7 and that, for example, image pixels 8 of different colors, which describe, for example, a parking space marking 25 or different background objects 27, remain unaffected.

Claims

1. A method for reducing color shift of image pixels of an image of a motor vehicle captured by a camera, the method comprising, for respective image pixels of the image captured by the camera: determining color information, the color information describing the color of a pixel of the image; checking whether the determined color information is greater than minimum color information and less than maximum color information, wherein the minimum color information and the maximum color information define a color information range within which a reference image pixel swing of a reference object is described; and Only when the determined color information is greater than the minimum color information and less than the maximum color information, the corrected color information of the image pixel is calculated taking into account the determined color information and the correction factor, and the calculated corrected color information is provided instead of the determined color information, wherein the corrected color information has a reduced color shift compared to the determined color information.

2. The method according to claim 1, wherein: The reference object is a grey road surface or a light box.

3. The method according to claim 1, wherein: The corresponding color information is described by the U color value and / or the V color value of the YUV color model, wherein the minimum color information is described by the minimum U color value and / or the minimum V color value, and the maximum color information is described by the maximum U color value and / or the maximum V color value.

4. The method according to claim 2, wherein: The minimum U color value and the maximum U color value and the minimum V color value and the maximum V color value define a color space region that is arranged around a center of a UV color space of the YUV color model.

5. The method according to claim 4, wherein: The greater the distance between the determined color information and the corrected color information in the UV color space, the farther the determined color information is from the center of the UV color space.

6. The method according to claim 4, wherein: Determining the number of image pixels for which the determined color information is greater than the minimum color information and less than the maximum color information, and checking whether the determined number is greater than the minimum number, wherein: In the case where the determined number is less than or equal to the minimum number, determining a substitute minimum color information and a substitute maximum color information by evaluating the image pixels, A color space region defined by the substituted minimum color information and the substituted maximum color information is arranged to be spaced apart from a center of a UV color space of the YUV color model; and Only if the determined number is greater than a minimum number is the corrected color information for the corresponding image pixel calculated.

7. The method according to claim 1, wherein: Before checking the determined color information of the respective image pixels, a harmonization function is applied to the image, wherein the harmonization function performs brightness harmonization and / or color harmonization on the image.

8. The method according to claim 7, wherein: The image is a partial image of an overall image which is composed of at least two partial images, wherein each partial image describes a sub-region of the motor vehicle surroundings in a front region, a rear region and / or a side region of the motor vehicle.

9. The method according to claim 1, wherein: After checking the determined color information for each image pixel of at least two images, an overall image is synthesized, which includes each of the at least two images as a partial image, and a coordination function is applied to the overall image, wherein the at least two images describe different sub-areas of the environment of the motor vehicle in the front area, rear area and / or side area of ​​the motor vehicle, and the coordination function performs brightness coordination and / or color coordination for the overall image.

10. The method according to claim 8, wherein: Each partial image is a window of a camera image captured by a camera, wherein the overall image shows a top view of the surroundings of the motor vehicle.

11. The method according to claim 1, wherein: By applying an object recognition algorithm to the image, at least one object in the image is recognized and only if the recognized object is a road surface, corresponding corrected color information for corresponding image pixels of the image is calculated.

12. The method according to claim 1, wherein: The correction factor is predetermined depending on the minimum color information and the maximum color information.

13. The method according to claim 1, wherein: When performing the calibration method, the reference image pixels are captured by several cameras, each camera comprising substantially the same camera lens as the camera used when capturing the images.

14. A computer program product comprising at least one instruction, which, when a computer executes the program, causes the computer to execute the method according to claim 1.

15. A control device for a motor vehicle, wherein: The control device is configured to carry out the method according to claim 1 .

Images