Method and apparatus for suppressing chromatic aberration in digital images

Abstract

The present invention relates to a method and apparatus for suppressing chromatic aberration in digital images. In particular, a method for suppressing chromatic aberration, especially blue or red fringing, in a digital image having multiple color channels is disclosed. The method comprises negatively correcting a first color channel by subtracting an overshoot component of the first color channel. The subtraction operation is limited to a lower threshold value, which depends on the local value of at least one other color channel.

Description

Technical Field

[0001] The present disclosure relates to the field of digital image processing, and in particular, to a method and apparatus for suppressing chromatic aberration. Background Art

[0002] An image captured by a camera through an imaging system (e.g., a lens system) will inevitably contain distortion. There are different types of distortion, the extent of which depends on several factors related to the characteristics of the imaging system and the scene being imaged, or a combination of both. Chromatic aberration can be attributed to the fact that light of different wavelengths will be refracted differently when passing through a refractive medium, and more specifically, through the lenses of an imaging system. There are two main types of chromatic aberration: axial (or longitudinal) chromatic aberration and lateral (or transverse) chromatic aberration.

[0003] Axial chromatic aberration corresponds to the axial spread of the focus of the various wavelength components of light incident on a lens parallel to its optical axis. As a rule, the focus of shorter wavelengths will be closer to the lens than the focus of relatively longer wavelengths. To illustrate, Figure 1 A lens 102 is shown with an optical axis 108 and an image plane 104 for a green wavelength component 112. It can be seen that the blue component 106 has a shorter focal length, while the red component 110 has a longer focal length. As a result, the sharpness of each of the red, green, and blue components will be uneven among them. In the design of complex optical instruments with multiple refractive elements, it is not uncommon to optimize the selection and spacing of these elements for the green component 112, which is located at the approximate center of the visible spectrum. Artifacts related to axial chromatic aberration include color areas in the image because not all three colors (red, green, blue) can be imaged in focus at the same time, resulting in edges in the image having a blurred appearance.

[0004] In lateral chromatic aberration, different wavelength components are focused on a common image plane, but exhibit lateral spread relative to the optical axis. Lateral chromatic aberration is generally more pronounced for light rays that are obliquely incident on the lens. Therefore, it does not appear at the center of the image plane. Figure 2 Some effects of lateral chromatic aberration are illustrated. Using the same Figure 1 As can be seen, light rays of different colors originating from a point in the scene will be refracted in such a way that the focal points of the different wavelengths are laterally offset in the image plane 104 .

[0005] Axial and lateral chromatic aberrations affect different optical systems to varying degrees. Both types can be significantly reduced by using advanced achromatic optics, however this may be prohibitively expensive for many important applications. In relatively simple optical systems the use of achromatic optics is inappropriate and the user may need to accept that the system provides imperfect raw images but that subsequent image processing can be applied to reduce the imperfections or minimize their interference.

[0006] The present invention addresses a type of chromatic aberration often referred to as blue fringing. The visible sign of blue fringing is the formation of a blue, purple, or magenta halo around the imaged object, which is particularly noticeable at the edges where dark areas meet bright areas in the image, for example, around backlit objects or high-contrast objects. The optical phenomena described above that produce blue fringing or light blue fringing can also produce red fringing or light red fringing. A given imaging system may be primarily affected by blue fringing or, less commonly, by red fringing. In imaging systems where blue fringing and red fringing are equally important, the chromatic aberration may be referred to as purple fringing.

[0007] Although blue fringing artifacts have been studied for a long time and are generally well understood, available image processing techniques intended to suppress or remove blue fringing still leave much to be desired. In fact, some techniques designed to compensate for blue fringing tend to do so at the expense of introducing new artifacts in the image. In the worst case, the new artifacts may be amplified by subsequent processing steps (e.g., dynamic-range compression (DRC), tone mapping, and sharpening), making the artifacts more troubling to human viewers. For example, DRC tends to brighten darker areas where blue fringes are typically located, which can make them more visible to viewers. Summary of the invention

[0008] It is an object of the present disclosure to provide a method and apparatus suitable for suppressing chromatic aberrations in digital images without introducing new visual artifacts. Another object is to provide such a method and apparatus that is not prone to introducing patterns that are easily converted into visual artifacts by downstream image processing steps. A specific object is to improve the suppression of blue fringing in digital images. In addition, it is also desirable to provide suppression of such local operations, i.e., without the need to calculate global quantities used to characterize the complete digital image. Finally, a further object is to propose such a method and apparatus suitable for use with digital cameras used for monitoring applications.

[0009] At least some of these objects are achieved by the invention as defined in the independent claims.The dependent claims relate to advantageous embodiments of the invention.

[0010] In a first aspect of the invention, a method for suppressing chromatic aberration in a digital image having a plurality of color channels is provided. The method comprises the step of negatively correcting a first color channel by subtracting an overshoot component of the first color channel. Rather than subtracting the overshoot component without restriction, the subtraction operation is limited to a lower threshold. The lower threshold depends on a local value of at least one other color channel.

[0011] The first color channel may correspond to such wavelengths for which the imaging system does not perform optimally and which are therefore relatively more exposed to imaging defects. For example, as explained in more detail above, the focus of the first color channel may deviate axially or laterally from the focus of wavelengths that are relatively more centrally located in the design spectrum of the imaging system. Depending on the color system used, the first color channel may be a blue or light blue hue (to address blue fringing) or a red or light red hue (to address red fringing). The inventors have realized that the compensation performed on the first color channel should be balanced with one or more additional color channels of the digital image. This avoids overcompensation and thereby avoids the formation of too many reverse edges; for blue fringing, the reverse edge is yellowish. One way to achieve this balance is to form a lower threshold based on the additional color channels and make the subtraction operation conditional on the lower threshold, in a sense that the resulting value of the first color channel should not be less than the lower threshold.

[0012] In some embodiments, the lower threshold is a local value of a second color channel.The second color channel may correspond to a center wavelength of a design spectrum for the imaging system, such as green or light green.

[0013] In other embodiments, the lower threshold is a combination of the local values ​​of the second and third (and any other) color channels. The combination can be an arithmetic mean, a geometric mean, or a harmonic mean, where each of these means can have a uniform or non-uniform weighting of the color channels involved. This can help ensure that a negative correction of the first color channel produces visually acceptable results, even when there are significant differences between the second and third color channels.

[0014] In some embodiments, the lower threshold has a granularity of one pixel, whereby the "local value" is a pixel value. In other embodiments, the lower threshold has a granularity of a group of multiple pixels, such as 4, 9, 16, 25 square, diamond or differently shaped pixel groups. This reduces the computational workload of forming the lower threshold. For example, the lower threshold of a pixel group can be set to the value of the pixel located at the center, or it can be set to the average or median of the pixel values ​​in the pixel group.

[0015] In some embodiments, the lower threshold is uniformly implemented throughout the digital image. That is, although the lower threshold is still based on the local value of the other color channel and may therefore vary with the digital image, the subtraction of the overshoot component is conditioned everywhere on the lower threshold, regardless of its value. Alternatively, the lower threshold is only implemented in a sub-region of the digital image, and the complete overshoot component is subtracted without restriction elsewhere. The total computational workload in such an embodiment can be lower.

[0016] In some embodiments, the overshoot component is calculated as the difference between a first color channel of a digital image and a low-pass filtered copy of the same color channel. The difference can be a pixel-by-pixel difference or a difference with higher granularity, for example, using one of the pixel groups outlined above. The difference can further undergo one or more of ramping (i.e., its positive or negative portion can be retained), linear and non-linear scaling. For the avoidance of doubt, the term "overshoot component" in this disclosure refers to a static quantity of a digital image or digital video frame; it has no direct connection to the transient of the same name in certain higher-order dynamical systems.

[0017] It will be appreciated that negative correction may be applied to more than one color channel. For example, according to some embodiments, the effect of purple fringing may be reduced by applying negative correction to the blue and red color channels of the same digital image.

[0018] It will also be appreciated that the negative correction of the first color channel may be performed together with the positive correction. The positive correction may be applied to the first color channel or to a different color channel in the digital image. The positive correction may include adding an undershoot component of the same color channel theme. Optionally, the addition of the undershoot component may be conditional on an upper threshold value that depends on the local value of at least one color channel different from the color channel to which the positive correction is applied.

[0019] In some embodiments, the negative correction (and any positive correction) is performed before any dynamic range compression, tone mapping, or sharpening of the digital image. This ordering is preferable because dynamic range compression, tone mapping, and sharpening may tend to amplify uncompensated artifacts associated with chromatic aberration.

[0020] In a second aspect of the present invention, an image processing device configured to suppress chromatic aberration in a digital image having multiple color channels is provided. The image processing device comprises a memory and a processing circuit configured to perform the method according to the first aspect. The image processing device may be an embedded component of a digital camera, in particular a camera for monitoring a relevant application scenario.

[0021] The second aspect shares the effects and advantages of the first aspect and can be implemented with corresponding degrees of technical changes.

[0022] The invention further relates to a computer program comprising instructions for causing a computer or in particular an image processing device to perform the above method. The computer program may be stored or distributed on a data carrier. As used herein, a "data carrier" may be a transient data carrier (e.g. a modulated electromagnetic wave or light wave), or a non-transient data carrier. Non-transient data carriers include volatile and non-volatile memories, such as permanent and non-permanent storage media of the magnetic, optical or solid-state type. Still within the scope of a "data carrier", such memories may be fixedly mounted or portable.

[0023] In this disclosure, "blue fringing" and similar terms indicating a particular color should be interpreted broadly, i.e., it should be understood that red fringing, magenta fringing, etc. are produced by common refraction-related phenomena. It should also be understood that artifacts of different colors can generally be repaired by similar techniques.

[0024] Generally, unless otherwise expressly defined herein, all terms used in the claims should be interpreted according to their ordinary meaning in the technical field. Unless otherwise expressly stated, all references to "an / an / the element, device, component, means, step, etc." should be openly interpreted as referring to at least one instance of the element, device, component, means, step, etc. Unless expressly stated, the steps of any method disclosed herein do not have to be performed in the exact order described. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Various aspects and embodiments will now be described by way of example with reference to the accompanying drawings, in which:

[0026] Figure 1 is a schematic diagram illustrating the influence of axial chromatic aberration;

[0027] Figure 2 is a schematic diagram illustrating the influence of lateral chromatic aberration;

[0028] Figure 3 is a flow chart of a method for suppressing chromatic aberration in a digital image according to an embodiment of the present invention;

[0029] Figure 4 is a block diagram of the imaging system;

[0030] Figure 5 , Figure 6 and Figure 7 shows different ways in which the image processing apparatus according to the invention can be arranged in or at an imaging system; and

[0031] Figure 8 is a schematic signal processing diagram illustrating some aspects of chromatic aberration suppression according to embodiments herein. DETAILED DESCRIPTION

[0032] Various aspects of the present disclosure will now be described more fully hereinafter with reference to the accompanying drawings in which certain embodiments of the invention are shown. However, these aspects may be embodied in many different forms and should not be construed as limiting; rather, these embodiments are provided by way of example so that this disclosure will be thorough and complete and will fully convey the scope of all aspects of the invention to those skilled in the art. The same reference numerals refer to the same elements throughout the specification.

[0033] A method for clarifying some aspects of chromatic aberration has been described above. Figure 1 and Figure 2 .

[0034] Figure 41 is a schematic diagram of an imaging system, in this case a digital camera. The imaging system has a lens system 120 that images a scene on an image sensor 122. The image sensor 122 is provided with a color filter array, such as a Bayer filter, so that different pixels will receive radiation of specific wavelength regions in a known, usually mosaic-like pattern. After the signals of the individual sensor pixels of the image sensor 122 are read, demosaicing 124 is performed. At this point, the color information in the scene has been extracted, that is, the detection signal from the scene has been separated into components of the color system for which the color filter array is designed. The color channels can be the wavelengths that make up the primary colors, which can be combined into desired points in a subset (color gamut) of the visible color space. The color channels can, for example, correspond to specified red R, green G, and blue B. In order to correct for imperfections in color reproduction, a color correction matrix 126 is applied to the sensor output after demosaicing. The color correction matrix can be described as a calibration matrix that takes into account the wavelength-dependent response for each pixel of the image sensor 122. In a simple process, the color correction matrix is ​​derived by imaging a known target using the imaging system, which is typically a color rendition card. Thus, there is a target of known color characteristics, a resulting image sensor response, and a color correction matrix that, when applied, should bring the sensor response closer to the known target, effectively completing the optimization task of tending to produce the best color correction matrix. Errors in color can be described in different ways, and there are many algorithms for deriving the minimum total error. The present invention does not advocate any particular color error measurement or optimization algorithm. When the color correction matrix 126 is applied, the response from the image sensor (after demosaicing 124) is converted to or at least closer to the value it should have according to the color rendition card. In the most advanced imaging systems, the actual step of performing color rendition card imaging can be omitted, and the color correction matrix can be derived by using the spectral response data for the specific sensor and imaging system.

[0035] After applying the color correction matrix 126, the response from the image sensor should have been corrected pixel by pixel to a true or at least more accurate color representation, where the total color error is minimized. The resulting image is forwarded to the image processing pipeline 128. It should be noted that the application of the color correction matrix 126 can be considered an essential component of the image processing pipeline 128. The image processing pipeline 128 can include noise filtering (for removing spatial and / or temporal noise), distortion correction (for removing the effects of, for example, barrel distortion), global and / or local tone mapping (for example, enabling imaging of scenes containing multiple intensities), transformations (for example, rotation), flat field correction (for example, for removing the effects of vignetting), application of overlays (for example, privacy masks, explanatory text), dynamic range correction (including dynamic range compression), sharpening, etc. These example operations can be arranged in an order deemed appropriate by the implementer. The image processing pipeline 128 can also be associated with an analysis engine for performing object detection, object recognition, alarm triggering, etc. After the image processing pipeline 128, the digital image may be forwarded to an encoder 130, which is responsible for encoding the image data according to an encoding protocol and forwarding it to a recipient, server, cloud storage, etc. in a desired image format.

[0036] Figure 3 is a flow chart of a method 300 for suppressing chromatic aberration in a digital image according to an embodiment of the present invention. The method 300 may be Figures 5 to 7 The type of image processing device 510 shown is implemented in FIG. Figure 4 In a sequential stage of an imaging system of the type described herein, it is advantageous to perform method 300 before image processing pipeline 128. Preferably, method 300 is performed after application of color correction matrix 126. Alternatively, method 300 may be integrated into image processing pipeline 128, wherein method 300 is preferably performed after any noise filtering and before any dynamic range compression, tone mapping, or sharpening operations therein. In double exposure systems and similar systems that merge two exposures of the same scene, method 300 is preferably performed after the exposures are merged, i.e., on the merged image.

[0037] At an overall level, method 300 is structured as a negative correction block 310 and an optional positive correction block 320. Each of blocks 310, 320 may be applied to one color channel or multiple color channels of a digital image. Positive correction 320, if present, may be applied to a color channel different from the color channel subject to negative correction 310. Preferably, correction 310, 320 is applied to a color channel representing a non-central wavelength in the design spectrum of the imaging system, such as a blue (light blue) color channel or a red (light red) color channel. The order of blocks 310, 320 may be as follows: Figure 3As shown or inverted. A staggered order is also possible, where one or more negative corrections alternate with one or more positive corrections. For example, a negative correction of the first color channel may be followed by a positive correction of the first component, which may be followed by a negative correction of the second color channel, with optional further positive and / or negative corrections.

[0038] When the negative correction 310 is applied to the first color channel of the digital image, it includes the step of extracting an overshoot component from the image data associated with the first color channel of the digital image (312). Since blue fringing is known to amplify sharp edges, the overshoot component can be selected to represent a high frequency component of the first color channel, where frequency refers to the spatial variation of the image data. One method of obtaining such a high frequency component is to first calculate a low-pass (LP) filtered copy of the first color channel and then subtract the LP filtered copy from the original first color channel.

[0039] Assume that a digital image is provided in the form of a spatial representation with three color channels {B(u,v), R(u,v), G(u,v)}, each color channel representing a corresponding value (e.g., intensity, brightness) as a function of an index pair (u,v) that is used to specify a pixel (as described above, a coarser granularity can be used, where the index pair (u,v) specifies a predefined group of pixels). The value function can be represented as a table, in particular a lookup table, where the entries can be 8-bit (4+4) floating point numbers. The three color channels are represented in common or comparable units. Preferably, the three color channels use a common linear unit, thereby simplifying cross-channel processing, for example, linear scaling of value triplets substantially preserves hue. The color channels can have original or near-original dynamic range, or they can have a pre-compressed dynamic range. Because method 300 does not rely on the presence of pixel saturation to detect image areas that may potentially be affected by aberrations, method 300 can also be applied to high dynamic range (HDR) images. In fact, method 300 may perform better when applied to HDR images because the local values ​​of the additional color channels on which the lower threshold depends are not altered by saturation.

[0040] One of these types of spatial representations is a simple format for exchanging images between components of a digital imaging system. If post-processing is performed using method 300 outside of an imaging system used to acquire a digital image, the digital image can be provided in a compressed format suitable for storage or data transmission. In this case, it may be necessary to decompress (or unpack) the image data into a spatial representation in which all color channels are in common or comparable units. A similar process can be a useful preparation for method 300, and if the digital image is a frame of a predictively encoded video sequence, an earlier or later frame in the sequence can be selected for use. In addition, it is worth noting that method 300 cannot be easily applied to image data in a YCbCr format, in which the luminance Y and chrominance Cb, Cr components have no direct relationship to color. Therefore, a preliminary conversion to a representation of color channels (e.g., RGB) is necessary.

[0041] Returning to step 312, the literature discloses several suitable algorithms for providing LP filtered copies (312.1) of the channels of a digital image encoded according to one of the above types. Examples include Fourier transforms. One option is to perform a Fourier transform on the spatial representation of the color channels (e.g., using a fast Fourier transform (FFT) or a direct Fourier transform (DFT)).

[0042] {B(u,v):u=1,2,…,U,v=1,2,…V}

[0043] Among them, U×V is the resolution of the digital image, which is converted into frequency representation:

[0044]

[0045] Thus, for |k|>K max , Then it is inversely transformed to include only K max The spatial representation of the frequency content is:

[0046] {B LP (u,v):u=1,2,…,U,v=1,2,…V}

[0047] In order to ensure that for |k|>K max , A Fourier-type transform can be designed so that the high-frequency components are not computed, or those frequency components can be subsequently eliminated before the inverse transform. max This can be done by comparing images of similar scenes and / or images with comparable brightness and color to a tentative K maxMethod 300 is performed to empirically determine, and adjustments are made until visually pleasing results are obtained. Alternatively, a non-perceptual evaluation may be utilized, instead comparing the frequency response and / or sharpness of the compensated first color channel with the same number of color channels corresponding to a centrally located wavelength (e.g., green or aqua).

[0048] Another option is to compute a low-pass filtered copy (312.1) by convolving the spatial representation B(u,v) with kernel k:

[0049] B LP (u,v)=(B*κ)(u,v)=∫∫B(u′,v′)κ(u′,v′;u,v)du′dv′

[0050] This convolution can correspond to a uniform blur, such as a Gaussian blur, in which case the kernel is symmetric with respect to the origin:

[0051] κ(u′,v′;u,v)=κ(|u′-u|,|v′-v|)

[0052] Alternatively or additionally, the kernel κ can be designed to compensate for the point-spread function (PSF) of the optics in the imaging system that acquires the digital image. If the PSF is different for on-axis and off-axis image points, the kernel may not have the symmetric properties described above; instead, the kernel depends on u,v. It is worth noting that the convolution operation can be designed to incorporate one or more steps of method 300, including the difference calculation (312.2) for a properly selected kernel κ to be described below.

[0053] In implementation, to reduce the need for runtime (e.g., on-chip) memory, a kernel κ supported over a large interval can be conveniently split into a sum of multiple sub-kernels κ=κ1+κ2+…, where each sub-kernel is supported over a relatively small interval. Optionally, the implementer can define a variable offset for the sub-kernel, where the offset (rather than the sub-kernel itself) can be determined based on the distance from the optical axis.

[0054] Next is the step of computing the difference between the color channels and the LP filtered copy (312.2):

[0055] B(u,v)-B LP (u,v)

[0056] The difference may optionally be further processed before being used in subsequent steps, for example by combining the difference function with at least one other function g (312.3), thereby obtaining the overshoot component as:

[0057] g(B(u,v)-BLP (u,v)

[0058] Preferably, the function g is almost non-decreasing everywhere or almost increasing everywhere. In a computer implementation of the method 300 , the function g can be represented as a piecewise linear (PWL) function, for which an effective software code is publicly available.

[0059] A first option is to use a ramp function, for example as a function of the positive or negative part of g, defined respectively as:

[0060]

[0061] Or locally smoothed versions of these.

[0062] A second option is to use a constant scaling factor g(x)=α1x, where the coefficient α1 can be used to adjust the magnitude of the negative compensation 310. Preferably, the coefficient α1 is constant for the first color channel in the entire digital image. The corresponding coefficient α2 for the second color channel can be independent of the first coefficient α1. max Similarly, coefficient α1 can be determined empirically by trying different tentative values ​​and selecting the most visually pleasing one.Non-perceptual evaluation is also possible, where the frequency response and / or sharpness indicator values ​​of the compensated first color channel are compared with those of a reference color channel corresponding to the center-located wavelength.

[0063] A third option is to use a nonlinear scaling function. For example, you can use a non-decreasing descaling function that is superlinear and asymptotically zero, such as:

[0064]

[0065] When |x|→0, for p>1, it goes to zero faster than any multiple of x. If the difference B(u,v)-B LP (u,v) consists of a scaling function with superlinear asymptotics, then a negative compensation 310 is applied to the following pixels (or pixel groups) in which the difference is significant, but not significant elsewhere. In fact, those image areas where the blue edge can be ignored can be left as is, to avoid the risk of introducing artifacts unnecessarily. If the scaling needs to be adaptive to the perception of "significant" differences and "negligible" edges in the current application scenario, the scaling function can be extended by a>0 to replace.

[0066] In various embodiments, the first, second and third options may be advantageously combined by incorporating corresponding functions into said g.

[0067] Next follows a step of subtracting the overshoot component from the first color channel (314). According to method 300, this subtraction is not unlimited, but is limited by a lower threshold value T B (u,v)(or below threshold T B (u,v) is the condition), the lower threshold depends on the local value of at least one other color channel:

[0068]

[0069] Where g is the function introduced in optional step 312.3. In embodiments where method 300 does not include optional step 312.3, g(x)=x can be conceptually set in equation (1). The maximum operation ensures that the pixel value B(u,v) will be replaced by a value at least equal to the threshold value T B The value of (u,v).

[0070] In some embodiments of method 300, the threshold may be set equal to the local value of another color channel, for example, T B (u,v)=G(u,v) or T B (u,v) = R(u,v). In other embodiments, the threshold is a combination of the local values ​​of two other color channels:

[0071] T B (u,v)=β1G(u,v)+β2R(u,v),β1,β2>0

[0072] The combination can be a plain average (β1=β2=1 / 2) or a weighted average (β1+β2=1). Another option is to use β1+β2=2 / 3, which under certain conditions (when G(u,v) and R(u,v) are approximately equal), will result in the three color channels having approximately equal intensities, so that the compensated pixel will have an unobtrusive light gray hue. The combination can also be the maximum or minimum of two other color channels:

[0073] T B (u,v)=max{G(u,v),R(u,v)},

[0074] T B (u,v)=min{G(u,v),R(u,v)}.

[0075] As described above, the method 300 can be performed at different granularities, i.e., the index pair (u, v) can point to a pixel or a group of multiple pixels. Specifically, a rule can be applied according to which, if the overshoot component is subtracted, once any pixel in the group eventually falls below the threshold, the subtraction step 314 assigns the threshold to all pixels in the pixel group.

[0076] Another option is to perform the subtraction step 314 at a one-pixel granularity, but calculate the threshold T at a coarser granularity. B One way to implement this is to partition the pixel space [1,U]×[1,V] into pixel groups Where I is the set of indices such that:

[0077]

[0078] And for any i≠j, Starting from a threshold with the granularity of pixel groups, T B =T B (W i ), despite the modified threshold However, it is still possible to express the subtraction operation 314 exactly with a granularity of one pixel:

[0079]

[0080] in, is a piecewise constant, that is, For i, such that (u,v)∈W i Alternatively, the modified threshold in (2) Can be used For example, if each W i If all are 9×9 pixel groups, the interpolation can have a finer granularity of 5×5 pixels. The threshold function with the granularity of pixel groups can be defined as T B (W i )=R(u i ,v i ), where (u i ,v i ) is W i Alternatively, the average of the local values ​​can be formed:

[0081]

[0082] Among them, |W i |It is W i The number of pixels in .

[0083] In addition, along the same lines, the method 300 can be modified so that the threshold T B has a granularity of one pixel, but the subtraction 314 is performed at a coarser granularity.

[0084] For a summary of the processing steps described so far, see Figure 8. Starting from a digital image with three color channels 890, one of the color channels (here B) is fed to a low-pass filter 810 and a signed sum component 820 forms a difference consisting of a ramp function 830 (alternatively, a scaling function 830' that asymptotically approaches zero superlinearly) and a scaling factor 840, from which an overshoot component is obtained. This overshoot component is subtracted from the original color channel at position 850, on which basis a maximum operator 860 ensures that the resulting difference is not less than a threshold value calculated from one or both of the two other color channels (here G and R). The original color channel is replaced by its value minus the overshoot component or, in necessary pixels, by the threshold value. In addition to components 830, 840 and 860, Figure 8 The processing map in can be described as an unsharp mask applied to the B color channel.

[0085] In some embodiments, the subtraction 314 subject to the lower threshold is performed by applying rule (1) uniformly across the first color channel of the entire digital image.

[0086] In other embodiments, rule (1) is applied only in some sub-regions of the digital image. In the rest of the image, the overshoot component may not be subtracted at all, or the overshoot component may be subtracted in an unrestricted manner without considering the lower threshold. The sub-region may be the area most likely to be affected by blue fringing. In one example, an edge filter can be used to locate sharp dark / bright edges and other image features in which blue fringes are known to occur frequently, and the sub-region can be defined as its environment. In another example, saturated pixels and / or pixels with typical blue-edge hues are located, and the sub-region is defined as the environment of these pixels. Alternatively, a trained machine learning model can be used to find dark / bright edges. Optionally, in order to avoid visible boundaries between the sub-regions of the image to which rule (1) is applied and the rest of the image to which rule (1) is not applied, one or more transition bands can be defined, wherein the modified rule is used:

[0087]

[0088] A properly chosen constant γ1 in the interval (0,1) will have the effect of adjusting the subtraction in a way that the substituted pixel value is closer to the original pixel value B(u,v).The transition band may be a neighborhood of each sub-region with a predefined thickness ∈>0.

[0089] return Figure 3If no more color channels should be processed by negative correction 310 in the current round (branch N of step 318), execution proceeds to the optional second block, positive correction 320. If negative correction 310 can be said to resolve the blue edge on the dark side of an edge in the image, then the effect of positive correction 320 is to restore the "missing" amount of blue on the bright side of the edge. Areas with defects in blue may have an unnatural yellowish appearance.

[0090] To perform a positive correction 320, the undershoot component (322) is extracted in a similar manner as the overshoot component, albeit with opposite signs inside the g function:

[0091] g(B LP (u,v)-B(u,v))

[0092] In the positive correction 320, the above-described conditional subtraction operation (314) corresponds to a conditional addition (324) that is limited by an upper threshold. The upper threshold depends on the local value of at least one other color channel. According to any of the above-described functions T B can also be used as an upper threshold. This can be summarized as the following substitution rule:

[0093]

[0094] The positive correction 320 may be modified by including the same optional features (with sign changes, if necessary) as already outlined with respect to the negative correction 310. That is, the details of the positive correction 320 (including adjustment and adaptation of numerical parameters, choice of function g (if any)) remain independent of the negative correction 310 and need not be similar to or match the negative correction 310.

[0095] Execution of method 300 may end when there are no more color channels to which positive correction 320 is applied (branch N of step 328). If desired, the digital image may then be processed by operations such as dynamic range compression, tone mapping, and sharpening.

[0096] An image processing device 510 having a memory 512 and a suitably configured processing circuit 514 may be used to perform the method 300. Figure 5 As shown in , the image processing device 510 may be an essential component of the camera 500. The camera 500 may be a camera suitable for monitoring an application scene. For example, it may be fixedly installed and configured to transmit its output as a real-time stream.

[0097] Alternatively, if Figure 6As shown in , the image processing device 510 can be configured to process the output of the camera 500. The image processing device 510 can be co-configured with the camera 500 at the same location or can receive its output remotely through a communication network. Preferably, the image processing device 510 operates in real time, so that the user can view the video stream captured by the camera 500 and processed by the image processing device 510 with a controllable (and usually small) delay.

[0098] Figure 7 A hybrid implementation is shown, in which the image processing device 510 is included in the camera 500 , and has the option of delegating some of its computational tasks to a networked (or cloud) processing resource 520 .

[0099] Aspects of the disclosure have mainly been described above with reference to a few embodiments. However, as readily appreciated by a person skilled in the art, other embodiments than the ones disclosed above are equally possible within the scope of the invention as defined by the appended patent claims.

Claims

1. A method for suppressing chromatic aberration in a digital image having multiple color channels, the method being performed by an image processing device comprising a processor and a memory, the method comprising: negatively correcting the first color channel by subtracting an overshoot component of the first color channel, wherein the subtraction operation is limited to a lower threshold value that depends on a local value of at least one further color channel, Characterized in that the lower threshold has a granularity of a group of multiple pixels.

2. The method according to claim 1, wherein: The lower threshold is a local value of the second color channel.

3. The method according to claim 1, wherein: The lower threshold is a combination of the local value of the second color channel and the local value of the third color channel.

4. The method according to claim 1, wherein: The subtraction has a granularity of one pixel.

5. The method according to claim 1, wherein: The first color channel represents a blue, light blue, red or light red primary color.

6. The method according to claim 1, wherein: The lower threshold is implemented uniformly throughout the digital image.

7. The method according to claim 1, wherein: The lower threshold is implemented only in a sub-region of the digital image.

8. The method according to claim 1, further comprising: The overshoot component is extracted by computing a difference between the first color channel of the digital image and a low pass filtered copy of the same color channel.

9. The method according to claim 8, wherein: The extraction of the overshoot component further comprises combining the difference with one or more of: Ramp function; constant scaling factor; A superlinear scaling function that asymptotically approaches zero.

10. The method according to claim 1, further comprising: The negative correction is applied to the second color channel.

11. The method according to claim 1, further comprising: The first color channel or a different color channel of the color channels is positively corrected by adding an undershoot component of the same color channel, the addition operation being limited by an upper threshold value that depends on a local value of at least one other color channel.

12. The method according to claim 1, wherein: The negative correction and any positive correction are performed prior to any dynamic range compression, tone mapping or sharpening of the digital image.

13. A non-transitory computer readable storage medium having stored thereon instructions for, when executed on a device having processing capabilities, implementing a method for suppressing chromatic aberration in a digital image having a plurality of color channels, the method comprising: negatively correcting the first color channel by subtracting an overshoot component of the first color channel, wherein the subtraction operation is limited to a lower threshold value that depends on a local value of at least one further color channel, Characterized in that the lower threshold has a granularity of a group of multiple pixels.

14. An image processing apparatus for suppressing chromatic aberration in a digital image having multiple color channels, the image processing apparatus comprising a memory and a processing circuit configured to execute a method for suppressing chromatic aberration in a digital image having multiple color channels, the method comprising: negatively correcting the first color channel by subtracting an overshoot component of the first color channel, wherein the subtraction operation is limited to a lower threshold value that depends on a local value of at least one further color channel, Characterized in that the lower threshold has a granularity of a group of multiple pixels.

Images