Image Processing Device

Abstract

The image processing device processes an image in which three types of color pixels are arranged in a Bayer pattern to provide a decorrelated image consisting of three types of components arranged in a pattern corresponding to the Bayer pattern. A first type of component of the decorrelated image is provided as a replacement for a first type of pixel in the image, the first type of component being a weighted combination of pixels in the image including the first type of pixel and neighboring pixels, with second and third types of neighboring pixels having positive weighting coefficients corresponding to addition to the first type of pixel. A second type of component is provided as a replacement for a second type of pixel, the second type of component being a weighted combination of pixels including the second type of pixel and neighboring pixels, with first type of neighboring pixel having negative weighting coefficients corresponding to subtraction from the second type of pixel. A third type of component is provided as a replacement for a third type of pixel, the third type of component being a weighted combination of pixels including the third type of pixel and neighboring pixels, with first type of neighboring pixel having negative weighting coefficients corresponding to subtraction from the third type of pixel.

Description

[Technical field]

[0001] One aspect of the present invention relates to an image processing device adapted to process an image including three types of color pixels arranged in a Bayer pattern. The image processing device can be used, for example, as a pre-processing device in an encoding system, and can improve encoding efficiency. Further aspects of the present invention relate to an encoding system, an image processing method, and a computer program for the image processing device. [Background technology]

[0002] US6154493 describes a technique for splitting raw image data into multiple channels, including a color plane difference channel. Each of these channels is compressed separately using a two-dimensional discrete wavelet transform. The compression utilizes quantization, so that decompression of the compressed channel data can result in a perceptually lossless image. The technique operates directly on images in Bayer pattern format. The quantization thresholds vary depending on the channel and discrete wavelet transform (DWT) subband being processed. Summary of the Invention

[0003] There is a need for improved image processing that allows for relatively highly decorrelated images based on images in which color pixels are arranged in a Bayer pattern, thereby enabling high code efficiency that provides better image quality for a given compression ratio.

[0004] According to one aspect of the invention as defined in claim 1, there is provided an image processing device adapted to process an image comprising three colour pixels arranged in a Bayer pattern to obtain a decorrelated image of three components arranged in a pattern corresponding to the Bayer pattern.

[0005] Thereby, in accordance with the Bayer pattern, the image is a rectangular array of 2x2 blocks of color pixels including two diagonally arranged pixels of a first type of color, pixels of a second type of color, and pixels of a third type of color.

[0006] Thereby, the decorrelated image is a rectangular array of 2x2 component blocks comprising two diagonally arranged first, second and third type components according to a pattern corresponding to the Bayer pattern.

[0007] The image processing device is adapted to provide the first type of components in the decorrelated image as a replacement for the first type of pixels in the image, whereby the components of the first type are weighted combinations of pixels in the image including the pixels of the first type and neighbouring pixels, the second and third types of neighbouring pixels having overall positive weighting coefficients corresponding to an overall addition of the second and third types of neighbouring pixels to the pixels of the first type.

[0008] providing the second type of components in the decorrelated image as replacements for the second type of pixels in the image, whereby the components of the second type are weighted combinations of pixels in the image including the pixels of the second type and neighboring pixels, the neighboring pixels of the first type having an overall negative weighting coefficient corresponding to an overall subtraction of the first type of neighboring pixels from the pixels of the second type.

[0009] providing the third type of component in the decorrelated image as a replacement for the third type of pixel in the image, whereby the component of the third type is a weighted combination of pixels in the image including the pixel of the third type and neighboring pixels, the neighboring pixels of the first type having an overall negative weighting coefficient corresponding to an overall subtraction of the first type of neighboring pixels from the pixel of the third type.

[0010] According to further aspects of the invention as defined in claims 12, 14 and 15 there is provided an encoding system, an image processing method and a computer program product, respectively.

[0011] The image processing device defined herein can effectively decorrelate an image including three types of color pixels arranged in a Bayer pattern, such as an image generated by an image sensor. The three types of color pixels may be, for example, red, green, and blue, whereby there are twice as many green pixels as red and blue pixels. As another example, the three types of color pixels may be, for example, yellow, cyan, and magenta, or according to any other color scheme. Since the image processing device efficiently decorrelates the type of image, the decorrelated image generated thereby can be efficiently coded. This allows a given image quality to be achieved with a relatively small amount of coded image data. In other words, the coded image has a relatively good image quality for a given amount of data that the coded image may constitute. Nevertheless, the image processing device does not need to introduce any loss of information. That is, the original form of the image can be completely reconstructed based on the decorrelated image generated by the image processing device.

[0012] For purposes of illustration, some embodiments of the invention will now be described in detail with reference to the accompanying drawings, in which additional features will be set forth, some of which are defined in the dependent claims, and whose advantages will become apparent. [Brief description of the drawings]

[0013] [Figure 1] FIG. 1 is a block diagram of the camera. [Diagram 2] FIG. 2 is a schematic diagram showing a raw image provided by an image sensor in a camera. [Diagram 3] FIG. 3 is a block diagram of an image processing device mounted on the camera. [Figure 4]FIG. 4 is a schematic diagram of a decorrelated image provided by a decorrelation module of an image processing device on the basis of a raw image. [Diagram 5] FIG. 5 is a block diagram of a decorrelation module that provides a decorrelated image. [Figure 6] FIG. 6 is a conceptual diagram illustrating the first decorrelation operation performed by the decorrelation module. [Figure 7] FIG. 7 is a schematic illustration of an intermediate decorrelated image obtained by a first decorrelation operation. [Figure 8] FIG. 8 is a conceptual diagram illustrating the second decorrelation operation performed by the decorrelation module. [Figure 9] FIG. 9 is a schematic diagram of a partial wavelet filtered decorrelated image provided by a quincunx wavelet filter module of an image processing device based on a decorrelated image. [Figure 10] FIG. 10 is a block diagram of a quincunx wavelet filter module that provides a partial wavelet filtered decorrelated image. [Figure 11] FIG. 11 is a conceptual diagram for explaining the first wavelet filter operation performed by the quincunx wavelet filter module. [Figure 12] FIG. 12 is a schematic illustration of an intermediate partial wavelet filtered decorrelated image obtained by the first wavelet filtering operation. [Figure 13] FIG. 13 is a conceptual diagram for explaining the second wavelet filter operation performed by the quincunx wavelet filter module. [Figure 14] FIG. 14 is a schematic diagram of a set of subbands provided by a two-dimensional wavelet filter module of an image processing device based on the same type of components in a partially wavelet filtered decorrelated image. [Figure 15]FIG. 15 is a schematic diagram of a Le Gall type 5 / 3 wavelet filter that may be applied by the two-dimensional wavelet filter module, which is conceptually related to the first and second decorrelation operations performed by the decorrelation module and the first and second wavelet filter operations performed by the quincunx wavelet filter module. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] FIG. 1 is a schematic diagram of a camera 100. FIG. 1 provides a block diagram of the camera 100. The camera 100 may be, for example, a photo camera or a video camera capable of capturing images or video or both, respectively, or a combination of both. The camera 100 may further store the captured images and / or video in an encoded form. The camera 100 may transmit an encoded version of the captured images and / or video. The camera 100 may be, for example, part of a smartphone or other type of electronic device.

[0015] 1 includes an image sensor 101, an image processing device 102, an encoder 103, a storage device 104, and a communication interface 105. The image sensor 101 may be configured with three types of color pixel sensors, for example, a green-sensitive pixel sensor, a red-sensitive pixel sensor, and a blue-sensitive pixel sensor. These color pixel sensors may be arranged in a so-called Bayer pattern, in which there are twice as many green-sensitive pixel sensors as red-sensitive pixel sensors and blue-sensitive pixel sensors.

[0016] FIG. 2 shows a raw image 200 that may be provided by the image sensor 101 in the camera 100. FIG. 2 is a schematic diagram of the raw image 200. The raw image 200 is composed of three types of color pixels, a first type of color pixel being a green pixel G, a second type of color pixel being a red pixel R, and a third type of color pixel being a blue pixel B, and these color pixels are arranged in a Bayer pattern. There are twice as many green pixels G as red pixels R and blue pixels B, and these three types of color pixels are interleaved as shown in FIG. 2. Thus, according to the Bayer pattern, the raw image 200 is a rectangular array of 2×2 color pixel blocks consisting of two diagonally arranged green pixels G, red pixels R, and blue pixels B.

[0017] The camera 100 basically works as follows: The image processor 102 provides a decorrelated and wavelet filtered version of the raw image 200, which may consist of a set of sub-bands for low frequency luminance components, a set of sub-bands for high frequency luminance components, a set of sub-bands for red chrominance components, and a set of sub-bands for blue chrominance components. The decorrelated and wavelet filtered version of the raw image 200 may consist of the same amount of information as the raw image 200. That is, the raw image 200 may be completely reconstructed based on the decorrelated and wavelet filtered version of the raw image 200. This means that the image processor 102 does not need to introduce any loss of information and therefore does not need to introduce any loss of image quality. The image processor 102 will be described in more detail below.

[0018] The encoder 103 encodes the decorrelated and wavelet filtered version of the raw image 200. In doing so, the encoder 103 may apply data compression, which may cause some loss of information and, therefore, some loss of quality. For example, the encoder 103 may operate in the manner described in U.S. Pat. No. 9,332,258.

[0019] The image processing device 102, which will be described in more detail below, enables the encoder 103 to achieve a relatively high data compression ratio for a given image quality. In other words, the image processing device 102 enables the encoder 103 to provide a relatively high quality encoded image for a given data compression ratio. That is, the image processing device 102 contributes to achieving a relatively high coding efficiency. The image processing device 102 and the encoder 103 can be jointly considered as an encoding system 106 capable of achieving a relatively high coding efficiency, as shown in FIG. 1. The encoding system 106 can be implemented, for example, in the form of an integrated circuit.

[0020] The storage device 104 may store the encoded images provided by the encoding system 106. The storage device 104 may comprise, for example, a solid-state storage device or a disk-based storage device, or a combination of both, as well as other types of storage devices.

[0021] The communication interface 105 can transmit the encoded images to an external device, which may be communicatively coupled to the camera 100, for example, via a direct communication link or a communication network. The communication interface 105 may consist of, for example, a USB port, a Bluetooth interface, or a Wi-Fi interface, or any combination of these, as well as other types of communication interfaces.

[0022] The decoding system may decode the encoded image such that a copy of the raw image 200 is obtained. The decoding system may be configured, for example, in the camera 100 or in an external device, or both. The decoding system is not shown in FIG. 1 for convenience and simplicity. In general, the decoding system may perform the inverse operations to those performed by the encoding system 106, as described previously herein and in more detail below.

[0023] A copy of raw image 200 obtained by decoding the encoded image may be subjected to various image processing operations, for example, to obtain an image suitable for display. These image operations may include, for example, debayering, gamma correction, white balance, and noise removal. The term "debayering" refers to an operation that effectively removes the Bayer pattern and replaces it with a pattern suitable for a display device, such as a conventional RGB pattern. An advantage of encoding raw image 200 as described herein, rather than encoding a debayered version of raw image 200, is that less information is lost and higher image quality can be achieved.

[0024] Fig. 3 is a schematic diagram of the image processing device 102 in the camera 100. Fig. 3 is a block diagram of the image processing device 102. The image processing device 102 is composed of a decorrelation module 301, a quincunx wavelet filter module 302, and a two-dimensional wavelet filter module 303. These modules may be implemented, for example, in the form of a dedicated circuit, a programmable circuit, or a suitably programmed processing device, or any combination thereof.

[0025] The image processing device 102 basically operates as follows: A decorrelation module 301 performs various pixel replacement operations so as to obtain a decorrelated image 304 on the basis of the raw image 200. More specifically, the decorrelation module 301 replaces the green pixels G of the raw image 200 with a luminance component Y, and further replaces the red pixels R and blue pixels B of the raw image 200 with red chromaticity components Cr and blue chromaticity components Cb, respectively.

[0026] Fig. 4 is a schematic diagram of the decorrelated image 304 provided by the decorrelation module 301. Fig. 3 is a schematic diagram of the decorrelated image 304. The luminance component Y, the red chrominance component Cr, and the blue chrominance component Cb are also arranged in a Bayer pattern. The luminance component Y is twice as abundant as the red chrominance component Cr and the blue chrominance component Cb. Thus, the decorrelated image 304 is a rectangular array of 2x2 component blocks consisting of two diagonally arranged luminance components Y, red chrominance components Cr, and blue chrominance components Cb.

[0027] More specifically, the luminance component Y in the decorrelated image 304 that constitutes a replacement for a green pixel G in the raw image 200 is a weighted combination of that green pixel G and its neighboring pixels in the raw image 200. In this weighted combination, the neighboring red and blue pixels R, B have an overall positive weighting factor, which corresponds to an overall addition of the neighboring red and blue pixels R, B to the replaced green pixel G. Conversely, the neighboring green pixel G may have an overall negative weighting factor, which corresponds to an overall subtraction of the neighboring green pixel G from the replaced green pixel G.

[0028] For example, the weighted combination that provides the luminance component Y may be as follows:

[0029] Y(i,j)=+7 / 8*G(i,j)-1 / 16*G(i-1,j-1)-1 / 16*G(i+1,j-1)-1 / 16*G(i-1,j+1)-1 / 16*G(i+1,j+1)-1 / 32*G(i-2,j)- 1 / 32*G(i,j-2)-1 / 32*G(i+2,j)-1 / 32*G(i,j+2)+1 / 8*B(i,j-1)+1 / 8*B(i,j+1)+1 / 8*R(i-1,j)+1 / 8*R(i+1,j).

[0030] In this equation, the two variables between the comma-separated parentheses indicate the position of the pixel in question in the raw image 200 illustrated in FIG. 2, whereby i and j respectively indicate the horizontal and vertical positions of the luminance component Y(i,j) being formed and the position of the green pixel G(i,j) it replaces.

[0031] In this embodiment, the four nearest blue and red pixels B(i,j-1), B(i,j+1), R(i-1,j), and R(i+1,j) are all added to the green pixel G(i,j) being replaced. The four nearest green pixels G(i-1,j-1), G(i+1,j-1), G(i-1,j+1), and G(i+1,j+1) are all subtracted from the green pixel G(i,j) being replaced. The four further neighboring green pixels G(i-2,j), G(i,j-2), G(i+2,j), and G(i,j+2) are also subtracted from the green pixel G(i,j) being replaced. The four nearest green pixels G(i-1,j-1), G(i+1,j-1), G(i-1,j+1), G(i+1,j+1) are in an x-position with respect to the replacing green pixel G(i,j). The four further green pixels G(i-2,j), G(i,j-2), G(i+2,j), G(i,j+2) are in a +-position with respect to the replacing green pixel G(i,j). The four nearest green pixels G(i-1,j-1), G(i+1,j-1), G(i-1,j+1), G(i+1,j+1) are weighted more heavily, i.e., -1 / 16, than the -1 / 32 weight of the four further neighboring green pixels G(i-2,j), G(i,j-2), G(i+2,j), G(i,j+2).

[0032] The red chromaticity component Cr of the decorrelated image 304 that constitutes a replacement for a red pixel R in the raw image 200 is a weighted combination of that pixel and its neighboring pixels, in which the neighboring green pixels G have a negative overall weighting coefficient, which corresponds to a total subtraction of the neighboring green pixels G from the red pixel R being replaced.

[0033] The weighted combination that provides the red chromaticity component Cr can be as follows:

[0034] Cr(i,j)=R(i,j)-1 / 4*G(i-1,j)-1 / 4*G(i,j-1)-1 / 4*G(i+1,j)-1 / 4*G(i,j+1).

[0035] In this equation, the two variables between the parentheses separated by a comma again indicate the position of the pixel in the raw image 200 illustrated in Fig. 2, whereby i and j respectively indicate the horizontal and vertical positions of the red chromaticity component Cr(i,j) to be formed and the horizontal and vertical positions of the red pixel R(i,j) to be replaced by it. In this embodiment, all of the four immediately adjacent green pixels G(i-1,j), G(i,j-1), G(i+1,j), and G(i,j+1) are subtracted from the red pixel R(i,j) to be replaced.

[0036] Similarly, the blue chromaticity component Cb in the decorrelated image 304 that constitutes a replacement for a blue pixel B in the raw image 200 is a weighted combination of that blue pixel B and its neighboring pixels, in which the neighboring green pixel G has a global negative weighting factor, which corresponds to a global subtraction of the neighboring green pixel G from the blue pixel B that it is replacing.

[0037] The weighted combination that provides the blue chromaticity component Cb can be as follows:

[0038] Cb(i,j)=B(i,j)-1 / 4*G(i-1,j)-1 / 4*G(i,j-1)-1 / 4*G(i+1,j)-1 / 4*G(i,j+1).

[0039] Again, the two variables between the parentheses separated by a comma indicate the position of the pixel in the raw image 200 illustrated in Fig. 2, whereby i and j respectively indicate the horizontal and vertical positions of the blue chromaticity component Cb(i,j) to be formed and the horizontal and vertical positions of the blue pixel B(i,j) to be replaced by it. In this embodiment, similar to the weighted combination to obtain the red chromaticity component Cr(i,j) shown above, all of the four immediately adjacent green pixels G(i-1,j), G(i,j-1), G(i+1,j), and G(i,j+1) are subtracted from the blue pixel B(i,j) to be replaced.

[0040] Figure 5 illustrates diagrammatically one embodiment of a decorrelation module 301 which forms part of the image processing apparatus 102 illustrated in Figure 3. Figure 5 provides a block diagram of this embodiment of the decorrelation module 301, which will hereafter be referred to simply as the decorrelation module 301. The decorrelation module 301 is composed of an input buffer memory 501, a chrominance component formation module 502, an intermediate buffer memory 503 and a luma component formation module 504.

[0041] The decorrelation module 301 basically operates as follows: The input buffer memory 501 temporarily stores at least a portion of the raw image 200 shown in Fig. 2. The chrominance component formation module 502 retrieves the red pixel R and the four immediately neighboring green pixels G of the raw image 200 from the input buffer memory 501 and forms a red-based chrominance component Cr by performing a weighted combination between the red pixel R and the four immediately neighboring green pixels G as shown herein. In this way, the chrominance component formation module 502 forms a respective red-based chrominance component Cr for the decorrelated image 304 constituting a replacement for the red pixel R in the raw image 200.

[0042] The chrominance component forming module 502 similarly forms the blue chrominance component Cb, i.e., the chrominance component forming module 502 takes the blue pixel B and the four immediately adjacent green pixels G of the raw image 200 from the input buffer memory 501, and forms the blue chrominance component Cb by weighted combinations between the blue pixel B and the four immediately adjacent green pixels G as shown herein.

[0043] FIG. 6 illustrates the formation of the red chrominance component Cr and the blue chrominance component Cb described herein. This can be considered as a first decorrelation operation. FIG. 6 is a conceptual diagram illustrating the formation of the chrominance components Cr, Cb for the decorrelated image 304 by this first decorrelation operation. In FIG. 6, a portion of the raw image 200 is shown in the left half of the figure, while a portion of the decorrelated image 304 is shown in the right half of the same figure. Also shown is a red pixel R and its four immediate green pixels G of the raw image 200, a blue pixel B and its four immediate green pixels G of the blue pixel B, the red chrominance component Cr of the decorrelated image 304 constituting the replacement of the red pixel R described above, and the blue chrominance component Cb of the decorrelated image 304 constituting the replacement of the blue pixel B described above. Also shown are weighting coefficients, which correspond to those shown above.

[0044] Fig. 7 is a schematic diagram of an intermediate decorrelated image 700 provided by the chrominance component formation module 502, Fig. 7 provides a schematic diagram of the intermediate decorrelated image 700. The intermediate decorrelated image 700 corresponds to a modified version of the raw image 200 in which each red and blue pixel B has been replaced by a respective red-based chrominance component Cr and a respective blue-based chrominance component Cb. The green pixels G of the raw image 200 have not yet been replaced and therefore remain present. At least a portion of the intermediate decorrelated image 700 is temporarily stored in the intermediate buffer memory 503 of the decorrelation module 301 illustrated in Fig. 5.

[0045] Referring again to Fig. 5, the luminance component forming module 504 retrieves the green pixel G and its four immediate chrominance pixels Cr, Cb, namely two red chrominance pixels Cr and two blue chrominance pixels Cb, in the intermediate decorrelation image 700 from the intermediate buffer memory 503. The luminance component forming module 504 then forms the luminance component Y by performing a weighted combination of the green pixel G and the four immediate chrominance pixels Cr, Cb. In this way, the luminance component forming module 504 forms a respective luminance component Y for the decorrelation image 304, which constitutes a replacement for the green pixel G in the raw image 200. In this way, the luminance component forming module 504 provides the decorrelation image 304 illustrated in Fig. 4 based on the intermediate decorrelation image 700 illustrated in Fig. 6, which is temporarily stored in the intermediate buffer memory 503 of the decorrelation module 301 illustrated in Fig. 5.

[0046] FIG. 8 shows the formation of the luminance component Y as described herein. This can be considered as a second decorrelation operation. FIG. 8 shows a conceptual diagram of how this second decorrelation operation results in the formation of the luminance component Y of the decorrelated image 304. In FIG. 8, a portion of the intermediate decorrelated image 700 is shown in the left half of the figure, while a portion of the decorrelated image 304 is shown in the right half of the figure. The green pixel G of the intermediate decorrelated image 700 is shown, as well as its four immediate chrominance components Cb, Cr. Also shown is the luminance component Y of the decorrelated image 304, which constitutes a replacement for the green pixel G mentioned above. When combined with the weighting coefficients shown in FIG. 6, the corresponding weighting coefficients are obtained for forming the luminance component Y.

[0047] The decorrelation module 301 shown in Fig. 5 can operate on integer values ​​resulting from the application of weighting coefficients to the pixels and round division. This allows a relatively simple and low-cost implementation. There are various ways to round the numbers. For example, the formation of the red chromaticity component Cr and the blue chromaticity component Cb can be matched as follows:

[0048] Cb(i,j)=B(i,j)-ROUNDDOWN(1 / 4*G(i-1,j)+1 / 4*G(i,j-1)+1 / 4*G(i+1,j)+1 / 4*G(i,j+1)).

[0049] Cr(i,j)=R(i,j)-ROUNDDOWN(1 / 4*G(i-1,j)+1 / 4*G(i,j-1)+1 / 4*G(i+1,j)+1 / 4*G(i,j+1)).

[0050] Similarly, the formation of the luminance component Y can be adapted as follows:

[0051] Y(i,j)=G(i,j)+ROUNDDOWN(1 / 8*Cr(i-1,j)+1 / 8*Cb(i,j-1)+1 / 8*Cr(i+1,j)+1 / 8*Cb(i,j+1)).

[0052] At the decoder, the counterpart module of the decorrelation module 301 can be adapted to exactly invert the above equation by implementing the following:

[0053] G(i,j)=Y(i,j)-ROUNDDOWN(1 / 8*Cr(i-1,j)+1 / 8*Cb(i,j-1)+1 / 8*Cr(i+1,j)+1 / 8*Cb(i,j+1)).

[0054] R(i,j)=Cr(i,j)+ROUNDDOWN(1 / 4*G(i-1,j)+1 / 4*G(i,j-1)+1 / 4*G(i+1,j)+1 / 4*G(i,j+1)).

[0055] B(i,j)=Cb(i,j)+ROUNDDOWN(1 / 4*G(i-1,j)+1 / 4*G(i,j-1)+1 / 4*G(i+1,j)+1 / 4*G(i,j+1)).

[0056] In these formulas, "ROUNDDOWN" represents the operator that rounds a number down to the nearest integer.

[0057] Referring back to the image processing device 102 shown in Fig. 3, the quincunx wavelet filter module 302 applies a wavelet filter operation to the luminance component Y of the decorrelated image 304 shown in Fig. 4. In effect, the quincunx wavelet filter module 302 decomposes the luminance component Y into a first set of arrays of luminance components Y and a second set of arrays of luminance components Y, where the arrays of luminance components Y belonging to the first set are interleaved with the arrays of luminance components Y of the second set. For example, the arrays of luminance components Y of the first set may correspond to even lines of the decorrelated image 304 illustrated in Fig. 4, while the arrays of luminance components Y of the second set may correspond to non-even lines of the decorrelated image 304, or vice versa.

[0058] The quincunx wavelet filter module 302 filters the high-frequency luminance components Y H In this case, the quincunx wavelet filter module 302 may provide a low-frequency luminance component Y L In fact, the high-frequency luminance component Y H , replace the luminance component Y belonging to the first set of arrays. Also, the low luminance component Y L replaces the luminance component Y that belongs to the second set of arrays.

[0059] 9 is a schematic diagram of a partial wavelet filtered decorrelated image 900 provided by the quincunx wavelet filter module 302. In this example, the even lines of the partial wavelet filtered decorrelated image 900 are dominated by the high frequency luminance component Y H while the non-even lines constitute the low-frequency luminance component Y LThus, in the partially wavelet filtered decorrelated image 900, the different types of components are arranged in a dyadic rather than a mosaic arrangement. This feature may be relevant, for example, if the encoder 103 of the camera 100 illustrated in FIG. 1 is specifically designed to handle components organized in a dyadic arrangement. This is the case, for example, if the encoder 103 operates in the manner described in U.S. Pat. No. 9,332,258, previously mentioned herein.

[0060] Fig. 10 is a diagram showing an embodiment of the quincunx wavelet filter module 302 constituting a part of the image processing device 102 shown in Fig. 3. Fig. 10 provides a block diagram of the quincunx wavelet filter module 302 of this embodiment, which will hereinafter be simply referred to as the quincunx wavelet filter module 302. The quincunx wavelet filter module 302 is composed of an input buffer memory 1001, a high frequency luminance component forming module 1002, an intermediate buffer memory 1003, and a low frequency luminance component forming module 1004.

[0061] The quincunx wavelet filter module 302 basically operates as follows: The input buffer memory 1001 temporarily stores at least a part of the decorrelated image 304 shown in FIG. 4. The high frequency luminance component forming module 1002 retrieves from the input buffer memory 1001 a luminance component Y from the decorrelated image 304 and retrieves four immediately neighboring luminance components Y in a quincunx configuration with respect to a first luminance component, hereafter referred to as the central luminance component Y for convenience. The central luminance component Y may belong to a first set of arrangements, while the immediately neighboring luminance components Y may belong to a second set of arrangements. For example, the central luminance component Y may be present on an even line in the decorrelated image 304, while two of the four immediately neighboring luminance components Y may be present on a non-even line directly above said even line, and the other two of the four immediately neighboring luminance components Y may be present on a non-even line directly below said even line.

[0062] The high-frequency luminance component forming module 1002 performs a weighted combination of the central luminance component Y on one side and the four immediately adjacent luminance components Y on the other side to form a high-frequency luminance component Y H The weighted combination can be expressed as follows:

[0063] Y H (i,j)=Y(i,j)-1 / 4*Y(i-1,j-1)-1 / 4*Y(i+1,j-1)-1 / 4*Y(i+1,j+1)-1 / 4*Y(i-1,j+1).

[0064] Here, as before, the two variables between the comma-separated parentheses indicate the position of the luminance component in question in the decorrelated image 304 illustrated in FIG. 4, whereby i and j respectively represent the high-frequency luminance components Y H 1 indicates the horizontal and vertical position of (i,j) and the position of the luminance component Y(i,j) it replaces.

[0065] FIG. 11 shows the high frequency luminance component Y, which can be considered as the first partial wavelet filter operation. H FIG. 11 illustrates the formation of the high frequency luminance component Y for the partial wavelet filtered decorrelated image 900 by this first partial wavelet filtering operation. H 11 provides a conceptual diagram of how the partial wavelet filtered decorrelated image 900 is formed. In Fig. 11, a portion of the decorrelated image 304 is represented in the left half of the figure, while a portion of the partially wavelet filtered decorrelated image 900 is represented in the right half of the same figure. Luminance components Y belonging to a first set of arrays of luminance components Y and luminance components Y belonging to a second set of arrays are shown in different shading in the decorrelated image 304. In this example, the arrays of luminance components Y correspond to lines in the decorrelated image 304.

[0066] 11 shows a luminance component Y in the decorrelated image 304 that belongs to a first set of arrays and its four immediate neighbors Y in the second set of arrays. In addition, the high frequency luminance components Y in the partially wavelet filtered decorrelated image 900 are H is shown, which constitutes a permutation of the luminance component mentioned at the beginning. Furthermore, the weighting coefficients are also shown, which correspond to those shown here.

[0067] 12 is a schematic diagram of an intermediate partial wavelet filtered decorrelated image 1200 provided by the high frequency luminance component formation module 1002. The image 1200 shows that each luminance component Y belonging to the first set of sequences is divided into respective high frequency luminance components Y H 10. The luminance components Y belonging to the second set of arrays in the decorrelated image 304 have not yet been replaced and therefore remain present. At least a portion of the intermediate partial wavelet filtered decorrelated image 1200 is temporarily stored in the intermediate buffer memory 1003 of the quincunx wavelet filter module 302 illustrated in FIG.

[0068] Referring again to FIG. 10, the low-frequency luminance component formation module 1004 extracts from the intermediate buffer memory 1003 the luminance component Y in the intermediate partially wavelet filtered decorrelated image 1200 and the four immediately neighboring high-frequency luminance components Y in a quincunx arrangement with respect to the luminance component Y. H In this example, the luminance component Y belongs to the second set of arrays, but its immediate neighbor, the high-frequency luminance component Y H are substituted for the luminance components Y belonging to the first set of arrays. For example, the luminance components Y may be present in the non-even lines in the decorrelated image 304 and thus in the intermediate partial wavelet filter version 1200, while the four immediately neighboring high frequency luminance components Y H Two of the four immediately adjacent luminance components Y may be on the even lines immediately above the non-even line, and the other two of the four immediately adjacent luminance components Y may be on the even lines immediately below the non-even line.

[0069] The low-frequency luminance component forming module 1004 generates a low-frequency luminance component Y and four adjacent high-frequency luminance components Y H The low-frequency luminance component Y L This weighted combination can be expressed as:

[0070] Y L (i,j)=Y(i,j)+1 / 8*Y H (i-1,j-1)+1 / 8*Y H (i+1,j-1)+1 / 8*Y H (i+1,j+1)+1 / 8*Y H (i-1,j+1).

[0071] Here, as before, the two variables between the comma-separated brackets represent the position of the luminance component in question in the intermediate partial wavelet filtered decorrelated image 1200 illustrated in FIG. 12. Thus, i and j respectively represent the position of the low-frequency luminance component Y L represents the horizontal and vertical position of (i,j) and the position of the luminance component Y(i,j) it replaces.

[0072] FIG. 13 shows the low-frequency luminance component Y, which can be considered as a second partial wavelet filter operation. L FIG. 13 illustrates the formation of the low frequency luminance component Y for the partial wavelet filtered decorrelated image 900 by this second partial wavelet filtering operation. L 13 is a conceptual diagram illustrating how an intermediate partial wavelet filtered decorrelated image 1200 is formed. In FIG. 13, a portion of the intermediate partial wavelet filtered decorrelated image 900 is represented in the left half of the figure, while a portion of the intermediate partial wavelet filtered decorrelated image 900 is represented in the right half of the same figure.

[0073] FIG. 13 shows the luminance component Y of the intermediate partial wavelet filtered decorrelated image 1200 and its four immediate high frequency luminance components Y HFurthermore, the low-frequency luminance component Y L are shown, which constitute a permutation of the luminance component mentioned above. Additionally, weighting coefficients are also shown, which correspond to those shown here.

[0074] Similar to the decorrelation module 301 illustrated in Fig. 5, the quincunx wavelet filter module 302 illustrated in Fig. 10 may operate with integer and round division resulting from applying weighting coefficients to the components. This also allows for a relatively simple and low-cost implementation. There are various ways to round numbers, such as rounding down to the nearest integer, as presented herein by way of example.

[0075] Referring again to the image processing device 102 shown in FIG. 3, the two-dimensional wavelet filter module 303 filters the red chromaticity component Cr, the blue chromaticity component Cb, and the low-frequency luminance component Y in the partially wavelet filtered decorrelated image 900 shown in FIG. L In other words, the red chrominance component Cr in this image essentially constitutes a sub-image that is subjected to the two-dimensional wavelet filter. Similarly, the blue chrominance component Cb essentially constitutes another sub-image that is subjected to the same filter, and the low-frequency luminance component Y L constitutes yet another sub-image that is subjected to a two-dimensional wavelet filter.

[0076] The two-dimensional wavelet filter may be a succession of two one-dimensional wavelet filter operations. One of the two one-dimensional wavelet filter operations may be performed in the horizontal direction, and the other one-dimensional wavelet filter operation may be performed in the vertical direction. For example, the components of the sub-image may first be applied line-wise to the one-dimensional wavelet filter. This first wavelet filter operation produces a horizontal high frequency subband and a horizontal low frequency subband. The components of the horizontal high frequency subband are then applied column-wise to the one-dimensional wavelet filter to produce horizontal high / vertical high frequency subbands and horizontal high / vertical low frequency subbands. The components of the horizontal low frequency subband are then applied column-wise to the one-dimensional wavelet filter to produce horizontal low / vertical high frequency subbands and horizontal low / vertical low frequency subbands.

[0077] 14 is a diagram illustrating a set of subbands provided by the 2D wavelet filter module 303. FIG. 14 is a 2D frequency diagram, with the horizontal axis representing the horizontal frequency F H The vertical axis represents the vertical frequency F V 14. In this example, there are four subbands, designated as 1401 horizontal high / vertical high frequency subband, designated as 1402 horizontal high / vertical low frequency subband, designated as 1403 horizontal low / vertical high frequency subband, and designated as 1404 horizontal low / vertical low frequency subband, as previously described herein. These four subbands result from a two-dimensional wavelet filter of the subimages, as previously described herein, whereby the subimages are formed by the same type of components in the partial wavelet filtered decorrelated image 900. Thus, the four subbands shown in FIG. 14 are either red chrominance components Cr, blue chrominance components Cb, or low frequency luma components Y. L can be generated from

[0078] FIG. 15 is a diagram illustrating a Regal 5 / 3 type wavelet filter. FIG. 15 is a conceptual diagram of this wavelet filter. The Regal 5 / 3 type wavelet filter described below with reference to FIG. 15 can be made to correspond to the above-mentioned one-dimensional wavelet filter operation performed by the two-dimensional wavelet filter module 303. Furthermore, the Regal 5 / 3 type wavelet filter is conceptually related to the first and second decorrelation operations performed by the decorrelation module 301, as well as the first and second wavelet filter operations performed by the quincunx wavelet filter module 302. This will be explained in more detail below.

[0079] The Le Gall type 5 / 3 wavelet filter is applied to a series of input samples, which are represented as a series of relatively small circles in the upper part of Figure 15. Some of these small circles are filled with white while other parts of these small circles are filled with black, hereafter referred to as the white input samples and the black input samples, respectively. The white input samples are interleaved with the black input samples as shown in Figure 15. The Le Gall type 5 / 3 wavelet filter provides high frequency output samples for the black input samples and low frequency output samples for the white input samples. The high frequency output samples are represented in the center of Figure 15 while the low frequency output samples are represented in the lower part of the figure.

[0080] The high frequency output sample is formed by making a weighted combination of one black input sample and the other two immediately neighboring white input samples, as shown in FIG. 15, in which the black input sample has a weighting factor of +1 and the two immediately neighboring white input samples have a weighting factor of -0.5. The weighted combination thus represents the difference between the black input sample and the two immediately neighboring white input samples. The high frequency output sample resulting from this weighted combination effectively takes the place of the black input sample from which it is formed.

[0081] A low-frequency output sample is formed for a white input sample by making a weighted combination of the white input sample and its two immediate high-frequency output samples. These two high-frequency output samples in the weighted combination are essentially substitutes for the two black input samples immediately adjacent to the white input sample, as shown in FIG. 15. In the weighted combination, the white input sample has a weighting factor of +1 and its immediate high-frequency output samples have a weighting factor of +0.25, as shown in FIG. 15. Thus, the weighted combination is a weighted sum of the white input sample and its two immediate high-frequency output samples. The low-frequency output sample resulting from the weighted combination is essentially substitutes for the white input sample from which it is formed.

[0082] As mentioned above, the first and second decorrelation operations performed in the image processing device 102 by the decorrelation module 301 to form the decorrelated image 304 are conceptually related to the Le Gall 5 / 3 type wavelet filter described above. More specifically, the formation of the red chrominance component Cr and the formation of the blue chrominance component Cb illustrated in FIG. 6 can be conceptually considered as a two-dimensional extension of the one-dimensional nature of the formation of the high-frequency output samples in the Le Gall 5 / 3 type wavelet filter illustrated in FIG. 15. Instead of taking two input samples immediately adjacent in one dimension to form the high-frequency output sample as illustrated in FIG. 15, the formation of the chrominance component involves taking four immediately adjacent pixels in two dimensions as illustrated in FIG. 6. The sum of the weighting coefficients in the weighted combination providing the high-frequency output sample is equal to the sum of the weighting coefficients in the weighted combination providing the chrominance component. In the first mentioned weighted combination, there are two neighbors, each with a weight coefficient of -0.5, and in the last mentioned weighted combination, there are four neighbors, each with a weight coefficient of -0.25.

[0083] Similarly, the formation of the luma component Y illustrated in FIG. 8 can be conceptually considered as a two-dimensional extension of the one-dimensional nature of the formation of the low-frequency output samples in the Le Gall 5 / 3 type wavelet filter illustrated in FIG. 15. Rather than taking two immediately neighboring high-frequency output samples in one dimension to form the low-frequency output sample as illustrated in FIG. 15, the formation of the luma component Y involves taking four immediately neighboring chroma pixels in two dimensions as illustrated in FIG. 8. Again, the sum of the weighting coefficients in the weighted combination providing the low-frequency output sample is equal to the sum of the weighting coefficients in the weighted combination providing the luma component. In the first-mentioned weighted combination, there are two neighbors each having a weighting coefficient of +0.25, and in the last-mentioned weighted combination, there are four neighbors each having a weighting coefficient of +0.125.

[0084] Also as described herein, the first and second wavelet filter operations performed in the image processor 102 by the quincunx wavelet filter module 302 to form the partial wavelet filtered decorrelated image 900 are conceptually related to the Le Gall 5 / 3 type wavelet filter described herein. More specifically, the high frequency luminance component Y H The formation of can be conceptually viewed as a two-dimensional extension of the one-dimensional character of the formation of high-frequency output samples in the Le Gall 5 / 3 type wavelet filter shown in Figure 15. As illustrated in Figure 15, to form the high-frequency output samples, instead of taking the two immediately adjacent input samples in one dimension, the high-frequency luminance component Y H The formation of involves taking the four immediate luminance pixel neighbors in two dimensions, as illustrated in Figure 11. The sum of the weighting coefficients in the weighted combination that provides the high frequency output sample is equal to the sum of the weighting coefficients in the weighted combination that provides the high frequency luminance component. In the first mentioned weighted combination, there are two neighbors, each with a weighting coefficient of -0.5, and in the last mentioned weighted combination, there are four neighbors, each with a weighting coefficient of -0.25.

[0085] Similarly, the low-frequency luminance component Y L Conceptually, the formation of the low-frequency output sample in the Le Gall 5 / 3 type wavelet filter illustrated in FIG. 15 can be considered as a two-dimensional extension of the one-dimensional nature of the formation of the low-frequency output sample. As illustrated in FIG. 15, instead of forming the low-frequency output sample by taking two immediately adjacent high-frequency output samples in one dimension, the low-frequency luminance component Y L The formation of the luminance component Y H in two dimensions. Again, the sum of the weighting coefficients in the weighted combination providing the low-frequency output sample is equal to the sum of the weighting coefficients in the weighted combination providing the low-frequency luminance component. In the first-mentioned weighted combination, there are two neighbors, each with a weighting coefficient of +0.25, and in the last-mentioned weighted combination, there are four neighbors, each with a weighting coefficient of +0.125.

[0086] In the camera 100 illustrated in FIG. 1, the image processor 102 thus processes the raw image 200 illustrated in FIG. 2 to obtain the partially wavelet filtered decorrelated image 900 illustrated in FIG. 10, which is a partial wavelet filtered image of the high frequency luminance component Y H A sub-image consisting of the low-frequency luminance component Y L 14. The raw image 200 can be considered as a combination of four sub-images: a sub-image consisting of red chromaticity components Cr and a sub-image consisting of blue chromaticity components Cb. The image processing device 102 further provides, for each of the last three sub-images, a set of four sub-bands as shown in FIG. 14. The encoder 103 then generates the following data capable of representing the raw image 200 shown in FIG. 2: H , low-frequency luminance component Y L In this way, the four subbands representing the red chromaticity components Cr, and the four subbands representing the blue chromaticity components Cb can be efficiently coded without losing any information.

[0087] (Note) The embodiments described herein with reference to the drawings have been presented for purposes of illustration. The invention can be put into practice in many different ways. To illustrate this, some alternatives will be briefly presented.

[0088] The present invention may be applied in many types of products or methods related to image processing and / or video processing. The presented embodiments provide examples where the present invention is applied for image and video compression. However, the present invention may also be applied for other types of image and video processing, such as providing multi-resolution views of images for segmented storage and / or transport of images, image analysis to achieve detection or segmentation of objects / features in images, and as an analysis filter in a debayering algorithm.

[0089] There are many different ways of implementing the image processing device according to the invention. Any of the modules in the presented embodiments can be implemented by electrical circuitry, which may be dedicated or programmable, or by a suitably programmed processing device, or by a combination thereof. A computer program may define one or more operations performed by the image processing device described with reference to Figures 1 to 15. In this regard, each block diagram may also be considered to represent, at least in part, a flow chart diagram of such a computer program, as well as a method that may be performed by the processing device when executing the computer program. For example, the decorrelation module illustrated in Figure 5 may be considered to represent a decorrelation step. Similarly, other modules may be considered to represent method steps.

[0090] There are many different ways of implementing the decorrelation pixel-to-component transformation scheme in a product or method according to the invention. In the presented embodiment, a pixel-to-component transformation scheme is applied that is conceptually related to a Le Gall 5 / 3 type wavelet filter, as described herein. More specifically, in this conceptual relationship, the second and third type of pixels, which are the red and blue pixels in the presented embodiment, can be considered as samples to which high frequency components are provided, essentially using a two-dimensional filter kernel that is an extension of the one-dimensional kernel of the wavelet filter, whereby a correction of weighting factors is applied. And the first type of pixels, which are the green pixels in the presented embodiment, can be considered as samples to which low frequency components are provided, essentially using a two-dimensional filter kernel that is an extension of the one-dimensional kernel of the wavelet filter, whereby a correction of weighting factors is applied.

[0091] In other embodiments, the pixel-to-component conversion scheme may be based on a different type of wavelet filter, for example a Daubechies 9 / 7 type wavelet filter or a 13 / 7 family wavelet filter. A pixel-to-component conversion scheme based on one of the aforementioned types of wavelet filters may be more complex to implement than the pixel-to-component conversion scheme of the presented embodiment. For example, a pixel-to-component conversion scheme based on a Daubechies 9 / 7 type wavelet filter may include more partial decorrelation operations than the first and second decorrelation operations described herein above with reference to Figures 5-8. That is, a decorrelation module based on a Daubechies 9 / 7 type wavelet filter may require more function blocks than the decorrelation module 301 shown in Figure 5.

[0092] A pixel-to-component transformation scheme based on a wavelet filter of a more complex type than the Le Gall 5 / 3 type wavelet filter would provide a more complex weighted combination that defines the component that replaces the pixel. This is because more neighboring pixels are taken into account to form the component that replaces the pixel. The general rules regarding weighted combinations as set out hereinabove and defined in the appended claims would nevertheless apply. That is, in a weighted combination that provides a first type of component, the second and third types of neighboring pixels have an overall positive weighting factor that corresponds to an overall addition of the second and third types of neighboring pixels to the first type of pixel that is being replaced. Conversely, in a weighted combination that provides a second or third type of component, the first type of neighboring pixels have an overall negative weighting factor that corresponds to an overall subtraction of the first type of neighboring pixels from the second or third type of pixel that is being replaced, respectively.

[0093] Furthermore, it should be noted that there are many different ways of implementing decorrelation pixel-to-component transformation schemes, conceptually related to a particular type of wavelet filter, such as the Le Gall 5 / 3 wavelet filter. For example, the implementation described in the presented embodiment can be simplified as follows: The weighted combination that provides the red chromaticity component Cr is simplified by considering only pixels on two consecutive lines. Note that if the red pixel R(i,j) is on the lower of two consecutive lines, the weighted combination can be expressed as follows:

[0094] Cr(i,j)=R(i,j)-1 / 4*G(i-1,j)-1 / 2*G(i,j-1)-1 / 4*G(i+1,j).

[0095] 6, in that case the green pixel G immediately below the red pixel R that is replaced by the red chromaticity component Cr is omitted. This omission is compensated for by doubling the weighting factor of the green pixel G immediately above the red pixel R, resulting in -0.5 instead of -0.25. This is equivalent to horizontally mirroring, so to speak, the aforementioned green pixel G, so that this same pixel is also immediately below the red pixel R to take the place of the omitted green pixel G.

[0096] If a red pixel R(i,j) is the upper one of two consecutive lines, the weighted combination can be expressed as:

[0097] Cr(i,j)=R(i,j)-1 / 4*G(i-1,j)-1 / 4*G(i+1,j)-1 / 2*G(i,j+1).

[0098] 6, in that case, the green pixel G immediately above the red pixel R that is replaced by the red chromaticity component Cr is omitted. This omission is compensated for by doubling the weighting coefficient of the green pixel G immediately below the red pixel R, resulting in -0.5 instead of -0.25.

[0099] Similarly, the weighted combination that gives the blue chromaticity component Cb is simplified by considering only the pixels on two consecutive lines. Note that if the blue pixel B(i,j) is the lower of the two consecutive lines, the weighted combination can be expressed as follows:

[0100] Cb(i,j)=B(i,j)-1 / 4*G(i-1,j)-1 / 2*G(i,j-1)-1 / 4*G(i+1,j).

[0101] 6, in that case, the green pixel G immediately below the blue pixel B that is replaced by the blue chromaticity component Cb is omitted. This omission is compensated for by doubling the weighting coefficient of the green pixel G immediately above the blue pixel B, resulting in -0.5 instead of -0.25.

[0102] If a blue pixel B(i,j) is above two consecutive lines, the weighted combination can be expressed as:

[0103] Cb(i,j)=B(i,j)-1 / 4*G(i-1,j)-1 / 4*G(i+1,j)-1 / 2*G(i,j+1).

[0104] 6, in that case, the green pixel G immediately above the blue pixel B that is replaced by the blue chromaticity component Cb is omitted. This omission is compensated for by doubling the weighting coefficient of the green pixel G immediately below the blue pixel B, resulting in -0.5 instead of -0.25.

[0105] 7, the weighted combination giving the luma component Y can be similarly simplified by considering only the pixels and chroma components on two consecutive lines. If the luma pixel Y(i,j) and the replaced green pixel G(i,j) are the lower of two consecutive lines having horizontally adjacent blue-based chroma components Cb and vertically adjacent red-based chroma components Cr, the weighted combination can be expressed as follows:

[0106] Y(i,j)=G(i,j)+1 / 8*Cb(i-1,j)+1 / 4*Cr(i,j-1)+1 / 8*Cb(i+1,j).

[0107] In the present variant having horizontally adjacent red chromaticity components Cr and vertically adjacent blue chromaticity components Cb, the weighted combination can be expressed as follows:

[0108] Y(i,j)=G(i,j)+1 / 8*Cr(i-1,j)+1 / 4*Cb(i,j-1)+1 / 8*Cr(i+1,j).

[0109] 8, in the latter case, the blue chrominance component Cb immediately below the green pixel G that is replaced by the luminance component Y is omitted. This omission is compensated for by doubling the weighting coefficient of the blue chrominance component Cb immediately above the green pixel G, so that it becomes +0.25 instead of +0.125. This is equivalent to horizontally mirroring the aforementioned blue chrominance component Cb so that this same component is also immediately below the green pixel G to replace the omitted blue chrominance component Cb.

[0110] In addition, when the luminance pixel Y(i,j) and green pixel G(i,j) to be replaced are located on the upper side of two consecutive lines having horizontally adjacent blue chromaticity components Cb and vertically adjacent red chromaticity components Cr, the weighted combination can be expressed as follows:

[0111] Y(i,j)=G(i,j)+1 / 8*Cb(i-1,j)+1 / 8*Cb(i+1,j)+1 / 4*Cr(i,j+1).

[0112] In the present variant having horizontally adjacent red chromaticity components Cr and vertically adjacent blue chromaticity components Cb, the weighted combination can be expressed as follows:

[0113] Y(i,j)=G(i,j)+1 / 8*Cr(i-1,j)+1 / 8*Cr(i+1,j)+1 / 4*Cb(i,j+1).

[0114] 8, in the latter case, the blue chrominance component Cb immediately above the green pixel G that is replaced by the luminance component Y is omitted. This omission is compensated for by doubling the weighting coefficient of the blue chrominance component Cb immediately below the green pixel G, resulting in +0.25 instead of +0.125.

[0115] The wavelet filter operation applied to the luminance component Y by the quincunx wavelet filter module 302 illustrated in Figure 10 can be simplified in a similar manner. That is, the simplified wavelet filter operation can consider only the luminance components on two consecutive lines in the decorrelated image 304 illustrated in Figure 4. With reference to Figure 11, this can be done by H This can mean ignoring the luminance components Y on the line immediately below the luminance component Y on which H The weighting coefficients for the remaining luminance components Y on the line immediately above the luminance component Y on which Y is formed may be doubled to -0.5 instead of -0.25. Conversely, the simplification is H This can mean ignoring the luminance components Y on the line immediately above the luminance component Y on which the weighting coefficient is formed. In this case, the weighting coefficient of the remaining luminance components Y is also doubled.

[0116] The wavelet filtering operation applied by the quincunx wavelet filter module 302 reduces the luminance component Y and the high frequency luminance component Y only on two consecutive lines in the intermediate partial wavelet filtered decorrelated image 1200 illustrated in FIG. H With reference to FIG. 13, such a simplification can be achieved by considering the low frequency luminance component Y L The high-frequency luminance component Y on the line immediately below the luminance component Y H In that case, the low-frequency luminance component Y L The remaining high-frequency luminance components Y on the line immediately above the luminance component Y on which H The weighting factor of Y may be doubled to +0.25 instead of +0.125. Conversely, the simplification is H The high-frequency luminance component Y on the line immediately above the luminance component Y H This can mean ignoring the remaining luminance component Y. In that case, the weighting factor for the remaining luminance component Y is still doubled.

[0117] The observations presented herein illustrate that there are many different ways to implement a decorrelation pixel-to-component transform in accordance with the present invention. The transform can be applied in a perfect lossless manner or in a non-perfect lossless manner. For example, in video coding with a relatively low compression rate, a perfect lossless transform may be preferred for high quality. Conversely, in video coding with a relatively high compression rate, a non-perfect lossless transform may be tolerated and even preferred.

[0118] In another aspect, there are many different ways of implementing the wavelet filter operation applied to the luminance component Y in the product or method according to the invention. In the presented embodiment, there is a line-wise decomposition of the luminance component Y into a first set of arrays and a second set of arrays. In other embodiments, this decomposition may be column-wise. More generally, instead of a line-wise processing as shown here and in the presented embodiment, a column-wise processing can be performed.

[0119] In general, there are many different ways of implementing the invention, whereby different implementations may have different topologies. In any given topology, a single entity may perform multiple functions, or multiple entities may jointly perform a single function. In this respect, the drawings are highly schematic.

[0120] The remarks presented herein show that the embodiments described with reference to the drawings illustrate the invention, rather than limit it. The invention can be implemented in many alternative ways, within the scope of the appended claims. All changes that come within the meaning and range of equivalency of the claims are to be embraced within their scope. Any reference signs in the claims should not be interpreted as limiting the claims. The verb "comprise" in the claims does not exclude the presence of other elements or other steps than those stated in the claims. The same applies to similar verbs such as "include" and "contain". In a claim relating to a product, the recitation of an element in the singular does not exclude the possibility that the product is composed of a plurality of such elements. Similarly, in a claim relating to a method, the recitation of a step in the singular does not exclude the possibility that the method is composed of a plurality of such steps. The mere fact that each dependent claim defines a respective additional feature does not exclude combinations of additional features other than those reflected in the claim.

Claims

1. 1. An image processing device configured to process an original image (200) comprising pixels of three different colors (R, G, B) arranged in a Bayer pattern to obtain a decorrelated version (304) of said original image consisting of three different types of components (Y, Cr, Cb) arranged in a pattern corresponding to said Bayer pattern, Thereby, according to the Bayer pattern, the original image (200) is a rectangular array of 2x2 blocks of pixels including two diagonally arranged first color pixels (G), second color pixels (R) and third color pixels (B), Thereby, according to a pattern corresponding to the Bayer pattern, the decorrelated version (304) of the original image is a rectangular array of 2×2 blocks of components comprising two diagonally arranged first type components (Y), second type components (Cr) and third type components (Cb); The image processing device includes: providing the first type of component (Y) in the decorrelated version (304) of the original image as a replacement for the pixel (G) of the first color in the original image (200), whereby the component (Y) of the first type is a weighted combination of pixels in the original image (200) including the pixel (G) of the first color and neighboring pixels (R,B) of the second and third colors having an overall positive weighting coefficient corresponding to an overall addition of the neighboring pixels (R,B) of the second and third colors to the pixel (G) of the first color; providing the second type of component (Cr) in the decorrelated version (304) of the original image as a replacement for the second color pixel (R) in the original image (200), whereby the second type of component (Cr) is a weighted combination of pixels in the original image (200) including the pixel (R) of the second color and neighboring pixels (G) of the first color, the neighboring pixels (G) having an overall negative weighting coefficient corresponding to an overall subtraction of the first color neighboring pixels (G) from the pixel (R) of the second color; 2. An image processing apparatus configured to provide a third type of component (Cb) in the decorrelated version (304) of the original image as a replacement for the pixel (B) of the third color in the original image (200), whereby the component (Cb) of the third type is a weighted combination of pixels in the original image (200) including the pixel (B) of the third color and neighboring pixels (G) of the first color having an overall negative weighting coefficient corresponding to an overall subtraction of the neighboring pixels (G) of the first color from the pixel (B) of the third color.

2. 2. The image processing device according to claim 1, comprising: First, provide a respective component of the second type (Cr) as a replacement for each pixel of the second color (R), and a respective component of the third type (Cb) as a replacement for each pixel of the third color (B), and then 1. An image processing apparatus configured to provide a respective component of the first type (Y) as a replacement for a respective pixel of the first type (G), whereby the component of the first type (Y) is provided based on a weighted combination of, on the one hand, the first colour pixel (G) which it replaces, and, on the other hand, the second type component (Cr) and the third type component (Cb) which are provided as replacements for the pixels of the second colour and the pixels of the third colour to form a pixel group with the pixel of the first colour.

3. The weighted bonds forming the first type of the component (Y) are a nearest pixel (G) of the first color in x-disposition to the pixel of the first color to be replaced; 3. The image processing apparatus according to claim 1, further comprising: an adjacent pixel of the first color in a + arrangement relative to the pixel of the first color to be replaced.

4. 4. An image processing apparatus as claimed in claim 3, wherein the nearest pixels (G) of the first colour in an x-configuration have a heavier weighting factor in the weighted combination than the further neighbouring pixels of the first colour in a +-configuration.

5. In the weighted bonds forming the first type of the component (Y), The pixel of the first color to be replaced (G) has a weighting factor of +7 / 8; the nearest pixel of the first color in the x-location has a weighting factor of −1 / 16; the further neighboring pixel of the first color in a + arrangement has a weighting coefficient of −1 / 32; The image processing apparatus of claim 4 , wherein the neighboring pixels of the second color (R) and the neighboring pixels of the third color (B) have a weighting factor of +1 / 8.

6. In the weighted bonds forming the second type of the component (Cr), The pixel of the second color to be replaced (R) has a weighting factor of +1; The image processing device according to any one of claims 1 to 5, wherein the neighbouring pixels (G) of the first colour are the four immediate neighbouring pixels of the first colour having a weighting factor of -1 / 4.

7. In the weighted bonds forming the third type of component (Cb), the pixel (B) of the third color to be replaced has a weighting factor of +1; The image processing device according to any one of claims 1 to 6, wherein the neighbouring pixels (G) of the first colour are the four immediate neighbouring pixels of the first colour having a weighting factor of -1 / 4.

8. The image processing device detects the first type of high frequency component (Y Ｈ ) and the first type of low frequency components (Y Ｌ The wavelet-based filter is configured to apply a wavelet-based filter to the first type of components (Y) in the decorrelated version (304) of the original image to obtain a first set of arrays of the first type of components (Y) and a second set of arrays of the first type of components, the arrays of the first type of components being interlaced with ... Ｈ ) is provided for the components of the first type belonging to the first set of sequences, whereby the low frequency components of the first type (Y Ｌ 8. The image processing apparatus of claim 1, wherein a first set of sequences is provided for the components of the first type that belong to the second set of sequences.

9. The first type of high frequency component (Y Ｈ ) is a weighted combination of the first type of component from the first set of sequences and the first type of adjacent components from the second set of sequences, and the first type of low frequency components (Y Ｌ 9. The image processing apparatus of claim 8, wherein the first type component and adjacent high frequency components belonging to the second set of sequences are weighted sums.

10. The first type of high frequency component (Y Ｈ In the weighted combination forming a weighted combination, the components of the first type belonging to the first set of sequences have a weighting factor of +1 and adjacent components of the first type belonging to the second set have a weighting factor of -1 / 4; The first type of low frequency component (Y Ｌ 10. The image processing apparatus of claim 9, wherein in the weighted sum forming a weighted sum (i.e., a weighted sum of the first type and the second set of sequences) the components of the first type have a weighting factor of +1 and adjacent high frequency components have a weighting factor of +1 / 8.

11. The image processing device detects the first type of low-frequency components (Y Ｌ 11. The image processing device according to claim 8, configured to apply a two-dimensional wavelet-based filter to the components of the second type (Cr) of the decorrelated version (304) of the original image and to the components of the third type (Cb) of the decorrelated version (304) of the original image.

12. 12. An encoding system comprising: an image processing device according to any one of claims 1 to 11; and an encoder configured to encode at least one of the decorrelated version (304) of the original image and a wavelet-based filtered version of the decorrelated version.

13. 12. An image processing apparatus according to claim 11, and an encoder configured to encode at least one of the decorrelated version (304) of the original image and a wavelet-based filtered version of the decorrelated version, the encoder comprising: The first type of high frequency component (Y Ｈ )and, The first type of low frequency component (Y Ｌ ) a subband generated by applying the two-dimensional wavelet-based filter to the subbands generated by applying the two-dimensional wavelet-based filter to the components of the second type (Cr) of the decorrelated version (304) of the original image; and a subband generated by applying the two-dimensional wavelet-based filter to the components of the third type (Cb) of the decorrelated version (304) of the original image.

14. 2. An inverse image processor configured to process the decorrelated version (304) of the original image generated by the image processor of claim 1, the inverse image processor configured to reconstruct the original image (200) including the pixels (R, G, B) of three different colors arranged in the Bayer pattern.

15. 1. A method of processing an original image (200) containing pixels of three different colors (R, G, B) arranged in a Bayer pattern to obtain a decorrelated version (304) of said original image consisting of three different types of components (Y, Cr, Cb) arranged in a pattern corresponding to said Bayer pattern, comprising the steps of: Thereby, according to the Bayer pattern, the original image (200) is a rectangular array of 2x2 blocks of pixels including two diagonally arranged first color pixels (G), second color pixels (R) and third color pixels (B), Thereby, according to a pattern corresponding to the Bayer pattern, the decorrelated version (304) of the original image is a rectangular array of 2×2 blocks of components comprising two diagonally arranged first type components (Y), second type components (Cr) and third type components (Cb); The method comprises: providing the first type of component (Y) in the decorrelated version (304) of the original image as a replacement for the pixel (G) of the first color in the original image (200), whereby the component (Y) of the first type is a weighted combination of pixels in the original image (200) including the pixel (G) of the first color and neighboring pixels (R, B) of the second and third colors having an overall positive weighting coefficient corresponding to an overall addition of the neighboring pixels (R, B) of the second and third colors to the pixel (G) of the first color; providing the second type of component (Cr) in the decorrelated version (304) of the original image as a replacement for the second color pixel (R) in the original image (200), whereby the second type of component (Cr) is a weighted combination of pixels in the original image (200) including the pixel (R) of the second color and neighboring pixels (G) of the first color having an overall negative weighting coefficient corresponding to an overall subtraction of the first color neighboring pixels (G) from the second color pixel (R); providing the third type of component (Cb) in the decorrelated version (304) of the original image as a replacement for the pixel (B) of the third color in the original image (200), whereby the component (Cb) of the third type is a weighted combination of pixels in the original image (200) including the pixel (B) of the third color and neighboring pixels (G) of the first color having an overall negative weighting coefficient corresponding to an overall subtraction of the neighboring pixels (G) of the first color from the pixel (B) of the third color.

16. A computer program product comprising a set of instructions enabling an image processing apparatus to carry out the method of claim 15.

Images