image processor

Abstract

An image processor processes an image comprising three color pixels (R, G, B) arranged in a Bayer pattern. The image processor provides a decorrelated image comprising three components (Y, Cr, Cb) arranged in a pattern corresponding to the Bayer pattern. The image processor provides a first type of component (Y) in the decorrelated image to replace a first type of pixel (G) in the image, whereby the first type of component (Y) is a weighted combination of a cluster of pixels in the image comprising the first type of pixel (G) and neighboring pixels, wherein the neighboring second and third type of pixels (R, B) have an overall positive weighting factor corresponding to adding both the neighboring second and third type of pixels (R, B) to the first type of pixel (G). The image processor provides a second type of component (Cr) in the decorrelated image to replace a second type of pixel (R) in the image, whereby the second type of component (Cr) is a weighted combination of a cluster of pixels comprising the second type of pixel (R) and neighboring pixels, wherein the neighboring first type of pixel (G) has an overall negative weighting factor corresponding to subtracting all the neighboring first type of pixel (G) from the second type of pixel (R). The image processor provides a third type of component (Cb) of the decorrelated image to replace a third type of pixel (B) in the image, whereby the third type of component (Cb) is a weighted combination of a cluster of pixels comprising the third type of pixel (B) and neighboring pixels, wherein the neighboring first type of pixel (G) has an overall negative weighting factor corresponding to subtracting all the neighboring first type of pixel (G) from the third type of pixel (B).

Description

Technical Field

[0001] One aspect of the invention relates to an image processor adapted to process an image comprising three color pixels arranged in a Bayer pattern. As an example, this image processor can be used as a preprocessor to enhance coding efficiency in an encoding system. Other aspects of the invention relate to encoding systems, image processing methods, and computer programs for image processors. Background Technology

[0002] U.S. Patent 6,154,493 describes a technique for splitting raw image data into multiple channels, including color plane difference channels. Each of these channels is individually compressed using a two-dimensional discrete wavelet transform. This compression employs quantization processing, thereby producing a perceptibly lossless image by recovering the compressed channel data. The technique operates directly on the image in the form of a Bell pattern. A quantization threshold is defined for the quantization processing, which can be varied according to the channels being processed and the Discrete Wavelet Transform (DWT) subbands. Summary of the Invention

[0003] There is a need for an improved image processing method that allows for the acquisition of relatively decorrelation-rich images based on images with color pixels arranged in a Bell pattern, thereby allowing for higher coding efficiency and thus providing better image quality at a given compression ratio.

[0004] According to one aspect of the invention as claimed in claim 1, an image processor is provided, which is adapted to process an image comprising three types of color pixels arranged in a Bell pattern, thereby obtaining a decorrelated image composed of three types of components arranged in a pattern corresponding to the Bell pattern.

[0005] Therefore, according to the Bell pattern, the image is a rectangular block array of 2×2 colored pixels, including two diagonally arranged first-type colored pixels, one second-type colored pixel, and one third-type colored pixel; and

[0006] Therefore, according to the pattern corresponding to the Bell pattern, the decorrelation image is a rectangular block array consisting of 2×2 components, including two diagonally arranged first-type components, one second-type component, and one third-type component.

[0007] The processor is adapted to:

[0008] The first type of component in the decorrelated image is provided to replace the first type of pixels in the image, whereby the first type of component is a weighted combination of a cluster of pixels in the image that includes the first type of pixels and neighboring pixels, wherein the neighboring second and third type pixels have an overall positive weighting factor corresponding to adding all the neighboring second and third type pixels to the first type of pixels.

[0009] The second type of component in the decorrelated image is provided to replace the second type of pixels in the image, wherein the second type of component is a weighted combination of pixels of the second type and a cluster of pixels of adjacent pixels, wherein adjacent pixels of the first type have a weighting factor that is generally negative corresponding to subtracting all adjacent pixels of the first type from the pixels of the second type; and

[0010] The third type component in the decorrelated image is provided to replace the third type of pixels in the image, wherein the third type component is a weighted combination of a cluster of pixels including the third type of pixels and neighboring pixels, wherein the neighboring first type of pixels have an overall negative weighting factor corresponding to subtracting all the neighboring first type of pixels from the third type of pixels.

[0011] According to further aspects of the invention as defined in claims 12, 14 and 15, an encoding system, an image processing method and a computer program product are provided respectively.

[0012] The image processor defined above can effectively decorrelate an image containing three color pixels arranged in a Bell pattern, such as an image generated by an image sensor. As an example, these three color pixels can be red, green, and blue, where green pixels appear twice as often as red and blue pixels. As another example, these three color pixels can be yellow, cyan, and magenta, or according to any other color scheme. Because the image processor effectively decorrelates the image of the correlated type, the resulting decorrelated image can be efficiently encoded. This allows a specified image quality to be achieved using an encoded image containing a relatively small amount of data. In other words, the encoded image will have relatively good quality for a specified amount of data that can be contained. However, the image processor does not need to introduce any information loss. That is, the original image can be perfectly reconstructed from the decorrelated image generated by the image processor.

[0013] For illustrative purposes, some embodiments of the invention have been described in detail with reference to the accompanying drawings. Additional features will be introduced in this description, some of which are defined in the dependent claims, and their advantages will be apparent. Attached Figure Description

[0014] Figure 1 This is a block diagram of a camera.

[0015] Figure 2 This is a schematic diagram of the raw image provided by the image sensor in the camera.

[0016] Figure 3 This is a block diagram of the image processor in a camera.

[0017] Figure 4 This is a schematic diagram of the decorrelation module of the image processor, based on the decorrelation image provided by the original image.

[0018] Figure 5 This is a block diagram of a decorrelated module that provides decorrelated images.

[0019] Figure 6 This is a conceptual diagram illustrating the first decorrelation operation performed by the decorrelation module.

[0020] Figure 7 This is a schematic diagram of the intermediate decorrelation image obtained through the first decorrelation operation.

[0021] Figure 8 This is a conceptual diagram illustrating the second decorrelation operation performed by the decorrelation module.

[0022] Figure 9 This is a schematic diagram of the decorrelated image provided by the image processor's plum blossom wavelet filtering module based on the decorrelated image after local wavelet filtering.

[0023] Figure 10 This is a block diagram of a plum blossom wavelet filtering module that provides decorrelation of the image after local wavelet filtering.

[0024] Figure 11 This is a conceptual diagram illustrating the first wavelet filtering operation performed by the plum blossom-shaped wavelet filtering module.

[0025] Figure 12 This is a schematic diagram of the intermediate decorrelated image obtained through the first wavelet filtering operation, after local wavelet filtering.

[0026] Figure 13 This is a conceptual diagram illustrating the second wavelet filtering operation performed by the plum blossom wavelet filtering module.

[0027] Figure 14This is a schematic diagram of the subband set provided by the two-dimensional wavelet filtering module of the image processor based on the same type of components in the decorrelated image after local wavelet filtering.

[0028] Figure 15 This is a schematic diagram of a Le Gall 5 / 3 type wavelet filtering process, which can be applied by a two-dimensional wavelet filtering module and is conceptually related to the first and second decorrelation operations performed by the decorrelation module, as well as the first and second wavelet filtering operations performed by the quincunx wavelet filtering module. Detailed Implementation

[0029] Figure 1 Camera 100 is shown schematically. Figure 1 A block diagram of camera 100 is provided. As an example, camera 100 may be a camera or a video camera, or a combination of both, or both, that captures images or videos. Camera 100 may further store the captured images or videos, or both, in encoded form. Camera 100 may transmit the captured images or videos, or both, in encoded form. As an example, camera 100 may be part of a smartphone or other type of electronic device.

[0030] Figure 1 The camera 100 shown includes an image sensor 101, an image processor 102, an encoder 103, a storage device 104, and a communication interface 105. The image sensor 101 may include three types of color pixel sensors, such as green-sensitive pixel sensors, red-sensitive pixel sensors, and blue-sensitive pixel sensors. These color pixel sensors may be arranged in a so-called Bell pattern, whereby the number of green-sensitive pixel sensors is twice that of red-sensitive and blue-sensitive pixel sensors.

[0031] Figure 2 The image sensor 101 in the camera 100 is shown as the raw image 200 that can be provided. Figure 2 A schematic diagram of the original image 200 is provided. The original image 200 includes three types of color pixels: a first type of color pixel, which is a green pixel G; a second type of color pixel, which is a red pixel R; and a third type of color pixel, which is a blue pixel B. These color pixels are arranged in a Bell pattern. The green pixel G appears twice as often as the red pixel R and the blue pixel B. Figure 2 As shown, these three types of color pixels are interleaved. It can be seen that, according to the Bell pattern, the original image 200 is a rectangular block array of 2×2 color pixels, which includes two diagonally arranged green pixels G, one red pixel R, and one blue pixel B.

[0032] In summary, camera 100 operates as follows. Image processor 102 provides a decorrelation- and wavelet-filtered version of the original image 200, which may include a low-frequency luminance component, a set of subbands for high-frequency luminance components, a set of subbands based on the red chromaticity component, and a set of subbands based on the blue chromaticity component. This decorrelation- and wavelet-filtered version of the original image 200 may include the same amount of information as the original image 200. That is, based on the decorrelation- and wavelet-filtered version of the original image 200, the original image 200 can be perfectly reconstructed. This means that image processor 102 does not need to introduce any information loss, and therefore does not need to introduce any image quality loss. Image processor 102 will be described in more detail below.

[0033] Encoder 103 encodes a decorrelation- and wavelet-filtered version of the original image 200. In this process, encoder 103 may apply data compression, which may result in some information loss and consequently some quality degradation. For example, encoder 103 may operate in the manner described in U.S. Patent 9,332,258.

[0034] The image processor 102, which will be described in more detail below, allows the encoder 103 to achieve a relatively high data compression rate for a given image quality. In other words, the image processor 102 allows the encoder 103 to provide a relatively high-quality encoded image for a given data compression rate. That is to say, the image processor 102 helps to achieve relatively high encoding efficiency. Figure 1 As shown, the image processor 102 and encoder 103 can be collectively considered as an encoding system 106 capable of achieving relatively high encoding efficiency. As an example, the encoding system 106 can be implemented as an integrated circuit.

[0035] Storage device 104 can store encoded images provided by encoding system 106. For example, storage device 104 may include solid-state storage devices or disk-based storage devices, or combinations thereof, as well as other types of storage devices.

[0036] The communication interface 105 can send encoded images to an external device that can be communicatively coupled to the camera 100 (e.g., via a direct communication link or a communication network). As an example, the communication interface 105 may include a USB port, a Bluetooth interface, or a Wi-Fi interface, or any combination of these interfaces, as well as other types of communication interfaces.

[0037] The decoding system can decode the encoded image to obtain a copy of the original image 200. As an example, this decoding system can be included in camera 100 or an external device, or both. For simplicity and convenience, in Figure 1The decoding system is not shown. Typically, the decoding system can perform the opposite operation to that performed by the encoding system 106, which is described above and in more detail below.

[0038] As an example, a copy of the original image 200 obtained by decoding the encoded image can undergo various image processing operations to obtain an image suitable for display. These image operations, for example, may include debayering, gamma correction, white balance, and noise reduction. The term "debayering" refers to an operation used to effectively remove the bel pattern and replace it with a pattern suitable for the display device (such as a conventional RGB pattern). The advantage of encoding the original image 200 as described above, rather than encoding a debayered version of the original image 200, is that less information is lost, thereby allowing for higher image quality.

[0039] Figure 3 The image processor 102 in the camera 100 is shown schematically. Figure 3 A block diagram of an image processor 102 is provided. The image processor 102 includes a decorrelation module 301, a quincunx wavelet filter module 302, and a two-dimensional wavelet filter module 303. As an example, these modules may be implemented as dedicated circuitry, programmable circuitry, or a suitably programmed processor, or any combination thereof.

[0040] In summary, the image processor 102 operates as follows: The decorrelation module 301 performs various pixel replacement operations to obtain a decorrelation image 304 based on the original image 200. More specifically, the decorrelation module 301 replaces the green pixels G in the original image 200 with the luminance component Y. The decorrelation module 301 further replaces the red pixels R and blue pixels B in the original image 200 with the red-based chromaticity component Cr and the blue-based chromaticity component Cb, respectively.

[0041] Figure 4 The decorrelation image 304 provided by the decorrelation module 301 is shown schematically. Figure 3 A schematic diagram of decorrelation image 304 is provided. The luminance component Y, the red-based chrominance component Cr, and the blue-based chrominance component Cb are also arranged in a Bell pattern. The number of luminance components Y is twice that of the red-based chrominance component Cr and the blue-based chrominance component Cb. It can be seen that decorrelation image 304 is a rectangular block array of 2 by 2 components, which includes two diagonally set luminance components Y, one red-based chrominance component Cr, and one blue-based chrominance component Cb.

[0042] More specifically, the luminance component Y in the decorrelated image 304 that replaces the green pixel G in the original image 20 is a weighted combination of the associated green pixel G and its neighboring pixels in the original image 200. In this weighted combination, the neighboring red and blue pixels R and B have generally positive weighting factors. This corresponds to adding all neighboring red and blue pixels R and B to the replaced green pixel G. Conversely, neighboring green pixels G can have generally negative weighting factors. This corresponds to subtracting all neighboring green pixels G from the replaced green pixel G.

[0043] For example, a weighted combination of the luminance component Y can be provided as follows:

[0044] Y(i,j)=+7 / 8*G(ij)-1 / 16*G(i-1,j-1)-1 / 16*G(i+1,j-1)-1 / 16*G(i-1,j+1)-1 / 16*G(i+1,j+1)-1 / 32*G(i-2,j) -1 / 32*G(i,j-2)-1 / 32*G(i+2,j)-1 / 32*G(i,j+2)+1 / 8*B(i,j-1)+1 / 8*B(i,j+1)+1 / 8*R(i-1,j)+1 / 8*R(i+1,j),

[0045] In this equation, the two variables separated by a comma between the parentheses represent: Figure 2 The positions of the relevant pixels in the original image 200 shown are given, where i and j represent the positions of the formed luminance component Y(i,j) and the green pixel G(i,j) that is replaced therefrom in the horizontal and vertical directions, respectively.

[0046] In this embodiment, the four directly adjacent blue and red pixels B(i,j-1), B(i,j+1), R(i-1,j), and R(i+1,j) are added to the replaced green pixel G(i,j). The four nearest-neighbor green pixels G(i-1,j-1), G(i+1,j-1), G(i-1,j+1), and G(i+1,j+1) are subtracted from the replaced green pixel G(i,j). Furthermore, four further neighboring green pixels G(i-2,j), G(i,j-2), G(i+2,j), and G(i,j+2) are also subtracted from the replaced green pixel G(i,j). The four nearest-neighbor green pixels G(i-1,j-1), G(i+1,j-1), G(i-1,j+1), and G(i+1,j+1) are in an x-shaped configuration opposite to the replaced green pixel G(i,j). The four further nearest-neighbor green pixels G(i-2,j), G(i,j-2), G(i+2,j), and G(i,j+2) are in a cross-shaped configuration opposite to the replaced green pixel G(i,j). Compared to the weighting factor (which is -1 / 32) of the four further nearest-neighbor green pixels G(i-2,j), G(i,j-2), G(i+2,j), and G(i,j+2), the four nearest-neighbor green pixels G(i-1,j-1), G(i+1,j-1), G(i-1,j+1), and G(i+1,j+1) have a heavier weighting factor of -1 / 16.

[0047] The red-based chromaticity component Cr in the decorrelated image 304, which replaces the red pixel R in the original image 200, is a weighted combination of the correlated red pixel R and its neighboring pixels. In this weighted combination, the neighboring green pixels G have an overall negative weighting factor. This corresponds to subtracting all neighboring green pixels G from the replaced red pixel R.

[0048] A weighted combination of the red chromaticity component Cr can be provided as follows:

[0049] Cr(i,j)=R(i,j)-1 / 4*G(i-1,j)-1 / 4*G(i,j-1)-1 / 4*G(i+1,j)-1 / 4*G(i,j+1),

[0050] In this equation, the two variables separated by a comma between the parentheses also represent the same thing. Figure 2The positions of the relevant pixels in the original image 200 are shown, so i and j represent the positions of the formed red-based chromaticity component Cr(i,j) and the red pixel R(i,j) that is thus replaced, in the horizontal and vertical directions, respectively. In this embodiment, four directly adjacent green pixels G(i-1,j), G(i,j-1), G(i+1,j), and G(i,j+1) are all subtracted from the replaced red pixel R(i,j).

[0051] Similarly, the blue-based chromaticity component Cb in the decorrelated image 304, which replaces the blue pixel B in the original image 200, is a weighted combination of the associated blue pixel B and its neighboring pixels. In this weighted combination, the neighboring green pixels G have an overall negative weighting factor. This corresponds to subtracting all neighboring green pixels G from the replaced blue pixel B.

[0052] A weighted combination of the blue-based chromaticity component Cb can be provided as follows:

[0053] Cb(i,j)=B(i,j)-1 / 4*G(i-1,j)-1 / 4*G(i,j-1)-1 / 4*G(i+1,j)-1 / 4*G(i,j+1),

[0054] In this equation, the two variables separated by a comma between the parentheses also represent the same thing. Figure 2 The positions of the relevant pixels in the original image 200 are shown. Therefore, i and j represent the horizontal and vertical positions of the formed blue-based chromaticity component Cb(i,j) and the blue pixel B(i,j) thus replaced, respectively. In this embodiment, as in the weighted combination of the red-based chromaticity component Cr(i,j) described above, the four directly adjacent green pixels G(i-1,j), G(i,j-1), G(i+1,j), and G(i,j+1) are all subtracted from the replaced blue pixel B(i,j).

[0055] Figure 5 The formation is schematically shown. Figure 3 An embodiment of the decorrelation module 301, which is a part of the image processor 102 shown. Figure 5 A block diagram of this embodiment of decorrelation module 301 is provided, which is hereinafter simply referred to as decorrelation module 301. The decorrelation module 301 includes an input buffer memory 501, a chromaticity component forming module 502, an intermediate buffer memory 503, and a luminance component forming module 504.

[0056] In summary, the relevant module 301 operates as follows: Input buffer memory 501 temporarily stores... Figure 2The image shows at least a portion of the original image 200. The chroma component forming module 502 retrieves the red pixels R and four directly adjacent green pixels G from the input buffer memory 501 of the original image 200. Then, the chroma component forming module 502 forms a red-based chroma component Cr by performing a weighted combination between the red pixels R and the four directly adjacent green pixels G as previously described. In this way, the chroma component forming module 502 forms a corresponding red-based chroma component Cr for the decorrelated image 304, which replaces the red pixels R in the original image 200.

[0057] The chroma component forming module 502 forms the blue-based chroma component Cb in a similar manner. That is, the chroma component forming module 502 retrieves the blue pixel B and four directly adjacent green pixels G from the input buffer memory 501 of the original image 200. Then, the chroma component forming module 502 forms the blue-based chroma component by performing a weighted combination between the blue pixel B and the four directly adjacent green pixels G as previously described.

[0058] Figure 6 The processes for forming the red-based chromaticity component Cr and the blue-based chromaticity component Cb, as described above, are shown. This process can be considered as the first decorrelation operation. Figure 6 A conceptual diagram of the first decorrelation operation for the chromaticity components Cr and Cb, upon which the decorrelated image 304 is formed, is provided. Figure 6 The left half of the figure shows a portion of the original image 200, while the right half shows a portion of the decorrelated image 304. Here, the red pixel R and its four directly adjacent green pixels G, and the blue pixel B and its four directly adjacent green pixels G in the original image 200 are indicated. Furthermore, the red-based chromaticity component Cr in the decorrelated image 304 replacing the aforementioned red pixel R, and the blue-based chromaticity component Cb in the decorrelated image replacing the aforementioned blue pixel B are also indicated. Moreover, the weighting factors corresponding to these pixels as previously indicated are also indicated here.

[0059] Figure 7 The intermediate decorrelation image 700 provided by the chromaticity component forming module 502 is schematically shown. Figure 7 A schematic diagram of an intermediate decorrelation image 700 is provided. This intermediate decorrelation image 700 corresponds to a modified version of the original image 200 in which the corresponding red and blue pixels B are replaced with the corresponding red-based chromaticity component Cr and the corresponding blue-based chromaticity component Cb. The green pixels G of the original image 200 are not replaced and are thus retained. At least a portion of the intermediate decorrelation image 700 is temporarily stored... Figure 5The intermediate buffer memory 503 of the decorrelation module 301 shown is used.

[0060] Refer again Figure 5 The luminance component forming module 504 retrieves the green pixel G and four directly adjacent chrominance pixels Cr and Cb from the intermediate decorrelated image 700 in the intermediate buffer memory 503; that is, two red-based chrominance pixels Cr and two blue-based chrominance pixels Cb. Then, the luminance component forming module 504 forms the luminance component Y by weighting the green pixel G and the four directly adjacent chrominance pixels Cr and Cb. In this way, the luminance component forming module 504 forms the corresponding luminance component Y for the decorrelated image 304, which replaces the green pixel G in the original image 200. Thus, the luminance component forming module 504, based on... Figure 6 The image shown is temporarily stored in Figure 5 The intermediate decorrelation image 700 in the intermediate buffer memory 503 of the decorrelation module 301 shown provides Figure 4 The decorrelated image 304 is shown.

[0061] Figure 8 The process for forming the luminance component Y, as described above, is shown. This process can be considered as a second decorrelation operation. Figure 8 A conceptual diagram is provided regarding the second decorrelation operation for the luminance component Y, from which the decorrelated image 304 is formed. Figure 8 The left half of the figure shows a portion of the middle decorrelation image 700, while the right half shows a portion of the decorrelation image 304. Here, the green pixel G in the middle decorrelation image 700 and its four directly adjacent chromaticity components Cb and Cr are indicated. Furthermore, the luminance component Y in the decorrelation image 304, which replaces the aforementioned green pixel G, is also indicated. Further, the weighting factor is also indicated here. When compared with... Figure 6 When the weighting factors shown are combined, a weighting factor corresponding to these pixels used to form the luminance component Y, as previously indicated, will be obtained.

[0062] Figure 5 The decorrelation module 301 shown can use integers generated by applying weighting factors to pixels, along with rounding division. This allows for a relatively simple, low-cost implementation. There are many ways to round numbers. For example, the process used to form the red-based chromaticity component Cr and the blue-based chromaticity component Cb can be adapted in the following way:

[0063] Cb(i,j)=B(i,j)-ROUNDDOWN(1 / 4*G(i-1,j)+1 / 4*G(i,j-1)+1 / 4*G(i+1,j)+1 / 4*G(i,j+1)); and

[0064] Cr(i,j)=R(i,j)-ROUNDDOWN(1 / 4*G(i-1,j)+1 / 4*G(i,j-1)+1 / 4*G(i+1,j)+1 / 4*G(i,j+1)).

[0065] Similarly, the processing for forming the luminance component Y can be adapted in the following way:

[0066] Y(i,j)=G(i,j)+ROUNDDOWN(1 / 8*Cr(i-1,j)+1 / 8*Cb(i,j-1)+1 / 8*Cr(i+1,j)+1 / 8*Cb(i,j+1)).

[0067] In the decoder, the module corresponding to the decorrelation module 301 can be adapted to precisely reverse the above equation by executing the following equation:

[0068] G(i,j)=Y(i,j)-ROUNDDOWN(1 / 8*Cr(i-1,j)+1 / 8*Cb(i,j-1)+1 / 8*Cr(i+1,j)+1 / 8*Cb(i,j+1));

[0069] R(i,j)=Cr(i,j)+ROUNDDOWN(1 / 4*G(i-1,j)+1 / 4*G(i,j-1)+1 / 4*G(i+1,j)+1 / 4*G(i,j+1)); and

[0070] B(i,j)=Cb(i,j)+ROUNDDOWN(1 / 4*G(i-1,j)+1 / 4*G(i,j-1)+1 / 4*G(i+1,j)+1 / 4*G(i,j+1)).

[0071] In these equations, ROUNDDOWN represents the operator that rounds a number down to the nearest integer.

[0072] Refer again Figure 3 The image processor 102 shown above, the quincunx wavelet filter module 302 applies wavelet filtering operations to... Figure 4 The image 304 shown is a decorrelation of the luminance component Y. In practice, the quincunx wavelet filter module 302 decomposes the luminance component Y into an array of first group luminance components Y and an array of second group luminance components Y. The arrays belonging to the first group luminance components Y and the arrays belonging to the second group luminance components Y are interleaved. For example, the array of the first group luminance components Y can correspond to... Figure 4 The even-numbered rows in the decorrelation image 304 shown correspond to the non-even-numbered rows in the decorrelation image 304, and vice versa.

[0073] The plum blossom-shaped wavelet filter module 302 can provide high-frequency brightness components Y for the brightness components Y belonging to the first array group. H In this case, the plum blossom-shaped wavelet filter module 302 can provide a low-frequency brightness component Y for the brightness component Y belonging to the second array group. L In fact, the high-frequency luminance component Y... H This replaced the luminance component Y belonging to the first array group. The low luminance component Y... L Replace the luminance component Y belonging to the second array group.

[0074] Figure 9 The diagram schematically illustrates the decorrelation image 900 provided by the plum blossom wavelet filtering module 302 after local wavelet filtering. Figure 9 A schematic diagram of a decorrelated image 900 after local wavelet filtering is provided. In this example, the even-numbered rows of the decorrelated image 900 after local wavelet filtering include the high-frequency brightness component Y. H The non-even rows include the low-frequency luminance component Y. L Therefore, in the decorrelated image 900 after local wavelet filtering, each different type of component is organized into a binary array, rather than a mosaic array. For example, if Figure 1 The encoder 103 in the camera 100 shown is specifically designed to process components organized in a binary array, so this feature can be relevant. As an example, this is possible if the encoder 103 operates as described in the aforementioned U.S. Patent 9,332,258.

[0075] Figure 10 It shows the formation Figure 3 An embodiment of a quincunx wavelet filter module 302, which is a part of the image processor 102 shown. Figure 10 A block diagram of this embodiment of a quincunx wavelet filter module 302 is provided, which is hereinafter simply referred to as the quincunx wavelet filter module 302. The quincunx wavelet filter module 302 includes an input buffer memory 1001, a high-frequency luminance component forming module 1002, an intermediate buffer memory 1003, and a low-frequency luminance component forming module 1004.

[0076] In summary, the plum blossom wavelet filter module 302 operates as follows: Input buffer memory 1001 temporarily stores... Figure 4The decorrelation image 304 shown represents at least a portion of the image. The high-frequency luminance component forming module 1002 retrieves the luminance component Y from the decorrelation image 304 and four directly adjacent luminance components Y arranged in a quincunx pattern relative to the first mentioned luminance component (hereinafter referred to as the center luminance component Y for convenience) from the input buffer memory 1001. The center luminance component Y may belong to a first array group, while the directly adjacent luminance components Y may belong to a second array group. For example, the center luminance component Y may exist in an even-numbered row of the decorrelation image 304, while two of the four directly adjacent luminance components Y may exist in a non-even-numbered row immediately above the aforementioned even-numbered row, and the other two of these four directly adjacent luminance components Y may exist in a non-even-numbered row immediately below the aforementioned even-numbered row.

[0077] The high-frequency luminance component forming module 1002 forms the high-frequency luminance component Y by weighting and combining the center luminance component Y (on one hand) with four directly adjacent luminance components Y (on the other hand). H The weighted combination can be expressed as follows:

[0078] Y H (i,j)=Y(i,j)-1 / 4*Y(i-1,j-1)-1 / 4*Y(i+1,j-1)-1 / 4*Y(i+1,j+1)-1 / 4*Y(i-1,j+1),

[0079] As in the previous text, the two variables separated by a comma are represented by parentheses. Figure 4 The positions of the correlated luminance components in the decorrelated image 304 are shown below; therefore, i and j represent the formed high-frequency luminance components Y, respectively. H The positions of (i,j) and the brightness component Y(i,j) that is replaced therefrom in the horizontal and vertical directions.

[0080] Figure 11 The method for forming the high-frequency luminance component Y is shown. H The processing can be considered as a first local wavelet filtering operation. Figure 11 Provides the high-frequency luminance component Y for forming the decorrelationed image 900 after local wavelet filtering. H This is a conceptual diagram of the first local wavelet filtering operation. Figure 11 The left half of the figure shows a portion of the decorrelation image 304, while the right half shows a portion of the decorrelation image 900 after local wavelet filtering. The luminance components Y belonging to the first group of luminance components Y and the luminance components Y belonging to the second group of luminance components Y are indicated by different shading in the decorrelation image 304. In this example, the array of luminance components Y corresponds to rows in the decorrelation image 304.

[0081] exist Figure 11 The image 304 indicates the luminance component Y belonging to the first array group and its four directly adjacent luminance components Y belonging to the second array group. It also indicates the high-frequency luminance component Y in the decorrelated image 900 after local wavelet filtering. H This replaces the luminance component mentioned earlier. Furthermore, the weighting factors corresponding to these components are also indicated here.

[0082] Figure 12 The diagram schematically illustrates the intermediate decorrelated image 1200 provided by the high-frequency luminance component forming module 1002, which has undergone local wavelet filtering. Figure 12 A schematic diagram of the decorrelationd image 1200 after local wavelet filtering is provided. This image 1200 is obtained by replacing the corresponding luminance component Y belonging to the first array group with a high-frequency luminance component Y. H This corresponds to a modified version of decorrelation image 304. The luminance component Y in decorrelation image 304, belonging to the second array, is not replaced and thus remains. At least a portion of the intermediate decorrelation image 1200, after local wavelet filtering, is temporarily stored... Figure 10 The intermediate buffer memory 1003 of the plum blossom-shaped wavelet filter module 302 shown is used.

[0083] Refer again Figure 10 The low-frequency luminance component forming module 1004 retrieves the luminance component Y from the intermediate decorrelation image 1200 after local wavelet filtering from the intermediate buffer memory 1003, as well as four directly adjacent high-frequency luminance components Y' arranged in a quincunx pattern relative to the luminance component Y. H In this example, the luminance component Y belongs to the second array group, while the directly adjacent high-frequency luminance component Y0 H The luminance component Y belonging to the first array group was replaced. For example, the luminance component Y could exist in the non-even rows of the decorrelated image 304, and thus exist in the locally wavelet-filtered version 1200 in the middle, while the four directly adjacent high-frequency luminance components Y H Two of the four luminance components Y can exist in the even rows immediately above the aforementioned even rows, and the other two of these four directly adjacent luminance components Y can exist in the even rows immediately below the aforementioned even rows.

[0084] The low-frequency luminance component forming module 1004 forms the luminance component Y (on one hand) and four directly adjacent high-frequency luminance components Y. H (On the other hand) weighted combination is performed to form the low-frequency luminance component Y. L The weighted combination can be expressed as follows:

[0085] Y L (i,j)=Y(i,j)+1 / 8*Y H (i-1,j-1)+1 / 8*Y H (i+1,j-1)+1 / 8*Y H (i+1,j+1)+1 / 8*Y H (i-1,j+1),

[0086] As in the previous text, the two variables separated by a comma between the parentheses represent variables in... Figure 12 The image shows the position of the correlated luminance component in the middle of the locally wavelet-filtered decorrelated image 1200. Therefore, i and j represent the formed low-frequency luminance component Y, respectively. L The positions of (i,j) and the brightness component Y(i,j) that is replaced therefrom in the horizontal and vertical directions.

[0087] Figure 13 The method for forming the low-frequency luminance component Y is shown. L The processing can be considered as a second local wavelet filtering operation. Figure 13 Provides the low-frequency luminance component Y for forming the decorrelationd image 900 after local wavelet filtering. L A conceptual diagram of the second local wavelet filtering operation. Figure 13 In the left half of the figure, a portion of the decorrelation image 1200 after local wavelet filtering is shown, while a portion of the decorrelation image 900 after local wavelet filtering is shown in the right half of the figure.

[0088] exist Figure 13 The image shows the luminance component Y in the decorrelated image 1200 after local wavelet filtering, and its four directly adjacent high-frequency luminance components Y0. H Furthermore, it also indicates the replacement of the aforementioned luminance component with the low-frequency luminance component Y in the locally wavelet-filtered decorrelationed image 900. L Furthermore, it also indicates the weighting factors corresponding to these components mentioned above.

[0089] and Figure 5 The de-related module 301 shown is the same. Figure 10 The illustrated cloverleaf wavelet filter module 302 can use integers generated by applying weighting factors to the components and rounding division. This also allows for a relatively simple and low-cost implementation. There are many ways to round numbers, such as rounding down to the nearest integer as exemplified above.

[0090] Refer again Figure 3The image processor 102 shown, and the two-dimensional wavelet filtering module 303 apply two-dimensional wavelet filtering processing to the image processor 102 and the two-dimensional wavelet filtering module 303 respectively. Figure 9 The image 900 shown after local wavelet filtering contains the red-based chromaticity component Cr, the blue-based chromaticity component Cb, and the low-frequency luminance component Y. L In other words, the red-based chromaticity component Cr in this image actually constitutes a sub-image that has undergone two-dimensional wavelet filtering. Similarly, the blue-based chromaticity component Cb actually constitutes another sub-image that has undergone this filtering process, along with the low-frequency luminance component Y. L This constitutes another sub-image that has undergone two-dimensional wavelet filtering.

[0091] Two-dimensional wavelet filtering can include two consecutively executed one-dimensional wavelet filtering operations. One of these operations can be performed horizontally, while the other can be performed vertically. For example, sub-image components can first be applied row-by-row to a one-dimensional wavelet filter. Then, the first wavelet filtering operation produces horizontal high-frequency and low-frequency subbands. Subsequently, components of the horizontal high-frequency subbands can be applied column-by-column to a one-dimensional wavelet filter, which produces horizontal high-frequency / vertical high-frequency and horizontal high-frequency / vertical low-frequency subbands. Components of the horizontal low-frequency subbands can be applied to a one-dimensional wavelet filter, which produces horizontal low-frequency / vertical high-frequency and horizontal low-frequency / vertical low-frequency subbands.

[0092] Figure 14 A set of subbands provided by the two-dimensional wavelet filtering module 303 is shown. Figure 14 It is a horizontal axis F that represents the frequency in the horizontal direction. H And the vertical axis F, representing the frequency in the vertical direction. Y The two-dimensional frequency map. In this example, there are four sub-bands mentioned above: a horizontal high-frequency and vertical high-frequency sub-band indicated by reference numeral 1401, a horizontal high-frequency and vertical low-frequency sub-band indicated by reference numeral 1402, a horizontal low-frequency and vertical high-frequency sub-band indicated by reference numeral 1403, and a horizontal low-frequency and vertical low-frequency sub-band indicated by reference numeral 1404. These four sub-bands are generated from the two-dimensional wavelet filtering process of the sub-image as described above, whereby the sub-image is formed from components of the same type in the decorrelated image 900 after local wavelet filtering. Therefore, Figure 14 The four sub-bands can be composed of the red-based chromaticity component Cr, the blue-based chromaticity component Cb, or the low-frequency luminance component Y. L produce.

[0093] Figure 15 The wavelet filtering process of the Le Gall 5 / 3 type is shown. Figure 15 A conceptual diagram of this wavelet filtering process is provided. See below for reference. Figure 15 The Le Gall 5 / 3 type wavelet filtering process described can correspond to the aforementioned one-dimensional wavelet filtering operation performed by the two-dimensional wavelet filtering module 303. Furthermore, the Le Gall 5 / 3 type wavelet filtering process is conceptually associated with the first and second decorrelation operations performed by the decorrelation module 301, and with the first and second wavelet filtering operations performed by the quincunx wavelet filtering module 302. This will be explained in more detail below.

[0094] Le Gall 5 / 3 type wavelet filtering is applied to a series of input samples, in Figure 15 The upper part of the image displays these samples as a series of relatively small circles. Some of these small circles are filled with white, while others are filled with black. In the following text, these samples will be referred to as white input samples and black input samples, respectively. Figure 15 As shown, white input samples are interleaved with black input samples. Le Gall 5 / 3 type wavelet filtering provides high-frequency output samples of the black input samples and low-frequency output samples of the white input samples. Figure 15 The middle part of the figure shows high-frequency output sampling, while the lower part of the figure shows low-frequency output sampling.

[0095] The high-frequency output sampling of black input samples is formed by weighting and combining the black input sample (on one hand) and two directly adjacent white input samples (on the other hand). For example... Figure 15 As shown, in this weighted combination, the black input sample has a weighting factor of +1, and the two directly adjacent white input samples have a weighting factor of -0.5. Therefore, this weighted combination represents the difference between the black input sample and the two directly adjacent white input samples. The high-frequency output sample generated from this weighted combination effectively replaces the black input sample that forms it.

[0096] The low-frequency output sample of the white input sample is formed by performing a weighted combination of the white input sample and two directly adjacent high-frequency output samples. For example... Figure 15 As shown, the two high-frequency output samples in the weighted combination actually replace the two black input samples that are directly adjacent to the associated white input sample. Figure 15 As shown, in this weighted combination, the weighting factor for the white input sample is +1, and the weighting factor for the directly adjacent high-frequency output sample is +0.25. Therefore, this weighted combination is a weighted sum of the white input sample and two directly adjacent high-frequency output samples. The low-frequency output sample generated by this weighted combination effectively replaces the white input sample that forms it.

[0097] As previously described, the first and second decorrelation operations performed by decorrelation module 301 in image processor 102 to form decorrelated image 304 are conceptually associated with wavelet filtering of the Le Gall 5 / 3 type as described above. More specifically, Figure 6 The process shown, which is based on the red chromaticity component Cr and the blue chromaticity component Cb, can conceptually be considered as having a one-dimensional characteristic. Figure 15 The two-dimensional extension of the process used to form high-frequency output samples in the Le Gall 5 / 3 type wavelet filtering process is shown. Figure 15 The process shown involves selecting two directly adjacent input samples in one dimension to form different high-frequency output samples, thus forming the chromaticity components. This includes processes such as... Figure 6 The diagram shows four directly adjacent components selected in two dimensions. The sum of the weighting factors in the weighted combination providing high-frequency output sampling is equal to the sum of the weighting factors in the weighted combination providing chromaticity components. The first weighted combination has two neighbors, each with a weighting factor of -0.5; the last weighted combination has four neighbors, each with a weighting factor of -0.25.

[0098] same, Figure 8 The process for forming the luminance component Y shown can conceptually be viewed as having one-dimensional characteristics. Figure 15 The two-dimensional extension of the process used to form the low-frequency output sample in the Le Gall 5 / 3 type wavelet filtering process is shown. Figure 15 The process shown involves selecting two directly adjacent high-frequency output samples in one dimension to form a different low-frequency output sample, thus forming the luminance component Y. This process includes, for example,... Figure 8 The diagram shows four directly adjacent chroma pixels selected in two dimensions. Similarly, the sum of the weighting factors in the weighted combination providing low-frequency output sampling is equal to the sum of the weighting factors in the weighted combination providing the luminance component. The first weighted combination has two neighbors, each with a weighting factor of +0.25; the last weighted combination has four neighbors, each with a weighting factor of +0.125.

[0099] Furthermore, as previously stated, the first and second wavelet filtering operations performed by the quincunx wavelet filtering module 302 in the image processor 102 to form the decorrelated image 900 after local wavelet filtering are conceptually associated with the Le Gall 5 / 3 type wavelet filtering process described above. More specifically, Figure 11 The high-frequency luminance component Y is shown. H The processing can be conceptually viewed as having one-dimensional properties. Figure 15The two-dimensional extension of the process used to form high-frequency output samples in the Le Gall 5 / 3 type wavelet filtering process is shown. Figure 15 The diagram shows how two directly adjacent input samples are selected in one dimension to form different high-frequency output samples, thus forming the high-frequency brightness component Y. H The processing includes, for example Figure 11 The diagram shows four directly adjacent luminance pixels selected in two dimensions. The sum of the weighting factors in the weighted combination providing high-frequency output sampling is equal to the sum of the weighting factors in the weighted combination providing high-frequency luminance components. The first weighted combination has two neighbors, each with a weighting factor of -0.5; the last weighted combination has four neighbors, each with a weighting factor of -0.25.

[0100] same, Figure 13 The low-frequency luminance component Y is shown. L The processing can be conceptually viewed as having one-dimensional properties in Figure 15 The two-dimensional extension of the process used to form low-frequency output samples in the Le Gall 5 / 3 type wavelet filtering process is shown. Figure 15 The diagram shows how two directly adjacent high-frequency output samples are selected in one dimension to form different low-frequency output samples, thus forming the low-frequency luminance component Y. L The processing includes, for example Figure 8 As shown, four directly adjacent high-frequency luminance components Y are selected in two dimensions. H Similarly, the sum of the weighting factors in the weighted combination providing low-frequency output sampling is equal to the sum of the weighting factors in the weighted combination providing low-frequency luminance components. The first weighted combination has two neighbors, each with a weighting factor of +0.25; the last weighted combination has four neighbors, each with a weighting factor of +0.125.

[0101] exist Figure 1 In the camera 100 shown, the image processor 102 thereby... Figure 2 The original image 200 shown is processed to obtain Figure 10 The decorrelated image 900 shown is processed by local wavelet filtering. This image can be viewed as a combination of four sub-images: one containing the high-frequency brightness component Y. H The sub-image contains the low-frequency brightness component Y. L The image processor 102 provides sub-images based on the red chromaticity component Cr and sub-images based on the blue chromaticity component Cb. Furthermore, the image processor 102 also provides for each of the three last-mentioned sub-images... Figure 14 The example shown has a set with four sub-bands. The encoder 103 can then efficiently encode the representation without any information loss. Figure 2 The following data from the original image 200 shown: high-frequency luminance component Y H , representing the low-frequency luminance component Y L The four subbands represent the four subbands based on the red chromaticity component Cr, and the four subbands based on the blue chromaticity component Cb.

[0102] Notes

[0103] The embodiments described above with reference to the accompanying drawings are given by way of example. The invention can be implemented in many different ways. To illustrate this, some alternatives are briefly indicated herein.

[0104] This invention can be applied to various types of products or methods related to image processing, video processing, or both. The given embodiments provide examples of applying the invention to image and video compression. However, the invention can be applied to other types of image and video processing, such as providing multi-resolution views of images for segmented storage and / or transmission, analyzing images to detect objects / attributes or achieve segmentation, and serving as an analysis filter for debayering algorithms.

[0105] According to the present invention, there are many ways to implement an image processor. Any module in the given embodiments can be implemented by means of electronic circuitry (which may be dedicated or programmable) or by means of a suitably programmed processor or a combination thereof. A computer program can be defined as referenced... Figure 1-15 The block diagram describes one or more operations performed by an image processor. In this respect, each block diagram can be at least partially considered as a flowchart representing such a computer program, and as representing a method that can be executed when the processor runs the computer program. For example, Figure 5 The decorrelation module shown can be considered as representing the decorrelation steps. Similarly, the other modules can be considered as representing the steps of the method.

[0106] According to the present invention, the decorrelation pixel-component transformation scheme in a product or method can be implemented in a variety of different ways. In the given embodiments, the applied pixel-component transformation scheme is conceptually related to wavelet filtering of the LeGall 5 / 3 type as described above. More specifically, in this conceptual relationship, pixels of the second and third types (which are red and blue pixels in the given embodiments) can be considered as being sampled for high-frequency components using a two-dimensional filter kernel, which is actually a one-dimensional kernel extension of the correlated wavelet filtering process, thereby applying weighted factor correction. Pixels of the first type (which are green pixels in the given embodiments) can be considered as being sampled for low-frequency components using the same two-dimensional filter kernel, which is actually an extension of the one-dimensional filter kernel of the correlated wavelet filtering process, thereby applying weighted factor correction.

[0107] In other embodiments, the pixel-component transformation scheme can be based on different types of wavelet filtering, such as Daubechies 9 / 7 type wavelet filtering, or wavelet filtering based on 13 / 7. Pixel-component transformation schemes based on one of the aforementioned wavelet filtering types may be more complex to implement, as illustrated in the pixel-component transformation schemes in the given embodiments. For example, compared with those referenced above... Figure 5-8 Compared to the first and second decorrelation operations described, the pixel-component transform scheme based on Daubechies 9 / 7 type wavelet filtering may include more local decorrelation operations. That is, compared with Figure 5 Compared to the decorrelation module 301 shown, the decorrelation module based on wavelet filtering of the Daubechies 9 / 7 type may require more function blocks.

[0108] Pixel-component transform schemes based on wavelet filtering of a more complex type than the Le Gall 5 / 3 type provide more complex weighted combinations for defining the components of the replacement pixel. This is because more neighboring pixels are considered to form the components of the replacement pixel. Nevertheless, the general rules relating to the weighted combinations indicated above and defined in the appended claims still apply. That is, in a weighted combination providing components of the first type, adjacent pixels of the second and third types have generally positive weighting factors corresponding to adding all adjacent pixels of the second and third types to the replaced first type pixel. Conversely, in a weighted combination providing components of the second or third type, adjacent pixels of the first type have generally negative weighting factors corresponding to subtracting all adjacent pixels of the first type from the replaced second or third pixel, respectively.

[0109] It should be further noted that decorrelation pixel-component transform schemes conceptually associated with specific types of wavelet filtering (e.g., Le Gall 5 / 3 type wavelet filtering) can be implemented in a variety of different ways. For example, the given embodiment describes an implementation that can be simplified as follows: Providing a weighted combination based on the red chromaticity component Cr is simplified by considering only pixels in two consecutive rows. If the red pixel R(i,j) is located in the lower row of the two consecutive rows, then the weighted combination can be expressed as follows:

[0110] Cr(i,j)=R(i,j)-1 / 4*G(i-1,j)-1 / 2*G(i,j-1)-1 / 4*G(i+1,j)

[0111] refer to Figure 6 In this case, the green pixel G immediately following the red pixel R, which is replaced by the red-based chromaticity component Cr, will be omitted. This omission is compensated for by doubling the weighting factor of the green pixel G immediately following the red pixel R, so that the weighting factor becomes -0.5 instead of -0.25. This is equivalent to horizontally mirroring the aforementioned green pixel G so that it is also immediately following the red pixel R, thereby replacing the omitted green pixel G.

[0112] If the red pixel R(i,j) is located in the upper row of two consecutive rows, then the weighted combination can be represented as follows:

[0113] Cr(i,j)=R(i,j)-1 / 4*G(i-1,j)-1 / 4*G(i+1,j)-1 / 2*G(i,j+1)

[0114] refer to Figure 6 In this case, the green pixel G immediately above the red pixel R, which is replaced by the red-based chromaticity component Cr, is omitted. This omission is compensated for by doubling the weighting factor of the green pixel G immediately below the red pixel R, so that the weighting factor becomes -0.5 instead of -0.25.

[0115] Similarly, providing a weighted combination based on the blue chromaticity component Cb is simplified by considering only pixels in two consecutive rows. If the blue pixel B(i,j) is located in the lower row of the two consecutive rows, then the weighted combination can be represented as follows:

[0116] Cb(i,j)=B(i,j)-1 / 4*G(i-1,j)-1 / 2*G(i,j-1)-1 / 4*G(i+1,j)

[0117] refer to Figure 6In this case, the green pixel G immediately following the blue pixel B, which is replaced by the blue-based chromaticity component Cb, will be omitted. This omission is compensated for by doubling the weighting factor of the green pixel G immediately above the blue pixel B, so that the weighting factor becomes -0.5 instead of -0.25.

[0118] If the blue pixel B(i,j) is located in the upper row of two consecutive rows, then the weighted combination can be represented as follows:

[0119] Cb(i,j)=B(i,j)–1 / 4*G(i-1,j)–1 / 4*G(i+1,j)-1 / 2*G(i,j+1)

[0120] refer to Figure 6 In this case, the green pixel G immediately above the blue pixel B, which is replaced by the blue-based chromaticity component Cb, will be omitted. This omission is compensated for by doubling the weighting factor of the green pixel G immediately below the blue pixel B, so that the weighting factor becomes -0.5 instead of -0.25.

[0121] Similarly, providing a weighted combination of the luminance component Y can also be achieved by considering only... Figure 7 The intermediate decorrelation image 700 is simplified by considering the pixels and chromaticity components on two consecutive rows. If the luminance pixel Y(i,j) and the replaced green pixel G(i,j) are located in the lower row of the two consecutive rows, then the weighted combination, with horizontally adjacent blue-based chromaticity components Cb and vertically adjacent red-based chromaticity components Cr, can be represented as follows:

[0122] Y(i,j)=G(i,j)+1 / 8*Cb(i-1,j)+1 / 4*Cr(i,j-1)+1 / 8*Cb(i+1,j)

[0123] In a variant of the above case, with horizontally adjacent red-based chromaticity components Cr and vertically adjacent blue-based chromaticity components Cb, the weighted combination can be expressed as follows:

[0124] Y(i,j)=G(i,j)+1 / 8*Cr(i-1,j)+1 / 4*Cb(i,j-1)+1 / 8*Cr(i+1,j)

[0125] refer to Figure 8In the latter case, the blue-based chromaticity component Cb immediately following the green pixel G, which is replaced by the luminance component Y, is omitted. This omission is compensated for by doubling the weighting factor of the blue-based chromaticity component Cb immediately following the green pixel G, so that the weighting factor becomes +0.25 instead of +0.125. This is equivalent to mirroring the aforementioned blue-based chromaticity component Cb horizontally, so that this component also immediately follows the green pixel G, thus replacing the omitted blue-based chromaticity component Cb.

[0126] If the luminance pixel Y(i,j) and the replaced green pixel G(i,j) are located in the upper row of two consecutive rows, then the weighted combination can be represented as follows, given the horizontally adjacent blue-based chromaticity component Cb and the vertically adjacent red-based chromaticity component Cr:

[0127] Y(i,j)=G(i,j)+1 / 8*Cb(i-1,j)+1 / 8*Cb(i+1,j)+1 / 4*Cr(i,j+1)

[0128] In a variant of the above case, with horizontally adjacent red-based chromaticity components Cr and vertically adjacent blue-based chromaticity components Cb, the weighted combination can be expressed as follows:

[0129] Y(i,j)=G(i,j)+1 / 8*Cr(i-1,j)+1 / 8*Cr(i+1,j)+1 / 4*Cb(i,j+1)

[0130] refer to Figure 8 In the latter case, the blue-based chromaticity component Cb immediately above the green pixel G that is replaced by the luminance component Y will be omitted. This omission is compensated for by doubling the weighting factor of the blue-based chromaticity component Cb immediately below the green pixel G, so that the weighting factor becomes +0.25 instead of +0.125.

[0131] Depend on Figure 10 The wavelet filtering operation applied to the luminance component Y by the plum blossom-shaped wavelet filter module 302 shown can be simplified in a similar way. That is, the simplified wavelet filtering operation can consider only... Figure 4 The brightness components on two consecutive lines in the decorrelated image 304 are shown. (Reference) Figure 11 This means that the high-frequency luminance component Y located immediately following it can be ignored. H The luminance component Y is located below the luminance component Y on the line. In this case, the luminance component Y is located immediately following the line where the high-frequency luminance component Y is formed. HThe weighting factor of the remaining luminance component Y on the row above the luminance component Y can be doubled, thus becoming -0.5 instead of -0.25. Conversely, simplification could mean ignoring the row immediately following the formed high-frequency luminance component Y. H The luminance component Y is above the luminance component Y on the line. In this case, the weighting factor of the remaining luminance component Y will also be doubled.

[0132] By only considering Figure 12 The image shown shows the luminance component Y1 and the high-frequency luminance component Y2 on two consecutive rows in the decorrelated image 1200 after local wavelet filtering. H This can further simplify the wavelet filtering operation applied by the plum blossom-shaped wavelet filter module 302. (Reference) Figure 13 This simplification could mean ignoring the low-frequency luminance component Y located immediately following it. L The high-frequency luminance component Y on the line below the luminance component Y H In this case, the low-frequency luminance component Y is located immediately following it. L The remaining high-frequency luminance component Y on the line above the luminance component Y H The weighting factor can be doubled, thus becoming +0.25 instead of +0.125. Conversely, this simplification could mean ignoring the area immediately following the high-frequency luminance component Y. H The high-frequency luminance component Y above the line luminance component Y H In this case, the weighting factor of the remaining luminance component Y will also be doubled.

[0133] The notes given above indicate that decorrelation pixel-component transformation can be implemented in various different ways according to the present invention. This transformation can be applied in a fully reversible or partially reversible manner. For example, in video coding with a relatively low compression ratio, a fully reversible transformation is preferred to achieve high quality. Conversely, in video coding with a relatively high compression ratio, a partially reversible transformation may be acceptable, and even preferred.

[0134] In another aspect, the wavelet filtering operation applied to the luminance component Y in the product or method according to the invention can be implemented in a variety of different ways. In the given embodiments, the luminance component Y is decomposed row-by-row into a first array and a second array. In other embodiments, the decomposition can be column-by-column. More generally, column-by-column processing can be implemented, rather than the row-by-row processing indicated above and in the given embodiments.

[0135] This invention can typically be implemented in a variety of different ways, and thus, different implementations can have different topologies. In any given topology, a single entity can perform multiple functions, or several entities can jointly perform a single function. For this purpose, the accompanying drawings are merely illustrative.

[0136] The foregoing notes indicate that the embodiments described with reference to the accompanying drawings are illustrative of the invention and not limiting of it. The invention may be practiced in various alternative ways within the scope of the appended claims. All variations within the equivalent meaning and scope of the claims should be included within their scope. Any reference numerals in the claims should not be construed as limiting the claims. The verb "comprising" in a claim does not exclude the presence of other components or steps besides those listed in the claim. This also applies to similar verbs such as "comprising" and "consisting of". In a claim relating to a product, reference to a singular form of a component does not exclude that the product may include a plurality of such components. Similarly, in a claim relating to a method, reference to a singular form of a step does not exclude that the method may include a plurality of such steps. The fact that the corresponding dependent claims define the corresponding additional features does not exclude combinations of additional features other than those reflected in the claims.

Claims

1. An image processor configured to process an image (200) comprising three different color pixels (R, G, B) arranged in a Bell pattern, thereby obtaining a decorrelated image (304) composed of three different type components (Y, Cr, Cb) arranged in a pattern corresponding to the Bell pattern. Thus, according to the Bell pattern, the image (200) is a rectangular block array of 2×2 pixels, including two diagonally arranged first color pixels (G), a second color pixel (R), and a third color pixel (B); and Thus, according to the pattern corresponding to the Bell pattern, the decorrelation image (304) is a rectangular block array consisting of 2×2 components, including two diagonally arranged first type components (Y), one second type component (Cr), and one third type component (Cb). The image processor is configured as follows: The first type of component (Y) in the decorrelated image (304) is provided to replace the first color pixel (G) in the image (200), whereby the first type of component (Y) is a weighted combination of a cluster of pixels in the image (200) that includes the first color pixel (G) and neighboring pixels, wherein the neighboring second and third color pixels (R, B) have an overall positive weighting factor corresponding to adding all the neighboring second and third color pixels (R, B) to the first color pixel (G). The second type of component (Cr) in the decorrelation image (304) is provided to replace the second color pixel (R) in the image (200), whereby the second type of component (Cr) is a weighted combination of a cluster of pixels in the image (200) that includes the second color pixel (R) and neighboring pixels, wherein the neighboring first color pixels (G) have an overall negative weighting factor corresponding to subtracting all the neighboring first color pixels (G) from the second color pixel (R); and The third type component (Cb) in the decorrelation image (304) is provided to replace the third color pixel (B) in the image (200), whereby the third type component (Cb) is a weighted combination of a cluster of pixels in the image (200) that includes the third color pixel (B) and neighboring pixels, wherein the neighboring first color pixel (G) has a corresponding overall negative weighting factor that is subtracted from the third color pixel (B) for all the neighboring first color pixels (G); The weighted combinations that form the components (Y) of the first type include: The nearest neighbor of the first color pixel (G) that is constructed in an x-shape relative to the first color pixel that is being replaced; as well as The next adjacent first color pixel that is constructed in a cross shape relative to the first color pixel that was replaced.

2. The image processor of claim 1, wherein the image processor is configured to: First, a corresponding second-type component (Cr) is provided to replace the corresponding second-color pixel (R), and a corresponding third-type component (Cb) is provided to replace the corresponding third-color pixel (B). Then, A corresponding first type component (Y) is provided to replace the corresponding first color pixel (G), thereby providing the first type component (Y) based on a weighted combination of the first color pixel (G) that is replaced with the first type component on one hand and the second type component (Cr) and the third type component (Cb) that are respectively provided to replace the second color pixel and the third color pixel that form a cluster with the first color pixel on the other hand.

3. The image processor of claim 1, wherein in the weighted combination, the nearest neighbor first color pixel (G) constructed in an x-shape has a heavier weighting factor than the further adjacent first color pixel constructed in a cross shape.

4. The image processor of claim 3, wherein in the weighted combination forming the component (Y) of the first type: The weighting factor for the replaced first color pixel (G) is +7 / 8; The weighting factor for the first nearest-neighbor color pixel in the x-shaped configuration is -1 / 16; The weighting factor for the further adjacent first color pixels, which are arranged in a cross shape, is -1 / 32; and The weighting factor for adjacent second color pixels (R) and adjacent third color pixels (B) is +1 / 8.

5. The image processor of claim 4, wherein in the weighted combination forming the components (Cr) of the second type: The weighting factor for the replaced second color pixel (R) is +1; and The adjacent first color pixel (G) is four directly adjacent first color pixels with a weighting factor of -1 / 4.

6. The image processor of claim 4, wherein in the weighted combination forming the components (Cb) of the third type: The weighting factor of the replaced third color pixel (B) is +1; and The adjacent first color pixel (G) is four directly adjacent first color pixels with a weighting factor of -1 / 4.

7. The image processor according to any one of claims 1 to 6, wherein the image processor is configured to apply wavelet-based filtering to the first type of component (Y) in the decorrelated image (304), thereby obtaining the high-frequency first type of component (Y). H ) and the first type of low-frequency component (Y) L The wavelet-based filtering process includes decomposing the first type of component (Y) into an array of first-type components in a first group and an array of first-type components in a second group. The arrays of first-type components in the first group and the arrays of first-type components in the second group are interleaved, thereby providing the high-frequency first-type component (Y) to the first-type components belonging to the first group array. H ), and thereby provide the low-frequency first-type components (Y) for the first type of components belonging to the second array group. L ).

8. The image processor of claim 7, wherein the first type of high-frequency component (Y) H ) is a weighted combination of the first type of components belonging to the first group of arrays and the adjacent first type of components belonging to the second group of arrays, and wherein the low-frequency first type of components (Y) L ) is the weighted sum of the first type of component belonging to the second group of arrays and the adjacent high-frequency components.

9. The image processor according to claim 8, wherein: In the first type of component (Y) that forms the high frequency H In the weighted combination of the first type of components belonging to the first array group, the weighting factor is +1, and the weighting factor of the adjacent first type components belonging to the second array group is -1 / 4. as well as In forming the first type of component (Y) of the low frequency L In the weighted sum of ), the weighting factor of the first type of component belonging to the second array is +1, and the weighting factor of the adjacent high-frequency component is +1 / 8.

10. The image processor of claim 7, wherein the image processor is configured to apply two-dimensional wavelet-based filtering to the low-frequency first type component (Y). L The second type component (Cr) in the decorrelation image (304) and the third type component (Cb) in the decorrelation image (304).

11. An encoding system comprising an image processor according to any one of claims 1 to 10, and an encoder configured to encode at least one of the following: the decorrelated image (304) and a wavelet-based filtered version of the decorrelated image (304).

12. The encoding system of claim 11, wherein the encoder is configured to encode the following: High-frequency type I components (Y H ); By applying two-dimensional wavelet-based filtering to the low-frequency type 1 component (Y) L The sub-bands obtained; The subband obtained by applying the two-dimensional wavelet-based filtering process to the second type of component (Cr) in the decorrelated image (304); and The subband is obtained by applying the two-dimensional wavelet-based filtering process to the third type of component (Cb) in the decorrelated image (304).

13. A reverse-operation image processor configured to process a decorrelated image (304) generated by the image processor according to claim 1, the reverse-operation image processor being configured to reconstruct an image (200) comprising three different color pixels (R, G, B) arranged in a Bell pattern.

14. A method for processing an image (200) comprising three different color pixels (R, G, B) arranged in a Bell pattern, thereby obtaining a decorrelated image (304) composed of three different types of components (Y, Cr, Cb) arranged in a pattern corresponding to the Bell pattern. Thus, according to the Bell pattern, the image (200) is a rectangular block array of 2×2 pixels, including two diagonally arranged first color pixels (G), a second color pixel (R), and a third color pixel (B); and Thus, according to the pattern corresponding to the Bell pattern, the decorrelation image (304) is a rectangular block array consisting of 2×2 components, including two diagonally arranged first type components (Y), one second type component (Cr), and one third type component (Cb). The method includes: The first type of component (Y) in the decorrelated image (304) is provided to replace the first color pixel (G) in the image (200), whereby the first type of component (Y) is a weighted combination of a cluster of pixels in the image (200) that includes the first color pixel (G) and neighboring pixels, wherein the neighboring second and third color pixels (R, B) have an overall positive weighting factor corresponding to adding all the neighboring second and third color pixels (R, B) to the first color pixel (G). The second type of component (Cr) in the decorrelation image (304) is provided to replace the second color pixel (R) in the image (200), whereby the second type of component (Cr) is a weighted combination of a cluster of pixels in the image (200) that includes the second color pixel (R) and neighboring pixels, wherein the neighboring first color pixels (G) have an overall negative weighting factor corresponding to subtracting all the neighboring first color pixels (G) from the second color pixel (R); and The third type component (Cb) in the decorrelation image (304) is provided to replace the third color pixel (B) in the image (200), whereby the third type component (Cb) is a weighted combination of a cluster of pixels in the image (200) that includes the third color pixel (B) and neighboring pixels, wherein the neighboring first color pixel (G) has a corresponding overall negative weighting factor that is subtracted from the third color pixel (B) for all the neighboring first color pixels (G); The weighted combinations that form the components (Y) of the first type include: The nearest neighbor of the first color pixel (G) that is constructed in an x-shape relative to the first color pixel that is being replaced; as well as The next adjacent first color pixel that is constructed in a cross shape relative to the first color pixel that was replaced.

15. A computer program product comprising an instruction set that enables an image processor to execute the method of claim 14.

Images