Image processor

Abstract

The present invention relates to an image processor for processing an image (200) comprising three types of color pixels (R, G, B) arranged in a Bayer pattern. The image processor provides an inversely correlated image composed of three types of components (Y, Cr, Cb) arranged in a pattern corresponding to the Bayer pattern. The image processor provides a first type of component (Y) of the inversely correlated image as a substitute for a first type of pixel (G) of the image, wherein the first type of component (Y) is a weighted combination of clusters of pixels of the image including the first type of pixel (G) and neighboring pixels, and the second and third types of neighboring pixels (R, B) have an overall positive (+) weighting factor corresponding to the total sum of the second and third types of neighboring pixels (R, B) with respect to the first type of pixel (G). The image processor provides a component (Cr) of the second type of the inversely correlated image as a substitute for a pixel (R) of the second type of the image, wherein the component (Cr) of the second type is a weighted combination of clusters of pixels of the image including the pixel (R) of the second type and neighboring pixels, and the neighboring pixels (G) of the first type have an overall negative (-) weighting factor corresponding to the overall subtraction of the neighboring pixels (G) of the first type from the pixel (R) of the second type. The image processor provides a component (Cb) of the third type of the inversely correlated image as a substitute for a pixel (B) of the third type of the image, wherein the component (Cb) of the third type is a weighted combination of clusters of pixels of the image including the pixel (B) of the third type and neighboring pixels, and the neighboring pixels (G) of the first type have an overall negative (-) weighting factor corresponding to the overall subtraction of the neighboring pixels (G) of the first type from the pixel (B) of the third type.

Description

Technology Field

[0001] One aspect of the present invention relates to an image processor configured to process an image composed of three types of color pixels arranged in a Bayer pattern. The image processor may be used, for example, as a preprocessor in an encoding system that improves encoding efficiency. Other aspects of the present invention relate to an encoding system, an image processing method, and a computer program for an image processor. Background Technology

[0002] U.S. Patent No. 6,154,493 describes a technique for splitting raw image data into multiple channels, including color plane difference channels. Each of these channels is individually compressed using a two-dimensional discrete wavelet transform. The compression utilizes quantization, thereby enabling the restoration of the compressed channel data to produce a perceptually lossless image. This technique operates directly on images in the form of Bayer patterns. Quantization thresholds are defined for quantization that may vary depending on the channel being processed and the discrete wavelet transform (DWT) subband.

[0003] Improved image processing is required to obtain a relatively highly inversely correlated image based on an image in which color pixels are arranged in a Bayer pattern, which enables the achievement of higher coding efficiency that provides better image quality at a given compression rate.

[0004] According to an embodiment of the present invention as defined in claim 1, an image processor is provided configured to process an image comprising three types of color pixels arranged in a Bayer pattern to obtain a de-corrected image composed of three types of components arranged in a pattern corresponding to the Bayer pattern.

[0005] Accordingly, according to the Bayer pattern, the image is a rectangular array of blocks of 2 x 2 color pixels comprising two diagonally arranged color pixels of type 1, color pixels of type 2, and color pixels of type 3;

[0006] Accordingly, according to the pattern corresponding to the Bayer pattern, the inversely correlated image is a rectangular array of blocks of 2 x 2 components comprising a component arranged diagonally on two sides of a first type, a component of a second type, and a component of a third type;

[0007] The above processor is:

[0008] - Configured to provide a component of the first type of an inversely correlated image as a substitute for a pixel of the first type of the image, wherein the component of the first type is a weighted combination of clusters of pixels of the image including the pixel of the first type and neighboring pixels, and the neighboring pixels of the second and third types have a total positive (+) weighting factor corresponding to the total addition of the neighboring pixels of the second and third types to the pixel of the first type;

[0009] - Configured to provide a component of the second type of an inversely correlated image as a substitute for a pixel of the second type of the image, wherein the component of the second type is a weighted combination of clusters of pixels of the image including the pixel of the second type and neighboring pixels, and the neighboring pixels of the first type have an overall negative (-) weighting factor corresponding to the total subtraction of the neighboring pixels of the first type from the pixel of the second type; and

[0010] - Configured to provide a component of the third type of an image that is inversely correlated as a substitute for a pixel of the third type of the image, wherein the component of the third type is a weighted combination of clusters of pixels of the image including the pixel of the third type and neighboring pixels, and the neighboring pixels of the first type have an overall negative (-) weighting factor corresponding to the total subtraction of the neighboring pixels of the first type from the pixel of the third type.

[0011] According to further aspects of the present invention as defined in claims 12, 14 and 15, each encoding system, a method for processing an image, and a computer program product are provided.

[0012] An image processor as defined below can effectively decorrelate an image composed of three types of color pixels arranged in a Bayer pattern, such as an image generated by an image sensor, for example. The three types of color pixels may be, for example, red, green, and blue, such that green pixels occur twice as often as red and blue pixels. As another example, the three types of color pixels may be, for example, yellow, cyan, and magenda, or follow any other color scheme. Because the image processor efficiently decorrelates the corresponding type of image, the decorreled image generated by it can be efficiently encoded. This enables obtaining a given image quality with an encoded image composed of a relatively small amount of data. In other words, the encoded image has relatively good quality for a given amount of data that the encoded image can contain. Nevertheless, the image processor does not need to cause any loss of information. That is, the image in its original form can be perfectly restored based on the decorreled image generated by the image processor.

[0013] Hereinafter, several embodiments given as examples of the present invention will be described in detail with reference to the accompanying drawings. Hereinafter, additional features will be presented, some of which are features defined in dependent claims, and these advantages will become more apparent. Brief explanation of the drawing

[0014] Figure 1 is a block diagram of a camera. Figure 2 is a schematic diagram of a raw image provided by the image sensor of a camera. Figure 3 is a block diagram of the image processor of a camera. Figure 4 is a schematic diagram of an inversely correlated image provided by the inverse correlation module of an image processor based on a raw image. Figure 5 is a block diagram of an inverse correlation module that provides an inversely correlated image. Figure 6 is a conceptual diagram showing the first negative correlation operation performed by the negative correlation module. Figure 7 is a schematic diagram of an intermediate inversely correlated image obtained by the first inverse correlation operation. Figure 8 is a conceptual diagram showing the second negative correlation operation performed by the negative correlation module. Figure 9 is a schematic diagram of a partially wavelet-filtered inversely correlated image provided by the quantum wavelet filtering module of an image processor based on the inversely correlated image. Figure 10 is a block diagram of a Conkense wavelet filtering module that provides a partially wavelet-filtered inversely correlated image. FIG. 11 is a conceptual diagram illustrating the first wavelet filtering operation performed by the Quankons wavelet filtering module. FIG. 12 is a schematic diagram of an intermediate partially wavelet-filtered inversely correlated image obtained by the first wavelet filtering operation. FIG. 13 is a conceptual diagram showing the second wavelet filtering operation performed by the Quankons wavelet filtering module. Figure 14 is a schematic diagram of a set of subbands provided by a two-dimensional wavelet filtering module of an image processor based on components of the same type in a partially wavelet-filtered inversely correlated image. FIG. 15 is a schematic diagram of a Le Gall 5 / 3 type wavelet filtering that can be applied by a 2D wavelet filtering module and is conceptually related to the first and second inverse correlation operations performed by the inverse correlation module as well as the first and second wavelet filtering operations performed by the convolution wavelet filtering module. Specific details for implementing the invention

[0015] Fig. 1 Figure 1 schematically illustrates a camera (100). Figure 1 provides a block diagram of the camera (100). The camera (100) may be, for example, a photo camera or a video camera, or a combination of a photo camera and a video camera capable of capturing images or videos, or both. The camera (100) may further store the captured images or videos, or both, in an encoded form. The camera (100) may transmit an encoded version of the captured images or videos, or both. The camera (100) may be, for example, part of a smartphone or other type of electronic device.

[0016] The camera (100) illustrated in FIG. 1 includes an image sensor (101), an image processor (102), an encoder (103), a storage device (104), and a communication interface (105). The image sensor (101) may include three types of color pixel sensors, such as, for example, a green-sensitive pixel sensor, a red-sensitive pixel sensor, and a blue-sensitive pixel sensor. These color pixel sensors may be arranged in a so-called Bayer pattern, thereby having twice as many green-sensitive pixel sensors as red-sensitive pixel sensors and blue-sensitive pixel sensors.

[0017] Fig. 2Figure 2 illustrates a raw image (200) that can be provided by the image sensor (101) of the camera (100). Figure 2 provides a schematic diagram of the raw image (200). The raw image (200) includes three types of color pixels: a first type of color pixel, which is a green pixel (G); a second type of color pixel, which is a red pixel (R); and a third type of color pixel, which is a blue pixel (B). These color pixels are arranged in a Bayer pattern. The green pixel (G) occurs twice as often as the red pixel (R) and the blue pixel (B). These three types of color pixels are interleaved as shown in Figure 2. As can be seen, according to the Bayer pattern, the raw image (200) is a rectangular array of 2x2 color pixel blocks containing green pixels (G), red pixels (R), and blue pixels (B) arranged along two diagonals.

[0018] The camera (100) basically operates as follows. An image processor (102) provides an inversely correlated and wavelet-filtered version of a raw image (200) that may include a low-frequency luminance component, a subband set for a high-frequency luminance component, and a subband set for a red-based chromaticity component. The inversely correlated and wavelet-filtered version of the raw image (200) may contain the same amount of information as the raw image (200). That is, the raw image (200) can be perfectly restored based on the inversely correlated and wavelet-filtered version of the raw image (200). This means that the image processor (102) does not need to cause information loss and therefore does not need to cause image quality loss. The image processor (102) will be described in more detail later.

[0019] The encoder (103) encodes an inversely correlated and wavelet-filtered version of the raw image (200). In doing so, the encoder (103) may apply data compression, thereby causing some loss of information and thus some loss of quality. For example, the encoder (103) may operate in the manner described in U.S. Patent No. 9,332,258.

[0020] The image processor (102), which will be described in more detail later, enables the encoder (103) to achieve a relatively high data compression rate for a given image quality. In other words, the image processor (102) enables the encoder (103) to provide an encoded image of relatively high quality for a given data compression rate. That is, the image processor (102) contributes to achieving a relatively high encoding efficiency. The image processor (102) and the encoder (103) together can be considered as an encoding system (106) capable of achieving a relatively high encoding efficiency, as illustrated in FIG. 1. The encoding system (106) can be implemented, for example, in the form of an integrated circuit.

[0021] The storage device (104) can store an encoded image provided by the encoding system (106). The storage device (104) may include, for example, a solid-state storage device or a disk-based storage device, or a combination thereof, as well as other types of storage devices.

[0022] The communication interface (105) can transmit the encoded image to an external device, and the external device may be connected to communicate with the camera (100) via, for example, a direct communication link or a communication network. The communication interface (105) may include, for example, a USB port, a Bluetooth interface, a Wi-Fi interface, or any combination thereof, as well as other types of communication interfaces.

[0023] The decoding system can decode the encoded image to obtain a copy of the raw image (200). The decoding system may be included, for example, in the camera (100) or an external device, or both. The decoding system is not illustrated in FIG. 1 for convenience and simplification. In general, the decoding system can perform operations opposite to those performed by the encoding system (106), as described above and later in more detail.

[0024] A copy of the raw image (200) obtained by decoding the encoded image may undergo various image processing operations to obtain an image suitable for display, for example. These image processing operations may include, for example, debayering, gamma correction, white balancing, and noise removal. The term 'debayering' refers to the operation of effectively removing the Bayer pattern and replacing it with a pattern suitable for a display device, such as, for example, an existing RGB pattern. The advantage of encoding the raw image (200) as described above, rather than encoding a debayered version of the raw image (200), is that higher image quality can be achieved with less information loss.

[0025] Fig. 3 Figure 102 schematically illustrates an image processor (102) of a camera (100). Figure 3 provides a block diagram of the image processor (102). The image processor (102) includes an inverse correlation module (301), a quantum wavelet filtering module (302), and a two-dimensional wavelet filtering module (303). These modules may be implemented, for example, in the form of a dedicated circuit, a programmable circuit, or a suitably programmed processor, or any combination thereof.

[0026] The image processor (102) basically operates as follows. The inverse correlation module (301) performs various pixel replacement operations to obtain an inversely correlated image (304) based on the raw image (200). More specifically, the inverse correlation module (301) replaces the green pixels (G) of the raw image (200) with the luminance component (Y). The inverse correlation module (301) additionally replaces the red pixels (R) and blue pixels (B) of the raw image (200) with the red-based chromaticity component (Cr) and the blue-based chromaticity component (Cb), respectively.

[0027] Fig. 4 Figure 3 schematically illustrates the inversely correlated image (304) provided by the inverse correlation module (301). Figure 3 provides a schematic diagram of the inversely correlated image (304). The luminance component (Y), the red-based chromaticity component (Cr), and the blue-based chromaticity component (Cb) are also arranged in a Bayer pattern. There is a luminance component (Y) that is twice the red-based chromaticity component (Cr) and the blue-based chromaticity component (Cb). As can be seen, the inversely correlated image (304) is a rectangular array of 2 x 2 block components containing the luminance component (Y), the red-based chromaticity component (Cr), and the blue-based chromaticity component (Cb) arranged along two diagonals.

[0028] More specifically, the luminance component (Y) in the inversely correlated image (304) constituting the replacement of the green pixel (G) in the original image (200) is a weighted combination of the relevant green pixel (G) and neighboring pixels in the original image (200). In the weighted combination, the neighboring red and blue pixels (R, B) have a total positive (+) weighting factor. This corresponds to adding the neighboring red and blue pixels (R, B) to the green pixel (G) being replaced. Conversely, the neighboring green pixel (G) may have a total negative (-) weighting factor. This corresponds to subtracting the neighboring green pixel (G) from the green pixel (G) being replaced.

[0029] For example, a weighted combination providing the luminance component (Y) may be as follows:

[0030] Y(i,j) = +7 / 8*G(ij) - 1 / 16*G(i-1, j-1) - 1 / 16* G(i+1, j-1) - 1 / 16* G(i-1, j+1) - 1 / 16* G(i+1, j+1) - 1 / 32*G(i-2,j) - 1 / 32*G(i,j-2) - 1 / 32*G(i+2,j) - 1 / 32*G(i,j+2) + 1 / 8*B(i,j-1) + 1 / 8*B(i,j+1) +1 / 8*R(i-1,j) + 1 / 8*R(i+1, j).

[0031] In this equation, the two variables between the comma-separated parentheses represent the positions of the relevant pixels in the raw image (200) shown in FIG. 2, whereby i and j represent the positions in the horizontal and vertical directions of the luminance component (Y)(i,j) formed and the green pixel (G)(i,j) replaced by it, respectively.

[0032] In this embodiment, four immediately adjacent blue and red pixels B(i,j-1), B(i,j+1), R(i-1,j), and R(i+1,j) are all added to the green pixel (G)(i,j) being replaced. The four closest green pixels G(i-1,j-1), G(i+1,j-1), G(i-1,j+1), and G(i+1,j+1) are all subtracted from the green pixel G(i,j) being replaced. Additionally, four additional adjacent green pixels G(i-2,j), G(i,j-2), G(i+2,j), and G(i,j+2) are also subtracted from the green pixel G(i,j) being replaced. The four nearest neighboring green pixels G(i-1, j-1), G(i+1, j-1), G(i-1, j+1), and G(i+1, j+1) are in an x-configuration with respect to the green pixel G(ij) being replaced. The four additional neighboring green pixels G(i-2,j), G(i,j-2), G(i+2,j), and G(i,j+2) are in an x-configuration with respect to the green pixel G(ij) being replaced. The four nearest neighboring green pixels G(i-1, j-1), G(i+1, j-1), G(i-1, j+1), G(i+1, j+1) have a higher weighting factor, i.e. -1 / 16, than the four additional neighboring green pixels G(i-2,j), G(i,j-2), G(i+2,j), G(i,j+2) which have a weighting factor of -1 / 32.

[0033] The red-based chromaticity component (Cr) of the inversely correlated image (304) constituting the replacement of the red pixel (R) of the original image (200) is a weighted combination of the pixels of the relevant red pixel (R) and neighboring pixels. In this weighted combination, the neighboring green pixel (G) has a total negative weighting factor. This corresponds to the total subtraction of the neighboring green pixel (G) from the red pixel (R) being replaced.

[0034] Weighted combinations providing a red-based chromaticity component (Cr) may be as follows.

[0035] Cr(i,j) = R(i,j) - 1 / 4*G(i-1,j) - 1 / 4*G(i,j-1) - 1 / 4*G(i+1,j) - 1 / 4*G(i,j+1).

[0036] In this equation as well, the two variables between the comma-separated parentheses represent the positions of the relevant pixels in the raw image (200) shown in FIG. 2, whereby i and j represent the horizontal and vertical positions of the red-based chromaticity component Cr(i,j) formed and the red pixel R(i,j) replaced by it, respectively. In this embodiment, four immediately adjacent green pixels (G)(i-1,j), G(i,j-1), G(i+1,j), and G(i,j+1) are all subtracted from the red pixel R(i,j) replaced.

[0037] Likewise, the blue-based chromaticity component (Cb) of the inversely correlated image (304) constituting the replacement of the blue pixel (B) of the original image (200) is a weighted combination of the relevant blue pixel (B) and neighboring pixels. In the weighted combination, the neighboring green pixels (G) have a total negative weighting factor. This corresponds to subtracting the neighboring green pixels (G) from the blue pixel (B) being replaced.

[0038] Weighted combinations providing a blue-based chromaticity component (Cb) may be as follows:

[0039] Cb(i,j) = B(i,j) - 1 / 4*G(i-1,j) - 1 / 4*G(i,j-1) - 1 / 4*G(i+1,j) - 1 / 4*G(i,j+1).

[0040] In this equation as well, the two variables between the comma-separated parentheses represent the positions of the relevant pixels in the raw image (200) shown in FIG. 2, whereby i and j represent the horizontal and vertical positions of the blue-based chromaticity component Cb(i,j) formed and the blue pixel B(i,j) replaced by it, respectively. In this embodiment, the four immediately adjacent green pixels G(i-1,j), G(i,j-1), G(i+1,j), and G(i,j+1) are all subtracted from the blue pixel B(i,j) replaced, as in the weighted combination providing the previously presented red-based chromaticity component Cr(i,j).

[0041] Fig. 5 FIG. 3 schematically illustrates an embodiment of an inverse correlation module (301) that forms part of the image processor (102) illustrated in FIG. 3. FIG. 5 provides a block diagram according to the present embodiment of an inverse correlation module (301), which will be simply referred to as the inverse correlation module (301) below. The inverse correlation module (301) includes an input buffer memory (501), a chromaticity component forming module (502), an intermediate buffer memory (503), and a luminance component forming module (504).

[0042] The inverse correlation module (301) basically operates as follows. The input buffer memory (501) temporarily stores at least a portion of the raw image (200) shown in FIG. 2. The chromaticity component forming module (502) retrieves the red pixel (R) of the raw image (200) as well as four immediately adjacent green pixels (G) from the input buffer memory (501). Then, the chromaticity component forming module (502) forms a red-based chromaticity component (Cr) by creating a weighted combination between the red pixel (R) and the four immediately adjacent green pixels (G) as previously mentioned. In this way, the chromaticity component forming module (502) forms each red-based chromaticity component (Cr) for the inversely correlated image (304) that constitutes a replacement for the red pixel (R) in the raw image (200).

[0043] The chromaticity component forming module (502) forms a blue-based chromaticity component (Cb) in a similar manner. That is, the chromaticity component forming module (502) retrieves the blue pixel (B) of the raw image (200) as well as the four immediately adjacent green pixels (G) from the input buffer memory (501). Then, the chromaticity component forming module (502) forms the blue-based chromaticity component (Cb) by creating a weighted combination between the blue pixel (B) and the four immediately adjacent green pixels (G), as previously mentioned.

[0044] Fig. 6 As described above, it illustrates the formation of a red-based chromaticity component (Cr) and a blue-based chromaticity component (Cb). This can be considered as a first inverse correlation operation. FIG. 6 provides a conceptual diagram of the first inverse correlation operation forming chromaticity components (Cr, Cb) for an inversely correlated image (304). In FIG. 6, a portion of the raw image (200) is shown in the left half of this figure, while a portion of the inversely correlated image (304) is shown in the right half of this same figure. In the raw image (200), a red pixel (R) and four immediately adjacent green pixels (G), as well as a blue pixel (B) and four immediately adjacent green pixels (G), are shown. Additionally, a red-based chromaticity component (Cr) constituting the replacement of the aforementioned red pixel (R) in the inversely correlated image (304) is displayed, along with a blue-based chromaticity component (Cb) constituting the replacement of the aforementioned blue pixel (B) in the inversely correlated image (304). Additionally, weighting factors are displayed, which correspond to those previously mentioned.

[0045] Fig. 7Figure 7 schematically illustrates an intermediate inversely correlated image (700) provided by a chromaticity component forming module (502). Figure 7 provides a schematic diagram of the intermediate inversely correlated image (700). The intermediate inversely correlated image (700) corresponds to a modified version of the original image (200) in which each red and blue pixel (B) is replaced with each red-based chromaticity component (Cr) and each blue-based chromaticity component (Cb). The green pixels (G) of the original image (200) are present as they have not yet been replaced. At least a portion of the intermediate inversely correlated image (700) is temporarily stored in the intermediate buffer memory (503) of the inversely correlated module (301) illustrated in Figure 5.

[0046] Referring again to FIG. 5, the luminance component forming module (504) retrieves the green pixel (G) of the intermediate inversely correlated image (700) from the intermediate buffer memory (503), as well as four immediately adjacent color pixels (Cr, Cb), namely two red-based color pixels (Cr) and two blue-based chromaticity pixels (Cb). Then, the luminance component forming module (504) forms a luminance component (Y) by creating a weighted combination of the green pixel (G) and the four immediately adjacent chromaticity pixels (Cr, Cb). In this way, the luminance component forming module (504) forms each luminance component (Y) for the inversely correlated image (304) that constitutes a replacement for the green pixel (G) in the raw image (200). Accordingly, the luminance component forming module (504) provides the inversely correlated image (304) shown in FIG. 4 based on the intermediate inversely correlated image (700) shown in FIG. 6, which is temporarily stored in the intermediate buffer memory (503) of the inversely correlated module (301) shown in FIG. 5.

[0047] Fig. 8Figure 8 illustrates the formation of the luminance component (Y) as described above. This can be considered as a second inverse correlation operation. Figure 8 provides a conceptual diagram of the second inverse correlation operation forming the luminance component (Y) for the inversely correlated image (304). In Figure 8, a portion of the intermediate inversely correlated image (700) is shown in the left half of this figure, while a portion of the inversely correlated image (304) is shown in the right half of this same figure. In the intermediate inversely correlated image (700), the green pixel (G) and four immediately adjacent chromaticity components (Cb, Cr) are shown. Additionally, the luminance component (Y) of the inversely correlated image (304) is shown, which constitutes a replacement for the aforementioned green pixel (G). Additionally, weighting factors are shown. When these are combined with the weighting factors shown in Figure 6, weighting factors corresponding to those previously mentioned are obtained to form the luminance component (Y).

[0048] The inverse correlation module (301) illustrated in FIG. 5 can operate on integers and rounding from the result of applying weighting factors to the pixels. This enables a relatively simple and low-cost implementation. There are various methods for rounding numbers. For example, the formation of the red-based chromaticity component (Cr) and the blue-based chromaticity component (Cb) can be adapted in the following way:

[0049] Cb(i,j) = B(i,j) - ROUNDDOWN(1 / 4*G(i-1,j) + 1 / 4*G(i,j-1) + 1 / 4*G(i+1,j) + 1 / 4*G(i,j+1)); and

[0050] Cr(i,j) = R(i,j) - ROUNDDOWN(1 / 4*G(i-1,j) + 1 / 4*G(i,j-1) + 1 / 4*G(i+1,j) + 1 / 4*G(i,j+1)).

[0051] Similarly, the formation of the luminance component (Y) can be adapted in the following way:

[0052] Y(i,j) = G(i,j) + ROUNDDOWN(1 / 8*Cr(i-1,j) + 1 / 8*Cb(i,j-1) + 1 / 8*Cr(i+1,j) + 1 / 8*Cb(i,j+1)).

[0053] In the decoder, the module corresponding to the inverse correlation module (301) can be adapted to accurately invert the aforementioned equation by performing one of the following:

[0054] G(i,j) = Y(i,j) - ROUNDDOWN(1 / 8*Cr(i-1,j) + 1 / 8*Cb(i,j-1) + 1 / 8*Cr(i+1,j) + 1 / 8*Cb(i,j+1));

[0055] R(i,j) = Cr(i,j) + ROUNDDOWN(1 / 4*G(i-1,j) + 1 / 4*G(i,j-1) + 1 / 4*G(i+1,j) + 1 / 4*G(i,j+1)); and

[0056] B(i,j) = Cb(i,j) + ROUNDDOWN(1 / 4*G(i-1,j) + 1 / 4*G(i,j-1) + 1 / 4*G(i+1,j) + 1 / 4*G(i,j+1)).

[0057] In these expressions, ROUNDDOWN represents an operator that rounds a number down to the nearest integer.

[0058] Referring again to the image processor (102) illustrated in FIG. 3, the convolution wavelet filtering module (302) applies a wavelet filtering operation to the luminance components (Y) of the inversely correlated image (304) illustrated in FIG. 4. In fact, the convolution wavelet filtering module (302) decomposes the luminance components (Y) into a first array set of luminance components (Y) and a second array set of luminance components (Y). The array of luminance components (Y) belonging to the first set is interleaved with the array of luminance components (Y) of the second set. For example, the first array set of luminance components (Y) may correspond to an even line of the inversely correlated image (304) illustrated in FIG. 4, while the second array set of luminance components (Y) may correspond to an uneven line of the inversely correlated image (304), and vice versa.

[0059] The convolution wavelet filtering module (302) is a high-frequency luminance component (Y) for a luminance component (Y) belonging to a first array set. H ) can be provided. In that case, the convolution wavelet filtering module (302) provides a low-frequency luminance component (Y) for a luminance component (Y) belonging to the second array set. L It can provide ). In fact, high-frequency luminance component (Y H ) replaces the luminance component (Y) belonging to the first array set. Low luminance component (Y L ) replaces the luminance component (Y) belonging to the second array set.

[0060] Fig. 9 Figure 9 schematically illustrates a partially wavelet-filtered inversely correlated image (900) provided by a convolution wavelet filtering module (302). Figure 9 provides a schematic diagram of the partially wavelet-filtered inversely correlated image (900). In this example, the even lines of the partially wavelet-filtered inversely correlated image (900) are high-frequency luminance components (Y H While it includes ), uneven lines include low-frequency luminance components (YL ...includes ). Accordingly, in a partially wavelet-filtered inversely correlated image (900), each different type of component is organized into dyadic arrays rather than mosaic arrays. This feature may be relevant if, for example, the encoder (103) of the camera (100) illustrated in FIG. 1 is specifically designed to process components organized into dyadic arrays. This may be, for example, if the encoder (103) operates in the manner described in the aforementioned U.S. Patent No. 9,332,258.

[0061] Fig. 10 FIG. 3 illustrates an embodiment of a quantum wavelet filtering module (302) that forms part of the image processor (102) illustrated in FIG. 3. FIG. 10 provides a block diagram according to this embodiment of a quantum wavelet filtering module (302), which will hereinafter simply be referred to as the quantum wavelet filtering module (302). The quantum wavelet filtering module (302) includes an input buffer memory (1001), a high-frequency luminance component forming module (1002), an intermediate buffer memory (1003), and a low-frequency luminance component forming module (1004).

[0062] The convolution wavelet filtering module (302) basically operates as follows. The input buffer memory (1001) temporarily stores at least a portion of the inversely correlated image (304) shown in FIG. 4. The high-frequency luminance component forming module (1002) searches the input buffer memory (1001) for four immediately adjacent luminance components (Y) that are in a convolutional configuration with respect to the luminance component (Y) from the inversely correlated image (304) as well as the luminance component (Y) mentioned earlier (hereinafter referred to as the central luminance component (Y) for convenience). The central luminance component (Y) may belong to a first array set, while the immediately adjacent luminance components (Y) may belong to a second array set. For example, the central luminance component (Y) may exist on an even line in the inversely correlated image (304), while two of the four immediately adjacent luminance components (Y) may exist on an uneven line immediately above the aforementioned even line, and the other two of the four immediately adjacent luminance components (Y) may exist on an uneven line immediately below the aforementioned even line.

[0063] The high-frequency luminance component forming module (1002) creates a weighted combination between the central luminance component (Y) on one side and the four immediately adjacent luminance components (Y) on the other side, thereby forming a high-frequency luminance component (Y H It forms ). Weighted combinations can be expressed as follows:

[0064] Y H (i,j) = Y(i,j) - 1 / 4*Y(i-1,j-1) - 1 / 4*Y(i+1,j-1) - 1 / 4*Y(i+1,j+1) - 1 / 4*Y(i-1,j+1).

[0065] Here, as previously mentioned, the two variables between the comma-separated parentheses represent the positions of the luminance components associated with the inversely correlated image (304) shown in FIG. 4, whereby i and j are respectively the formed high-frequency luminance components Y HIt indicates the horizontal and vertical positions of (i,j) and the luminance component Y(i,j) replaced by it.

[0066] Fig. 11 is a high-frequency luminance component (Y) that can be considered as the first partial wavelet filtering operation. H Figure 11 illustrates the formation of a high-frequency luminance component (Y) for a partially wavelet-filtered inversely correlated image (900). H A conceptual diagram of a first partial wavelet filtering operation forming ) is provided. In FIG. 11, a portion of the inversely correlated image (304) is represented in the left half of this figure, while a portion of the partially wavelet-filtered inversely correlated image (900) is represented in the right half of this same figure. The luminance components (Y) belonging to the first array set of luminance components (Y) and the luminance components (Y) belonging to the second array set are indicated by different shades in the inversely correlated image (304). In this example, the array of luminance components (Y) corresponds to the lines of the inversely correlated image (304). The luminance components (Y) belonging to the first array set of the inversely correlated image (304) and four immediately adjacent luminance components (Y) belonging to the second array set are shown in FIG. 11. Additionally, the high-frequency luminance components (Y) of the partially wavelet-filtered inversely correlated image (900) H ) is displayed, which constitutes a replacement for the luminance component mentioned earlier. Additionally, weighting factors corresponding to those previously displayed are displayed.

[0067] Fig. 12 Figure 12 schematically illustrates an intermediate partially wavelet-filtered inversely correlated image (1200) provided by a high-frequency luminance component forming module (1002). Figure 12 provides a schematic diagram of an intermediate partially wavelet-filtered inversely correlated image (1200). In this image (1200), each luminance component (Y) belonging to a first set of arrays is each high-frequency luminance component (Y HIt corresponds to a modified version of the inversely correlated image (304) that has been replaced with ). The luminance component (Y) of the inversely correlated image (304) belonging to the second array set has not yet been replaced and therefore exists. At least a portion of the intermediate partially wavelet-filtered inversely correlated image (1200) is temporarily stored in the intermediate buffer memory (1003) of the convolution wavelet filtering module (302) shown in FIG. 10.

[0068] Referring again to FIG. 10, the low-frequency luminance component forming module (1004) obtains from the intermediate buffer memory (1003) the luminance component (Y) of the intermediate partially wavelet-filtered inverse-correlated image (1200) as well as four immediately adjacent high-frequency luminance components (Y) in a sequence configuration. H Searches for ). In this example, the luminance component (Y) belongs to the second array set, while the immediately adjacent high-frequency luminance component (Y H ) replaces the luminance component (Y) belonging to the first array set. For example, the luminance component (Y) may exist on an uneven line in the inversely correlated image (304) and thus may exist in the intermediate partially wavelet-filtered version (1200), while the four immediately adjacent high-frequency luminance components (Y H Two of the four luminance components (Y) may exist on the even line immediately above the aforementioned uneven line, and the remaining two of the four adjacent luminance components may exist on the even line immediately below the aforementioned uneven line.

[0069] The low-frequency luminance component forming module (1004) has, on one hand, a luminance component (Y) and on the other hand, four immediately adjacent high-frequency luminance components (Y H By creating a weighted combination of ), the low-frequency luminance component (Y L It forms ). Weighted combinations can be expressed as follows.

[0070] Y L (i,j) = Y(i,j) + 1 / 8*Y H(i-1,j-1) + 1 / 8*Y H (i+1,j-1) + 1 / 8*Y H (i+1,j+1) + 1 / 8*Y H (i-1,j+1).

[0071] Here, as previously mentioned, the two variables between the comma-separated parentheses represent the positions of luminance components related to the intermediate partially wavelet-filtered inversely correlated image (1200) shown in FIG. 12, whereby i and j are respectively the low-frequency luminance components Y that are formed. L It indicates the horizontal and vertical positions of (i,j) and the luminance component Y(i,j) replaced by it.

[0072] Fig. 13 is a low-frequency luminance component (Y) that can be considered as a second partial wavelet filtering operation L ) exemplifies the formation of. FIG. 13 shows a low-frequency luminance component (Y) for a partially wavelet-filtered inversely correlated image (900). L A conceptual diagram of a second partial wavelet filtering operation forming ) is provided. In FIG. 13, a portion of the intermediate partially wavelet filtered inversely correlated image (1200) is shown in the left half of this figure, while a portion of the partially wavelet filtered inversely correlated image (900) is shown in the right half of this same figure.

[0073] The luminance component (Y) of the intermediate partially wavelet-filtered inversely correlated image (1200), as well as four immediately adjacent high-frequency luminance components (Y H ) is shown in FIG. 13. Also, in the partially wavelet-filtered inversely correlated image (900), the low-frequency luminance component (Y L ) is displayed, which constitutes a replacement for the aforementioned luminance component, and also a weighting factor corresponding to that previously shown is displayed.

[0074] As with the inverse correlation module (301) illustrated in FIG. 5, the convolution wavelet filtering module (302) illustrated in FIG. 10 can operate as an integer and round as a result of applying weighting factors to the components. This also enables a relatively simple and inexpensive implementation. For example, there are various methods of rounding such numbers, such as rounding to the nearest integer as previously presented as an example.

[0075] Referring again to the image processor (102) illustrated in FIG. 3, the two-dimensional wavelet filtering module (303) filters the red-based chromaticity component (Cr), the blue-based chromaticity component (Cb), and the low-frequency luminance component (Y) in the partially wavelet-filtered inversely correlated image (900) illustrated in FIG. 9. L 2D wavelet filtering is applied individually to ). That is, in this image, the red-based chromaticity component (Cr) effectively constitutes a sub-image undergoing 2D wavelet filtering. Similarly, the blue-based chromaticity component (Cb) effectively constitutes another sub-image undergoing this filtering, and the low-frequency luminance component (Y L ) constitutes another sub-image that undergoes 2D wavelet filtering.

[0076] Two-dimensional wavelet filtering may include two one-dimensional wavelet filtering operations performed sequentially. One of the two one-dimensional wavelet filtering operations may be performed in a horizontal direction, while the other one-dimensional wavelet filtering operation may be performed in a vertical direction. For example, sub-image components may first be applied to a one-dimensional wavelet filter in a line manner. This first wavelet filtering operation then generates a horizontal high-frequency subband and a horizontal low-frequency subband. Subsequently, components of the horizontal high-frequency subband may be applied to a one-dimensional wavelet filter in a column manner to generate a horizontal high-frequency / vertical high-frequency subband and a horizontal high-frequency / vertical low-frequency subband. Components of the horizontal low-frequency subband may be applied to a one-dimensional wavelet filter to generate a horizontal low-frequency / vertical high-frequency subband and a horizontal low-frequency / vertical low-frequency subband.

[0077] Fig. 14 illustrates a set of subbands provided by a two-dimensional wavelet filtering module (303). FIG. 14 shows the horizontal frequency (F H The horizontal axis representing ) and the frequency (F) in the vertical direction V It is a two-dimensional frequency plot having a vertical axis representing ). In this example, there are the previously mentioned four subbands: the horizontal high frequency / vertical high frequency subband designated by reference number (1401), the horizontal high frequency / vertical low frequency subband designated by reference number (1402), the horizontal low frequency / vertical high frequency subband designated by reference number (1403), and the horizontal low frequency / vertical low frequency subband. These four subbands are the result of two-dimensional wavelet filtering of the sub-image as described above, thereby forming the sub-image by components of the same type in the partially wavelet-filtered inversely correlated image (900). Thus, the four subbands shown in FIG. 4 are the red-based chromaticity component (Cr), the blue-based chromaticity component (Cb), or the low-frequency luminance component (Y LIt can be generated by ).

[0078] Fig. 15 Figure 15 illustrates a wavelet filtering of the Le Gall 5 / 3 type. Figure 15 provides a conceptual diagram of this wavelet filtering. The wavelet filtering of the Le Gall 5 / 3 type, described later with reference to Figure 15, can correspond to the aforementioned one-dimensional wavelet filtering operation performed by the two-dimensional wavelet filtering module (303). Furthermore, the wavelet filtering of the Le Gall 5 / 3 type is conceptually related not only to the first and second inverse correlation operations performed by the inverse correlation module (301), but also to the first and second wavelet filtering operations performed by the convolution wavelet filtering module (302). This will be explained in more detail below.

[0079] Le Gall 5 / 3 type wavelet filtering is applied to a series of input samples represented by a series of relatively small circles in the upper part of FIG. 15. Some of these small circles have white fill, while other parts of these small circles have black fill. These are referred to hereinafter as the white input sample and the black input sample, respectively. The white input sample is interleaved with the black input sample as shown in FIG. 15. Le Gall 5 / 3 type wavelet filtering provides a high-frequency output sample for the black input sample and a low-frequency output sample for the white input sample. The high-frequency output sample is shown in the middle part of FIG. 15, while the low-frequency output sample is shown in the lower part of this figure.

[0080] A high-frequency output sample for a black input sample is formed by creating a weighted combination of a black input sample on one side and two immediately adjacent white input samples on the other. In this weighted combination, the black input sample has a weighting factor of +1, and the two immediately adjacent white input samples have a weighting factor of -0.5, as shown in FIG. 15. Thus, the weighted combination represents the difference between the black input sample and the two immediately adjacent white input samples. The high-frequency output sample resulting from this weighted combination effectively replaces the black input sample from which it is formed.

[0081] A low-frequency output sample for a white input sample is formed by creating a weighted combination of a white input sample and two immediately adjacent high-frequency output samples. In the weighted combination, these two high-frequency output samples effectively replace the positions of the two black input samples immediately adjacent to the associated white input sample, as shown in FIG. 15. In the weighted combination, as shown in FIG. 15, the white input sample has a weighting factor of +1, and its immediately adjacent high-frequency output samples have a weighting factor of +0.25. Thus, the weighted combination is the weighted sum of the white input sample and the two immediately adjacent high-frequency output samples. The low-frequency output sample resulting from the weighted combination effectively replaces the white input sample from which it is formed.

[0082] As previously mentioned, the first and second inverse correlation operations performed by the inverse correlation module (301) in the image processor (102) to form an inversely correlated image (304) are conceptually related to the Le Gall 5 / 3 type wavelet filtering discussed earlier. More specifically, the formation of the red-based chromaticity component (Cr) and the blue-based chromaticity component (Cb) illustrated in FIG. 6 can be conceptually considered as a two-dimensional extension of the formation of high-frequency output samples in the Le Gall 5 / 3 type wavelet filtering illustrated in FIG. 15, which has one-dimensional characteristics. Instead of taking two immediately adjacent input samples in one dimension to form high-frequency output samples as illustrated in FIG. 15, the formation of the chromaticity components involves taking four immediately adjacent pixels in two dimensions as illustrated in FIG. 6. The sum of the weighting factors in the weighted combination providing the high-frequency output samples is equal to the sum of the weighting factors in the weighted combination providing the chromaticity components. The first mentioned weighted combination has 2 neighbors with a weight factor of -0.5 each, and the last mentioned weighted combination has 4 neighbors with a weight factor of -0.25 each.

[0083] Similarly, the formation of the luminance component (Y) illustrated in FIG. 8 can be conceptually considered as a two-dimensional extension of the formation of low-frequency output samples in the Le Gall 5 / 3 type wavelet filtering illustrated in FIG. 15, which has one-dimensional characteristics. Instead of taking two immediately adjacent high-frequency output samples in one dimension to form the low-frequency output samples as illustrated in FIG. 15, the formation of the luminance component (Y) involves taking four immediately adjacent chromaticity pixels in two dimensions as illustrated in FIG. 8. Again, the sum of the weighting factors in the weighted combination providing the low-frequency output samples is equal to the sum of the weighting factors in the weighted combination providing the luminance component. The first mentioned weighted combination has two neighbors with a weighting factor of +0.25 each, and the last mentioned weighted combination has four neighbors with a weighting factor of +0.125 each.

[0084] As previously mentioned, the first and second wavelet filtering operations performed by the Conquer wavelet filtering module (302) in the image processor (102) to form a partially wavelet-filtered inversely correlated image (900) are conceptually related to the aforementioned Le Gall 5 / 3 type wavelet filtering. More specifically, the high-frequency luminance component (Y shown in FIG. 11) H The formation of ) can be conceptually regarded as a two-dimensional extension of high-frequency output sample formation in the Le Gall 5 / 3 type wavelet filtering illustrated in FIG. 15, which has one-dimensional characteristics. Instead of taking two immediately adjacent input samples in one dimension to form the high-frequency output sample as shown in FIG. 15, the high-frequency luminance component (Y HThe formation of ) involves taking four neighboring luminance pixels in two dimensions as shown in FIG. 11. The sum of the weighting factors in the weighted combination providing high-frequency output samples is equal to the sum of the weighting factors in the weighted combination providing high-frequency luminance components. The first mentioned weighted combination has two neighbors with a weighting factor of -0.5 each, and the last mentioned weighted combination has four neighbors with a weighting factor of -0.25 each.

[0085] Similarly, the low-frequency luminance component (Y shown in FIG. 13) L The formation of ) can be conceptually regarded as a two-dimensional extension of the low-frequency output sample formation in the Le Gall 5 / 3 type wavelet filtering illustrated in FIG. 15, which has one-dimensional characteristics. Instead of taking two immediately adjacent high-frequency output samples in one dimension to form the low-frequency output sample as illustrated in FIG. 15, the formation of the low-frequency luminance component (YL) is achieved by taking four adjacent high-frequency luminance components (Y) as illustrated in FIG. 8. H It includes taking ) as a two-dimensional form. Again, the sum of the weighting factors in the weighted combinations providing low-frequency output samples is equal to the sum of the weighting factors in the weighted combinations providing low-frequency luminance components. The first mentioned weighted combination has two neighbors with a weighting factor of +0.25 each, and the last mentioned weighted combination has four neighbors with a weighting factor of +0.125 each.

[0086] In the camera (100) illustrated in FIG. 1, the image processor (102) thus processes the raw image (200) illustrated in FIG. 2 to obtain a partially wavelet-filtered inversely correlated image (900) illustrated in FIG. 10, which is four sub-images, namely high-frequency luminance components (Y H Sub-image including ), low-frequency luminance component (Y LIt can be considered as a combination of a sub-image containing ), a sub-image containing a red-based chromaticity component (Cr), and a sub-image containing a blue-based chromaticity component (Cb). Additionally, the image processor (102) provides a set of four sub-bands for each of the last three sub-images, as shown in FIG. 14. Then, the encoder (103) provides the following data, namely the high-frequency luminance component (Y), which can represent the raw image (200) shown in FIG. 2 without information loss. H ), low-frequency luminance component (Y L It is possible to efficiently encode four subbands representing ), four subbands representing red-based chromaticity components (Cr), and four subbands representing blue-based chromaticity components (Cb).

[0087] Notes

[0088] The embodiments described above with reference to the drawings are presented as examples. The present invention can be implemented in numerous different ways. To illustrate this, some alternatives are briefly mentioned.

[0089] The present invention may be applied to various types of products or methods related to image processing, video processing, or both. The presented embodiments provide examples of the present invention applied to image and video compression. However, the present invention may be applied to other types of image and video processing, such as, for example, providing multi-resolution views of an image for segmented storage and / or transmission of the image, image analysis for detecting objects / attributes in an image and achieving segmentation, and analysis filters for debayering algorithms.

[0090] There are various methods for implementing an image processor according to the present invention. All modules of the presented embodiments may be implemented by dedicated or programmable electrical circuits, or by a suitably programmed processor, or by a combination thereof. A computer program may define one or more operations performed by the image processor described with reference to FIGS. 1 through 15. In this regard, each of the block diagrams may also be considered, at least partially, as representing a flowchart of such a computer program, as well as a method that the processor may perform when executing the computer program. For example, the inverse correlation module shown in FIG. 5 may be considered to represent an inverse correlation step. Similarly, other modules may be considered to represent steps of the method.

[0091] There are numerous different methods for implementing the inverse correlation pixel-component transformation scheme in the product or method according to the present invention. In the presented embodiments, a pixel-component transformation scheme conceptually related to the Le Gall 5 / 3 type wavelet filtering as described above is applied. More specifically, in this conceptual relationship, the pixels of the second and third types, which are red and blue pixels in the presented embodiments, can be considered as samples provided with high-frequency components using a two-dimensional filter kernel that is essentially an extension of the one-dimensional kernel of the related wavelet filtering, thereby applying a modification of the weighting factor. The pixels of the first type, which are green pixels in the presented embodiments, can be considered as samples provided with low-frequency components using a two-dimensional filter kernel that is essentially an extension of the one-dimensional kernel of the related wavelet filtering, thereby applying a modification of the weighting factor.

[0092] In other embodiments, the pixel-component transformation method may be based on different types of wavelet filtering, such as, for example, Daubechies 9 / 7 type wavelet filtering or 13 / 7 based wavelet filtering. A pixel-component transformation method based on one of the aforementioned types of wavelet filtering may be more complex to implement than the pixel-component transformation method in the presented embodiments. For example, a pixel-component transformation method based on Daubechies 9 / 7 type wavelet filtering may include more partial inverse correlation operations than the first and second inverse correlation operations described above with reference to FIGS. 5 through 8. That is, an inverse correlation module based on Daubechies 9 / 7 type wavelet filtering may require more functional blocks than the inverse correlation module (301) shown in FIG. 5.

[0093] A pixel-component transformation method based on a type of wavelet filtering more complex than Le Gall 5 / 3 type wavelet filtering provides a more complex weighting combination that defines the component replacing the pixel. This is because more neighboring pixels are considered to form the component replacing the pixel. Nevertheless, the general rules regarding weighting combinations defined in the foregoing and appended claims will still apply. That is, in a weighting combination providing a component of the first type, neighboring pixels of the second and third types have a total positive weighting factor corresponding to the total addition of the neighboring pixels of the second and third types to the pixel of the first type being replaced. Conversely, in a weighting combination providing a component of the second or third type, neighboring pixels of the first type each have a total negative weighting factor corresponding to the total subtraction of the neighboring pixels of the first type from the pixel of the second or third type being replaced.

[0094] It should also be noted that there are numerous other ways to implement inverse-correlated pixel-component transform schemes conceptually related to specific types of wavelet filtering, such as, for example, Le Gall 5 / 3 type wavelet filtering. For example, the presented embodiments describe an implementation that can be simplified as follows. The weighted combination providing the red-based chromaticity component (Cr) is simplified by considering only pixels in two consecutive lines. If the red pixel R(i,j) is in the lower of two consecutive lines, the weighted combination can be expressed as follows:

[0095] Cr(i,j) = R(i,j) - 1 / 4*G(i-1,j) - 1 / 2*G(i,j-1) - 1 / 4*G(i+1,j)

[0096] Referring to FIG. 6, in this case, the green pixel (G) located immediately below the red pixel (R) that has been replaced by a red-based chromaticity component (Cr) is omitted. This omission is compensated by doubling the weighting factor for the green pixel (G) located immediately above the red pixel (R). Thus, it becomes 0.5 instead of 0.25. Since this is equivalent to horizontally mirroring the aforementioned green pixel (G), this same pixel is also located immediately below the red pixel (R), thereby replacing the omitted green pixel (G).

[0097] If the red pixel R(i,j) is above two consecutive lines, the weighted combination can be expressed as follows:

[0098] Cr(i,j) = R(i,j) - 1 / 4*G(i-1,j) - 1 / 4*G(i+1,j) - 1 / 2*G(i,j+1)

[0099] Referring to Fig. 6, in this case, the green pixel (G) immediately above the red pixel (R) replaced by the red-based chromaticity component (Cr) is omitted. This omission is compensated by doubling the weighting factor for the green pixel (G) immediately below the red pixel (R). Thus, it becomes 0.5 instead of 0.25.

[0100] In the same way, the weighted combination providing the blue-based chromaticity component (Cb) is simplified by considering only pixels in two consecutive lines. If the blue pixel B(i,j) is in the lower of two consecutive lines, the weighted combination can be expressed as follows:

[0101] Cb(i,j) = B(i,j) - 1 / 4*G(i-1,j) - 1 / 2*G(i,j-1) - 1 / 4*G(i+1,j).

[0102] Referring to FIG. 6, in that case, the green pixel (G) immediately below the blue pixel (B) replaced by the blue-based chromaticity component (Cb) is omitted. This omission is compensated by doubling the weighting factor for the green pixel (G) immediately above the blue pixel (B). Thus, it becomes 0.5 instead of 0.25.

[0103] If the blue pixel B(i,j) is above two consecutive lines, the weighted combination can be expressed as follows:

[0104] Cb(i,j) = B(i,j) - 1 / 4*G(i-1,j) - 1 / 4*G(i+1,j) - 1 / 2*G(i,j+1).

[0105] Referring to FIG. 6, in this case, the green pixel (G) immediately above the blue pixel (B) replaced by the blue-based chromaticity component (Cb) is omitted. This omission is compensated by doubling the weighting factor for the green pixel (G) immediately below the blue pixel (B), so that it becomes 0.5 instead of 0.25.

[0106] The weighted combination providing the luminance component (Y) can be similarly simplified by considering only the pixels and chromaticity components on two consecutive lines in the intermediate inversely correlated image (700) exemplified in FIG. 7. When the substituted luminance pixel Y(i,j) and green pixel G(i,j) are located below two consecutive lines along with a horizontally adjacent blue-based chromaticity component (Cb) and a vertically adjacent red-based chromaticity component (Cr), the weighted combination can be expressed as follows:

[0107] Y(i,j) = G(i,j) + 1 / 8*Cb(i-1,j) + 1 / 4*Cr(i,j-1) + 1 / 8*Cb(i+1,j).

[0108] In a variation of the previous case having horizontally adjacent red-based chromaticity components (Cr) and vertically adjacent blue-based chromaticity components (Cb), the weighted combination can be expressed as follows:

[0109] Y(i,j) = G(i,j) + 1 / 8*Cr(i-1,j) + 1 / 4*Cb(i,j-1) + 1 / 8*Cr(i+1,j).

[0110] Referring to FIG. 8, in the latter case, the blue-based color component (Cb) located immediately below the green pixel (G), which is replaced by the luminance component (Y), is omitted. This omission is compensated by doubling the weighting factor for the blue-based chromaticity component (Cb) located immediately above the green pixel (G), so that it becomes +0.25 instead of +0.125. This is equivalent to horizontally mirroring the aforementioned blue-based chromaticity component (Cb), so that the same component is located immediately below the green pixel (G) to replace the omitted blue-based chromaticity component (Cb).

[0111] When the replaced luminance pixel Y(i,j) and green pixel G(i,j) are at the top of two consecutive lines along with a horizontally adjacent blue-based chromaticity component (Cb) and a vertically adjacent red-based chromaticity component (Cr), the weighted combination can be expressed as follows:

[0112] Y(i,j) = G(i,j) + 1 / 8*Cb(i-1,j) + 1 / 8*Cb(i+1,j) + 1 / 4*Cr(i,j+1).

[0113] In a variation of the previous case having horizontally adjacent red-based chromaticity components (Cr) and vertically adjacent blue-based chromaticity components (Cb), the weighted combination can be expressed as follows:

[0114] Y(i,j) = G(i,j) + 1 / 8*Cr(i-1,j) + 1 / 8*Cr(i+1,j) + 1 / 4*Cb(i,j+1)

[0115] Referring to Fig. 8, in the latter case, the blue-based chromaticity component (Cb) located immediately above the green pixel (G), which is replaced by the luminance component (Y), is omitted. This omission is compensated by doubling the weighting factor for the blue-based chromaticity component (Cb) located immediately below the green pixel (G), so that it becomes +0.25 instead of +0.125.

[0116] The wavelet filtering operation applied to the luminance component (Y) by the convolution wavelet filtering module (302) illustrated in FIG. 10 can be simplified in a similar manner. That is, the simplified wavelet filtering operation can consider only two consecutive lines of luminance components in the inversely correlated image (304) illustrated in FIG. 4. Referring to FIG. 11, this involves the high-frequency luminance component (Y H This may mean ignoring the luminance component (Y) located on the line immediately below the luminance component (Y) where ) is formed. In this case, the high-frequency luminance component (Y H The weighting factor for the remaining luminance component (Y) immediately above the luminance component (Y) where ) is formed can be doubled so that it becomes -0.5 instead of -0.25. Conversely, simplification is the high-frequency luminance component (Y H This may mean ignoring the luminance component (Y) located on the line immediately above the luminance component (Y) where ) is formed. In that case, the weighting factor for the remaining luminance components (Y) is also doubled.

[0117] The wavelet filtering operation applied by the convolution wavelet filtering module (302) applies only to the luminance component (Y) and high-frequency luminance component (Y) for two consecutive lines in the intermediate partially wavelet-filtered inverse-correlated image (1200) illustrated in FIG. 12. H This can be further simplified by considering the low-frequency luminance component (Y L The high-frequency luminance component (Y) located on the line immediately below the luminance component (Y) where ) is formed H This may mean ignoring the ). In this case, the low-frequency luminance component (Y L The remaining high-frequency luminance component (Y) located immediately above the luminance component (Y) where ) is formed H The weighting factor for ) can be doubled to +0.25 instead of +0.125. Conversely, simplification is the high-frequency luminance component (Y H The high-frequency luminance component (Y) located on the line immediately above the luminance component (Y) where ) is formed H This may mean ignoring ). In that case, the weighting factor for the remaining luminance component (Y) is also doubled.

[0118] The previously presented description illustrates that there are numerous different methods for implementing the inverse correlation pixel-component transformation according to the present invention. The transformation may be applied in a perfectly reversible manner or in a partially reversible manner. For example, in video encodings with relatively low compression ratios, a perfectly reversible transformation may be preferred for high quality. Conversely, in video encodings with relatively high compression ratios, a partially reversible transformation may be acceptable and even preferred.

[0119] In other embodiments, there are numerous different ways of implementing wavelet filtering operations applied to the luminance component (Y) in the product or method according to the present invention. In the presented embodiment, there is a line-based decomposition of the luminance component (Y) into a first array set and a second array set. In other embodiments, such decomposition may be column-based. More generally, it is possible to implement column-based processing instead of line-based processing as shown above and in the presented embodiment.

[0120] In general, there are numerous different ways to implement the present invention, and thus different implementations may have different topologies. In a given topology, a single entity may perform multiple functions, or multiple entities may jointly perform a single function. In this regard, the drawings are highly exemplary.

[0121] The foregoing demonstrates that the embodiments described with reference to the drawings are illustrative of the invention, not limiting it. The invention may be embodied in numerous alternative ways that fall within the scope of the appended claims. Any modifications occurring within the meaning and scope of the claims' equivalences should be encompassed within that scope. Reference numerals in the claims should not be interpreted as limiting the claims. The verb "comprise" in the claims does not exclude the existence of other elements or steps other than those described in the claims. The same applies to similar verbs such as "include" and "contain." A singular reference to a component in a claim relating to a product does not exclude the product from including multiple such elements. Likewise, a singular reference to a step in a claim relating to a method does not exclude the method from including multiple such steps. The mere fact that each dependent claim defines a separate additional feature does not exclude combinations of additional features other than those reflected in the claims.

Claims

Claim 1 An image processor configured to process an image (200) containing three different colored pixels (R, G, B) arranged in a Bayer pattern to obtain a de-correlated image (304) composed of three different types of components (Y, Cr, Cb) arranged in a pattern corresponding to the Bayer pattern, wherein, according to the Bayer pattern, the image (200) is a rectangular array of blocks of 2 x 2 color pixels including two diagonally arranged pixels of a first color (G), a pixel of a second color (R), and a pixel of a third color (B); and according to the pattern corresponding to the Bayer pattern, the de-correlated image (304) is a rectangular array of blocks of 2 x 2 components including two diagonally arranged components of a first type (Y), a component of a second type (Cr), and a component of a third type (Cb); The image processor comprises an inverse correlation module (301), wherein the inverse correlation module (301) is configured to execute a first function to provide a first type component (Y) of an inversely correlated image (304) as a substitute for a first color pixel (G) of the image (200), and the first type component (Y) according to the first function is a weighted combination of clusters of pixels of the image (200) including the first color pixel (G) and neighboring pixels, and the neighboring pixels (R, B) of the second and third colors have an overall positive (+) weighting factor corresponding to the total addition of the neighboring pixels (R, B) of the second and third colors to the first color pixel (G);- A second function is configured to be executed to provide a second type component (Cr) of an inversely correlated image (304) as a substitute for a second color pixel (R) of the image (200), wherein the second type component (Cr) according to the second function is a weighted combination of clusters of pixels of the image (200) including the second color pixel (R) and neighboring pixels, and the first color neighboring pixels (G) have an overall negative (-) weighting factor corresponding to the total subtraction of the first color neighboring pixels (G) from the second color pixel (R); and - configured to execute a third function to provide a third type component (Cb) of an inversely correlated image (304) as a substitute for a third color pixel (B) of the image (200), wherein the third type component (Cb) according to the third function is a weighted combination of clusters of pixels of the image (200) including the third color pixel (B) and neighbor pixels, and the first color neighbor pixels (G) have a total negative (-) weighting factor corresponding to the total subtraction of the first color neighbor pixels (G) from the third color pixel (B), and the weighted combination forming the first type component (Y) comprises: - the nearest neighbor pixels (G) of the first color that are in an x-configuration with respect to the first color pixel being substituted; and, - additional neighbor pixels of the first color that are at a distance of two pixel positions with respect to the first color pixel being substituted and are in a +- configuration; an image processor.; Claim 2 In claim 1, the image processor is configured to: - first, provide each component (Cr) of a second type as a substitute for each pixel (R) of a second color and each component (Cb) of a third type as a substitute for each pixel (B) of a third color, and, subsequently, - provide each component (Y) of a first type as a substitute for each pixel (G) of a first color, so that the component (Y) of the first type is provided on one hand as a substitute for the pixel (G) of the first color which is a substitute for the component of the first type, and on the other hand as a substitute for the pixels of the second color and the pixels of the third color, respectively, and is provided based on a weighted combination of the component (Cr) of the second type and the component (Cb) of the third type which forms a cluster with the pixel of the first color. Claim 3 An image processor according to claim 1, wherein the nearest neighbor pixels (G) of the first color in the x-configuration are at a distance of two pixel positions from the first color pixel being replaced and have a larger weight factor in the weight combination than the additional neighbor pixels of the first color in the +-configuration. Claim 4 In paragraph 3, in the weighted combination forming the component (Y) of the first type: - the pixel (G) of the first color being replaced has a weighting factor of +7 / 8; - the nearest neighboring pixels of the first color in an x-configuration have a weighting factor of -1 / 16; - additional neighboring pixels of the first color in a +-configuration at a distance of two pixel positions from the pixel of the first color being replaced have a weighting factor of -1 / 32; and - the neighboring pixels (R) of the second color and the neighboring pixels (B) of the third color have a weighting factor of +1 / 8, image processor. Claim 5 An image processor according to any one of claims 1 to 4, wherein in the weighted combination forming the component (Cr) of the second type: - the pixel (R) of the second color being replaced has a weighting factor of +1; and - the neighboring pixels (G) of the first color are four immediately neighboring pixels of the first color having a weighting factor of -1 / 4. Claim 6 An image processor according to any one of claims 1 to 4, wherein in the weighted combination forming the component (Cb) of the third type: - the pixel (B) of the third color being replaced has a weighting factor of +1; and - the neighboring pixels (G) of the first color are four immediately neighboring pixels of the first color having a weighting factor of -1 / 4. Claim 7 In any one of claims 1 to 4, the image processor comprises a first type of high-frequency component (Y H ) and Type 1 low-frequency component (Y L To obtain the first type of components (Y) of the inversely correlated image (304), wavelet-based filtering is configured to be applied to the first type of components (Y), wherein the wavelet-based filtering comprises decomposing the first type of frequency components (Y) into an array of a first set of first type components and an array of a second set of first type components, wherein the array of first type components belonging to the first set is interlaced with the array of first type components belonging to the second set, thereby thereby the first type of high frequency components (Y H ) is provided for components of the first type belonging to the array of the first set, and low-frequency components of the first type (Y L ) is an image processor provided for components of the first type belonging to the second set of arrays. Claim 8 In paragraph 7, the first type of high-frequency component (Y H ) is a weighted combination between a component of type 1 belonging to the first array set and neighboring components of type 1 belonging to the second array set, and a low-frequency component of type 1 (Y L An image processor, which is a weighted sum of the first type components belonging to the second set of arrays and neighboring high-frequency components. Claim 9 In paragraph 8, - the above-mentioned first type high-frequency component (Y H In a weighted combination forming ), the first type component belonging to the first set of arrays has a weighting factor of +1, and neighboring first type components belonging to the second set of arrays have a weighting factor of -1 / 4; and,- the first type low frequency component (Y L An image processor, wherein in a weighted sum forming ), the first type component belonging to the second set of arrays has a weighting factor of +1 and the neighboring high-frequency component has a weighting factor of +1 / 8. Claim 10 In paragraph 7, the processor comprises the first type of low-frequency components (Y L An image processor configured to apply two-dimensional wavelet-based filtering to the second type of components (Cr) of the inversely correlated image (304) and to the third type of components (Cb) of the inversely correlated image (304). Claim 11 An encoding system comprising an image processor according to any one of claims 1 to 4, and an encoder configured to encode at least one of an inversely correlated image (304) and a wavelet-based filtered version of the inversely correlated image. Claim 12 In paragraph 11, the encoder comprises: - a first type of high-frequency component (Y H );- Type 1 low-frequency component (Y L An encoding system configured to encode a subband generated by applying 2D wavelet-based filtering to ) - a subband generated by applying 2D wavelet-based filtering to a second type component (Cr) of the inversely correlated image (304); and - a subband generated by applying 2D wavelet-based filtering to a third type component (Cb) of the inversely correlated image (304). Claim 13 A method for processing an image (200) containing three different colored pixels (R, G, B) arranged in a Bayer pattern to obtain a de-corrected image (304) composed of three different types of components (Y, Cr, Cb) arranged in a pattern corresponding to the Bayer pattern, wherein, according to the Bayer pattern, the image (200) is a rectangular array of blocks of 2 x 2 colored pixels including two diagonally arranged pixels of a first color (G), a pixel of a second color (R), and a pixel of a third color (B); and, according to the pattern corresponding to the Bayer pattern, the de-corrected image (304) is a rectangular array of blocks of 2 x 2 components including two diagonally arranged components of a first type (Y), a component of a second type (Cr), and a component of a third type (Cb); The above method comprises the following steps executed by a processor circuit: - providing a first type component (Y) of an inversely correlated image (304) as a substitute for a first color pixel (G) of the image (200), wherein the first type component (Y) is a weighted combination of a cluster of pixels of the image (200) including the first color pixel (G) and neighboring pixels, and the neighboring pixels of the second and third colors (R, B) have a total positive (+) weighting factor corresponding to the total addition of the neighboring pixels of the second and third colors (R, B) to the first color pixel (G);- A step of providing a second type component (Cr) of an inversely correlated image (304) as a substitute for a second color pixel (R) of the image (200), wherein the second type component (Cr) is a weighted combination of clusters of pixels of the image (200) including the second color pixel (R) and neighboring pixels, and the first color neighboring pixels (G) have an overall negative (-) weighting factor corresponding to the overall subtraction of the first color neighboring pixels (G) from the second color pixel (R); and, - providing a third type component (Cb) of the inversely correlated image (304) as a substitute for a third color pixel (B) of the image (200), wherein the third type component (Cb) is a weighted combination of clusters of pixels of the image (200) including pixels adjacent to the third color pixel (B), and the first color adjacent pixels (G) have an overall negative (-) weighting factor corresponding to the total subtraction of the first color adjacent pixels (G) from the third color pixel (B); and the weighted combination forming the first type component (Y) comprises: - the nearest neighbor pixels (G) of the first color that are in an x-configuration with respect to the first color pixel being replaced; and, - additional neighbor pixels of the first color that are at a distance of two pixel positions from the first color pixel being replaced and are in a +- configuration. Claim 14 A computer program storage medium comprising an instruction set that enables an image processor to perform the method according to paragraph 13. Claim 15 A decoder comprising a module corresponding to an inverse correlation module (301) of an image processor according to claim 1, wherein the module is configured to process an inversely correlated image (304) generated by the image processor by inversely calculating the first function, second function, and third function executed by the inverse correlation module (301) to reconstruct an image (200) comprising three different colored pixels (R, G, B) arranged in a Bayer pattern. Claim 16 delete

Images