Decorrelation transform for color filter array pattern images

Abstract

A concept for encoding a CFA pattern image is provided, wherein for each of a sample cluster of a CFA pattern image comprising first to fourth sample positions, first to fourth color coordinates are calculated to provide an image representation of the CFA pattern image.

Description

Technical Field

[0001] Embodiments of the present disclosure relate to an apparatus for encoding a CFA mode image into an image representation and an apparatus for decoding the image representation into a CFA mode image. Embodiments of the present disclosure also relate to a transform for a CFA mode image, and in particular, to color decorrelation or combined color and spatial decorrelation for a CFA mode image. Background Art

[0002] Typical digital cameras are equipped with a so-called CFA pattern sensor (e.g., a Bayer sensor), which consists of a regular grid of sensor elements overlaid with a regular pattern (or array) of color filters to produce three color channels (see Figure 1 Typically, the filters correspond to red, green, and blue light, but other filter configurations are possible, for example, red, clear (no filter), blue. In this way, three independent color channels can be captured, but they are not aligned with each other and require additional post-processing to produce a full-color image from the captured image.

[0003] To save bandwidth and processing power near the sensor, it is desirable to compress the CFA sensor data as is, i.e., without reconstruction (or post-processing) into a full-resolution RGB (true color) image, transmit the compressed CFA image signal, and reconstruct the data at the receiver. The full-resolution true color image generation, i.e., post-processing, is then performed on the reconstructed CFA image data at the decoder. This reduces bandwidth by requiring the transmission of only one component per sensor instead of three.

[0004] Efficient transmission and bandwidth reduction typically further include lossy compression of the CFA data, which consists of multiple steps: first, decorrelation across components, which is denoted "color decorrelation", spatial signal decorrelation - for example by wavelets or discrete cosine transform (DCT); followed by quantization (irreversible reduction of precision); followed by entropy encoding of the data.

[0005] Both the color decorrelation process and the spatial decorrelation process are aimed at facilitating an efficient encoding process and a high compression rate combined with a high quality of the decoded images. Summary of the Invention

[0006] Therefore, it is an object of the present disclosure to provide a more efficient technique for encoding CFA mode images. This object is achieved by the subject matter of the independent claims appended hereto.

[0007] Embodiments of the present disclosure according to the first aspect of the present invention are based on the discovery that CFA pattern images can be encoded more efficiently if the energy of the two channels of the CFA pattern image is compacted to, or primarily constitutes, a single output coordinate for the output image representation. Compared to conventional transformations that aim to compact the energy of the three channels of the CFA pattern image to a single coordinate, the disclosed techniques achieve higher coding efficiency, for example in terms of an improved signal-to-noise ratio of the recovered CFA pattern image.

[0008] Furthermore, embodiments of the present disclosure according to a second aspect of the present disclosure, which can be combined with the first aspect, are based on the discovery that the quality of encoded CFA pattern images can be improved by, for example, considering, for the encoding of a sample cluster, such as a superpixel, samples of the CFA pattern image located in a portion of the CFA pattern image that intersects rows and columns of the CFA pattern image comprising at least one column, and at least one row offset from samples of the sample cluster currently to be encoded, for spatial decorrelation. By considering a larger portion of the CFA pattern image, decorrelation can be improved and energy can therefore be packed more efficiently, thereby allowing for a higher compression rate and a lower signal-to-noise ratio.

[0009] According to a first aspect of the present disclosure, this discovery is used to encode a CFA mode image, which is segmented into sample clusters, each having a first sample from a first channel, a second sample from a second channel, and two third samples from a third channel. Encoding the CFA mode image includes: for the first and second sample positions of each sample cluster, calculating first and second color coordinates, respectively, by forming the difference between the first or second sample of the corresponding sample cluster and the first or second filtered third sample value, respectively. Another step of encoding the CFA mode image includes: for the third and fourth sample positions of each sample cluster, calculating third and fourth color coordinates, respectively, by forming the sum of the third or fourth sample of the corresponding sample cluster and the first or second filtered color coordinate, respectively. The first and second filtered color coordinates are obtained by forming third and fourth weighted sums of the first and second color coordinates adjacent to the respective third and fourth sample positions, where the sum of the weights of each of the third and fourth weighted sums is greater than one-half. It may be equal to or greater than one. In one example, the sum is greater than one-quarter for the weight associated with the first color coordinate, and greater than one-quarter for the weight associated with the second color coordinate. It may be equal to or greater than 0.5. Due to this choice of weights used to calculate the third and fourth weighted sums of the third and fourth color coordinates, the first and second channels (specifically, the low-frequency components) are overweighted in the third and fourth color coordinates relative to the first channel. In other words, the energy of the third and fourth color coordinates is primarily determined by the energy of the first and second channels of the CFA pattern image. The end result is better decorrelation and thereby leads to a higher compression rate or lower quantization loss in the subsequent quantization and entropy encoding process. In the case of an RGB pattern with green as the third channel and equal weights for the first and second samples, the third and fourth color coordinates primarily correspond to a magenta or a mixture of blue and red of a low-pass filtered version of the CFA image, with no green or only a portion of green at a lower energy.

[0010] According to an embodiment, the weights of the third and fourth weighted sums are composed of individual weights for weighting the first and second color coordinates, so that the weights can be adapted to the sensitivity of the corresponding channels. By selecting the weights of the first and second color coordinates so that the first and second channels are equally weighted in the third and fourth color coordinates, encoding efficiency is improved.

[0011] According to an embodiment, the weights used to weight the first and second color coordinates in the third and fourth weighted sums are each implemented as a power of 2, where the exponent of each power of 2 is an integer. Therefore, the multiplication of the first and second color coordinates and their corresponding weights can be performed as a computationally efficient bit shift operation.

[0012] According to an embodiment, a set of operating modes may be used to encode a CFA pattern image, wherein the first through third samples of the image are arranged in columns and rows. In the context of columns and rows, horizontal directions refer to directions within a row of the CFA pattern image, and vertical directions refer to directions along a column of the CFA pattern image. Furthermore, the definitions of columns and rows can be interchanged by rotating the CFA pattern image 90°, so that the attributes of columns and rows should be understood as non-limiting. For some examples, encoding of the CFA pattern image may be performed row by row, and a first row referred to as being above a second row should be understood to be encoded before the second row. Therefore, the second row is below the first row. The set of operating modes includes one or more of an isotropic operating mode, a vertical causal operating mode, and a striped operating mode, each of which differs in that a portion of the CFA pattern image is considered for calculating color coordinates for selected sample positions of a selected sample cluster. In all named operating modes, samples located in the row in which the selected sample cluster is located are included in the calculation of color coordinates. In the isotropic operating mode, samples located in rows above and below the row in which the selected sample cluster is located (designated as the current row) are included in the calculation of color coordinates of the corresponding type for the selected sample position. By including additional rows of the current row of the CFA pattern image in the calculation, the low-frequency and high-frequency components of the CFA pattern image in the vertical direction can be separated, enabling better energy compaction of the CFA pattern image into one or more color coordinates. In contrast, using only samples within the current row allows for energy compaction, primarily in the horizontal direction. Therefore, including additional rows above and / or below the current row provides more efficient data compression and a lower signal-to-noise ratio. In the vertical causal mode of operation, samples in rows above, but not below, the current row are included when calculating the corresponding type of color coordinate for a selected sample position. Because samples in rows below the current row are not considered, encoding can be performed immediately after receiving data signaling samples in the current row, without having to wait for data signaling samples in subsequent rows. Furthermore, by considering samples in rows above the current row, a larger portion of the CFA pattern image is considered when calculating color coordinates, improving quality. Therefore, this mode of operation provides a combination of high code throughput and fast encoding. In the striped mode of operation, samples within the current row are exclusively included when calculating color coordinates. Therefore, compared to the vertical causal and isotropic operation modes, samples at a particular sample position are overwritten or discarded when calculating the color coordinates of that sample position, and less data referring to the rows above the current row needs to be stored. Thus, the striped operation mode combines the advantages of the fast vertical causal operation mode with low memory requirements.

[0013] According to an embodiment, encoding of a CFA pattern image includes further steps of calculating a differential color coordinate, a third and fourth color coordinates, and calculating a combined color coordinate based on the fourth and differential color coordinates. These steps are provided to highly compact the energy of the CFA pattern image into a single color coordinate, namely the combined color coordinate, thereby allowing for a high degree of compression of the image representation via an entropy coding process.

[0014] According to a second aspect of the present disclosure, the techniques described above are used to encode a CFA pattern image, the pattern image having a first sample of a first channel, a second sample of a second channel, and a third sample of a third channel, the first through third samples being arranged in an array having columns and rows. Encoding the CFA pattern image includes, for each first sample position at which the first sample is located, calculating a first color coordinate by forming a difference between the first sample at the corresponding first sample position and a first filtered third sample value, the first filtered third sample value being derived by forming a first weighted sum of third samples adjacent to the first sample position. The third samples used for the third weighted sum include third samples located in rows adjacent to one or more rows (hereinafter referred to as the current row) in which the sample cluster at the corresponding first sample position is located. For example, the adjacent rows may be rows above the current row and may have been previously encoded, making them available for encoding. Including the third samples of rows adjacent to the current row in calculating the third weighted sum requires an additional row, but improves encoding quality, as explained above with respect to vertical causal and isotropic modes of operation.

[0015] According to an embodiment, a set of operating modes can be used to encode CFA mode images, wherein a third of the operating modes includes a first operating mode, such as one of the vertical causal and isotropic operating modes described above, and a slice operating mode. In the slice operating mode, the third sample having the first weighted sum is composed of samples located within the current row. Thus, it is possible to select between the operating modes, with the slice operating mode providing fast encoding and low memory requirements, and the first operating mode providing higher quality for greater coding efficiency.

[0016] Embodiments of the present disclosure according to the first and second aspects described above may be used to encode a CFA mode image into an image representation, which may be provided as such or as an encoded representation of the image representation, and to obtain the CFA mode image from the image representation or the encoded representation, respectively, by decoding the image representation. Therefore, the features, functionality, and advantages described with respect to encoding of the CFA mode image also apply to both the encoding and decoding procedures. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Advantageous embodiments of the disclosure are the subject of the dependent claims, and preferred embodiments are described in more detail below with reference to the figures, in which:

[0018] Figure 1 FIG. 4 shows an apparatus for encoding CFA mode images according to an embodiment.

[0019] Figure 2 FIG. 4 shows an apparatus for encoding CFA mode images according to another embodiment.

[0020] Figure 3 shows an example of a CFA mode image,

[0021] Figure 4 FIG. 4 shows an apparatus for encoding CFA mode images according to another embodiment.

[0022] Figure 5 An apparatus for decoding an image representation into a CFA mode image according to an embodiment is shown.

[0023] Figure 6 An apparatus for decoding an image representation into a CFA mode image according to another embodiment is shown.

[0024] Figure 7 An apparatus for decoding an image representation into a CFA mode image according to another embodiment is shown.

[0025] Figure 8 shows an example of a CFA mode image,

[0026] Figure 9 An example is shown for encoding the current row of a CFA mode image in slice operation mode. DETAILED DESCRIPTION

[0027] Hereinafter, embodiments are discussed in detail, however, it should be understood that the embodiments provide many applicable techniques that can be embodied in a wide variety of image and video processing and encoding. The specific embodiments discussed merely illustrate specific ways of implementing and using the technology of the present invention and do not limit the scope of the embodiments. In the following description, a number of details are set forth to provide a more thorough explanation of the embodiments of the present disclosure. However, it will be apparent to those skilled in the art that other embodiments can be practiced without these specific details. In other cases, well-known structures and devices are shown in block diagram form rather than in detail to avoid confusing the examples described herein. In addition, unless otherwise specifically noted, the features of the different embodiments described herein may be combined with each other.

[0028] In the following description of the embodiments, identical or similar components or components having identical functionality are provided with identical reference numerals or identified by identical names, and repeated description of components provided with identical reference numerals or identified by identical names is generally omitted. Therefore, the descriptions provided for components provided with identical or similar reference numerals or identified by identical names may be interchangeable or applicable to each other in different embodiments.

[0029] A variety of color transforms can be applied to color decorrelation to encode CFA pattern images. These transforms can also include spatial decorrelation.

[0030] One example is the RCTD transform, which is a four-component variant of the RCT transform applied to true color images for lossy or lossless compression purposes. Figure 8 The CFA pattern seen in [1] is divided into 2×2 “superpixels”, each consisting of one red value, one blue value, and two green values, and from these values, four components are formed within each superpixel:

[0031]

[0032]

[0033] C b :=Bt

[0034] C r :=Rt

[0035] Δ∶=G1-G2

[0036] Indicates rounding to an integer, such as rounding down to the next lower integer or rounding down operation, or rounding up to the next higher integer or rounding up operation. This expression applies to formulas throughout the application. Then, the quaternion (Y, C b ,C r ,Δ) is input to other compression steps. It is easy to see that this transformation is completely reversible without loss.

[0037] Recently, the Belgian company intoPix proposed a variant of the above transformation that includes a spatial decorrelation step along with the color transformation. To this end, it divides the CFA image into horizontal strips, each of which is two pixels or one superpixel high and as wide as the entire CFA image. Within each strip, the color transformation is performed in four lifting steps, each of which is itself reversible.

[0038] The first step generates two chromaticity coordinates from the red and blue samples and the adjacent green sample, such as Figure 9 As shown in .

[0039] To define the transform, the sample value to the left of the current position is indicated by sub-index l and the sample value to the right of the current position is indicated by sub-index r. Similarly, sub-index t indicates the current position to the top, and sub-index b indicates the sample position below the current sample. Next, the first lifting step subtracts the weighted sum of the green sample values ​​from the red and blue values, and thus forms C similar to the above case but from the weighted neighborhood of the green value. b and C r aisle:

[0040]

[0041]

[0042] At the left and right edges, the non-existing left green channel is replaced by the right green channel, and at the right image edge, the non-existing right green channel is replaced by the left sample values.

[0043] In the second lifting step, the two luminance channels Y1 and Y2 are transformed from C b 、C r And green channel formation:

[0044]

[0045]

[0046] Where the Y1 samples are located on the top row of a two-row pixel configuration, and the Y2 samples are located on the bottom row, as shown in Figure 2 As indicated in .

[0047] The third lifting step now calculates the luma difference channel Δ on the top row from Y1 and Y2. Similar to the above case, the sub-index rt indicates the sample value to the upper right of the diagonal, lt indicates the sample value to the upper left, rb indicates the sample value to the lower right, and lb indicates the sample value to the lower left. Therefore:

[0048]

[0049] Finally, the last lifting step generates an output luma signal from Y1 and Y2 as an average by using the difference signal Δ:

[0050]

[0051] Then the quaternion (Y,C b ,C r ,Δ) is input to the further compression step, where Δ is optionally excluded from any spatial decorrelation since it already consists of the high-pass filtered difference signal.

[0052] Similar transforms (i.e., transforms that combine spatial and color decorrelation) have been discussed previously, for example by T. Suzuki in "Lossless compression of CFA-sampled images using YDgCoCg transforms with CDF wavelets," in the Proceedings of the International Conference on Image Processing (ICIP) 2018. However, the transform discussed there is based on the YCgCo transform first proposed by H.S. Malvar and G.J. Sullivaan in "Progressive-to-lossless compression of color-filter-array images using macropixel spectral spatial transformations," in the Proceedings of DCC'12, April 2012 (Snowbird, UT), pp. 3-12. The Suzuki transform is also based on lifting, similar to the lifting described by intoPix, but not limited to two rows.

[0053] The above-described intoPix transform aims to compact the energy of three channels into one luma channel, namely Y, and is further limited to horizontal stripes. As previously introduced, the idea according to the first aspect of the present disclosure is to compact the energy of two of the three channels. According to the second aspect of the present disclosure, the transform includes at least one row of samples beyond the row of the currently encoded sample cluster.

[0054] The first part of the detailed description introduces the general technology implemented by the embodiments according to the two aspects of the present disclosure. Subsequently, different embodiments according to the first and second aspects are specified with reference to the general technology.

[0055] Figure 1 A schematic representation of an example of an apparatus 100 for encoding a CFA pattern image 10 into an image representation 70 is shown. The CFA pattern image 10 includes a first sample 12 for a first channel, a second sample 14 for a second channel, and a third sample 16 for a third channel, the third sample including third samples 16A and 16B. For example, the first through third channels consist of red, blue, and green channels, or red, white, and blue channels.

[0056] For example, the CFA image pattern 10 is obtained from a plurality of sensor elements, each of which is associated with one of a first channel, a second channel, and a third channel, wherein the sensor elements associated with a common channel are configured to detect light within a common wavelength range specific to the channel. For example, the CFA pattern image may have a Bayer pattern, such as Figure 8 As shown in .

[0057] The CFA mode image 10 is segmented into sample clusters 20 , each of which includes one of the first samples 12 located at a first sample position 22 , one of the second samples 14 located at a second sample position 24 , a third sample 16A located at a third sample position 26 , and another third sample 16B located at a fourth sample position 28 .

[0058] The apparatus 100 is configured to encode a CFA pattern image into an image representation 70 by calculating first to fourth color coordinates 110, 120, 130, 140 for the first to fourth sample positions 22, 24, 26, 28 of each of the sample clusters 20. In step 1, which may be referred to as a first lifting step, for each of the sample clusters 20, the apparatus 100 calculates a first color coordinate 110', such as a first chroma difference channel coordinate, for the first sample position 22' of the corresponding sample cluster 20'. The first color coordinate is understood to represent the difference between the first sample of the corresponding sample cluster 20' and the first sample position 22' and a first filtered third sample value 112'. The first filtered third sample value 12' is derived by forming a first weighted sum 114' of the third samples 16 adjacent to the first sample position 22'. It should be noted that the samples adjacent to a particular sample position may be part of the same sample cluster as the particular sample position, but may also be part of another sample cluster. Furthermore, the apparatus 100 is configured to calculate a second color coordinate 120′, for example, a second chromaticity difference channel coordinate, for a second sample position 24′ of the corresponding sample cluster 20′, such that the second color coordinate 120′ represents a difference between the second sample 14′ and the second sample position 14′ of the corresponding sample cluster 20′ and a second filtered third sample value 122′. The second filtered third sample value 122′ is obtained by forming a second weighted sum 124′ of the third samples 16 adjacent to the second sample position 24′.

[0059] In a subsequent step 2, which may be referred to as a second lifting step, the device 100 is configured to calculate, for each of the sample clusters 20, a third color coordinate 130' by forming a sum (e.g., an equally weighted sum) of the third sample 16A' at the third sample position 26' of the respective sample cluster 20' and a first filtered color coordinate 132', the first filtered color coordinate being obtained by forming a third weighted sum 134' of the first color coordinate 110 and the second color coordinate 120 calculated for the first sample position 22 and the second sample position 24 adjacent to the third sample position 26'. The device 100 is configured to calculate, for the fourth sample position 28′ of each sample cluster 20′, a fourth color coordinate 140′ by forming a sum (e.g., an equally weighted sum) of the third sample 16B′ at the fourth sample position 28′ of the corresponding sample cluster 20′ and a second filtered color coordinate 142′, the second filtered color coordinate being obtained by a fourth weighted sum 144′ of the first color coordinate 110 and the second color coordinate 120 calculated for the first sample position 22 and the second sample position 24 adjacent to the fourth sample position 28′.

[0060] For example, for each of sample clusters 20, device 100 may provide respective first through fourth color coordinates 110, 120, 130, 140 in image representation 70. A representation of image representation 70 based on first through fourth color coordinates 110, 120, 130, 140 may be referred to as a transformed representation 149.

[0061] Alternatively, the apparatus 100 may be configured to use the third color coordinate 130 and the fourth color coordinate 140 as intermediate values ​​for calculating the differential color coordinate 250 and the combined color coordinate 260 for each of the sample clusters 20 . Figure 2 A schematic representation of another example of the device 100 is shown, which, in addition to calculating the first to fourth color coordinates 110, 120, 130, 140, also includes two additional steps for calculating a differential color coordinate 250 and a combined color coordinate 260 based on the third color coordinate 130 and the fourth color coordinate 140. In this case, the device 100 can provide, for each of the sample clusters 20, a corresponding first color coordinate 110, second color coordinate 120, differential color coordinate 250, and combined color coordinate 260 in the image representation 70. The representation of the image representation 70 based on the first color coordinate 110, second color coordinate 120, differential color coordinate 250, and combined color coordinate 260 can be referred to as a compacted transformed representation 249.

[0062] Thus, in a third step 3, which may be referred to as a third lifting step, for each of the sample clusters 20, the device 100 may calculate a differential color coordinate 250' for the third sample position 26' of the corresponding sample cluster 20' by forming a difference between the third color coordinate 130' at the third sample position 26' of the corresponding sample cluster 20' and a filtered fourth color coordinate 252'. The filtered fourth color coordinate 252' is derived by forming a fifth weighted sum 254' (e.g., an equally weighted sum) of the fourth color coordinates 140 calculated for the fourth sample position 28 adjacent to the third sample position 26'.

[0063] In a fourth step 4, which may be referred to as a fourth lifting step, for each of the sample clusters 20, the device 100 may calculate a combined color coordinate 260 for a fourth sample position 28′ of the corresponding sample cluster 20′ by forming a sum of the fourth color coordinate 140′ at the fourth sample position 28′ of the corresponding sample cluster 20′ and a filtered differential color coordinate 262′. The filtered differential color coordinate 262′ is derived by forming a sixth weighted sum 264′ (e.g., an equally weighted sum) of the differential color coordinates 250 calculated for the third sample position 26 adjacent to the fourth sample position 28′.

[0064] According to, for example, Figure 1 In the embodiment illustrated in the CFA imaging pattern in FIG, the first through fourth sample positions are arranged in an array having a first type of row 52 and a second type of row 54. The first type of row 52 includes alternating first sample positions 22 and third sample positions 26, and the second type of row 54 includes alternating second sample positions 24 and fourth sample positions 28. The first type of rows 52 and the second type of rows are arranged alternately along the columns of the array. The first sample position 22 of the first type of row 52 and the second sample position 24 of the second type of row 54 are located in different columns 62, 64 of the array.

[0065] Thus, in the described array arrangement, each of the first sample positions 22 is adjacent to four third samples 16, two of the third samples 16A being located at a third sample position 26 adjacent to the first sample position 22 in the horizontal direction, and two of the third samples 16B being located at a fourth sample position 28 adjacent to the first sample position 22 in the vertical direction. In this context, the horizontal direction refers to the top and the bottom, and the vertical direction refers to the left and the right. Similarly, each of the second sample positions 24 is adjacent to four third samples 16, two of the third samples 16 being located at a third sample position 26 adjacent to the second sample position 24 in the vertical direction, and two of the third samples 16 being located at a fourth sample position 28 adjacent to the second sample position 24 in the horizontal direction. According to the described example of the arrangement of the sample positions, each of the sample clusters 20 is located within a pair of two adjacent sample rows comprising a row 52 of the first type and a row 54 of the second type. It should be noted that, as Figure 1 The view arrangement of the sample cluster 20 shown in relation to the first type of rows 52 and the second type of rows 54 and to the first type of columns 62 and the second type of columns 64 is illustrative, that is, the upper and lower positions as well as the left and right positions of the rows and columns can be exchanged, respectively, which can result in four different arrangements of the first to fourth sample positions within the sample cluster 20.

[0066] For example, the device 100 may perform calculations of color coordinates, i.e., the first to fourth color coordinates 110, 120, 130, 140 and, optionally, the differential color coordinates 250 and the combined color coordinates 260, in a row-by-row manner (i.e., row by row). Different processing schemes or operating modes may be possible, depending on the size of the portion of the CFA pattern image that serves as the source of the respective input values ​​for the first to sixth weighted sums 114', 124', 134', 144', 254', 264'.

[0067] For example, device 100 may store the first to fourth color coordinates and / or differential and combined color coordinates calculated during the respective first to fourth lifting steps in a respective data buffer so that a subsequent lifting step can utilize the stored color coordinates. Buffered color coordinates 110, 120, 130, 140, 250, 260 or samples 12, 14, 16 may be released or overwritten when not needed for calculating color coordinates for other sample clusters. Therefore, the amount of data that needs to be stored in a respective data buffer may depend on the number of adjacent sample positions considered in a lifting step utilizing the respective data buffer.

[0068] Figure 3An example of a portion of a CFA mode image 10 including a sample cluster 20' for calculating color coordinates is illustrated. The sample cluster 20' is located in a row 52' of a first type and a row 54' of a second type, which is referred to as a current row 56.

[0069] For example, the device 100 may operate in a stripe mode of operation or an intra-row mode of operation, wherein only samples located at sample positions within the current row 56 may contribute to the first through sixth weighted sums 114', 124', 134', 144', 254', 264'. Alternatively, the device 100 may operate in a vertical causal mode of operation, wherein samples in one or more rows located above the current row 56 may additionally contribute to one or more of the first through sixth weighted sums 114', 124', 134', 144', 254', 264'. In yet another experiment, the device 100 may operate in an isotropic mode of operation, wherein, in addition to the vertical causal mode of operation, samples in one or more rows located below the current row 56 may also contribute to one or more of the first through sixth weighted sums 114', 124', 134', 144', 254', 264'.

[0070] It should be noted that for sample positions that are not at the edge of the CFA pattern image, samples contributing to the first to sixth weighted sums 114', 124', 134', 144', 254', 264' are described in detail. For edge sample positions, the missing sample positions may be replaced by sample positions that are opposite to the missing sample positions with respect to the sample positions used to calculate the corresponding weighted sum.

[0071] In the stripe mode of operation, only samples located within the current row 56 are considered as input values ​​for the first through sixth weighted sums 114', 124', 134', 144', 254', 264'. For example, the third sample 16 for the first weighted sum 114' may be composed of the third samples 16 located at the third sample positions 26-1 and 26' horizontally adjacent to the first sample position 22' of the corresponding sample cluster 20', and the third sample 16 located at the fourth sample position 28'. Correspondingly, the third sample 16 for the second weighted sum 124' may be composed of the third samples 16 located at the fourth sample positions 28-1 and 28' horizontally adjacent to the second sample position 24' of the corresponding sample cluster 20', and the third sample 16 located at the third sample position 26'. The first color coordinates 110 and the second color coordinates 120 used to calculate the fourth weighted sum 144′ may include the second color coordinates 120 located at the second sample position 24′, 24-1 adjacent to the fourth sample position 28′ of the corresponding sample cluster 20′ in the horizontal direction, and the first color coordinates 110 located at the first sample position 22′ adjacent to the third sample position 26′ in the first vertical direction (e.g., the top direction). In addition, the first color coordinates 110 and the second color coordinates 120 used to calculate the third weighted sum 134′ may include the first color coordinates 110 located at the first sample positions 22′, 22-1 adjacent to the third sample position 26′ of the corresponding sample cluster 20′ in the horizontal direction, and the second color coordinates 120 located at the second sample position 24′ adjacent to the third sample position 26′ in the second vertical direction (e.g., the bottom direction).

[0072] The fourth color coordinate 140 and the differential color coordinate 250 used to calculate the fifth weighted sum 254' and the sixth weighted sum 264' are adjacent to the corresponding third sample position 26' and fourth sample position 28' in the diagonal direction of the array, respectively. The first and second diagonal directions are diagonal directions having a vertical component pointing in the first vertical direction or the top direction. The third and fourth diagonal directions are diagonal directions having a vertical component pointing in the second vertical direction or the bottom direction.

[0073] In the stripe mode of operation, the fourth color coordinates 140 used to calculate the fifth weighted sum 254′ may be composed of the fourth color coordinates 140 calculated for the fourth sample positions 28′, 28-1 that are adjacent to the third sample position 26′ of the corresponding sample cluster 20 in the third and fourth diagonal directions of the array (e.g., the lower left and lower right diagonal directions). Accordingly, the differential color coordinates 250 of the sixth weighted sum 264′ may be composed of the differential color coordinates 250 calculated for the third sample positions 26′, 26-1 that are adjacent to the corresponding sample cluster 20′ and the fourth sample position 28′ in the first and second diagonal directions (e.g., the upper left in the upper right diagonal direction).

[0074] Because in striped mode, samples are explicitly used only to calculate the color coordinates for the current row 56, specific sample positions can be replaced by the color coordinates calculated for that specific sample position. Consequently, this mode of operation has significantly lower memory requirements. Furthermore, by compacting the energy of the first and second channels into a single color coordinate for the image representation, it can improve performance by approximately 0.3 dB at low bit rates compared to current state-of-the-art transforms.

[0075] Another example of the current row 56 for calculating the corresponding color coordinates for the case where the first, second and third channels are red, blue and green channels is depicted in Figure 9 middle.

[0076] Another possible mode of operation is to see Figure 3 In the vertical causal mode of operation, sample positions within one or more rows adjacent to the current row 56 in a first vertical direction may be included in the calculation of one or more of the first through sixth weighted sums 114', 124', 134', 144', 254', 264'. Preferably, the first vertical direction is directed toward a row of the array received by apparatus 100 that is temporally preceding the current row 56. Thus, CFA mode image 10 may be encoded without additional time delay relative to the stripe mode of operation.

[0077] For example, in the vertical causal mode of operation, the second weighted sum 124', the fourth weighted sum 144', and the sixth weighted sum 264' may be calculated as described with respect to the stripe mode of operation. The third sample 16 used for the first weighted sum 114' may be composed of the third sample 16 located at the third sample positions 26-1, 26' horizontally adjacent to the first sample position 22' of the corresponding sample cluster 20', and the third sample 16 located at the fourth sample positions 28', 28-2 vertically adjacent to the first sample position 22'. Additionally, the first color coordinates 110 and the second color coordinates 120 used to calculate the third weighted sum 134' may be composed of the first color coordinates 110 located at the first sample positions 22', 22-1 horizontally adjacent to the third sample position 26' of the corresponding sample cluster 20', and the second color coordinates 120 located at the second sample positions 24', 24-2 vertically adjacent to the third sample position 26'. The fourth color coordinate 140 used to calculate the fifth weighted sum 254' may be composed of the fourth color coordinates 140 calculated for the fourth sample positions 28', 28-1, 28-3, 28-4 that are adjacent to the third sample position 26' of the corresponding sample cluster 20 in the four diagonal directions of the array. In other words, in the example of the vertical causal mode, only the top neighbor is accessed, and thus no additional delay is incurred, but each lifting step must buffer an additional row. That is, the encoder (e.g., device 100), the input buffer requirement may increase by four rows, and the decoder (e.g., Figure 5 500), the output buffer requirement may be increased by four rows.

[0078] In the stripe mode of operation and the vertical causal mode of operation, the first to sixth weighted sums 114', 124', 134', 144' may be weighted so that the total weight of the contributions from the sample positions adjacent to the sample position for calculating the corresponding color coordinate in the horizontal direction is equal to the total weight of the contributions from the sample positions adjacent to the sample position for calculating the corresponding color coordinate in the vertical direction. That is, when only one vertically adjacent sample position is considered, the corresponding value may be double-weighted.

[0079] Another possible operating mode is an isotropic operating mode, in which samples located in rows adjacent to the current row 56 in both vertical directions are considered when calculating one or more or all of the color coordinates 110, 120, 130, 140, 250, 260. The first weighted sum 114', the third weighted sum 134', and the fifth weighted sum 254' can be calculated as described with respect to the vertical causal operating mode. The third samples 16 used for the second weighted sum 124' can include third samples 16 located at fourth sample positions 28-1, 28' horizontally adjacent to the second sample position 24' of the corresponding sample cluster 20', and third samples 16 located at third sample positions 26', 26-2 vertically adjacent to the second sample position 24'. In this case, one additional row below the current row 56 with respect to the previously described operating mode is required for calculating the second color coordinate 120. Alternatively, for example, depending on the maximum acceptable time delay introduced by considering additional rows below the current row 56, the first color coordinates 110 and second color coordinates 120 used to calculate the fourth weighted sum 144' may include the second color coordinate 120 at the second sample position 24', 24-1 located horizontally adjacent to the fourth sample position 28' of the corresponding sample cluster 20', and the first color coordinate 110 at the first sample position 22', 22-2 located vertically adjacent to the third sample position 26'. In this case, two additional rows below the current row 56 are required with respect to the previously described operating mode. Alternatively, the differential color coordinates 250 of the sixth weighted sum 264' may consist of the differential color coordinates 250 calculated for the third sample positions 26', 26-1, 26-2, 26-3 adjacent to the fourth sample position 28' of the corresponding sample cluster 20' in the four diagonal directions. In this case, four additional rows below the current row 56 are required with respect to the previously described operating mode. In other words, the encoder transform delays its output 70 by 4 lines or rows, and the decoder includes an additional delay of 4 lines or rows, so the end-to-end delay may increase by 8 lines or rows.

[0080] According to an embodiment, the apparatus 100 includes a set of operating modes including one or more of an isotropic operating mode, a vertical causal operating mode, and a striped operating mode. Thus, one of the set of operating modes may be selected depending on available memory and on a maximum acceptable time delay for encoding and decoding the CFA pattern image 10.

[0081] According to an embodiment, the apparatus 100 may select one of the set of operating modes for encoding the CFA mode image 10 and cause the selected operating mode to be signaled in the data stream in which the image representation 70 is signaled.

[0082] According to an embodiment, first filtered third sample value 112′, second filtered third sample value 122′, first filtered color coordinate 132′, second filtered color coordinate 142′, filtered fourth color coordinate 252′, and filtered differential color coordinate 262′ are obtained from first weighted sum 114′, second weighted sum 124′, third weighted sum 134′, fourth weighted sum 144′, fifth weighted sum 254′, and sixth weighted sum 264′, respectively, by rounding operations. This allows for efficient calculation and storage of the corresponding color coordinates. For example, the rounding operation may be a floor or ceiling operation, which may round a number to the next lower or higher integer, respectively.

[0083] According to an embodiment, the first weighted sum 114' and the second weighted sum 124' each represent an average of their respective third samples. Thus, the first color value 110' and the second color value 120' may represent the difference between the spatially filtered averages of the first sample 12' and the second sample 14', respectively, and the respective third samples adjacent to the first position 22' and the second sample position 24'.

[0084] According to an embodiment, the weights of the third weighted sum 134' and the fourth weighted sum 144' are composed of a first weight for weighting the first color coordinate and a second weight for weighting the second color coordinate. For example, the first weight may be implemented as a first power of 2, and the second weight may be implemented as a second power of 2, where the exponents of the first and second powers of 2 are integers, such as a negative integer, zero, or a positive integer.

[0085] Below, examples of calculating color coordinates 110, 120, 130, 140, 250, and 260 are described in more detail. For illustrative purposes and according to some examples, it is assumed that the first channel is the red channel, the second channel is the blue channel, and the third channel is the green channel. However, the described transformation is also advantageous for different configurations. For example, in other embodiments, the first and second channels can be swapped and / or the third channel can be the white channel. In the following formulas, R represents the first sample 12, B represents the second sample 22, and G represents the third sample 24, or more precisely, their sample values. Also for illustrative purposes and not limiting, the first vertical direction is selected as the top direction, the second vertical direction is selected as the bottom direction, and the horizontal directions are selected as the left and right directions. The left-neighboring samples of the sample position for which the corresponding color coordinate is to be calculated are indexed by l, the right-neighboring samples are indexed by r, the top-neighboring samples are indexed by t, and the bottom-neighboring samples are indexed by b. The corresponding diagonal directions are indexed by lt, lb, rt, and rb. Therefore, the first color coordinate 110 can be referred to as the first chromaticity difference channel C r The second color coordinate 120 may be referred to as a second chromaticity difference channel C b, the third color coordinate 130 may be referred to as the first magenta channel M1, the fourth color coordinate 140 may be referred to as the second magenta channel M2, the difference color coordinate 250 may be referred to as the magenta difference channel Δ, and the combined color coordinate 260 may be referred to as the average magenta channel M.

[0086] According to an embodiment, in each of the third weighted sum 134' and the fourth weighted sum 144', the total weight for the first color coordinate 110 is equal to the total weight for the second color coordinate 120, and the sum of the weights of the respective weighted sums is equal to one. For this particular case and in combination with the first weighted sum 114' and the second weighted sum 124' being equally weighted sums, this may form a spatial high pass of the third channel (e.g. the green channel), and the high frequency component or DC component of the resulting combined color coordinate contains mainly or even only components of the first and second channels, such as red and blue components, but no components of the third channel, such as green components. Thus, in the case where the first and second channels represent the remaining channels and the blue channel, the combined color coordinate may be referred to as the magenta channel (M). In other words, the DC component of M may be the average of the red and blue channels, thereby obtaining its name "magenta". However, lifting steps such as the third and fourth enumeration steps may ensure that the DC component of the C b 、C r Compressing the energy of the CFA pattern image into the magenta channel can result in more efficient coding, thereby reducing the signal-to-noise ratio by, for example, 0.3 dB.

[0087] For example, in a first enumeration step, the first color coordinate 110 and the second color coordinate 120 may be calculated according to the following formula, which is best shown for the isotropic mode of operation:

[0088]

[0089]

[0090] In the "vertical causality" model, C b G in the calculation b By G t Replace. In "strip operation mode", the previous replacement is performed, and C r G in the calculation t Also by G b replace.

[0091] In the second lifting step, the third color coordinate 130 and the fourth color coordinate 140 may be calculated in the isotropic operation mode according to the following equations:

[0092]

[0093]

[0094] For example, the M1 samples are located at the top row, and the M2 channels are located on the bottom row. In the vertical causal mode of operation, the C r,b Can be obtained by C r,t Replace, in stripe operation mode, additionally, C b,t Can be obtained by C b,b Replacement. The replacement rule is similar to that of the first lifting step, i.e., unusable samples are created by reflecting similar samples from above or below.

[0095] In the above formulas for M1 and M2, the first color coordinate 110 (C r ) is implemented as a first power of 2 and is used for the second color coordinate 120 (C b ) is implemented as the second power of 2. For the case where r=b=1, the exponent of the first power of 2 and the exponent of the second power of 2 are both equal to -2, resulting in a weight of 1 / 4 for each of the first and second color coordinates, so that the sum of the weights of the third weighted sum 134' and the fourth weighted sum 144' is 1 in this case, thus providing an exemplary implementation for compacting the energy of the CFA pattern image into the magenta channel when the first and second channels are the red and blue channels, respectively.

[0096] According to an embodiment, the fifth weighted sum 254′ is an equally weighted average of the fourth color coordinates 140 of the fifth weighted sum 254′. For example, in this case, the differential color coordinate 250 may primarily represent the differential components of the first two third channels, so that the differential color coordinate 250 may be particularly small, which allows for efficient encoding.

[0097] For example, in isotropic and vertical causal modes of operation, the differential color coordinate 250 may be calculated according to the following formula:

[0098]

[0099] In stripe operation mode, M lt M lb Replace and M rt By M rb replace.

[0100] According to an embodiment, the sixth weighted sum 264 ′ is an equally weighted average of the fourth color coordinates of the sixth weighted sum 264 ′, and the sum of the weights of the sixth weighted sum 264 ′ is equal to one-half.

[0101] For example, the combined color coordinates 260 may be calculated according to the following formula:

[0102]

[0103] Thus, the described transform can contact the energy of the first and second channels in the combined color coordinate 216. In vertical causal mode and stripe mode of operation, Δ lb Can be determined by Δ lt Replace and Δ rb Can be determined by Δ rt replace.

[0104] For example, the final output of the transformation (eg, the compacted transformed representation 249) may be (M, C b ,C r ,Δ) samples. These samples can be the input for further spatial decorrelation. Alternatively, only Δ may not require additional spatial decorrelation.

[0105] Figure 4 Another example of a device 100 for encoding a CFA mode image 10 is illustrated. As shown, the device 100 may further include a quantizer 480 and an entropy encoder 490. The quantizer 480 may quantize the transformed representation 149 (i.e., the first through fourth color coordinates 110, 120, 130, 140 of the sample cluster 20) or, alternatively, the compacted transformed representation 249 (i.e., the first and second color coordinates 110, 120, and the differential color coordinates 250 and the combined color coordinates 260 of the sample cluster 20) to produce a quantized representation 482. The entropy encoder 490 may encode the quantized representation 482 to obtain an encoded representation 495 that may be provided as the image representation 70.

[0106] Optionally, the device 100 may further include a spatial decorrelation stage 475 that may use a spatial decorrelation transform (e.g., a discrete cosine transform or a discrete wavelet transform) to further spatially decorrelate the transformed representation 149 or the compacted transformed representation 249. In one example, the spatial decorrelation stage 475 applies the spatial decorrelation transform to the first to fourth color coordinates 110, 120, 130, 140 or the first color coordinate 110, the second color coordinate 120, the differential color coordinate 250, and the combined color coordinate 260 to obtain a spatially decorrelated representation 478, based on which the quantizer 480 may derive a quantized representation 482. In another example, the spatial decorrelation stage 475 applies the spatial decorrelation transform to the first color coordinate 110, the second color coordinate 120, and the combined color coordinate 260 to obtain the spatially decorrelated representation 478, and the quantizer 480 may derive the quantized representation 482 from the spatially decorrelation representation 478 and the differential color coordinate 250 of the sample cluster 20.

[0107] Figure 5A schematic representation of an apparatus 500 for decoding an image representation 70 in order to obtain a CFA mode image 10 is shown. The apparatus 500 is configured to calculate, for each sample cluster 20' of the sample clusters 20, a first sample 12', a second sample 14' and two third samples 16A', 16B' from the first to fourth color coordinates 110, 120, 130, 140.

[0108] The device 500 can calculate, for the third sample position 26' of the respective sample cluster 20', a third sample 16A' at the respective third sample position 26' by forming a difference between the third color coordinate 130' at the third sample position 26' of the respective sample cluster 20' and the first filtered color coordinate 132'. Additionally, the device 500 can calculate, for the fourth sample position 28' of each sample cluster 20', a third sample 16B' at the respective fourth sample position 28' by forming a difference between the fourth color coordinate 140' at the fourth sample position 28' of the respective sample cluster 20' and the second filtered color coordinate 142'. The first filtered color coordinate 132 and the second filtered color coordinate 142 can be obtained from the first and second color coordinates, as described with respect to the device 100.

[0109] In a subsequent step, the device 500 may calculate, for each sample cluster 20', a first sample 12' at the first sample position 22' by forming a sum of the first color coordinate 110' at the first sample position 22' of the corresponding sample cluster 20' and a first filtered third sample value 112'. Additionally, the device 500 may calculate, for each sample cluster 20', a second sample 14' at the second sample position 14' by forming a sum of the second color coordinate 120' at the second sample position 24' of the corresponding sample cluster 20' and a second filtered third sample value 122'. The first filtered third sample value 112 and the second filtered third sample value 122 may be obtained from the third sample 16 calculated for the third sample position 26 and the fourth sample position 28 according to the previous step as described with respect to the device 100.

[0110] Equivalent to apparatus 100, apparatus 500 may include a set of operating modes for calculating the first to sixth weighted sums. According to an embodiment, apparatus 500 may derive the operating mode to be used for decoding the image representation from a data stream signaling the image representation, and may select one of the set of operating modes accordingly.

[0111] Figure 6Illustrating another example of an apparatus 500 , the apparatus is configured to obtain first to fourth samples 12 , 14 , 16A, 16B from first and second color coordinates 110 , 120 and differential and combined color coordinates 250 , 260 that may be signaled within an image representation 70 . Figure 6 The embodiment shown in Figure 5 , for calculating the third color coordinate 130 and the fourth color coordinate 140 from the differential color coordinate 250 and the combined color coordinate 260. For that purpose, the device 500 can calculate, for the fourth sample position 28' of the respective sample cluster 20', the fourth color coordinate 140' at the respective fourth sample position 28' by forming a difference between the combined color coordinate 260' at the fourth sample position 28' of the respective sample cluster 20' and the filtered differential color coordinate 262'. The filtered differential color coordinate 262 can be obtained from the differential color coordinate 250 as described with respect to the device 100.

[0112] In a subsequent step, the device 500 may calculate the third color coordinate 130′ at the third sample position 26′ of the corresponding sample cluster 20′ by forming the sum of the differential color coordinate 250′ at the third sample position 26′ of the corresponding sample cluster 20′ and the filtered fourth color coordinate 252′. The filtered fourth color coordinate 252 may be obtained from the fourth color coordinate 140, as described with respect to the device 100.

[0113] Figure 7 A schematic representation of another example of a device 500 further comprising an entropy decoder 590 and an inverse quantizer 580 is shown. The entropy decoder 590 may decode the encoded representation 495, which may represent the image representation 70, to obtain a quantized representation 482. The inverse quantizer 580 may use an inverse quantization process to obtain, from the quantized representation 482, the first to fourth color coordinates 110, 120, 130, 140, which may be indicated as a transformed representation 149, or alternatively, the first color coordinate 110, the second color coordinate 120, the differential color coordinate 250, and the combined color coordinate 260, which may be indicated as a packed transformed representation 249.

[0114] Optionally, the inverse quantizer 580 provides a method as described with respect to the apparatus 100 (see Figure 4 ) and the device 500 further includes an inverse spatial decorrelation stage 575, which can use an inverse spatial decorrelation transform (for example, an inverse discrete cosine transform or an inverse discrete wavelet transform) to obtain the first to fourth color coordinates 110, 120, 130, 140 or the first color coordinate 110, the second color coordinate 120, the differential color coordinate 250 and the combined color coordinate 260.

[0115] According to another option, the inverse quantizer 580 provides the spatially decorrelated representation 478 and the differential color coordinates 250 , and the device 500 further includes an inverse spatial decorrelation stage 575 that can use an inverse spatial decorrelation transform to derive the first color coordinate 110 , the second color coordinate 120 and the combined color coordinate 260 .

[0116] For further details of the apparatus 500 for decoding the image representation 70 into the CFA pattern image 10, reference is made to the description of the apparatus 100. Although signals indicated by the same reference numerals may differ, for example due to quantization losses, the features of the apparatus 500 indicated by the same reference numerals as with respect to the apparatus 100 have equivalent functionality and may be implemented equivalently, as introduced in the corresponding description.

[0117] Hereinafter, embodiments of the present disclosure according to the first and second aspects are described. Figure 1 described.

[0118] In an embodiment according to the first aspect of the present disclosure, the third weighted sum 134 ′ is calculated such that the sum of its weights is greater than one-half, and the fourth weighted sum 144 ′ is derived such that the sum of its weights is greater than one-half.

[0119] In an embodiment according to the second aspect of the present disclosure, the third sample position 26 of the third sample 16 of the first weighted sum 114' is beyond the current row 56 of the array in which the corresponding sample cluster 20' is located. For example, the first weighted sum 114' can be calculated as described with respect to the isotropic operating mode or the vertical causal operating mode. Alternatively, the calculation of the second to sixth weighted sums 124', 134', 144', 254', 264' can also be performed according to one of the vertical causal operating mode or the isotropic operating mode. In other words, according to the second aspect of the present disclosure, the device 100, 500 may include one of the isotropic operating mode and the vertical causal operating mode. Different possible implementations of these operating modes are described above.

[0120] Other embodiments according to the second aspect may additionally include a striped operating mode. For example, apparatus 100, 500 may select between a striped operating mode and a first operating mode, which may correspond to an isotropic or vertical causal operating mode, for example. According to an embodiment, apparatus 100 may be configured to signal the operating mode used to compute the image representation 70 in the data stream, signaling the image representation 70 in the data stream. Thus, apparatus 500 may be configured to derive from the data stream the operating mode to be used for decoding the image representation 70 to obtain the CFA mode image.

[0121] Although some aspects have been described as features in the context of an apparatus, it is clear that this description may also be seen as a description of corresponding features of a method. Although some aspects have been described as features in the context of a method, it is clear that this description may also be seen as a description of corresponding features with respect to the functionality of the apparatus.

[0122] Some or all of the method steps may be performed by (or using) a hardware device such as a microprocessor, a programmable computer or an electronic circuit. In some embodiments, one or more of the most important method steps may be performed by this device.

[0123] The encoded image signal of the present invention may be stored on a digital storage medium, or may be transmitted on a transmission medium such as a wireless transmission medium or a wired transmission medium such as the Internet.

[0124] Depending on certain implementation requirements, embodiments of the present invention may be implemented in hardware or software, or at least partially in hardware or at least partially in software. Embodiments may be implemented using a digital storage medium such as a floppy disk, DVD, Blu-ray, CD, ROM, PROM, EPROM, EEPROM, or flash memory, the digital storage medium having electronically readable control signals stored thereon that cooperate (or are capable of cooperating) with a programmable computer system to cause the corresponding method to be performed. Thus, the digital storage medium may be computer readable.

[0125] Some embodiments according to the invention comprise a data carrier having electronically readable control signals, which are capable of cooperating with a programmable computer system, such that one of the methods described herein is performed.

[0126] Generally, embodiments of the present invention can be implemented as a computer program product with a program code, the program code being operative for performing one of the methods when the computer program product runs on a computer. The program code may, for example, be stored on a machine readable carrier.

[0127] Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.

[0128] In other words, an embodiment of the inventive method is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.

[0129] A further embodiment of the inventive method is, therefore, a data carrier (or a digital storage medium, or a computer-readable medium) comprising, recorded thereon, the computer program for performing one of the methods described herein. The data carrier, digital storage medium, or recorded medium are typically tangible and / or non-transitory.

[0130] A further embodiment of the inventive method is, therefore, a data stream or a sequence of signals representing the computer program for performing one of the methods described herein.The data stream or the sequence of signals may, for example, be configured to be transmitted via a data communication connection (for example, via the Internet).

[0131] A further embodiment comprises processing means, for example a computer or a programmable logic device configured or adapted to perform one of the methods described herein.

[0132] A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.

[0133] Another embodiment according to the invention comprises an apparatus or system configured to transfer (for example, electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver may be, for example, a computer, a mobile device, a memory device, etc. The apparatus or system may, for example, comprise a file server for transferring the computer program to the receiver.

[0134] In some embodiments, a programmable logic device (e.g., a field programmable gate array) may be used to perform some or all of the functionality of the methods described herein. In some embodiments, the field programmable gate array may cooperate with a microprocessor to perform one of the methods described herein. In general, the methods are preferably performed by any hardware device.

[0135] The devices described herein may be implemented using hardware devices or using computers or using a combination of hardware devices and computers.

[0136] The methods described herein may be performed using a hardware device or using a computer or using a combination of a hardware device and a computer.

[0137] In the foregoing detailed description, it can be seen that various features are grouped together in the examples for the purpose of streamlining the disclosure. The method of the present disclosure should not be interpreted as reflecting the intention that the claimed examples require more features than are explicitly described in the various technical solutions. On the contrary, as reflected in the following claims, the subject matter may lie in less than all the features of a single disclosed example. Therefore, the following claims are hereby incorporated into the detailed description, wherein each claim may serve as a separate example in its own right. Although each claim may serve as a separate example in its own right, it should be noted that although a dependent claim may refer to a specific combination with one or more other claims in the claims, other examples may also include a combination of a dependent claim with the subject matter of each other dependent claim or a combination of each feature with other dependent or independent claims. Unless it is stated that a specific combination is not desired, such combinations are proposed herein. In addition, it is intended that the features of a claim for any other independent claim be included, even if this claim is not directly attached to the independent claim.

[0138] The embodiments described above are merely illustrative of the principles of the present disclosure. It should be understood that modifications and variations of the configurations and details described herein will be readily apparent to those skilled in the art. Accordingly, the present invention is intended to be limited only by the scope of the following claims and not by the specific details presented herein through the description and explanation of the embodiments.

Claims

1. An apparatus for decoding an image representation into a color filter array pattern image, wherein the color filter array pattern image comprises a first sample of a first channel, a second sample of a second channel, and a third sample of a third channel, wherein the color filter array pattern image is segmented into sample clusters, each of the sample clusters comprising a first sample position to a fourth sample position, wherein one of the first samples is located at the first sample position, one of the second samples is located at the second sample position, and two third samples are located at the third sample position and the fourth sample position, respectively; wherein the apparatus is configured to derive, for each sample cluster, a first color coordinate of a first sample position of the corresponding sample cluster, a second color coordinate of a second sample position of the corresponding sample cluster, a third color coordinate of a third sample position of the corresponding sample cluster, and a fourth color coordinate of a fourth sample position of the corresponding sample cluster, The apparatus is configured to decode the image representation into the color filter array pattern image by: for the third sample position of each sample cluster, calculating a third sample at the corresponding third sample position by forming a difference between the third color coordinate at the third sample position and a first filtered color coordinate, the first filtered color coordinate being obtained by forming the third weighted sum in such a manner that the sum of weights of a third weighted sum of the first and second color coordinates at first and second sample positions adjacent to the third sample position is greater than one-half, for the fourth sample position of each sample cluster, calculating a third sample at the corresponding fourth sample position by forming a difference between the fourth color coordinate at the fourth sample position and a second filtered color coordinate, the second filtered color coordinate being obtained by forming the fourth weighted sum in such a manner that the sum of weights of a fourth weighted sum of first and second color coordinates at first and second sample positions adjacent to the fourth sample position is greater than one-half, for the first sample position of each sample cluster, calculating the first sample at the corresponding first sample position by forming a sum of the first color coordinate at the first sample position and a first filtered third sample value, the first filtered third sample value being obtained by forming a first weighted sum of third samples calculated for third and fourth sample positions adjacent to the first sample position, For the second sample position of each sample cluster, a second sample at the corresponding second sample position is calculated by forming a sum of the second color coordinate at the second sample position and a second filtered third sample value, the second filtered third sample value being derived by forming a second weighted sum of third samples calculated for third and fourth sample positions adjacent to the second sample position. 2 . The apparatus of claim 1 , wherein the weights of the third weighted sum and the fourth weighted sum consist of a first weight for weighting a first color coordinate and a second weight for weighting a second color coordinate.

3. The apparatus of claim 2, wherein the first weight is implemented as a first power of 2, and wherein the second weight is implemented as a second power of 2, the exponents of the first and second powers of 2 being integers.

4. The apparatus of claim 1 , wherein the first to fourth sample positions are arranged in an array having rows of a first type and rows of a second type, the rows of the first type each comprising first and third sample positions arranged alternately, the rows of the second type comprising second and third sample positions arranged alternately, wherein the rows of the first type and the rows of the second type are arranged alternately along columns of the array, and wherein the third sample position within a row of the first type on the one hand and the third sample position within a row of the second type on the other hand are located in different columns.

5. The apparatus of claim 4 , wherein the apparatus comprises a set of operating modes comprising at least one of an isotropic operating mode, a vertical causal operating mode, and a striped operating mode, wherein, except for sample clusters located at edges of the color filter array pattern image, the apparatus is configured to exclusively include the following to calculate one or more of the color coordinates for one of the sample positions of the corresponding sample cluster: In the isotropic mode of operation, samples at sample positions within the current row or rows in which the corresponding sample cluster is located and samples in rows above and below the current row, In the vertical causal mode of operation, samples at sample positions within the current row and samples in one or more rows above the current row, and In the stripe mode of operation, the samples at the sample positions within the current row.

6. An apparatus as claimed in claim 5, wherein the apparatus is configured to select one of the set of operating modes and wherein the apparatus is configured to derive the operating mode to be used for decoding the image representation from a data stream signaling the image representation.

7. The apparatus of claim 1, wherein the third weighted sum and the fourth weighted sum are equally weighted averages of the corresponding first and second color coordinates.

8. The apparatus of claim 1 , wherein the apparatus is configured to derive the first filtered third sample value from the first weighted sum, derive the second filtered third sample value from the second weighted sum, derive the first filtered color coordinate from the third weighted sum, and derive the second filtered color coordinate from the fourth weighted sum using a rounding operation.

9. The apparatus of claim 1 , wherein the apparatus is configured to derive the third color coordinate and the fourth color coordinate by: for the fourth sample position of each sample cluster, calculating the fourth color coordinate at the corresponding fourth sample position by forming a difference between the combined color coordinate at the fourth sample position and a filtered differential color coordinate, the filtered differential color coordinate being obtained by forming a fifth weighted sum of the differential color coordinates at third sample positions adjacent to the fourth sample position, For the third sample position of each sample cluster, the third color coordinate at the corresponding third sample position is calculated by forming a sum of the differential color coordinate at the third sample position and a filtered fourth color coordinate, the filtered fourth color coordinate being obtained by forming a sixth weighted sum of fourth color coordinates calculated for fourth sample positions adjacent to the third sample position.

10. The apparatus of claim 9, wherein the apparatus is configured to derive the filtered differential color coordinate from the fifth weighted sum and the filtered fourth color coordinate from the sixth weighted sum using a rounding operation.

11. The device of claim 9 , wherein the first to fourth sample positions are arranged in an array having rows of a first type and rows of a second type, the rows of the first type each comprising a first sample position and a third sample position arranged alternately, the rows of the second type comprising a second sample position and a third sample position arranged alternately, wherein the rows of the first type and the rows of the second type are arranged alternately along columns of the array, and wherein the third sample position within a row of the first type on the one hand and the third sample position within a row of the second type on the other hand are located in different columns, and wherein the device has an isotropic mode of operation, wherein, The third sample of the first weighted sum includes the third sample adjacent to the first sample position in the vertical and horizontal directions of the array, The third sample of the second weighted sum includes the third sample adjacent to the second sample position in the vertical and horizontal directions of the array, the first and second color coordinates of the third weighted sum include the first and second color coordinates calculated for first and second sample positions adjacent to the third sample position in vertical and horizontal directions of the array, the first and second color coordinates of the fourth weighted sum include the first and second color coordinates calculated for first and second sample positions adjacent to the four sample positions in vertical and horizontal directions of the array, the fourth color coordinate of the fifth weighted sum includes the fourth color coordinate calculated for a fourth sample position adjacent to the third sample position in a diagonal direction of the array, The differential color coordinates of the sixth weighted sum include the differential color coordinates calculated for a third sample position adjacent to the fourth sample position in a diagonal direction of the array.

12. The device of claim 9 , wherein the first to fourth sample positions are arranged in an array having rows of a first type and columns of a second type, the rows of the first type each comprising a first sample position and a third sample position arranged alternately, the rows of the second type comprising a second sample position and a third sample position arranged alternately, wherein the rows of the first type and the rows of the second type are arranged alternately along the columns of the array, and wherein the third sample position within a row of the first type on the one hand and the third sample position within a row of the second type on the other hand are located in different columns, and wherein the device has a vertical causal mode of operation, wherein, The third sample of the first weighted sum consists of the third samples adjacent to the first sample position in the vertical and horizontal directions of the array, the third samples of the second weighted sum are composed of the third samples adjacent to the second sample position in the horizontal direction and the first vertical direction of the array, wherein the sum of the weights of the second weighted sum of the third samples adjacent to the second sample position in the horizontal direction is equal to the weight of the second weighted sum of the third samples adjacent to the second sample position in the first vertical direction, the first and second color coordinates of the third weighted sum are composed of the first and second color coordinates calculated for first and second sample positions adjacent to the third sample position in vertical and horizontal directions of the array, the first and second color coordinates of the fourth weighted sum being composed of the first and second color coordinates calculated for first and second sample positions adjacent to the fourth sample position in the horizontal direction and the first vertical direction of the array, the fourth color coordinate of the fifth weighted sum is composed of the fourth color coordinate calculated for a fourth sample position adjacent to the third sample position in a diagonal direction of the array, The differential color coordinates of the sixth weighted sum are composed of the differential color coordinates calculated for a third sample position adjacent to the fourth sample position in a first diagonal direction and a second diagonal direction of the array, wherein the vertical components of the first diagonal direction and the second diagonal direction point in the same direction as the first vertical direction.

13. The device of claim 9 , wherein the first to fourth sample positions are arranged in an array having rows of a first type and rows of a second type, the rows of the first type each comprising a first sample position and a third sample position arranged alternately, the rows of the second type comprising a second sample position and a third sample position arranged alternately, wherein the rows of the first type and the rows of the second type are arranged alternately along columns of the array, and wherein the third sample position within a row of the first type on the one hand and the third sample position within a row of the second type on the other hand are located in different columns, and wherein the device has a stripe operation mode, wherein the third sample of the first weighted sum consists of the third samples adjacent to the first sample position in the horizontal direction and the second vertical direction of the array, wherein the sum of the weights of the weighted sum of the third samples adjacent to the second sample position in the horizontal direction is equal to the weight of the weighted sum of the third samples adjacent to the second sample position in the second vertical direction, The third sample of the second weighted sum consists of the third samples adjacent to the second sample position in the horizontal direction and the first vertical direction of the array, wherein the sum of the weights of the weighted sum of the third samples adjacent to the second sample position in the horizontal direction is equal to the weight of the weighted sum of the three samples adjacent to the first sample position in the first vertical direction, the first and second color coordinates of the third weighted sum being composed of the first and second color coordinates calculated for first and second sample positions adjacent to the third sample position in the horizontal direction and the second vertical direction of the array, the first and second color coordinates of the fourth weighted sum being composed of the first and second color coordinates calculated for first and second sample positions adjacent to the fourth sample position in the horizontal direction and the first vertical direction of the array, the fourth color coordinate of the fifth weighted sum is composed of the fourth color coordinate calculated for a fourth sample position adjacent to the third sample position in a third diagonal direction and a fourth diagonal direction of the array, wherein the vertical components of the third diagonal direction and the fourth diagonal direction point in the same direction as the second vertical direction, The differential color coordinates of the sixth weighted sum are composed of the differential color coordinates calculated for a third sample position adjacent to the fourth sample position in a first diagonal direction and a second diagonal direction of the array, wherein the vertical components of the first diagonal direction and the second diagonal direction point in the same direction as the first vertical direction.

14. The apparatus of claim 9, wherein the sixth weighted sum is an equal-weighted sum, wherein a sum of weights of the sixth weighted sum is equal to one-half.

15. The apparatus of claim 9, wherein the fifth weighted sum is an equally weighted average of the fourth color coordinates.

16. The apparatus of claim 1, wherein the apparatus is configured to receive an encoded representation as the image representation, and wherein the apparatus further comprises an entropy decoder configured to decode the encoded representation to obtain a quantized representation of the image representation, and The inverse quantizer is configured to obtain the first to fourth color coordinates or the first color coordinate, the second color coordinate, the differential color coordinate and the combined color coordinate through an inverse quantization process.

17. The apparatus of claim 1, wherein the apparatus is configured to receive an encoded representation as the image representation, and wherein the apparatus further comprises an entropy decoder configured to decode the encoded representation so as to obtain a quantized representation of the image representation, an inverse quantizer configured to derive a spatially decorrelated representation based on the quantized representation by an inverse quantization procedure, and An inverse spatial decorrelation stage is configured to derive the first to fourth color coordinates or the first color coordinate, the second color coordinate, the differential color coordinate and the combined color coordinate from the spatial decorrelation representation by using an inverse spatial decorrelation transform.

18. The apparatus of claim 9, wherein the apparatus is configured to receive an encoded representation as the image representation, and wherein the apparatus further comprises an entropy decoder configured to decode the encoded representation so as to obtain a quantized representation of the image representation, an inverse quantizer configured to derive a spatially decorrelated representation and the differential color coordinates based on the quantized representation by an inverse quantization procedure, and An inverse spatial decorrelation stage is configured to derive the first color coordinate, the second color coordinate, and the combined color coordinate from the spatially decorrelated representation by using an inverse spatial decorrelation transform.

19. An apparatus for encoding a color filter array pattern image into an image representation, wherein the color filter array pattern image comprises a first sample of a first channel, a second sample of a second channel, and a third sample of a third channel, wherein the color filter array pattern image is segmented into sample clusters, each of the sample clusters comprising a first sample position to a fourth sample position, wherein one of the first samples is located at the first sample position, one of the second samples is located at the second sample position, and two third samples are located at the third sample position and the fourth sample position, respectively, wherein the apparatus is configured to encode the color filter array pattern image into the image representation by: for the first sample position of each sample cluster, calculating a first color coordinate such that the first color coordinate represents a difference between the first sample at the first sample position and a first filtered third sample value, the first filtered third sample value being obtained by forming a first weighted sum of third samples adjacent to the first sample position, for the second sample position of each sample cluster, calculating a second color coordinate such that the second color coordinate represents a difference between the second sample at the second sample position and a second filtered third sample value, the second filtered third sample value being obtained by forming a second weighted sum of third samples adjacent to the second sample position, for the third sample position of each sample cluster, calculating a third color coordinate by forming a sum of the third sample at the third sample position and a first filtered color coordinate, the first filtered color coordinate being formed by forming the third weighted sum in such a manner that the sum of weights of third weighted sums of first and second color coordinates calculated for first and second sample positions adjacent to the third sample position is greater than one-half, For the fourth sample position of each sample cluster, a fourth color coordinate is calculated by forming a sum of the third sample at the fourth sample position and a second filtered color coordinate, the second filtered color coordinate being formed in such a manner that a sum of weights of a fourth weighted sum of the first and second color coordinates calculated for first and second sample positions adjacent to the fourth sample position is greater than one-half.

20. The apparatus of claim 19, wherein the weights of the third weighted sum and the fourth weighted sum consist of a first weight for weighting a first color coordinate and a second weight for weighting a second color coordinate.

21. The apparatus of claim 20, wherein the first weight is implemented as a first power of 2, and wherein the second weight is implemented as a second power of 2, the exponents of the first and second powers of 2 being integers.

22. The apparatus of claim 19 , wherein the first to fourth sample positions are arranged in an array having rows of a first type and rows of a second type, the rows of the first type each comprising first and third sample positions arranged alternately, the rows of the second type comprising second and third sample positions arranged alternately, wherein the rows of the first type and the rows of the second type are arranged alternately along the rows of the array, and wherein the third sample position within the rows of the first type on the one hand and the third sample position within the rows of the second type on the other hand are located in different columns.

23. The apparatus of claim 22 , wherein the apparatus comprises a set of operating modes comprising at least one of an isotropic operating mode, a vertical causal operating mode, and a striped operating mode, wherein, except for sample clusters located at edges of the color filter array pattern image, the apparatus is configured to exclusively include the following to calculate one or more of the color coordinates for one of the sample positions of the corresponding sample cluster: In the isotropic mode of operation, samples at sample positions within the current row or rows in which the corresponding sample cluster is located and samples in rows above and below the current row, In the vertical causal mode of operation, samples at sample positions within the current row and samples in one or more rows above the current row, and In the stripe mode of operation, the samples at the sample positions within the current row.

24. An apparatus as claimed in claim 23, wherein the apparatus is configured to select one of the set of operating modes and wherein the apparatus is configured to signal the operating mode used to obtain the image representation in a data stream, the image representation being signaled in the data stream.

25. The apparatus of claim 19, wherein the third weighted sum and the fourth weighted sum are equally weighted averages of the corresponding first and second color coordinates.

26. The apparatus of claim 19, wherein the apparatus is configured to derive the first filtered third sample value from the first weighted sum, derive the second filtered third sample value from the second weighted sum, derive the first filtered color coordinate from the third weighted sum, and derive the second filtered color coordinate from the fourth weighted sum using a rounding operation.

27. The apparatus of claim 19, wherein the apparatus is configured to use the third color coordinate and the fourth color coordinate as intermediate values ​​to for the third sample position of each sample cluster, calculating a differential color coordinate by forming a difference between the third color coordinate at the third sample position and a filtered fourth color coordinate, the filtered fourth color coordinate being obtained by forming a fifth weighted sum of fourth color coordinates calculated for fourth sample positions adjacent to the third sample position, For the fourth sample position of each sample cluster, a combined color coordinate is calculated by forming a sum of the fourth color coordinate at the fourth sample position and a filtered differential color coordinate obtained by forming a sixth weighted sum of differential color coordinates calculated for third sample positions adjacent to the fourth sample position.

28. The apparatus of claim 27, wherein the apparatus is configured to derive the filtered differential color coordinate from the fifth weighted sum and the filtered fourth color coordinate from the sixth weighted sum using a rounding operation.

29. The device of claim 27 , wherein the first to fourth sample positions are arranged in an array having rows of a first type and rows of a second type, the rows of the first type each comprising a first sample position and a third sample position arranged alternately, the rows of the second type comprising a second sample position and a third sample position arranged alternately, wherein the rows of the first type and the rows of the second type are arranged alternately along columns of the array, and wherein the third sample position within a row of the first type on the one hand and the third sample position within a row of the second type on the other hand are located in different columns, and wherein the device has an isotropic mode of operation, wherein The third sample of the first weighted sum includes the third sample adjacent to the first sample position in the vertical and horizontal directions of the array, The third sample of the second weighted sum includes the third sample adjacent to the second sample position in the vertical and horizontal directions of the array, the first and second color coordinates of the third weighted sum include the first and second color coordinates calculated for first and second sample positions adjacent to the third sample position in vertical and horizontal directions of the array, the first and second color coordinates of the fourth weighted sum include the first and second color coordinates calculated for first and second sample positions adjacent to the fourth sample position in vertical and horizontal directions of the array, the fourth color coordinate of the fifth weighted sum includes the fourth color coordinate calculated for a fourth sample position adjacent to the third sample position in a diagonal direction of the array, The differential color coordinates of the sixth weighted sum include the differential color coordinates calculated for a third sample position adjacent to the fourth sample position in a diagonal direction of the array.

30. The device of claim 27 , wherein the first to fourth sample positions are arranged in an array having rows of a first type and rows of a second type, the rows of the first type each comprising a first sample position and a third sample position arranged alternately, the rows of the second type comprising a second sample position and a third sample position arranged alternately, wherein the rows of the first type and the rows of the second type are arranged alternately along columns of the array, and wherein the third sample position within a row of the first type on the one hand and the third sample position within a row of the second type on the other hand are located in different columns, and wherein the device has a vertical causal mode of operation, wherein, The third sample of the first weighted sum consists of the third samples adjacent to the first sample position in the vertical and horizontal directions of the array, the third samples of the second weighted sum are composed of the third samples adjacent to the second sample position in the horizontal direction and the first vertical direction of the array, wherein the sum of the weights of the second weighted sum of the third samples adjacent to the second sample position in the horizontal direction is equal to the weight of the second weighted sum of the third samples adjacent to the second sample position in the first vertical direction, the first and second color coordinates of the third weighted sum are composed of the first and second color coordinates calculated for first and second sample positions adjacent to the third sample position in vertical and horizontal directions of the array, the first and second color coordinates of the fourth weighted sum being composed of the first and second color coordinates calculated for first and second sample positions adjacent to the fourth sample position in the horizontal direction and the first vertical direction of the array, the fourth color coordinate of the fifth weighted sum is composed of the fourth color coordinate calculated for a fourth sample position adjacent to the third sample position in a diagonal direction of the array, The differential color coordinates of the sixth weighted sum are composed of the differential color coordinates calculated for a third sample position adjacent to the fourth sample position in a first diagonal direction and a second diagonal direction of the array, wherein the vertical components of the first diagonal direction and the second diagonal direction point in the same direction as the first vertical direction.

31. The device of claim 27 , wherein the first to fourth sample positions are arranged in an array having rows of a first type and rows of a second type, the rows of the first type each comprising a first sample position and a third sample position arranged alternately, the rows of the second type comprising a second sample position and a third sample position arranged alternately, wherein the rows of the first type and the rows of the second type are arranged alternately along columns of the array, and wherein the third sample position within a row of the first type on the one hand and the third sample position within a row of the second type on the other hand are located in different columns, and wherein the device has a stripe mode of operation, wherein the third sample of the first weighted sum consists of the third samples adjacent to the first sample position in the horizontal direction and the second vertical direction of the array, wherein the sum of the weights of the weighted sum of the third samples adjacent to the second sample position in the horizontal direction is equal to the weight of the weighted sum of the third samples adjacent to the second sample position in the second vertical direction, the third samples of the second weighted sum are composed of the third samples adjacent to the second sample position in the horizontal direction and the first vertical direction of the array, wherein the sum of the weights of the weighted sum of the third samples adjacent to the second sample position in the horizontal direction is equal to the weight of the weighted sum of the third samples adjacent to the first sample position in the first vertical direction, the first and second color coordinates of the third weighted sum being composed of the first and second color coordinates calculated for first and second sample positions adjacent to the third sample position in the horizontal direction and the second vertical direction of the array, the first and second color coordinates of the fourth weighted sum being composed of the first and second color coordinates calculated for first and second sample positions adjacent to the fourth sample position in the horizontal direction and the first vertical direction of the array, the fourth color coordinate of the fifth weighted sum is composed of the fourth color coordinate calculated for a fourth sample position adjacent to the third sample position in a third diagonal direction and a fourth diagonal direction of the array, wherein the vertical components of the third diagonal direction and the fourth diagonal direction point in the same direction as the second vertical direction, The differential color coordinates of the sixth weighted sum are composed of the differential color coordinates calculated for a third sample position adjacent to the fourth sample position in a first diagonal direction and a second diagonal direction of the array, wherein the vertical components of the first diagonal direction and the second diagonal direction point in the same direction as the first vertical direction.

32. The apparatus of claim 27, wherein the sixth weighted sum is an equal-weighted sum, wherein a sum of weights of the sixth weighted sum is equal to one-half.

33. The apparatus of claim 27, wherein the fifth weighted sum is an equally weighted average of the fourth color coordinates.

34. The apparatus of claim 19, wherein the color filter array image pattern is obtained from a plurality of sensor assemblies, each of the sensor assemblies being associated with one of the first channel, the second channel, and the third channel, wherein the sensor assemblies associated with a common channel are configured to detect light within a common wavelength range specific to the channel.

35. The apparatus of claim 19, further comprising: a quantizer configured to derive a quantized representation based on the first to fourth color coordinates or based on the first color coordinate, the second color coordinate, the differential color coordinate, and the combined color coordinate, and An entropy encoder is configured to encode the quantized representation to obtain an encoded representation.

36. A device as described in claim 35, wherein the device further comprises a spatial decorrelation stage, which is configured to derive a spatially decorrelated representation based on the first to fourth color coordinates or based on the first color coordinates, the second color coordinates, the differential color coordinates and the combined color coordinates by using a spatial decorrelation transform, and wherein the quantizer is configured to derive the quantized representation from the spatially decorrelated representation.

37. The apparatus of claim 27, further comprising: a spatial decorrelation stage configured to derive a spatially decorrelated representation based on the first color coordinate, the second color coordinate and the combined color coordinate by using a spatial decorrelation transform, a quantizer configured to derive a quantized representation from the spatially decorrelated representation and the differential color coordinates, and An entropy encoder is configured to encode the quantized representation to obtain an encoded representation.

38. A method for decoding an image representation into a color filter array pattern image, wherein the color filter array pattern image comprises a first sample of a first channel, a second sample of a second channel, and a third sample of a third channel, wherein the color filter array pattern image is segmented into sample clusters, each of the sample clusters comprising a first sample position to a fourth sample position, wherein one of the first samples is located at the first sample position, one of the second samples is located at the second sample position, and two third samples are located at the third sample position and the fourth sample position, respectively; wherein the method comprises deriving, for each sample cluster, a first color coordinate of the first sample position of the corresponding sample cluster, a second color coordinate of the second sample position of the corresponding sample cluster, a third color coordinate of the third sample position of the corresponding sample cluster, and a fourth color coordinate of the fourth sample position of the corresponding sample cluster, The method comprises decoding the image representation into the color filter array pattern image by: for the third sample position of each sample cluster, calculating the third sample at the corresponding third sample position by forming a difference between the third color coordinate at the third sample position and a first filtered color coordinate, the first filtered color coordinate being obtained by forming the third weighted sum in such a manner that the sum of weights of a third weighted sum of the first and second color coordinates at first and second sample positions adjacent to the third sample position is greater than one-half, for the fourth sample position of each sample cluster, calculating the third sample at the corresponding fourth sample position by forming a difference between the fourth color coordinate at the fourth sample position and a second filtered color coordinate, the second filtered color coordinate being obtained by forming the fourth weighted sum in such a manner that the sum of weights of a fourth weighted sum of first and second color coordinates at first and second sample positions adjacent to the fourth sample position is greater than one-half, for the first sample position of each sample cluster, calculating the first sample at the corresponding first sample position by forming a sum of the first color coordinate at the first sample position and a first filtered third sample value, the first filtered third sample value being obtained by forming a first weighted sum of third samples calculated for third and fourth sample positions adjacent to the first sample position, For the second sample position of each sample cluster, a second sample at the corresponding second sample position is calculated by forming a sum of the second color coordinate at the second sample position and a second filtered third sample value, the second filtered third sample value being derived by forming a second weighted sum of third samples calculated for third and fourth sample positions adjacent to the second sample position.

39. A method for encoding a color filter array pattern image into an image representation, wherein the color filter array pattern image comprises a first sample of a first channel, a second sample of a second channel, and a third sample of a third channel, wherein the color filter array pattern image is segmented into sample clusters, each of the sample clusters comprising a first sample location to a fourth sample location, wherein one of the first samples is located at the first sample location, one of the second samples is located at the second sample location, and two third samples are located at a third sample location and a fourth sample location, respectively, wherein the method comprises encoding the color filter array pattern image into the image representation by: for a first sample position of each sample cluster, calculating a first color coordinate such that the first color coordinate represents a difference between the first sample at the first sample position and a first filtered third sample value, the first filtered third sample value being obtained by forming a first weighted sum of third samples adjacent to the first sample position, for the second sample position of each sample cluster, calculating a second color coordinate such that the second color coordinate represents a difference between the second sample at the second sample position and a second filtered third sample value, the second filtered third sample value being obtained by forming a second weighted sum of third samples adjacent to the second sample position, for the third sample position of each sample cluster, calculating a third color coordinate by forming a sum of the third sample at the third sample position and a first filtered color coordinate, the first filtered color coordinate being formed by forming the third weighted sum in such a manner that the sum of weights of third weighted sums of first and second color coordinates calculated for first and second sample positions adjacent to the third sample position is greater than one-half, For the fourth sample position of each sample cluster, a fourth color coordinate is calculated by forming a sum of the third sample at the fourth sample position and a second filtered color coordinate, the second filtered color coordinate being derived from the fourth weighted sum in such a manner that a sum of weights of a fourth weighted sum of the first and second color coordinates calculated for first and second sample positions adjacent to the fourth sample position is greater than one-half.

40. A non-transitory storage medium comprising a computer program for implementing the method of claim 38 or 39 when executed on a computer or a signal processor.

Images