Method and apparatus for performing color image compression - Patents.com

Abstract

The present invention relates to a method for compressing a color image into a packed data set, the color image comprising color image samples arranged on a reference grid according to a sampling pattern, the method comprising the steps of (a) performing a decorrelation transform to provide component images, (b) performing a number of discrete wavelet decompositions on said component images, and (c) forming a packet of the packed data set by processing each successive row of the reference grid and each component and grouping all samples of said component of said row into one or more packets, and adding said one or more packets to a packed data set. The present invention also relates to a device for compression, a computer program for compression, a method for decompression, a device for decompression and a computer program for decompression. The present invention also relates to a packed data set generated by the method for compression.

Description

[Technical field]

[0001] The present invention relates to a method, device and computer program for compressing and decompressing color images, and more particularly to the packetization of samples into packed data sets. The present invention also relates to packed data sets generated by said method for compression. [Background technology]

[0002] Color image sensors that produce color images are known in the industry. Video bandwidth requirements are growing rapidly due to a constantly increasing number of video resolutions, frame rates and streams to be managed. Currently, a new standard for low-latency and lightweight image coding with visually lossless quality, the JPEG XS standard (formally known as ISO / IEC JT1 / SC29 / WG1), is being defined by the JPEG committee, JPEG XS: A low-latency, lightweight image coding system - Part 1: Core coding system (hereafter, Reference 1) (ISO / IEC JTC 1 / SC 29 / WG 1 N91029-91st Meeting-Online-April 2021) JPEG XS: A low-latency, lightweight image coding system - Part 2: Profiles and buffer models (hereafter, Reference 2) (ISO / IEC JTC 1 / SC 29 / WG 1 N92027 92nd Meeting - Online - July 2021) This is explained in. The JPEG XS standard defines a progression order that is formally defined in sub-clause B.7 of Annex B of Reference 1. This progression order is called "Resolution-Line-Band-Component" (RLBC). Note that "bands" in the JPEG XS standard are called "subbands" in this document. This means that the subbands are serialized first component-by-component, then subband-by-subband, then subband-line-by-subband, then resolution-by-resolution. Resolution levels in the JPEG XS standard are, in this document, equivalent to decomposition levels. The lowest resolution level is numbered 0. Resolution levels increase by one for each bidirectional (i.e. vertical and horizontal) inverse DWT (Discrete Wavelet Transform) decomposition. The document “RICHTER THOMAS ET AL: “Bayer CFA Pattern Compression With JPEG XS”, IEEE TRANSACTIONS ON IMAGE PROCESSING, IEEE, USA, vol. 30, 16 July 2021, pages 6557-6569” is an academic paper that explains in an informal manner the contents defined in the JPEG XS standard as defined in References 1 and 2 above.

[0003] The document WO2021084118 discloses an image processor for processing an image comprising pixels arranged in a Bayer pattern and providing a highly decorrelated image.

[0004] The document US9332258 discloses a method and device for indicating stream compression, in which coefficients are grouped and for each group a maximum coded row index (GCLI) is determined and only the GCLI least significant bit of the coefficient is copied to the output stream together with the value of the GCLI. GCLI is also known in the art as bit plane count. Summary of the Invention

[0005] It is an object of the present invention to provide a more efficient method for compressing and decompressing color images.

[0006] The invention is defined by the independent claims. The dependent claims define advantageous embodiments.

[0007] According to a first aspect of the invention, there is provided a method for compressing a colour image, the colour image comprising colour image samples Sij arranged in a reference grid according to a sampling pattern, the reference grid comprising a number of rows and columns, the row index i starting at 0 and increasing with increasing rows and the column index j starting at 0 and increasing with increasing columns resulting in a packed data set, the method comprising: a) performing a decorrelation transformation, in which for each position (i,j) of a reference grid a component sample Ckij is determined, where k is a component index depending on the sampling pattern and Ckij is a linear combination of image samples Si'j' at position (i',j'), where i' is included between i-4 and i+4 and j' is included between j-4 and j+4, and component images C0, C1, C2 and, depending on the sampling pattern, an optional component image C3 are generated; b) repeating the generation of decomposed component images for each component image Ck, Nlky is the number of vertical DWT decompositions applied to a component image Ck, an integer ≥ 0; Nlkx is the number of horizontal DWT decompositions applied to a component image Ck, an integer ≧Nlky, - considering the component images Ck as auxiliary subbands; - If Nlky>0, for n=0 to Nlky-1, - performing a vertical DWT decomposition on the auxiliary subbands by performing a lifting scheme with a (5,3) filter bank on the sequence of the auxiliary subbands, where the resulting even samples are the LLn subbands and the resulting odd samples are the LHn subbands, and where for each of the resulting samples a reference is maintained to the position (i,j) of the reference grid from which they were determined; - performing a horizontal DWT decomposition in the LLn subband by performing a lifting scheme with a (5,3) filter bank on a row of the LLn subband, where the resulting even samples are in the LLn subband and the resulting odd samples are in the HLn subband, and for each of the resulting samples a reference is maintained to the location (i,j) of the reference grid from which they were determined; - performing a horizontal DWT decomposition in the LHn subbands by performing a lifting scheme with a (5,3) filter bank on rows of the LHn subbands, where the resulting even samples are the LHn subbands and the resulting odd samples are the HHn subbands, and where for each of the resulting samples a reference is maintained to the location (i,j) of the reference grid from which they were determined; - considering subband LLn as an auxiliary subband; Repeating the above steps, - If Nlkx>Nlky, for n=Nlky~Nlkx-1, - performing a horizontal DWT decomposition in the LLn-1 subband by performing a lifting scheme with a (5,3) filter bank on rows of the LLn-1 subband, where the resulting even samples are the LLn subband and the resulting odd samples are the HLn subband, and for each of the resulting samples a reference is maintained to the position (i,j) of the reference grid from which they were determined; - considering subband LLn as an auxiliary subband; Repeating the above and In accordance with the present invention, c) For each successive row i of the reference grid, For each component k, (i) grouping all samples of said component k of said row i into one or more packets; (ii) adding said one or more packets to a packed data set; A method is provided, comprising: In processing step c), the method processes all rows of the reference grid in order of increasing row index i. For a given row, all components may be processed in any order, provided that this order is the same when compressing and when decompressing the packed data set. This order may be in order of increasing component index k. In the context of the present invention: - Packetization is the formation and addition of packets to a packed data set. - the progression order results from the way component samples are grouped into packets and the way said packets are added to the packed data set. The inventive progression order results from step c) of the inventive method for compressing. The progression order of the JPEG XS standard is defined by subclause B.7 of reference 1. a packed data set is a data set that can be obtained by compressing a color image using the method for compressing of the present invention, the packed data set having the following properties that can be deduced from the method for compressing of the present invention, namely: - each packet contains only data from a single decomposed component image; and - each packet contains only data from a single row of a decomposed component image; and - all packets containing data from a given row of the decomposed component image are located in the packed data set before packets containing data from a subsequent row of the decomposed component image; The sequence includes a sequence of packets having Determining whether the original dataset is a packed dataset according to the invention is performed as follows. - Applying the method for decompression of the present invention on the original data set to generate an uncompressed data set. applying the method for compression of claim 1 on an uncompressed data set to generate a packed data set. If the packed dataset matches the original dataset, then this original dataset is the packed dataset according to the invention. In an embodiment of the invention, the packed data set may only contain packets as formed and assembled in step c) of the method for compressing. In this embodiment, the device for compressing and the device for decompressing may be configured with the same operating parameters, such as image size, number of vertical and horizontal DWT decompositions, type of decorrelation transform, sampling pattern and others. In another embodiment, the packed data set may contain metadata specifying the operating parameters used for compression in addition to the packets formed in step c) of the method for compressing. Such metadata may be located in one header associated with the entire packed data set, in several headers each associated with one packet of the packed data set, or both. Grouping all samples of component k located in row i of the reference grid into one or more packets may include considering all samples of component k located in row i (if they exist) and forming one or more new packets with these samples, where said one or more new packets are not yet present in the packed data set, and each sample is assigned one-by-one to a packet. If there are no samples of component k located in row i, no new packet is formed for this component k of this row i. Step c) of the method for compressing according to the invention is expressed in pseudocode. The following is an equivalent expression of the same step c) of the method in a more everyday language: For each successive row i of the reference grid (i.e. for row index i from row index 0 to the row index of the last row), a first iterative loop is executed. For each execution of this first loop, a second iterative loop is executed for each component k (i.e. for each value of component index k). For each execution of this second loop, two steps are executed successively: (i) first, all samples of the current component k located in the current row i of the reference grid are grouped into one or more packets not yet present in the packed data set, each sample is assigned one by one to a packet, and (ii) said one or more packets formed in step (i) are then added to the packed data set. The fact that the packets formed in step (i) are not yet present in the packed data set can be inferred from the method for compressing according to the invention. If there is no sample of the current component k located in the current row i, then no new packet is formed in method step (i) and therefore no packet is added to the packed data set in method step (ii). Subbands, component images and decomposed component images consist of even rows of even and odd samples followed by odd rows of even and odd samples. When performing decorrelation transform and lifting scheme operations on pixel locations near an image border, if a sample outside the image border is required to perform the operation, the value of a sample at the mirror position of the missing sample relative to the image border may be substituted. When performing a lifting scheme operation, allocating low frequency samples to even locations and high frequency samples to odd locations is equivalent to allocating high frequency samples to even locations and low frequency samples to odd locations. When performing a bidirectional (i.e., vertical and horizontal) DWT decomposition, performing a vertical DWT decomposition followed by a horizontal DWT decomposition is equivalent to performing a horizontal DWT decomposition followed by a vertical DWT decomposition. - when performing the decorrelation transformation of step a), linear combinations can be taken from the nearest pixels (i' comprised between i-1 and i+1 and j' comprised between j-1 and j+1), from further neighbouring pixels (i' comprised between i-2 and i+2 and j' comprised between j-2 and j+2) or from even further neighbouring pixels (i' comprised between i-3 and i+3 and j' comprised between j-3 and j+3).

[0008] In a preferred embodiment, the number of vertical DWT decompositions Nlky is equal to 0 or 1.

[0009] Between steps b) and c) entropy coding of the data may be performed.

[0010] The entropy encoding may include entropy encoding of bit-plane counts of a group of samples.

[0011] The steps of grouping bit-plane counts of groups of samples and grouping data representing the bit-planes may be performed.

[0012] In the sampling pattern, each pixel location may include samples arranged in a Bayer pattern that juxtaposes a red pixel R at position (i,j), a first green pixel G1 at position (i,j+1), a second green pixel G2 at position (i+1,j), and a blue pixel B at position (i+1,j+1) for each even i and j coordinate of a reference grid, and the decorrelation transform generates a component image C0 for the red pixel location, a component image C1 for the first green pixel location, a component image C2 for the second green pixel location, and a component image C3 for the blue pixel location.

[0013] In the sampling pattern, each pixel location may include samples arranged in a 4:4:4 pattern, where for each i and j coordinate of a reference grid, a luminance pixel Y, a blue chrominance pixel Cb, and a red chrominance pixel Cr are superimposed at location (i,j), or a red pixel R, a green pixel G, and a blue pixel B are superimposed at location (i,j), and the decorrelation transform generates a component image C0 for the luminance or red pixel locations, a component image C1 for the blue chrominance or green pixel locations, and a component image C2 for the red chrominance or blue pixel locations.

[0014] In the sampling pattern, each pixel location may include samples arranged in a 4:2:2 pattern such that for each i and every even j coordinate of the reference grid, a luminance pixel Y, a blue chrominance pixel Cb, and a red chrominance pixel Cr are located at location (i,j), and a luminance pixel Y is located at location (i,j+1), and the decorrelation transform generates a component image C0 for the luminance pixel locations, a component image C1 for the blue chrominance pixel locations, and a component image C2 for the red chrominance pixel locations.

[0015] The decorrelation transform may be an identity mapping, where each sample in the color image is mapped to a corresponding location on the component images.

[0016] The decorrelation transform can also be an in-line or full Star-Tetrix transform. In-line and full Star-Tetrix transforms are described in Appendix F of Reference 1.

[0017] According to a second aspect of the invention, a device for compressing a colour image, the colour image comprising colour image samples Sij arranged in a reference grid according to a sampling pattern, the reference grid comprising a number of rows and columns, the row index i starting at 0 and increasing with increasing rows and the column index j starting at 0 and increasing with increasing columns resulting in a packed data set, a) an electronic component configured to perform step a) of the compression method; b) an electronic component configured to perform step b) of the compression method; c) an electronic component configured to perform step c) of the compression method; A device is provided that includes:

[0018] According to a third aspect of the present invention there is provided a computer program comprising instructions which, when executed by a computer, cause the computer to perform the steps of the method for compressing of the present invention.

[0019] According to a fourth aspect of the present invention, in a packed data set obtainable by compressing a color image using the method for compression of the present invention, the following properties are satisfied: - each packet contains only data from a single decomposed component image; and - each packet contains only data from a single row of a decomposed component image; and - all packets containing data from a given row of the decomposed component image are located in the packed data set before packets containing data from a subsequent row of the decomposed component image; A packed data set is provided that includes a sequence of packets having:

[0020] According to a fifth aspect of the present invention, there is provided a method for decompressing a packed data set obtainable by executing a method for compressing according to the present invention, comprising the steps of: - unpacking packets contained in the packed data set; - performing an inverse DWT decomposition; - performing an inverse decorrelation transform; A method is provided, comprising:

[0021] According to a sixth aspect of the present invention, there is provided a device for decompressing a packed data set obtainable by executing a method for compressing according to the present invention, comprising: - unpacking packets contained in the packed data set; - performing an inverse DWT decomposition; - performing an inverse decorrelation transform; A device is provided that includes an electronic component configured to perform the steps of:

[0022] According to a seventh aspect of the present invention there is provided a computer program comprising instructions which, when executed by a computer, cause the computer to carry out the steps of the method for decompressing of the present invention. [Brief description of the drawings]

[0023] These and further aspects of the invention will now be described in further detail, by way of example, with reference to the accompanying drawings in which:

[0024] [Figure 1] FIG. 1 shows a schematic representation of a portion of a color image, where the sampling pattern is the Bayer pattern. [Figure 2a] FIG. 2a represents diagrammatically the component images resulting from the decorrelation transformation of the color image of FIG. 1, with each component sample Ckij represented at the position (i,j) of the reference grid from which it is determined. [Figure 2b]FIG. 2b represents diagrammatically component images resulting from a decorrelation transform of the color image of FIG. 1, where the component image samples are grouped in the component images. [Diagram 3] FIG. 3 shows diagrammatically parts of the same component images as FIG. 2, with a single vertical DWT decomposition applied to component images C0, C1 and C3, but no decomposition applied to component image C2, and with each component sample Ckij represented at position (i,j) of the reference grid from which it was determined. [Figure 4] Figure 4 shows diagrammatically parts of the same component images as Figure 3, with a single horizontal DWT decomposition applied to component images C0, C1 and C3, but no decomposition applied to component image C2, and with each component sample Ckij represented at position (i,j) of the reference grid from which it was determined. [Diagram 5] FIG. 5 shows generally the same portions of component images as FIG. 3, with a single horizontal DWT decomposition applied to component images C0, C1 and C3, but no decomposition applied to component image C2, the component image samples are grouped together in the component images, the samples are grouped by subband within each component image, and only the data for component image C0 is shown. [Figure 6] FIG. 6 shows a schematic representation of how the lifting scheme is performed using a (5,3) filter bank on rows of samples with a DWT decomposition. [Figure 7] FIG. 7 illustrates diagrammatically the subbands generated after applying five horizontal DWT decompositions without applying any vertical DWT decomposition of the component images. [Figure 8]FIG. 8, like FIG. 7, illustrates diagrammatically the subbands in the case where component images C0, C1, C3 are decomposed and component image C2 is not decomposed, as well as numbered arrows separating samples for the formation of packets and serialization of those packets in a packed data set in accordance with the invention. [Figure 9] FIG. 9 illustrates diagrammatically the subbands generated after one application of a vertical DWT decomposition followed by five applications of a horizontal DWT decomposition of the component images. [Figure 10] FIG. 10, like FIG. 9, illustrates diagrammatically the subbands in the case where component images C0, C1, C3 are decomposed and component image C2 is not decomposed, as well as numbered arrows separating samples for the formation of packets and serialization of those packets in a packed data set in accordance with the invention. [Figure 11] FIG. 11, like FIG. 9, illustrates diagrammatically the sub-bands in the case where component images C0, C1, C3 are decomposed and component image C2 is not decomposed, as well as numbered arrows separating samples for the formation of packets and their serialization in a packed data set, according to the JPEG XS standard. [Figure 12] FIG. 12 shows a schematic representation of the subbands generated for a three-component image with a 4:4:4 sampling pattern, where one vertical DWT decomposition and five horizontal DWT decompositions are performed for each component image, and illustrates how packets are formed and serialized in a packed data set according to the JPEG XS standard (top) and the present invention (bottom). [Figure 13] FIG. 13 shows a schematic representation of the subbands generated for a three-component image with a 4:2:2 sampling pattern, where one vertical DWT decomposition and five horizontal DWT decompositions are performed for each component image, and illustrates how packets are formed and serialized in a packed data set according to the JPEG XS standard (top) and the present invention (bottom). [Figure 14] FIG. 14 illustrates a device for compressing a color image. [Figure 15] FIG. 15 depicts a device for decompressing color images. [Figure 16] Figure 16a is a panel which diagrammatically represents the grouping of samples from the first four rows of the image of Figure 4 in packets, and Figure 16b is a panel which diagrammatically represents the order in which said packets are added to a packed data set according to the progression order of the present invention. [Figure 17] Figure 17a is a panel which illustrates diagrammatically the grouping of samples from the first four rows of the image of Figure 4 in packets, and Figure 17b is a panel which illustrates diagrammatically the order in which said packets are added to the packed data set according to the progression order of the JPEG XS standard. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0025] FIG. 1 is a schematic example of a color image obtained from an image sensor with a Bayer pattern. The reference grid represented by a square includes 8 columns (column index j from column index 0 to column index 7) and 6 rows (row index i from row index 0 to row index 5), with the upper left corner having coordinates (0,0). Each pixel position (i,j) may include a red, a first green, a second green or a blue sample according to the Bayer pattern. For example, reference number 1 represents a blue sample at reference grid position (1,1) and reference number 3 represents a red sample at reference grid position (2,4). In a real image, the number of columns and rows may be larger (e.g. 4096×2160).

[0026] Fig. 2a is a representation of the data obtained after a decorrelation transformation, in which the red, first green, second green and blue image samples are replaced by samples of component images C0, C1, C2, C3, respectively. As depicted in Fig. 2a, the decorrelation transformation may generate a grid of component samples of the same dimensions as the reference grid. The transformation resulting in a given component sample is centered on the corresponding color image sample. Thus, the spatial arrangement of the input samples is preserved through the transformation. The following is a non-exhaustive list of such decorrelation transforms: - Identity mapping, where each sample in the input grid is mapped to a corresponding position in the output grid (i.e., for any position (i,j) in the input / output grid, C0ij=Rij, C1ij=G1ij, C2ij=G2ij, C3ij=Bij). - Inline Star-Tetrix conversion - Complete Star-Tetrix conversion The inline and full Star-Tetrix transforms are described in Annex F of Reference 1. In Fig. 2a, the samples of the component images are located at the positions of the corresponding color image samples of the reference grid. In Fig. 2b, the four distinct component images C0, C1, C2, C3 are represented without reference to the reference grid. However, knowledge of the sampling pattern and the decorrelation transform allows one to determine the reference grid position (i,j) from which a given component sample of the component images is derived. For example, reference number 2 of sample C2 in Fig. 2b corresponds to the sample of component image C2 at reference grid position (3,2) in Fig. 2a.

[0027] FIG. 3 is a representation of the data obtained after one application of the vertical DWT decomposition on component images C0, C1, C3, but no decomposition on component image C2. For component images C0, C1, C3, subbands LL0 and LH0 are created. For component images C0, C1, subband LL0 appears in grid rows 0, 4, and subband LH0 appears in grid row 2. For component image C3, these subbands appear one row below. For component image C2, no DWT decomposition is performed, so the original samples of component image C2 are preserved. The subscripts mean: LL=low frequency subband, LH=high frequency subband, E=even row, O=odd row.

[0028] FIG. 4 is a representation of the data obtained after one horizontal DWT decomposition and one vertical DWT decomposition have been applied on component images C0, C1, C3 (i.e., after one horizontal DWT decomposition has been applied on component images C0, C1, C3 in the data of FIG. 3), but no decomposition has been applied on component image C2. The LL0 subband from each component image is decomposed into new LL0 and HL0 subbands. The LH0 subband from each component image is decomposed into new LH0 and HH0 subbands. Each sample is located at position (i,j) of the reference grid from which it was determined.

[0029] FIG. 5 is a representation of the data obtained after one horizontal DWT decomposition and one vertical DWT decomposition have been applied on component images C0, C1, C3 (i.e., one horizontal DWT decomposition has been applied on component images C0, C1, C3 on the data of FIG. 3), but no decomposition has been applied on component image C2, where first the samples are grouped by component, and then all samples from a given component are grouped by subband. In this figure, only the data for component 0 is represented. The LL0 subband appears in the four positions in the top left corner. The HL0 subband appears in the four positions in the top right corner. The LH0 subband appears on the left side of the bottom row, and the HH0 subband appears on the right side of the bottom row. Similar representations can be derived for components 1, 3. For the undecomposed component 2, the image is the C2 part of FIG. 2b. FIG. 5 presents the decomposed component image for component 0.

[0030] When performing the decorrelation transform in method step a) and the successive DWT decomposition in method step b), it is required that the samples resulting from the transform and the decomposition maintain a reference to the positions (i,j) of the reference grid from which they are determined. This can be achieved in different ways, and three possible embodiments are described below. - in a first embodiment, each sample resulting from the transform or decomposition can be stored together with the value (i,j) of the position of the reference grid from which it is determined. This is done for each operation of the decorrelation transform and for each successive DWT decomposition. In a second embodiment, the resulting samples are stored grouped per component and per subband, as represented in FIG. - in a third embodiment, the samples resulting from the transformation or decomposition are stored in an array of positions (i,j) of the reference grid from which they are determined (i.e. in the same row and column coordinates as the original data). Said array may be of the same dimension as the original color image or may be the original color image itself, the transformation and decomposition being so-called "in-place" transformation and decomposition. Such arrays of the same dimension as the original color image are represented in figures 2a, 3, 4 and 16a. According to the above embodiment, step c) of the method is performed as follows. In a first embodiment, step c) of the method is carried out by extracting from the storage a sample corresponding to each successive row and each component of the reference grid. In a second embodiment, step c) of the method is performed based on an address conversion between addresses in the storage or on the position (i,j) of the reference grid. For example, if the color image is the image of FIG. It can be determined that the pixel at position (1,1) of the color image (reference number 1 in FIG. 1) corresponds to the 30th sample from the start of the component image of FIG. 2b (reference number 2 in FIG. 2b); For the storage of the subbands in FIG. 5, a similar correspondence can be determined: the pixel at position (2,4) of the color image (reference number 3 in FIG. 1) corresponds to the sample at position (2,4) of the image after DWT decomposition (reference number 4 in FIG. 4, sample of component image C0 of subband LH0) and also to the sample at position (2,1) of the subband image for component image C0 (reference number 5 in FIG. 5). In a third embodiment, step c) of the method is performed by scanning the storage row by row, for example if the colour image is the image of figure 1, step c) can be performed by scanning the array of figure 4 row by row.

[0031] Figure 16a provides a representation of data obtained after applying a series of image transforms on an exemplary color image comprising color image samples obtained from an image sensor having a Bayer pattern, arranged on a reference grid, as shown in the first four rows of the image in Figure 1. The series of image transforms includes: a) a decorrelation transformation on said color image, as performed in step a) of the method for compressing according to the invention, generating component images C0, C1, C2, C3, as shown in the first four rows of the image in FIG. 2a, b) one vertical DWT decomposition and one horizontal DWT decomposition on the component images C0, C1, C3 (but no decomposition on the component image C2) as performed in step b) of the method for compressing of the present invention, generating decomposed component images as shown in the first four rows of images in FIG. Includes. Each sample is located at position (i,j) of the reference grid from which it is determined. Figures 16a and 16b together provide a representation of the execution of step c) of the method for compressing of the invention, with Figure 16a showing step (i) and Figure 16b showing step (ii). c) For each successive row i of the reference grid, For each component k, (i) grouping all samples of said component k of said row i into one or more packets; (ii) adding said one or more packets to the packed data set; First, the first row of the reference grid is scanned. Four samples of component 0, C0_LL0, C0_HL0, C0_LL0, C0_HL0, are found and grouped into one or more packets. In this example, a single packet, packet 0, is formed and added to the packed data set. Then, four samples of component 1 are found in the first row and processed in the same way to form packet 1. No other components are found in the first row of the reference grid. Then, the second row of the reference grid is scanned and packets 2 and 3 are formed in the same way from the samples of components 2 and 3. This is done until all rows of the reference grid have been processed and a packed data set containing eight packets is obtained. The order in which the packets are formed and added to the packed data set is according to the inventive progression order, which is to say that for each successive row i of the reference grid, for each component k, group all samples of said component k in said row i into one or more packets and add said one or more packets to the packed data set. For each row of the reference grid, the filled background of a cell of the reference grid indicates the packet that the sample associated with the cell is grouped in. The numbers associated with the packets indicate the order in which the packets are serialized in the packed data set, with packets with lower associated numbers being added earlier in the packed data set. This example applies to a four-component image with four rows and eight columns, where one vertical DWT decomposition and one horizontal DWT decomposition are performed on three component images, and no decomposition is performed on one component image. Other examples in which images have a different number of components, rows or columns, and where a different number of vertical or horizontal DWT decompositions are performed, will exhibit similar characteristics, including differences from the JPEG XS standard discussed below. In this example, step (i) of the method generates a single packet for each component k present in each row i of the reference grid. In another example of the method, step (i) may generate multiple packets for each component k present in each row i of the reference grid. For example, packet 0 in Figure 16a may be replaced by two packets, with the first packet containing samples C0_LL0, C0_LL0 and the second packet containing samples C0_HL0, C0_HL0. Figure 16a provides an illustration of the inventive progression order comparable to Figure 10, with the notable difference that in Figure 10, the component images C0, C1, C3 are decomposed by one vertical DWT decomposition and five horizontal DWT decompositions, whereas in Figure 16a, they are decomposed by one vertical DWT decomposition and one horizontal DWT decomposition. Another difference is that in Figure 10, the samples are grouped by component and by subband, whereas in Figure 16a, the samples are represented on a reference grid.

[0032] Figure 17a provides a representation of the data obtained after applying the same image transformation on the same colour image as represented in Figure 16a, where each sample is located at position (i,j) of the reference grid from which it is determined. Figure 17 further provides a representation of the grouping of the data into packets (Figure 17a) and the order in which the packets are added to the packed data set (Figure 17b) according to the progression order of the JPEG XS standard, as adapted from Table B.10 of Reference 1. The difference between Table B.10 from Reference 1 and Figure 17 is the number of horizontal DWT decompositions for the decomposed component images: 5 horizontal DWT decompositions in Table B.10 versus 1 horizontal DWT decomposition in Figure 17. The filled background of a cell in the reference grid indicates the packet that the sample associated with the cell is grouped in. The numbers associated with the packets indicate the order in which the packets are serialized in the packed dataset, with packets with lower associated numbers being added earlier in the packed dataset. Several differences can be observed between packets formed according to the inventive progression order as shown in figure 16 and packets formed according to the JPEG XS standard progression order as shown in figure 17. The first difference is that according to the inventive progression order, a packet contains only samples from a single component, whereas according to the JPEG XS standard progression order, a packet may contain samples from several components. The second difference is that according to the inventive progression order, a packet contains only samples from a single row of the reference grid, whereas according to the JPEG XS standard progression order, a packet may contain samples from several rows of the reference grid. Due to these differences, the inventive method has advantages over the JPEG XS standard method, which are explained below. Figure 17a provides an illustration of the progression order of the JPEG XS standard that is equivalent to Figure 11, with the notable difference that in Figure 11, the component images C0, C1, C3 are decomposed by one vertical DWT decomposition and five horizontal DWT decompositions, whereas in Figure 17a, they are decomposed by one vertical DWT decomposition and one horizontal DWT decomposition. Another difference is that in Figure 11, the samples are grouped by component and by subband, whereas in Figure 17a, the samples are represented on a reference grid.

[0033] FIG. 6 illustrates how a horizontal DWT decomposition is performed in a row of samples containing a sequence of even (E) and odd (O) samples by decomposing the sequence into a high-pass (H) subband and a low-pass (L) subband. First, in the row of samples, the odd samples are replaced by implementing a lifting scheme with a (5, 3) filter bank, and the resulting odd samples are represented by the equation H i =O i -1 / 2 * (E i-1 +Ei+1 ) Then, the H (high spatial frequency) subband is calculated. L i =E i +1 / 4 * (H i-1 +H i+1 ) to replace the even samples and generate the L (low spatial frequency) subband. As is well known, the formulas above where odd and even samples are swapped are equivalent. When a sample outside the image is needed, the corresponding sample inside the image, mirrored across the border, is taken. This means that the coefficients on the lower left side of the diagram (arrow H to L) are +1 / 2 instead of +1 / 4, and the coefficients on the upper right side of the diagram (arrow E to H) are -1 instead of -1 / 2.

[0034] In Figures 8, 10, 11, 12 and 13, the numbered arrows represent packets and the associated numbers indicate the order in which they are serialized in the packed data set (i.e., the order in which they are added to the packed data set), with packets associated with lower numbers being serialized before packets associated with higher numbers. A packet regroups all samples from all subbands covered by the corresponding arrow, starting at the end of the arrow and extending to the tip of the arrow.

[0035] 14 depicts a device 20 for compressing a color image, including a module 21 for performing a decorrelation transform on the color image to generate component images, a module 22 for performing a DWT decomposition on the component images to generate decomposed component images, and a module 23 for packetizing the decomposed component images to generate a packed data set. Each of these modules may include hardware such as gates, registers, FPGAs, ASICs, and / or microcontrollers.

[0036] 15 depicts a device 30 for decompressing a packed dataset, including a module 31 for depacketizing the packed dataset to generate decomposed component images, a module 32 for performing an inverse DWT decomposition on the decomposed component images to generate component images, and a module 33 for performing an inverse decorrelation transform on the component images to generate a color image. Each of these modules may include hardware such as gates, registers, FPGAs, ASICs, and / or microcontrollers.

[0037] Example 1 In a first exemplary embodiment of the invention, the color image is sampled according to the Bayer pattern. In step b) of the method, the number of vertical decompositions Nlky is equal to zero for all component images, and the number of horizontal decompositions Nlkx is equal to 5 for component images C0, C1, C3 and equal to zero for component image C2. Figure 7 is a representation of the subbands resulting from this decomposition for component images C0, C1, C3. Figure 8 represents the four decomposed component images together. According to the invention, sequential packets are formed containing samples corresponding to the four numbered arrows "0", "1", "2" and "3", representing the result of the execution of step c) of the method. The first row of the reference grid is analyzed, which is the one containing components 0, 1. All samples of component 0 are represented as arrow "0". These samples are packed into the packed data set as packet "0". Then the samples of component 1 (arrow "1") are packed into the packed data set as packet "1". The second row of the reference grid is analyzed in the same manner, as it contains components 2 and 3, corresponding to arrows "2" and "3", and their samples are packed into packets "2" and "3", respectively. Subsequent rows of the reference grid are processed in the same manner, up to the final row of the reference grid.

[0038] Example 2 In a second exemplary embodiment of the invention, the color image is again sampled according to the Bayer pattern. The number of vertical decompositions Nlky is equal to 1 for the component images C0, C1, C3 and zero for the component image C2, and the number of horizontal decompositions Nlkx is equal to 5 for the component images C0, C1, C3 and zero for the component image C2. Figure 9 is a representation of the subbands resulting from this decomposition for the component images C0, C1, C3. Figure 10 represents the four decomposed component images together. The numbered arrows "0" to "7" represent the results of the execution of step c) of the method. The first row of the reference grid is analyzed, which includes components 0, 1, and the samples of the arrows "0", "1" are treated and packed into the packed data set as packets "0", "1", respectively. The second row of the reference grid is analyzed, which includes components 2, 3, and the samples of the arrows "2", "3" are treated and packed into the packed data set as packets "2", "3", respectively. The third row of the reference grid is analyzed, which includes components 0 and 1, and the samples with arrows "4" and "5" are treated and packed into the packed data set as packets "4" and "5", respectively. The fourth row of the reference grid is analyzed, which includes components 2 and 3, and the samples with arrows "6" and "7" are treated and packed into the packed data set as packets "6" and "7", respectively. The subsequent rows of the reference grid are processed in the same way, up to the last row of the reference grid. Figure 11 shows a schematic representation of the subbands for a similar case to Figure 10, but the numbered arrows separating the samples for packet formation and serialization are determined according to the JPEG XS standard. It can be observed that in the method of the invention a given packet always contains samples from a single component, whereas in the JPEG XS standard a packet may contain samples from different components.

[0039] Example 3 In a third exemplary embodiment of the invention, the color image is a three-component image sampled according to a 4:4:4 sampling pattern, and one vertical and five horizontal DWT decompositions are performed for each component image. Figure 12 shows a schematic representation of the subbands resulting from the above DWT decompositions for all three component images, depicting at the top the packets formed and serialized in the packed data set according to the JPEG XS standard, and at the bottom the packets formed and serialized in the packed data set according to the method of the invention. As in Figure 8, the numbered arrows represent the grouping of component samples into packets and the order in which the packets are added to the packed data set.

[0040] Example 4 In a fourth exemplary embodiment of the invention, the color image is a three-component image sampled according to a 4:2:2 sampling pattern, and one vertical and five horizontal DWT decompositions are performed for each component image. Figure 13 shows a schematic representation of the subbands resulting from the above DWT decompositions for all three component images, depicting at the top the packets formed and serialized in the packed data set according to the JPEG XS standard, and at the bottom the packets formed and serialized in the packed data set according to the method of the invention. As in Figure 8, the numbered arrows represent the grouping of component samples into packets and the order in which the packets are added to the packed data set.

[0041] To compare how packets are formed and added to the packed data set in the JPEG XS standard and in the present invention, Table 1 shows the numbered subbands for an image with three components, where each component image is decomposed by one vertical DWT decomposition and five horizontal DWT decompositions. This is true for the images in Figures 12 and 13. It is also applicable to a four-component image, where three component images are decomposed by one vertical DWT decomposition and five horizontal DWT decompositions, and the fourth component image is not decomposed, as in the cases of Figures 10 and 11. TIFF0007608005000001.tif109170

[0042] Table 2 shows the JPEG XS progression order for the images in Figures 12 (top) and 13 (top), as adapted from Table B.6 in Reference 1. The packet indexes correspond to the numbered arrows in Figures 12 (top) and 13 (top). TIFF0007608005000002.tif63170

[0043] Table 3 shows the progression order for the images of Figures 12 (bottom) and 13 (bottom), as obtained from the method of the present invention. The packet indexes correspond to the numbered arrows in Figures 12 (bottom) and 13 (bottom). TIFF0007608005000003.tif83170

[0044] Table 4 shows the JPEG XS progression order for a four-component image, where three component images are decomposed by one vertical and five horizontal DWT decompositions, and the fourth component image is not decomposed. It is adapted from Table B.6 in Reference 1. A component image that is not decomposed is treated as a single subband with two subband lines. Each subband line of a non-decomposed component image is contained in its own packet. See Table 1 for subband numbering. The packet index corresponds to the numbered arrows in Figure 11. TIFF0007608005000004.tif83170

[0045] Table 5 shows the progression order resulting from the present invention for the same case as Table 4, i.e., a four-component image where three component images are decomposed by one vertical DWT decomposition and five horizontal DWT decompositions and the fourth component image is not decomposed, with packet indices corresponding to the numbered arrows in FIG. 10. TIFF0007608005000005.tif101170

[0046] Table 6 provides a comparison of packet characteristics between the JPEG XS standard and the present invention for the example given in the previous paragraph. TIFF0007608005000006.tif67170

[0047] The compression method according to the present invention differs from, and offers advantages over, the method of the JPEG XS standard in several aspects that are detailed below. First, the present invention allows for the generation of more packets than prior art solutions while maintaining a maximum packet size equal to or less than the minimum packet size of the prior art solutions (see Table 6). Having smaller packets allows for a reduction in the size of the packet buffer. A packet buffer is a storage for accumulating entire packets before making them available at its output; such a buffer is needed, for example, to smooth the data rate of the packed data set. However, the smaller the packets in a given packed data set, the more packets there are, which increases the fixed cost required to keep track of packet-related metadata in the packed data set, such as the location and size of each packet, which may take the form of a packet header, similar to the JPEG XS standard. The metadata does not include any information related to the original color image. Thus, the more metadata, the worse the image quality for a given size of the packed data set. The present invention allows for a trade-off between packet size and number of packets to be made, thereby allowing for a reduction in the size of the packet buffer while maintaining approximately the same amount of packet-related metadata, as compared to the prior art. Secondly, according to the present invention, a given packet contains only samples from a single row of a given decomposed component image, and therefore, said packet contains only samples derived from a single row of the original color image. In contrast, the JPEG XS standard specifies that any packet containing samples from any component image subjected to DWT decomposition must contain samples from all component images subjected to DWT decomposition. Thus, such a packet may contain samples derived from several rows of the original color image, for example in the case of the Bayer sampling pattern (see FIG. 11). By having only samples derived from a single row of the original color image in a given packet, the size of the reordering buffer can be reduced compared to the prior art. The reordering buffer is a storage that accumulates samples after forward decorrelation transform and forward DWT decomposition at the encoder side and outputs them in an order suitable for inverse decorrelation transform and inverse DWT decomposition at the decoder side. In fact, due to algorithmic dependencies between DWT decompositions, at the encoder side, samples resulting from higher-numbered DWT decompositions are generated after samples resulting from lower-numbered DWT decompositions. However, on the decoder side, such samples from a higher-numbered DWT decomposition are required before samples from a lower-numbered DWT decomposition. Thus, a reordering of samples is required between the encoder and the decoder, which is performed by the reordering buffer. An example of reducing the reordering buffer size in the case of the Bayer sampling pattern is described below. Since the JPEG XS standard requires samples from all component images for which DWT decomposition has been performed to be included in a packet, the reordering buffer needs to accumulate at least two rows of data to output a given packet, and samples from one of these component images for which DWT decomposition has been performed are only available after the two rows of data have been processed by the DWT decomposition.In addition, the fact that samples from non-decomposed component images are packed after samples from decomposed component images means that they must be stored in the reordering buffer until all samples from the decomposed component images have been processed. In contrast, in accordance with the present invention, since a given packet contains only data derived from a single row of the original color image, the reordering buffer only needs to accumulate a single row of data. In addition, samples from non-decomposed component images are treated in the same manner as samples from decomposed component images, thus eliminating the need to store them until all samples from the decomposed component images have been processed. These two features allow for a reduction in reordering buffer size compared to the prior art. Finally, in a hardware implementation of the method in the form of a hardware device, the reduced buffer size reduces the die area and therefore reduces the manufacturing cost of the hardware device, thereby providing a more efficient method for compressing and decompressing color images. In a software implementation of the method in the form of a computer program, the reduced buffer size reduces memory usage and therefore reduces the cost and power consumption of the computer running the computer program, again providing a more efficient method for compressing and decompressing color images.

[0048] The drawings of the illustrations are not to scale or to scale. Generally, in the drawings, identical components are designated by the same reference numerals.

Claims

1. 1. A method for compressing a color image, the color image comprising color image samples S i arranged in a reference grid according to a sampling pattern, the reference grid comprising a number of rows and columns, an index i starting at 0 and increasing with increasing rows, and an index j starting at 0 and increasing with increasing columns, resulting in a packed data set, a) performing a decorrelation transformation in which, for each position (i,j) of the reference grid, a component sample Ckij is determined, where k is a component index according to the sampling pattern, and Ckij is a linear combination of image samples Si'j' at positions (i',j'), where i' is inclusive between i-4 and i+4, and j' is inclusive between j-4 and j+4, and component images C0, C1, C2 and an optional component image C3 are generated according to the sampling pattern; b) for each component Ck, - Nlky is the number of vertical DWT decompositions applied to component Ck, an integer >= 0; - Nlkx is the number of horizontal DWT decompositions applied to component Ck, an integer ≥ Nlky; - considering said component Ck as an auxiliary subband; - If Nlky>0, for n=0 to Nlky-1, performing a vertical DWT decomposition on said auxiliary subbands by performing a lifting scheme with a (5,3) filter bank on said columns of said auxiliary subbands, whereby the resulting even samples are the LLn subbands and the resulting odd samples are the LHn subbands, and where for each of the resulting samples a reference is maintained to the position (i,j) of the reference grid from which they were determined; performing a horizontal DWT decomposition in the LLn subbands by performing a lifting scheme with a (5,3) filter bank on the rows of the LLn subbands, the resulting even samples being the LLn subbands and the resulting odd samples being the HLn subbands, and for each of the resulting samples a reference is maintained to the position (i,j) of the reference grid from which they were determined; performing a horizontal DWT decomposition in the LHn subbands by performing a lifting scheme with a (5,3) filter bank on the rows of the LHn subbands, the resulting even samples being the LHn subbands and the resulting odd samples being the HHn subbands, and for each of the resulting samples a reference is maintained to the position (i,j) of the reference grid from which they were determined; - considering the subband LLn as an auxiliary subband; Repeating the above steps, - If Nlkx>Nlky, for n=Nlky to Nlkx-1, performing a horizontal DWT decomposition in the LLn-1 subband by performing a lifting scheme with a (5,3) filter bank in the rows of the LLn-1 subband, the resulting even samples being the LLn subband and the resulting odd samples being the HLn subband, and for each of the resulting samples a reference is maintained to the position (i,j) of the reference grid from which they were determined; - considering the subband LLn as an auxiliary subband; Repeating the above Repeating the above steps, c) for each successive row of said reference grid: For each component Ck, grouping all samples of said component Ck of said row into one or more packets; adding said one or more packets to said packed data set; A method comprising:

2. 2. The method of claim 1, wherein Nlky, the number of vertical DWT decompositions, is equal to 0 or 1.

3. 2. The method according to claim 1, characterized in that between steps b) and c) an entropy coding of the data set is performed.

4. 4. The method of claim 3, wherein said entropy encoding comprises entropy encoding of bit-plane counts of groups of samples.

5. 5. The method of claim 4 further comprising grouping bit-plane counts of groups of samples and grouping data representative of said bit-plane counts.

6. 2. The method of claim 1, wherein the sampling pattern includes samples arranged in a Bayer pattern that juxtaposes a red pixel R at position (i,j), a first green pixel G1 at position (i,j+1), a second green pixel G2 at position (i+1,j), and a blue pixel B at position (i+1,j+1) for each even i and j coordinate of the reference grid, and wherein the decorrelation transform generates a component image C0 for the location of the red pixel R, a component image C1 for the location of the first green pixel G1, a component image C2 for the location of the second green pixel G2, and a component image C3 for the location of the blue pixel B.

7. 2. The method of claim 1 , wherein the sampling pattern includes samples arranged in a 4:4:4 pattern, where for each i and j coordinate of the reference grid, a luminance pixel Y, a blue chrominance pixel Cb, and a red chrominance pixel Cr are superimposed at position (i,j) or a red pixel R, a green pixel G, and a blue pixel B are superimposed at position (i,j), and wherein the decorrelation transform generates a component image C0 for the luminance pixel Y or red pixel location, a component image C1 for the blue chrominance pixel Cb or green pixel location, and a component image C2 for the red chrominance pixel Cr or blue pixel location.

8. 2. The method of claim 1 , wherein the sampling pattern includes samples arranged in a 4:2:2 pattern such that, for each i and every even j coordinate of the reference grid, a luminance pixel Y, a blue chrominance pixel Cb, and a red chrominance pixel Cr are located at position (i,j) and a luminance pixel Y is located at position (i,j+1), and wherein the decorrelation transform generates a component image C0 for the location of the luminance pixel Y, a component image C1 for the location of the blue chrominance pixel Cb, and a component image C2 for the location of the red chrominance pixel Cr.

9. 2. The method of claim 1, wherein the decorrelation transformation is an identity mapping, whereby each sample of the color image is mapped to a corresponding location on the component images C1, C2, C3.

10. 2. The method of claim 1, wherein the decorrelation transform is an in-line Star-Tetrix transform.

11. 2. The method of claim 1, wherein the decorrelation transform is a full Star-Tetrix transform.

12. 1. A device for compressing a color image, the color image comprising color image samples S i arranged in a reference grid according to a sampling pattern, the reference grid comprising a number of rows and columns, an index i starting at 0 and increasing with increasing rows and an index j starting at 0 and increasing with increasing columns resulting in a packed data set, One or more electronic components configured to perform step a) of the method of claim 1; One or more electronic components configured to perform step b) of the method of claim 1; one or more electronic components configured to perform step c) of the method of claim 1; A device comprising:

13. A non-transitory computer readable medium containing instructions, the program being configured to, when executed by a computer, cause the computer to perform the method of claim 1.

14. A method for decompressing a packed data set obtainable by carrying out the method according to claim 1, comprising the steps of: - unpacking said packets contained in said packed data set; - performing an inverse DWT decomposition; - performing an inverse decorrelation transformation; 23. A method for decompressing comprising:

15. A non-transitory computer readable medium containing instructions, the program being configured to, when executed by a computer, cause the computer to perform the method of claim 14.

16. A device for decompressing a packed data set obtainable by carrying out the method according to claim 1, comprising: - unpacking said packets contained in said packed data set; - performing an inverse DWT decomposition; - performing an inverse decorrelation transformation; 16. A device comprising one or more electronic components configured to:

17. 1. A method for compressing a color image, the color image comprising color image samples S i arranged in a reference grid according to a sampling pattern, the reference grid comprising a number of rows and columns, an index i starting at 0 and increasing with increasing rows, and an index j starting at 0 and increasing with increasing columns, resulting in a packed data set, a) performing a decorrelation transformation in which, for each position (i,j) of the reference grid, a component sample Ckij is determined, where k is a component index according to the sampling pattern, and Ckij is a linear combination of image samples Si'j' at positions (i',j'), where i' is inclusive between i-4 and i+4, and j' is inclusive between j-4 and j+4, and component images C0, C1, C2 and an optional component image C3 are generated according to the sampling pattern; b) for each component Ck, Nlky is the number of vertical DWT decompositions applied to component Ck, an integer ≧0; Nlkx is the number of horizontal DWT decompositions applied to component Ck, with an integer ≧Nlky; considering said component Ck as an auxiliary subband; When Nlky>0, for n=0 to Nlky-1, performing a vertical DWT decomposition on the auxiliary subbands by performing a lifting scheme with a (5,3) filter bank on the columns of the auxiliary subbands, where the resulting even samples are LLn subbands and the resulting odd samples are LHn subbands, and where for each of the resulting samples a reference is maintained to the location (i,j) of the reference grid from which they were determined; performing a horizontal DWT decomposition in the LLn subbands by performing a lifting scheme with a (5,3) filter bank on the rows of the LLn subbands, where the resulting even samples are in the LLn subbands and the resulting odd samples are in the HLn subbands, and where for each of the resulting samples a reference is maintained to the location (i,j) of the reference grid from which they were determined; performing a horizontal DWT decomposition in the LHn subband by performing a lifting scheme with a (5,3) filter bank on the rows of the LHn subband, wherein the resulting even samples are the LHn subband and the resulting odd samples are the HHn subband, and for each of the resulting samples a reference is maintained to the location (i,j) of the reference grid from which they were determined; Considering subband LLn as an auxiliary subband; Repeating the above steps, In the case of Nlkx>Nlky, for n=Nlky to Nlkx-1, performing a horizontal DWT decomposition in the LLn-1 subband by performing a lifting scheme with a (5,3) filter bank in the rows of the LLn-1 subband, the resulting even samples being the LLn subband and the resulting odd samples being the HLn subband, and for each of the resulting samples a reference is maintained to the location (i,j) of the reference grid from which they were determined; Considering subband LLn as an auxiliary subband; Repeating the above Repeating the above steps, c) for each successive row of said reference grid: For each component Ck, grouping all samples of said component Ck of said row into one or more packets; In adding the one or more packets to the packed data set, The packed data set is adding a sequence of said packets, each of said packets containing only data from a single component image, each of said packets containing only data from a single row of a component image, and all of said packets containing data from a given row of a component image being located in said packed data set before packets containing data from a subsequent row of a component image; A method comprising:

Images