Single channel encoding into a multi-channel container and subsequent image compression

Abstract

An encoding method and apparatus for packing single-channel data into a multi-channel container, such as an MP4, TIFF, or JPEG container, to achieve at least good utilization of the data capacity of the container. In some examples, the encoding method includes converting a plurality of scalar values ​​of a received data stream into a corresponding plurality of n-dimensional values, the conversion being performed using a mapper; assigning, using the processor, each of the n-dimensional values ​​as a pixel value to each pixel of a virtual image frame, where n is an integer greater than 1; and compressing the virtual image frame according to a type of container for image data. The mapper may map a plurality of 2D values ​​of an n-dimensional curve or n-dimensional space. n It is configured to map scalar values ​​to corresponding n-dimensional values ​​based on the relationships represented by the way-tree partitions.

Description

[Technical Field]

[0001] 1. CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of priority to U.S. Provisional Application No. 63 / 407,885, filed September 19, 2022, and European Application No. 23151686.5, filed January 16, 2023, each of which is incorporated herein by reference in its entirety.

[0002] 2. Disclosure Areas Various exemplary embodiments relate generally to video / image compression and, more particularly, but not exclusively, to video / image encoding and decoding. [Background technology]

[0003] 3.Background Compression reduces the memory required to store video and images and the bandwidth required to transmit them. The Motion Picture Experts Group (MPEG) codec, based on the H.264 compression standard, is a typical example of a codec used for this purpose. Other codecs are available on the market, many of which are designed to be compatible with a variety of professional, consumer, and mobile phone cameras. Summary of the Invention [Problem to be solved by the invention]

[0004] Overview of specific embodiments Disclosed herein are various methods and apparatus for packing single-channel data into a multi-channel container, such as an MP4, TIFF, or JPEG container, to achieve at least good utilization of the data capacity of the container. In various embodiments, the packing is based on a mapping relationship represented by an n-dimensional curve, or a mapping relationship between multiple 2D regions in an n-dimensional space. n way(2 n 2 nThe implementation is based on a mapping relationship represented by a n=3 octree (e.g., n=3 octree) partition. At least some embodiments are compatible with existing hardware in legacy video / image delivery pipelines, i.e., do not inherently rely on hardware / infrastructure modifications. Rather, such embodiments can be advantageously implemented as software and / or firmware, such as by interfacing a corresponding add-on data processing module with an existing codec, without modifying the codec's native container format(s). [Means for solving the problem]

[0005] According to an exemplary embodiment, there is provided an encoding method including: using a processor to convert a plurality of scalar values ​​of a received data stream into a corresponding plurality of n-dimensional values, said conversion being performed using a mapper; using the processor to assign each of the n-dimensional values ​​as a pixel value to each pixel of a virtual image frame, where n is an integer greater than 1; and using the processor to compress the virtual image frame according to a container type for image data, wherein the mapper is configured to convert a plurality of scalar values ​​of an n-dimensional curve or a plurality of 2-dimensional values ​​of an n-dimensional space. n It is configured to map scalar values ​​to corresponding n-dimensional values ​​based on the relationships represented by the way-tree partitions.

[0006] According to another exemplary embodiment, a non-transitory computer-readable medium is provided having stored thereon instructions that, when executed by an electronic processor, cause the electronic processor to perform operations including the method.

[0007] According to yet another exemplary embodiment, there is provided an apparatus for encoding image data, the apparatus comprising at least one processor and at least one memory containing program code, the at least one memory and the program code being configured to, using the at least one processor, cause the apparatus to at least: use an electronic mapper to convert a plurality of scalar values ​​of a received data stream into a corresponding plurality of n-dimensional values; assign each of the n-dimensional values ​​as a pixel value to each pixel of a virtual image frame, where n is an integer greater than 1; and compress the virtual image frame according to a container type of image data, the electronic mapper being configured to convert a plurality of scalar values ​​of a received data stream into a corresponding plurality of n-dimensional values, where n is an integer greater than 1; n It is configured to map scalar values ​​to corresponding n-dimensional values ​​based on the relationships represented by the way-tree partitions. [Brief explanation of the drawings]

[0008] Other aspects, features, and advantages of the various disclosed embodiments will become more fully apparent from the following detailed description and the accompanying drawings, in which:

[0009] Figure 1 shows an example of the processing of the video / image distribution pipeline;

[0010] FIG. 2 is a flowchart of an encoding method usable in the video / image distribution pipeline of FIG. 1 according to an embodiment;

[0011] 3 is a diagram illustrating the working principle of a 1D to 3D mapper that can be used in the encoding method of FIG. 2 according to an embodiment;

[0012] FIG. 4 illustrates the working principle of a 1D to 3D mapper that can be used in the encoding method of FIG. 2 according to another embodiment;

[0013] 5A-5B illustrate the working principle of a 1D-to-3D mapper that can be used in the encoding method of FIG. 2 according to yet another embodiment;

[0014] FIG. 6 is a flowchart of a decoding method usable in the video / image distribution pipeline of FIG. 1 according to an embodiment.

[0015] FIG. 7 is a block diagram illustrating a computing device according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0016] Detailed Description The present disclosure and aspects thereof may be embodied in various forms, including computer-implemented methods, computer program products, computer systems and networks, user interfaces, and hardware, devices, or circuits controlled by application programming interfaces, as well as hardware-implemented methods, signal processing circuits, memory arrays, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), etc. The foregoing is intended only to provide a general idea of ​​various aspects of the present disclosure and is not intended to limit the scope of the disclosure in any way.

[0017] 1 is a diagram illustrating an example process for a video / image delivery pipeline 100, showing various stages from video / image capture to video / image content display, according to an embodiment. A series of video / image frames 102 may be captured or generated using an image generation block 105. The frames 102 may be digitally captured (e.g., by a digital camera) or computer-generated (e.g., using computer animation) to provide video and / or image data 107. Alternatively, the frames 102 may be captured on film by a silver halide camera. The film may then be scanned and converted into a digital format to provide the video / image data 107.

[0018] In the production stage 110, data 107 may be edited to provide a video / image production stream 112. Data in the video / image production stream 112 may be provided to a processor (e.g., one or more processors, such as a central processing unit, CPU, etc.) for post-production editing in a post-production block 115. The post-production editing in block 115 may include, for example, adjusting or modifying the color or brightness of specific areas of the image to improve image quality or achieve the appearance of the video image according to the videographer's creative intent. This portion of post-production editing is sometimes referred to as "color timing" or "color grading." Other edits (e.g., scene selection and sequencing, image cropping, addition of computer-generated visual special effects, artifact removal, etc.) may be performed in block 115 to result in a "final" version 117 of the production for distribution. During post-production editing 115, the video and / or images may be optimized for viewing on a reference display 125.

[0019] Following post-production 115, the final version 117 data may be delivered to an encoding block 120 for delivery to further downstream decoding and playback devices, such as television sets, set-top boxes, movie theaters, etc. In some embodiments, encoding block 120 may include audio and video encoders, such as those defined by ATSC, DVB, DVD, Blu-Ray, and other distribution formats, to generate an encoded bitstream 122. At the receiver, encoded bitstream 122 is decoded by a decoding unit 130 to generate a corresponding decoded signal 132 that represents a replica or exact approximation of signal 117. The receiver may be attached to a target display 140 that may have slightly or completely different characteristics than the reference display 125. In such cases, a display management (DM) block 135 may be used to generate a display mapping signal 137 to map the decoded signal 132 to the characteristics of the target display 140. In some embodiments, the decoding unit 130 and the display management block 135 may include separate processors or may be based on a single integrated processing unit. Various embodiments disclosed below can be used to implement the encoding block 120 and / or the decoding unit 130.

[0020] The codecs used in the encoding block 120 and / or the decoding unit 130 enable video / image data processing and compression / decompression. Compression is used in the encoding block 120 to make the corresponding file(s) smaller. The decoding process performed by the decoding unit 130 typically involves decompressing the received video / image data file(s) into a form that can be used for playback and / or further editing. Examples of codecs that can be used in the encoding block 120 and the decoding unit 130 include, but are not limited to, XviD / DivX codecs, MPEG codecs, and H.264 codecs.

[0021] A container is a digital file that packages video / audio data and corresponding metadata. The metadata may include subtitles, resolution information, creation date, device type, language information, etc. Container files interleave different data types in a manner that allows the components to be easily accessed by the decoding unit 130. Different types of containers are typically identified by their file extensions, which include MP4, WAV, AIFF, AVI, MOV, WMV, MKV, TIFF, JPEG, HEVC, FLV, F4V, SWF, etc.

[0022] For example, the JPEG image file format is a popular choice for storing and transmitting photographic images, such as still images and individual video frames. Many operating systems have viewers that support the visualization of JPEG image files saved with the JPG or JPEG extension. Many web browsers also support the visualization of JPEG image files. JPEG encoding typically involves the following operations: (1) Conversion: Color images are converted from RGB (red, green, blue) color space to luminance / chrominance space. (2) Downsampling: Downsampling is typically performed on the chrominance components but not on the luminance component. For example, an image frame can be downsampled by a ratio of 2:1 horizontally and 1:1 vertically (2h1v). (3) Organization into Groups: The pixels for each color component are organized into groups of 8x8 pixels, often called "data units." If the number of rows is not an integer multiple of 8, the bottom row is duplicated one or more times to create the required multiple. Similar duplication may be applied to the rightmost column. (4) Discrete Cosine Transformation (DCT): A DCT is applied to each data unit to create an 8x8 map of transformed components. The DCT typically involves loss of information due to the limited precision of machine-based operations. (5) Quantization: Each of the 64 transformed components in the data unit is divided by another number called the quantization coefficient (QC) and rounded to an integer. This operation generally causes additional loss of information; larger QC values ​​tend to result in greater loss. Many encoders rely on the QC table recommended by the JPEG standard for quantization. (6) Encoding: The 64 quantized transform coefficients (which are integers) of each data unit are coded using a combination of run-length encoding (RLE) and Huffman coding. (7) Header: The final operation adds a header listing all relevant JPEG parameters. A corresponding JPEG decoder uses the inverse operation to produce an image that approximates the originally encoded image.

[0023] As another example, a TIFF image frame consists of a rectangular grid of pixels. The two axes of this geometry are called horizontal (or X or width) and vertical (or Y or length). Horizontal and vertical resolution need not be equal. A baseline TIFF image divides the vertical extent of the image into one or more strips, which are encoded and compressed separately. The TIFF format is an alternative to tiled image formats, in which both the horizontal and vertical extents of the image are divided into smaller units. The data for one pixel contains one or more samples. For example, an RGB image typically has one red, one green, and one blue sample per pixel, while a grayscale image has only one sample per pixel. The TIFF format can be used in both additive color models (e.g., RGB) and subtractive color models (e.g., cyan, magenta, yellow, and black, or CMYK). In at least some instances, interpretation of the channel data occurs outside the TIFF container. Interpretation can be aided by metadata, such as an International Color Consortium (ICC) profile. The TIFF format does not constrain the number of samples per pixel, nor does it constrain how many bits are coded for each sample. For example, three samples per pixel is the lower limit for multispectral imaging supported by TIFF, while hyperspectral imaging (which TIFF also supports) may use 100 or more samples per pixel. The TIFF format's support for custom selection of the number of samples per pixel is utilized in at least some embodiments disclosed herein below. TIFF images can be uncompressed, compressed using a lossless compression scheme, or compressed using a lossy compression scheme. An example of a lossless compression scheme compatible with the TIFF format is the Lempel-Ziv-Welch (LZW) compression scheme.

[0024] Augmented reality (AR) applications, virtual reality (VR) applications, and various applications involving the rendering of three-dimensional (3D) scenes may typically generate additional image data streams, each of which may take the form of a corresponding data sequence transmitted over a corresponding dedicated data channel. Examples of such data streams include, but are not limited to, depth data, vertex data, and index data. However, many of the above-mentioned types of containers (file formats) and corresponding conventional hardware for the encoding block 120 and decoding unit 130 do not inherently support single-channel transmission, which disadvantageously makes efficient encoding / decoding and compression / decompression of the above-mentioned additional data streams difficult.

[0025] Various embodiments disclosed herein address at least some of the above-identified problems in the art by providing various schemes for packing single-channel data into multi-channel containers to achieve one or more of: (i) efficient utilization of the data capacity of the multi-channel container; (ii) preservation of spatial and / or temporal coherence of the single-channel data streams in the packed form; and (iii) suppression or avoidance of excessive compression artifacts in the decompressed single-channel data at decoder unit 130. At least some embodiments are fully compatible with existing hardware of a corresponding video / image delivery pipeline (e.g., 100, FIG. 1 ), i.e., do not inherently rely on modifications to that hardware / infrastructure. Rather, such embodiments can be advantageously implemented in software and / or firmware without modifying the codec's native container format(s), e.g., by interfacing a corresponding relatively small data processing module with an existing codec.

[0026] 2 is a flowchart of an encoding method 200 usable in the encoding block 120 according to an embodiment. The encoding method 200 allows encoding a single-channel data stream (e.g., a data sequence) into an n-channel container, where n is an integer greater than 1. The single-channel data stream can be, for example, a depth data stream corresponding to a 3D scene. The n-channel container can be, for example, one of the multi-channel containers mentioned above.

[0027] The encoding method 200 includes receiving (at block 202) a next value of a single channel data stream. In the first instance of block 202, the next value received is the first value of the stream. In subsequent instances of block 202, the next value received is the value of the stream that follows the previously received value.

[0028] The encoding method 200 also includes selecting (at block 204) a next pixel of a virtual image frame. Here, the term "virtual image frame" refers to an image frame similar to a conventional image frame. However, unlike the latter, the virtual image frame does not represent a conventional image. Rather, different pixels of the virtual image frame can be assigned any desired pixel value, e.g., values ​​generated by an appropriate mapper (e.g., see block 206 and FIGS. 3-5). The pixels of such a virtual image frame can be spatially arranged, for example, in a rectangular array having its pixels in rows and columns, similar to a conventional image frame.

[0029] In some examples, in the first instance of block 204, the upper left corner pixel of the frame is selected. In any subsequent instances of block 204, the selection process may follow a raster pattern, or any other suitable predetermined pattern. Processing of an entire virtual image frame in encoding method 200 is typically complete after each pixel of the frame has been selected once. For example, in the case of a raster pattern, the pixel selected in the last instance of block 204 may be the pixel in the lower right corner of the frame.

[0030] In other embodiments, pixel processing orders other than the sequential order described above are used. In one embodiment, random pixel selection is performed. In various additional embodiments, pixels of the virtual image frame are processed in parallel using single instruction, multiple data (SIMD) vectorization; multiple instruction, multiple data (MIMD) multithreading; or a combination thereof. Such alternatives may also be implemented in various embodiments of the decoding method 600 (see FIG. 6).

[0031] The encoding method 200 also includes converting (at block 206) the one-dimensional (single-dimensional, scalar) values ​​received at block 202 into corresponding n-dimensional (nD) values ​​using a selected 1D to nD mapper. In some implementations, the nD values ​​can be represented by vectors in nD space. In a Cartesian coordinate system, an origin-based vector v in nD space can be represented by a sequence of values ​​(x1, x2, ..., x n ), where x iis the length of the projection of vector v onto the coordinate axis corresponding to the ith dimension of nD space. The number n is an algorithm parameter that depends on the container used in encoding method 200. For example, for JPEG and MP4 containers, the number n is n=3. For TIFF containers, the number n is the number of samples per pixel, and as explained above, in at least some examples, n>3. Some non-limiting examples of 1D to nD mappers that can be used in block 206 are described in detail below with reference to Figures 3-5.

[0032] The encoding method 200 also includes assigning (at block 208) the nD value generated at block 206 to the pixel selected at block 204. For example, for a container supporting an RGB color scheme with n=3, the x1, x2, and x3 components of the corresponding 3D vector are assigned to the selected pixel as the R, G, and B values ​​of that pixel, respectively (at block 208). As another example, for n=16 and a TIFF container, the x1, x2, ..., x3 components of the corresponding 16D vector are assigned to the selected pixel as the R, G, and B values ​​of that pixel, respectively. 16 The components are assigned to the pixels as 16 samples each corresponding to multispectral imaging (at block 208). In other examples of block 208, other suitable assignment schemes can also be used.

[0033] Encoding method 200 also includes determining (at decision block 210) whether the end of the corresponding virtual image frame has been reached. If it is determined that the end of the frame has not been reached ("No" at decision block 210), operation of encoding method 200 loops back to block 202. Otherwise ("Yes" at decision block 210), the (fully pixel-allocated) virtual image frame is compressed in a conventional manner according to the container format (at block 212). The container can then be directed from encoding block 120 to decoding unit 130, as previously described (see also FIG. 1). Once the operation of block 212 is completed, encoding method 200 ends.

[0034] 3-5 illustrate some non-limiting examples of 1D-to-nD mappers that can be used in block 206 of encoding method 200 according to various embodiments. For ease of explanation and not meant to be limiting, the illustrated examples correspond to the number n=3. Based on the provided description, one of ordinary skill in the art will be able to create and use various 1D-to-nD mappers that correspond to other values ​​of n, e.g., n=2 and n>3, without undue experimentation. In at least some examples, the corresponding 1D-to-nD mappers are implemented using one or more look-up tables (LUTs).

[0035] Figure 3 illustrates the operating principle of a 1D to 3D mapper that can be used in block 206 of encoding method 200 according to one embodiment. More specifically, Figure 3 shows a three-dimensional logarithmic spiral 302 in a Cartesian coordinate system with coordinate axes labeled X1, X2, and X3. This spiral 302 can be used to map a range [0,D] of scalar values ​​d to a 3D vector (Y,Cb,Cr), where Y, Cb, and Cr represent luminance, blue differential chrominance, and red differential chrominance components, respectively.

[0036] In some embodiments, a scalar value d (D≧d≧0) is mapped onto the helix 302 by finding a point on the helix that is a distance d along the helix from the origin O of the helix. The Cartesian coordinates (x3, x2, x1) of the found point are then used to determine the corresponding values ​​of Y, Cb, and Cr, respectively. In some implementations, the following equations (1)-(3) are used to program the processor of the encoding block 120 to perform the corresponding operations on the fly in block 206 of the encoding method 200: JPEG2025531920000002.jpg24147 where a and b are parameters that determine how the radius of spiral 302 increases, Cr0 and Cb0 are (offset) constants, and F(d) is a function of d that determines the distance scale along the Y (luminance) axis. In some implementations, the function F(d) is a quadratic function of d. In some other implementations, equations (1)-(3) are used to pre-calculate a corresponding LUT, which is then accessed by the processor to perform the corresponding operations in block 206.

[0037] When the mapping performed in block 206 is implemented based on the helix 302, such mapping is spatially and temporally coherent. In particular, such mapping is distance-preserving. Furthermore, the mapping is unique for any two points located on the helix 302. Furthermore, for any three points located on the helix 302, the distance relationship in 3D space is the same as the distance relationship along the helix 302 (i.e., in 1D space). For example, if point B is located between points A and C on the helix 302, the mapping is such that the distance between points A and C in 3D space is greater than the distance between points A and B, which is greater than the distance between points B and C. These properties resulting from the spatial and temporal coherence of the mapping are typically beneficial for achieving efficient compression, e.g., in JPEG, HEVC, MP4, and many other formats, and for suppressing the undesirable manifestation of compression artifacts in the decompressed single-channel data computed by the decoder unit 130.

[0038] 4 illustrates the operating principle of a 1D to 3D mapper that can be used in block 206 of encoding method 200 according to another embodiment. More specifically, FIG. 4 illustrates a 3D Peano curve 402 in a Cartesian coordinate system with axes labeled X1, X2, and X3, respectively. The granularity of Peano curve 402 is 2 bits per dimension. In additional embodiments, Peano curves of other granularities can be used as well. In other additional embodiments, various nD Peano curves (where n=2 or n>3) can be used to implement corresponding 1D to nD mappers of various granularities.

[0039] Peano curve 402 is used to map a range of scalar values ​​d [0,D] to 64 different 3D vectors (Y,Cb,Cr) or (R,G,B). In some examples, a scalar value d (D≧d≧0) is mapped to curve 402 by finding a point on the curve whose distance along the curve from the curve's origin O is rounded to d. The Cartesian coordinates (x3,x2,x1) of the found point are then used to determine the corresponding value of Y, Cb, Cr, respectively, or the corresponding value of R, G, B, respectively, for the pixel in question.

[0040] A Peano curve, such as Peano curve 402, is an example of a space-filling curve (with endpoints) whose range covers the entire range of a corresponding n-dimensional hypercube. Other examples of space-filling curves include, but are not limited to, Hilbert curves and Morton curves. A space-filling curve is a special case of a fractal curve. Various 1D → nD mappers suitable for implementing block 206 of encoding method 200 can be constructed using various suitable space-filling and / or fractal curves (with endpoints) in a manner similar to that described above with reference to Peano curve 402 and FIG. 4.

[0041] Figures 5A-5B illustrate the working principle of a 1D to 3D mapper that can be used in block 206 of encoding method 200 according to yet another embodiment. More specifically, Figure 5A shows an octree-partitioned 3D cube 500. Figure 5B shows an octree 510 used to partition 3D cube 500.

[0042] In general, an octree, such as octree 510 in Figure 5B, is a tree data structure in which each internal node has exactly eight children. Octrees can be used to partition a bounded three-dimensional space (such as 3D cube 500) by recursively subdividing the bounded space into eight octants. In a point-region (PR) octree, a node stores an explicit three-dimensional point that is the "center" of that node's subdivision. This center point defines one of the corners of each of the eight children. In a matrix-based (MX) octree, the subdivision point is implicitly the center of the space that the node represents. The root node of a PR octree can represent an infinite space. The root node of an MX octree represents a finite bounded space, and the implicit center is clearly defined. For example, the root node R of octree 510 (see Figure 5B) represents 3D cube 500. Octree 510 can be a 2D octant with n=3. n One skilled in the art would be able to derive various 2-way tree results for other values ​​of n without undue experimentation. n It will be easy to see how to construct a way tree. In additional examples, the values ​​of n are 2, 4, 5, etc.

[0043] The eight children of the root node R of the octree 510 are nodes 0 through 7 (see FIG. 5B). The subcubes (octants) of the 3D cube 500 corresponding to the child nodes 0 through 7 are similarly labeled 0 through 7 in FIG. 5A. Subcube 7 is not directly visible in the view shown in FIG. 5A. For illustrative purposes, only the children of child node 1 are explicitly shown in FIG. 5B. These grandchild nodes are labeled 10 through 17 in FIG. 5B. The subcubes (octants) of subcube 1 corresponding to grandchild nodes 10 through 17 are similarly labeled 10 through 17 in FIG. 5A. Subcube 17 is not directly visible in the view shown in FIG. 5A. For illustrative purposes, only the children of grandchild node 11 are explicitly shown in FIG. 5B. These great-grandchild nodes are labeled 110 through 117 in FIG. 5B. The subcubes (octants) of subcube 11 corresponding to great-grandchild nodes 110-117 are similarly labeled 110-117 in FIG. 5A. Subcube 117 is not directly visible in the diagram shown in FIG. 5A. Those skilled in the art will readily recognize that child nodes 0 and 2-7 each have a grandchild node that also has a great-grandchild node (not explicitly shown in FIG. 5A). Furthermore, those skilled in the art will recognize that grandchild nodes 10 and 12-17 each also have a great-grandchild node (not explicitly shown in FIG. 5A). Octree 510 has a total of 256 great-grandchild nodes. 3D cube 500 also has a total of 256 corresponding great-grandchild subcubes. The 256 great-grandchild subcubes partition 3D cube 500 into 256 non-overlapping portions, each of which has a cube shape.

[0044] In some implementations, to implement a 1D to 3D mapper for block 206 of encoding method 200, the three dimensions X1, X2, X3 of 3D cube 500 are (3)are assigned to represent R, G, B pixel values ​​or Y, Cb, Cr pixel values, respectively. The range [0, D] of scalar d values ​​is divided into 256 intervals. Each interval is assigned to 256 great-grandchild subcubes of the 3D cube 500. The Cartesian coordinates (x3, x2, x1) of the centers of the corresponding great-grandchild subcubes are then used to determine the R, G, B (or Y, Cb, Cr) pixel values ​​to which the scalar d values ​​map. In some implementations, the above-described relationships between the scalar d values ​​and the coordinates (x3, x2, x1) of the subcube centers are pre-calculated and compiled into a corresponding LUT, which is then accessed by the encoder to perform the appropriate operations in block 206.

[0045] In various embodiments, the encoding granularity of the octree, i.e., the number of subcubes in the 3D cube 500, is selected based on the desired accuracy and compression ratio intended for the encoder. The finer the cube subdivision, the more likely it is that the lossy compression performed in block 212 of the encoding method 200 will result in errors during the corresponding decoding performed in the decoding unit 130 (see also FIG. 6). Therefore, the selection of the encoding granularity of the octree may also need to be based on the amount of errors introduced by the lossy compression.

[0046] 6 is a flowchart of a decoding method 600 usable in the decoding unit 130 according to an embodiment. The decoding method 600 and the encoding method 200 are compatible with each other. In this way, the decoding method 600 allows substantial recovery of a single-channel data stream that has been encoded for transmission using a selected multi-channel container format and the encoding method 200.

[0047] The decoding method 600 includes decompressing (at block 602) a virtual image frame that has been compressed according to a container format. The decompression performed at block 602 is the inverse operation of the compression performed at block 212 of the encoding method 200.

[0048] The decoding method 600 also includes selecting a next pixel in the decompressed virtual image frame and reading the nD pixel value of the selected pixel (at block 604). In a representative example, the selection operation at block 604 of the decoding method 600 is implemented similarly to the selection operation at block 204 of the encoding method 200 described above. As such, the reader is referred to the above description of block 204 for relevant details of the pixel selection process.

[0049] The decoding method 600 also includes converting (at block 606) the nD pixel values ​​read in block 604 to corresponding scalar values ​​using an appropriately selected nD → 1D demapper such that the demapping performed by it is the inverse of the mapping performed by the 1D → nD mapper used in block 206 of the encoding method 200. In at least some examples, the nD → 1D demapper used in block 606 of the decoding method 600 and the 1D → nD mapper used in block 206 of the encoding method 200 are implemented based on the same LUT. More specifically, the nD → 1D demapper used in block 606 is configured to look up a scalar value in its LUT based on a given nD value, while the 1D → nD mapper used in block 206 of the encoding method 200 is configured to look up an nD value in the same LUT based on a given scalar value. In various embodiments, the nD → 1D demapper operates to account for errors introduced by lossy compression. For example, if a virtual frame pixel RGB value (100, 200, 30) comes out as an RGB value of (94, 203, 31) after decoding due to errors introduced by lossy compression, the nD → 1D demapper operates to correct the error to return the original (100, 200, 30) RGB value. Such error correction is achieved, for example, by selecting the granularity of the 1D → nD mapping such that the typical scattering of decoded points around an original constellation point is within the range closest (e.g., in the Euclidean distance sense) to the original constellation point rather than other constellation points. This feature of the demapper is often referred to as maximum likelihood detection.

[0050] The decoding method 600 also includes outputting (at block 608) the scalar value determined in block 606 as the next value of the corresponding data stream (data sequence). In the absence of loss and / or significant compression artifacts, the latter data stream is a duplicate or near duplicate of the data stream received in block 202 of the encoding method 200.

[0051] The decoding method 600 also includes determining (at decision block 610) whether the end of the corresponding virtual image frame has been reached. If it is determined that the end of the frame has not been reached (“No” at decision block 610), operation of the decoding method 600 loops back to block 604. Otherwise (“Yes” at decision block 610), the decoding method 600 ends.

[0052] 7 is a block diagram illustrating a computing device 700 according to an embodiment. The device 700 can be used, for example, in the encoding block 120. A computing device similar to the device 700 can also be used in the decoding unit 130. Based on the following description of the device 700, one skilled in the art will readily understand how to manufacture, configure, and use a similar computing device for the decoding unit 130.

[0053] The device 700 includes an input / output (I / O) device 710, an encoding engine 720, and a memory 730. The I / O device 710 can be used to enable the device 700 to receive at least a portion of the video / image stream 117 and output at least a portion of the encoded bitstream 122. The memory 730 may include, for example, a buffer for receiving encoded and compressed image data via the video / image stream 117. The received image data may include, among other things, the single-channel data stream described above. Once the data is buffered, the memory 730 can provide portions of the data to the encoding engine 720 for processing therein. The encoding engine 720 includes a processor 722 and a memory 724. The memory 724 can store program code therein that, when executed by the processor 722, enables the encoding engine 720 to perform various encoding operations, including, but not limited to, those described above with reference to some or all of Figures 2-5. The memory 724 can also store the above-mentioned LUT therein, which can be accessed by the processor 722 as needed.

[0054] Various aspects of the present invention can be further understood from the following enumerated example embodiments (EEE).

[0055] EEE(1): An encoding method, comprising: using a processor to convert a plurality of scalar values ​​of a received data stream into a corresponding plurality of n-dimensional values, the conversion being performed using a mapper; using the processor to assign each of the n-dimensional values ​​as a pixel value to each pixel of a virtual image frame, where n is an integer greater than 1; and using the processor to compress the virtual image frame according to a container type of image data, the mapper using an n-dimensional curve or a plurality of 2-dimensional spaces. nThe n-dimensional space is configured to map scalar values ​​to corresponding n-dimensional values ​​based on the relationships represented by the way-tree partitions. Note that an n-dimensional line is not an example of the "n-dimensional curve" described above. For example, a "curve" includes at least two segments that are not colinear with each other in the corresponding n-dimensional space.

[0056] EEE(2): The method according to EEE(1), where n is greater than 3.

[0057] EEE(3): The method of EEE(1) or EEE(2), wherein the corresponding n-dimensional values ​​are a set including a red value, a green value, and a blue value, or a set including a cyan value, a magenta value, and a yellow value, or a set including a luminance value, a blue differential chroma value, and a red differential chroma value.

[0058] EEE(4): The method according to any one of EEE(1) to EEE(3), wherein the plurality of scalar values ​​are depth data, vertex data, or index data representing a three-dimensional scene.

[0059] EEE(5): The method of EEE(1), EEE(3), or EEE(4), wherein the n-dimensional curve is a spiral and n=2 or n=3.

[0060] EEE(6): The method according to any one of EEE(1) to (5), wherein the n-dimensional curve is a space-filling curve having two endpoints.

[0061] EEE(7): The method of EEE(6), wherein the space-filling curve is selected from the group consisting of a Peano curve, a Hilbert curve, a Morton curve, and a fractal curve.

[0062] EEE(8): The method of any one of EEE(1) to EEE(6), wherein the mapper is configured to determine the corresponding n-dimensional value by: finding a location on the n-dimensional curve having a distance measured along the n-dimensional curve from its end point that represents the scalar value; and representing each component of the corresponding n-dimensional value by a set of coordinates of the location in the n-dimensional space.

[0063] EEE(9): The mapper is n and determining the corresponding n-dimensional value by identifying one partition among the way tree partitions that represents the scalar value, and representing each component of the corresponding n-dimensional value by a set of coordinates of the one partition in the n-dimensional space.

[0064] EEE(10): The mapper is configured to map the n-dimensional curve or the plurality of 2D regions of the n-dimensional space. n The method according to any one of EEE(1) to EEE(9), configured to use a look-up table, pre-calculated based on the way tree partition.

[0065] EEE(11): The method of any of EEE(1) to EEE(10), further comprising: using the processor or another processor to decompress a compressed image frame to generate a decompressed image frame, the decompression being performed according to a type of the container, the compressed image frame being generated by compression; and using the processor or another processor to convert a plurality of n-dimensional pixel values ​​of the decompressed image frame into a plurality of other scalar values, the conversion being performed using a demapper, wherein the demapper is configured to perform a demapping operation that is inverse to a corresponding mapping operation of the mapper.

[0066] EEE(12): Both the mapper and the demapper aren The method according to EEE (11), configured to use the same lookup table pre-calculated based on the way tree partition.

[0067] EEE(13): A non-transitory computer-readable medium storing instructions that, when executed by an electronic processor, cause the electronic processor to perform operations including any one of EEE(1) through EEE(12).

[0068] EEE (14): An apparatus for encoding image data, comprising at least one processor and at least one memory containing program code, the at least one memory and the program code configured to cause the apparatus, using the at least one processor, to at least: convert a plurality of scalar values ​​of a received data stream into a corresponding plurality of n-dimensional values ​​using an electronic mapper; assign each of the n-dimensional values ​​as a pixel value to each pixel of a virtual image frame, where n is an integer greater than 1; and compress the virtual image frame according to a container type of the image data, the electronic mapper being configured to convert a plurality of scalar values ​​of an n-dimensional curve or a plurality of 2-dimensional values ​​of an n-dimensional space. n It is configured to map scalar values ​​to corresponding n-dimensional values ​​based on the relationships represented by the way-tree partitions.

[0069] EEE(15): The apparatus of EEE(14), wherein the electronic mapper is configured to determine a position on the n-dimensional curve having a distance measured along the n-dimensional curve from an end point thereof that represents the scalar value, and to represent each component of the corresponding n-dimensional value by a set of coordinates of the position in the n-dimensional space.

[0070] EEE (16): The electronic mapper nThe apparatus of EEE (14) is configured to identify one partition among the way tree partitions that represents the scalar value, and to represent each component of the corresponding n-dimensional value by a set of coordinates of the one partition in the n-dimensional space.

[0071] EEE(17): The electronic mapper is a function of the n-dimensional curve or the n-dimensional space. n The apparatus of any one of EEE(14) to EEE(16), configured to use a look-up table, pre-calculated based on a way tree partition.

[0072] EEE(18): The apparatus of any one of EEE(14) to EEE(17), wherein the at least one memory and the program code, using the at least one processor, further cause the apparatus to: generate a decompressed image frame by decompressing the compressed image frame in accordance with the container type; and convert, using an electronic demapper, a plurality of n-dimensional pixel values ​​of the decompressed image frame into another plurality of scalar values, wherein the electronic demapper is configured to perform a demapping operation that is inverse to a corresponding mapping operation of the electronic mapper.

[0073] EEE(19): Both the electronic mapper and the electronic demapper are connected to the n-dimensional curve or the n-dimensional space. n The apparatus of EEE (18) is configured to use a common lookup table (e.g., each copy of the same) that is pre-calculated based on a way tree partition.

[0074] EEE(20): The apparatus according to any one of EEE(14) to EEE(19), wherein the plurality of scalar values ​​are depth data, vertex data, or index data representing a three-dimensional scene.

[0075] With respect to processes, systems, methods, heuristics, etc. described herein, the steps of such processes, etc. have been described as occurring according to a certain ordered sequence, but it will be understood that such processes can be practiced with the described steps performed in an order other than the order described herein. Furthermore, it will be understood that certain steps can be performed simultaneously, other steps can be added, or certain steps described herein can be omitted. In other words, the process descriptions herein are provided for the purpose of illustrating particular embodiments and should not be construed as limiting the scope of the claims in any way.

[0076] Accordingly, it should be understood that the above description is intended to be illustrative, and not limiting. Many embodiments and applications other than the examples provided will become apparent from reading the above description. The scope of the claims should be determined not with reference to the above description, but should instead be determined with reference to the appended claims, along with the full scope of equivalents to which such claims are entitled. It is anticipated and intended that future developments will occur in the technology discussed herein, and that the disclosed systems and methods will be incorporated into such future embodiments. In other words, it should be understood that modifications and variations are possible in this application.

[0077] All terms used in the claims are intended to be given their broadest reasonable interpretation and ordinary meaning as understood by one of ordinary skill in the art described herein, unless expressly stated to the contrary in the specification. In particular, the use of singular articles such as "a," "the," and "said" means that there are one or more of the described element, unless expressly limited to the contrary in the claim.

[0078] The Abstract of the Disclosure is provided to allow the reader to quickly grasp the nature of the technical disclosure. It is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. Furthermore, in the foregoing Detailed Description, it can be seen that various features are grouped together in various embodiments for clarity of disclosure. This method of disclosure does not reflect an intention that the claimed embodiments include more features than are expressly recited in each claim. Rather, as reflected in the following claims, inventive subject matter lies in fewer than all features of a single disclosed embodiment. Accordingly, the following claims are hereby incorporated into the Detailed Description, with each claim standing on its own as separately claimed subject matter.

[0079] While this disclosure includes reference to exemplary embodiments, this specification is not intended to be construed in a limiting sense. Various modifications of the described embodiments, as well as other embodiments within the scope of the present disclosure that are apparent to those skilled in the art to which the present disclosure pertains, are deemed to be within the principles and scope of the present disclosure, as expressed, for example, in the following claims.

[0080] Some embodiments may be implemented as circuit-based processes, including possible implementation on a single integrated circuit.

[0081] Some embodiments may be embodied in the form of methods and apparatuses for practicing those methods. Some embodiments may also be embodied in the form of program code recorded on a tangible medium, such as a magnetic recording medium, an optical recording medium, a solid-state memory, a floppy disk, a CD-ROM, a hard drive, or any other non-transitory machine-readable storage medium, which, when loaded and executed by a machine, such as a computer, makes the machine an apparatus for practicing various embodiments described herein. Some embodiments may also be embodied in the form of program code stored on a non-transitory machine-readable storage medium, including, for example, being loaded into and / or executed by a machine, which, when loaded and executed by a machine, such as a computer or processor, makes the machine an apparatus for practicing various embodiments described herein. When implemented on a general-purpose processor, the program code segments combine with the processor to provide a unique device that operates analogously to specific logic circuits.

[0082] Unless otherwise specified, each numerical value and range should be construed as approximate, as if the word "about" or "approximately" were before the numerical value or range.

[0083] Use of figure numbers and / or figure reference labels in the claims is intended to identify one or more possible embodiments of the claimed subject matter to facilitate claim interpretation, and such use should not be construed as necessarily limiting the scope of those claims to the embodiments shown in the corresponding figures.

[0084] Although the elements recited in the following method claims, if any, are presented in a particular order with corresponding labeling, these elements are not necessarily intended to be limited to being performed in that particular order, unless the claims imply a particular order for performing some or all of these elements.

[0085] References herein to "one embodiment" or "an embodiment" mean that a particular feature, structure, or characteristic described in connection with that embodiment may be included in at least one embodiment of the present disclosure. Appearances of the phrase "in one embodiment" in various places throughout this specification do not necessarily all refer to the same embodiment, nor are separate or additional embodiments necessarily mutually exclusive of other embodiments. The same applies to the term "implementation."

[0086] Unless otherwise specified herein, the use of ordinal adjectives "first," "second," "third," etc. to refer to one object of a plurality of similar objects merely indicates that different instances of such similar objects are being referenced and does not imply that the similar objects so referenced must be in a corresponding order or sequence, whether in time, space, ranking, or otherwise.

[0087] Unless otherwise specified herein, in addition to its plain meaning, the conjunction "if" can also be interpreted to mean "when," "upon," "in response to determining," or "in response to detecting," and the interpretation may depend on the particular context in which it is used. For example, the phrase "if it is determined" or "if [a stated condition] is detected" can be interpreted to mean "upon determining" or "in response to determining," or "upon detecting [the stated condition or event]" or "in response to detecting [the stated condition or event]."

[0088] Also, as used herein, the terms "couple," "coupling," "coupled," "connect," "connected," and "connected" refer to any implementation known in the art or developed in the future in which energy is transferred between two or more elements, and the interposition of one or more additional elements is intended, but not required. Conversely, terms such as "directly coupled," "directly connected," and the like imply the absence of such additional elements.

[0089] The terms "compatible" and "in accordance with," as used herein in connection with elements and standards, mean that the element communicates with other elements in a manner defined, in whole or in part, by the standard, and will be recognized by other elements as being fully capable of communicating with them in the manner defined by the standard. A compatible element need not operate internally in the manner defined by the standard.

[0090] The functions of the various elements illustrated in the figures, including functional blocks labeled "processor" and / or "controller," may be provided not only through the use of dedicated hardware, but also through the use of hardware capable of executing software in association with appropriate software. When provided by a processor, the functions may be provided by a single dedicated processor, a single shared processor, or multiple individual processors, some of which may be shared. Furthermore, the explicit use of the terms "processor" or "controller" should not be construed to refer solely to hardware capable of executing software, but may implicitly include, without limitation, digital signal processor (DSP) hardware, network processors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), read-only memory (ROM) for storing software, random access memory (RAM), and non-volatile storage. Other hardware, conventional and / or custom, may also be included. Similarly, the switches illustrated in the figures are conceptual only. These functions may be performed through the operation of program logic, through dedicated logic, through the interaction of program control and dedicated logic, or manually, with the particular technique being selectable by the implementer as more particularly understood from the context.

[0091] As used herein, the term "circuit" may refer to one or more or all of the following: (a) a hardware-only circuit implementation (such as an implementation with only analog and / or digital circuitry); (b) a combination of hardware circuitry and software, such as (where applicable): (i) a combination of analog and / or digital hardware circuitry(s) with software / firmware, (ii) a hardware processor(s) with software (including digital signal processor(s)), software, and any portion of memory(s) that cooperate to cause a device such as a cell phone or server to perform various functions, and (c) a hardware circuit(s) and / or processor(s), such as a microprocessor(s) or portion of a microprocessor(s), that requires software (e.g., firmware) to operate, but that may not be present when the software is not necessary for operation. This definition of circuit applies to all uses of the term in this application, including the claims. As a further example, as used herein, the term circuitry may cover merely a hardware circuit or processor (or processors), or a portion of a hardware circuit or processor together with its (or their) accompanying software and / or firmware implementation. The term circuitry may also cover, by way of example and where applicable to particular claim elements, a baseband or processor integrated circuit for a mobile device, or a similar integrated circuit in a server, cellular network device, or other computing or network device.

[0092] Those skilled in the art will appreciate that any block diagrams and flowcharts herein represent conceptual views of illustrative circuitry embodying the principles of the present disclosure. Similarly, any flowcharts, flow diagrams, state transition diagrams, pseudocode, etc. will be understood to represent various processes that may be substantially represented in a computer-readable medium and executed by a computer or processor, whether or not such a computer or processor is explicitly shown.

[0093] The "BRIEF SUMMARY OF SOME SPECIFIC EMBODIMENTS" herein is intended to introduce some exemplary embodiments; additional embodiments are described in the "DETAILED DESCRIPTION" and / or with reference to one or more drawings. The "BRIEF SUMMARY OF SOME SPECIFIC EMBODIMENTS" is not intended to identify essential elements or features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter.

Claims

1. 1. An encoding method for encoding a single-channel data stream into an n-channel image container, comprising: converting, using a processor, a plurality of scalar values ​​of the received single-channel data stream into a corresponding plurality of n-dimensional values, where n is an integer greater than 1 and is dependent on the format of the n-channel image container used for said encoding, said conversion being performed using a mapper; using the processor to assign each of the n-dimensional values ​​as a pixel value to each pixel of a virtual image frame; compressing, with the processor, the virtual image frames in accordance with an n-channel image container format; The mapper may be a function of a plurality of 2D curves or n-dimensional spaces. ｎ and a mapping relationship representing the scalar value to a corresponding n-dimensional value based on the mapping relationship represented by a way tree partition, the mapping relationship being configured to, for each of the scalar values, represent a position on the n-dimensional curve having a distance measured along the n-dimensional curve from its end point, or a position on the n-dimensional curve having a distance measured along the n-dimensional curve from its end point, ｎ determining a partition of the way tree partitions; and expressing each component of the corresponding n-dimensional value by the location or a set of coordinates of the partition in the n-dimensional space; method.

2. The method of claim 1 , wherein n is greater than 3.

3. 3. The method of claim 1, wherein the corresponding n-dimensional values ​​are a set including a red value, a green value, and a blue value, or a set including a cyan value, a magenta value, and a yellow value, or a set including a luminance value, a blue differential chroma value, and a red differential chroma value.

4. The method according to any one of claims 1 to 3, wherein the plurality of scalar values ​​are depth data, vertex data, or index data representing a three-dimensional scene.

5. 5. The method of claim 1, 3 or 4, wherein the n-dimensional curve is a spiral, and n=2 or n=3.

6. 10. A method according to any preceding claim, wherein the n-dimensional curve is a space-filling curve having two endpoints.

7. 7. The method of claim 6, wherein the space-filling curve is selected from the group consisting of a Peano curve, a Hilbert curve, a Morton curve, and a fractal curve.

8. The mapper may be configured to map the n-dimensional curve or the plurality of two dimensional spaces. ｎ 10. The method of any preceding claim, configured to use a look-up table, pre-computed based on a way-tree partition.

9. generating decompressed image frames by decompressing compressed image frames using the processor or another processor, the decompression being performed in accordance with the container format, the compressed image frames being generated by compression; and converting, using the processor or another processor, a plurality of n-dimensional pixel values ​​of the decompressed image frame into a plurality of scalar values, the conversion being performed using a demapper; wherein the demapper is configured to perform a demapping operation that is inverse to a corresponding mapping operation of the mapper.

10. A method according to any preceding claim.

10. Both the mapper and the demapper are connected to a plurality of 2D regions of the n-dimensional curve or the n-dimensional space. ｎ 10. The method of claim 9, configured to use a common lookup table pre-calculated based on way tree partitions.

11. A non-transitory computer readable medium storing instructions that, when executed by an electronic processor, cause the electronic processor to perform operations including any of the preceding claims.

12. 1. An apparatus for encoding image data, comprising: at least one processor; at least one memory containing program code; The at least one memory and the program code are configured, using the at least one processor, to cause the device to at least: converting, using an electronic mapper, a plurality of scalar values ​​of the received single-channel data stream into a corresponding plurality of n-dimensional values, where n is an integer greater than 1 and is dependent on the format of the n-channel image container used for said encoding; assigning each of the n-dimensional values ​​as a pixel value to each pixel of a virtual image frame; compressing the virtual image frames according to an n-channel image container format; wherein the electronic mapper is a function of a plurality of two-dimensional n-dimensional curves or n-dimensional spaces. ｎ and a mapping relationship representing the scalar value to a corresponding n-dimensional value based on the mapping relationship represented by a way tree partition, the mapping relationship being configured to, for each of the scalar values, represent a position on the n-dimensional curve having a distance measured along the n-dimensional curve from its end point, or a position on the n-dimensional curve having a distance measured along the n-dimensional curve from its end point, ｎ determining a partition of the way tree partitions; and expressing each component of the corresponding n-dimensional value by the location or a set of coordinates of the partition in the n-dimensional space.

13. The electronic mapper may be configured to map the n-dimensional curve or the plurality of two-dimensional space. ｎ 13. The apparatus of claim 12, configured to use a look-up table that is pre-calculated based on a way tree partition.

14. The at least one memory and the program code, when used with the at least one processor, further cause the apparatus to: decompressing the compressed image frame in accordance with the container format to generate a decompressed image frame; converting the n-dimensional pixel values ​​of the decompressed image frame into another plurality of scalar values ​​using an electronic demapper; 14. Apparatus according to claim 12 or 13, wherein the electronic demapper is configured to perform a demapping operation that is inverse to a corresponding mapping operation of the electronic mapper.

15. Both the electronic mapper and the electronic demapper are adapted to compute a plurality of 2D regions of the n-dimensional curve or the n-dimensional space. ｎ 15. The apparatus of claim 14, configured to use a common lookup table that is pre-calculated based on a way tree partition.

16. The apparatus according to any one of claims 12 to 15, wherein the plurality of scalar values ​​are depth data, vertex data, or index data representing a three-dimensional scene.

Images