Multiplane Image Compression

Abstract

Exemplary methods, apparatus, systems, and articles of manufacture (e.g., physical storage media) for implementing multi-plane image (MPI) compression are disclosed. The exemplary apparatus disclosed herein includes an interface that accesses an input multi-plane image stack corresponding to a source camera viewpoint, the input multi-plane image stack including a plurality of texture images and a corresponding plurality of alpha images, some of the alpha images including pixel values ​​representing the transparency of corresponding pixels in each of some of the texture images. The exemplary apparatus also includes a compressed image encoder that performs at least one of: (i) converting the plurality of texture images into a single composite texture image to generate the compressed multi-plane image stack; or (ii) converting the plurality of alpha images into a single composite alpha image to generate the compressed multi-plane image stack. In some disclosed examples, the interface outputs the compressed multi-plane image stack.

Description

Technical Field

[0001] [Related Applications] This patent claims the benefit of U.S. Provisional Patent Application No. 63 / 041,589, filed on June 19, 2020, entitled "COMPRESSION OF MULTIPLANE IMAGES FOR PARALAX-ENABLED VIDEO RENDERING". Priority is claimed to U.S. Provisional Patent Application No. 63 / 041,589, which is hereby incorporated by reference in its entirety.

[0002] The present disclosure generally relates to image compression, and more specifically, to multi-plane image compression.

Background Art

[0003] In 3D image rendering technology, multi-plane images (MPIs) that form a stack of semi-transparent image planes may be used to represent different depths of a 3D scene. Each plane of the MPI stack includes a texture image and an alpha image. The texture image provides texture pixel values (e.g., red-blue-green, or RGB values), and the alpha image includes alpha pixel values indicating the transparency of each respective texture pixel. When the alpha value is large, the texture pixel is opaque and the background cannot be seen. When the alpha value is small, the texture pixel is transparent and the background can be seen. In some examples, the MPI stack is generated at one platform (e.g., where the source camera is located), and then transmitted to another platform (e.g., the target / client platform), where a 3D image is rendered corresponding to the desired viewpoint.

Brief Description of the Drawings

[0004]

Figure 1

[0005]

Figure 2

[0006]

Figure 3

[0007]

Figure 4

[0008]

Figure 5

[0009]

Figure 6

[0010]

Figure 7

[0011]

Figure 8

[0012]

Figure 9

[0013]

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

[0014]

Figure 15

Figure 16

Figure 17

[0015]

Figure 18

[0016]

Figure 19

[0017]

Figure 20

[0018]

Figure 21

[0019] The drawings are not to scale. In general, the same reference numbers will be used throughout the drawings and the accompanying specification to refer to the same or similar parts, elements, etc. As used herein, references to connection (e.g., attachment, coupling, connection, joining) may, unless otherwise indicated, include intermediate members between the elements referred to by the reference to connection and / or relative movement between those elements. Thus, a reference to connection does not necessarily imply that the two elements are directly connected and / or in a fixed relationship to each other. As used herein, the expression that any part is "in contact with" another part is defined to mean that there is no intermediate part between the two parts.

[0020] Unless otherwise specified, descriptors such as "first", "second", "third", etc. are used in this specification without imparting or otherwise indicating a meaning of priority, physical order, arrangement in a list, and / or ordering in any way, but merely as labels and / or arbitrary names to distinguish elements for the purpose of facilitating an exemplary understanding of the disclosure. In some examples, the descriptor "first" is used to refer to an element within a detailed description, while the same element may be referred to in a claim with a different descriptor such as "second" or "third". In such cases, it should be understood that such descriptors are merely used, for example, to clearly identify elements having the same name. As used herein, "about" and "approximately" refer to dimensions that may not be exact due to manufacturing tolerances and / or other real-world imperfections. As used herein, "substantially real-time" refers to occurring almost instantaneously, although it should be recognized that in reality there are delays such as computational time, transmission, etc. Thus, unless otherwise specified, "substantially real-time" refers to real-time ±1 second.

Best Mode for Carrying Out the Invention

[0021] Exemplary methods, apparatuses, systems, and articles of manufacture (e.g., physical storage media) for implementing multi-plane image (MPI) compression are disclosed. Generally, an MPI system uses machine learning to create a volumetric representation of a scene as a stack of translucent images or planes that include textures obtained from the original camera view. When these planes are stacked on top of each other, the original camera view is restored. These stacks can be used to render scenes that would be visible from novel viewpoints with a visual quality that exceeds many other view interpolation techniques.

[0022] As described above, the MPI stack includes a stack of texture images (also referred to herein as a texture image plane or texture plane) and a stack of alpha images (also referred to herein as an alpha image plane or alpha plane). The texture images and alpha images collectively represent different depths in the source camera's field of view. The texture image for a given depth provides texture pixel values (e.g., red-blue-green, or RGB values), and the alpha image includes alpha pixel values indicating the transparency of each texture pixel, where increasing alpha values represent increasing opacity (or decreasing transparency), and decreasing alpha values represent decreasing opacity (or increasing transparency). In some examples, the MPI stack is generated at a source platform (e.g., where the camera is located) and then transmitted to a target platform (e.g., a client platform) where a 3D image is rendered corresponding to a desired viewpoint.

[0023] The exemplary MPI compression techniques disclosed herein maintain the quality of the rendered image while reducing the amount of MPI data transferred to a target platform for image rendering. Thus, the exemplary MPI compression techniques disclosed herein enable low-bandwidth transmission to a target or rendering node relative to the transmission bandwidth required by other (e.g., conventional) techniques. Accordingly, the disclosed exemplary MPI compression techniques can enable the execution of virtual reality and immersive media applications on platforms with limited sources, but can also be used in any number of image processing systems and applications.

[0024] In some examples, rather than transmitting a stack of N textures and alpha planes (e.g., N = 32 or some other value), the disclosed exemplary MPI compression technique generates a single composite texture frame and / or a single composite alpha frame (also referred to as an alpha map frame) that collectively contains sufficient information obtained from the MPI stack for an MPI image rendering system to provide a high-quality rendering of the stack information. In some examples, the single composite alpha frame and the single composite texture are generated from the MPI stack by the disclosed exemplary encoder on a source (e.g., transmission) platform. At a target (e.g., receiving) platform, the disclosed exemplary decoder uses the single composite alpha frame and the single composite texture frame to reconstruct the stack of N planes of the MPI stack that is used by the MPI renderer to render the target image.

[0025] It will be appreciated that the disclosed exemplary MPI compression technique can reduce the bitrate and pixel rate used for MPI data transmission (e.g., by N times such as N = 32 or some other value) compared to a system that transmits all of the N textures and alpha planes. As a result, the bandwidth and computational load in a target (e.g., client) image renderer are reduced. Thus, with the disclosed exemplary techniques, a video system can provide an immersive media experience on a resource-constrained platform, including, for example, tablets, laptops, smartphones, and other such devices that may be limited with respect to transmission bandwidth and / or computational bandwidth.

[0026] Returning to the figures, FIG. 1 shows a block diagram of an exemplary MPI stack generator 100 that generates an MPI stack from source images. In the MPI stack generator 100, one or more machine learning models are implemented to generate an MPI stack from a plurality of exemplary source camera image frames 110 and 120. Any suitable machine learning model may be implemented by the MPI stack generator 100. In some examples, a local light field fusion (LLFF) model may be used as the machine learning model implemented by the MPI stack generator 100. In some examples, the LLFF model uses images from several adjacent cameras as samples of the light field. These samples are used to generate a stack of MPI planes and can be used to expand the allowable field of view where a target (e.g., virtual) camera can be placed. Also, this approach enables the capture and reproduction of viewpoint-dependent lighting effects such as specular highlights, refraction, and reflection. In the example shown in FIG. 1, the MPI stack generator 100 implements a trained deep learning LLFF model that generates exemplary MPI stacks 130 and 135 corresponding to adjacent camera viewpoints respectively associated with the source cameras that provided the source camera image frames 110 and 120 from the source camera image frames 110 and 120.

[0027] In the example shown in FIG. 1, the generated MPI stacks, such as MPI stacks 130 and 135, are represented by a number of planes N (where N = 32 or some other value, etc.) placed in the frustum of the source camera's viewing cone such that the reciprocals of the depths of the planes are evenly spaced over the range [1 / d max ...1 / d min , and d min and d maxare the minimum and maximum depths of the frustum with respect to the camera center. As shown in the example of FIG. 1, the generated MPI stack 130 includes an exemplary texture image stack 140 associated with the frustum (or field of view) of the first camera corresponding to the source image frame 110, and an exemplary alpha stack 150 associated with the frustum (or field of view) of the first camera corresponding to the source image frame 110. Similarly, the generated MPI stack 135 includes an exemplary texture image stack 160 associated with the frustum (or field of view) of the second camera corresponding to the source image frame 120, and an exemplary alpha stack 170 associated with the frustum (or field of view) of the second camera corresponding to the source image frame 115. In the following disclosure, the processing of N = 32 planes will be described, but in practice, any number of planes can be used.

[0028] To render a scene for a desired viewpoint of a target virtual camera, the MPI stack is warped to the coordinate system of the virtual camera, and the planes are synthesized from back to front using alpha values as weighting factors applied to the texture values of each texture plane. Thus, in MPI-based techniques, view synthesis can be performed with less computational effort than conventional 3D mesh projection, but at the cost of reduced quality as the virtual camera moves away from the source camera. Also, a 32-plane MPI stack using 8-bit red, green, blue, alpha (RGBA) samples per pixel at a resolution of 1920×1080 contains 0.265 gigabits (GB) of data. When encoding and rendering a video sequence at 30 frames per second (fps) using conventional MPI-based techniques, a bandwidth of 63.7 Gb / s is utilized for a single camera source, and proportionally higher bandwidths are utilized for multiple camera inputs. This bandwidth utilization can be prohibitively high for some applications, rendering MPI-based techniques impractical. Also, the high pixel rate can impose a large computational load on the encoder of the source (e.g., transmission) platform and the decoder / renderer of the target (e.g., client) platform, making conventional MPI-based techniques unsuitable for some client video applications.

[0029] A block diagram of an exemplary MPI renderer 200 for rendering an exemplary target image 205 from one or more MPI stacks is shown in FIG. 2. In the example shown in FIG. 2, the MPI renderer 200 is implemented on a target platform and receives the MPI stacks 130 and 135 described above in connection with FIG. 1. As described above, the MPI stack 130 includes a texture image stack 140 and an alpha image stack 150, and the MPI stack 135 includes a texture image stack 160 and an alpha image stack 170. The MPI renderer 200 of the example shown weights the texture images 140 and 160 by the respective alpha images 1150 and 170, and then blends the weighted images to create a rendered image 205 corresponding to the target view point. In some examples, the blending performed by the MPI renderer 200 includes warping the texture image stacks 140 and 160, and the alpha image stacks 150 and 170, to the coordinate system of the target camera.

[0030] A block diagram of an example video encoding and decoding system 300 including the MPI stack generator 100 of Figure 1 and the MPI renderer 200 of Figure 2 is shown in Figure 3. The illustrated example video encoding and decoding system 300, also referred to as video system 300, has an example source platform 305 including the MPI stack generator 100 and an example MPI stack encoder 310, and an example target platform 315 including an example MPI stack decoder 320 and the MPI renderer 200. The source platform 305 can be implemented by any computing platform / device, such as, but not limited to, one or more servers, personal computers, workstations, mobile devices (e.g., mobile phones, smartphones, tablets such as iPads), personal digital assistants (PDAs), Internet appliances, cameras, etc. Similarly, the target platform 315 can be implemented by any computing platform / device such as, but not limited to, one or more servers, personal computers, workstations, mobile devices (e.g., mobile phones, smartphones, tablets such as iPad™), PDAs, Internet appliances, DVD players, CD players, digital video recorders, Blu-ray players, game consoles, personal video recorders, set-top boxes, etc.

[0031] In the example shown in FIG. 3, the MPI stack generator 100 of the source platform 305 generates respective MPI stacks, such as MPI stacks 130 and 135, for each input video frame encoded and stored / transmitted by the source platform 305 as described above. The generated MPI stacks 130 and 135 each have texture stacks 140 and 160 in which pixels are represented in the RGB format. The MPI stack encoder 310 of the example shown converts the MPI stacks 130 and 135 into respective exemplary texture image stacks 325 and respective exemplary alpha image stacks 330, where the pixels of the texture image 325 are converted into a chrominance format such as luminance and YUV format, and the pixels of the alpha image 330 are unchanged. The MPI stack encoder 310 uses any video encoder, such as an encoder implementing High Efficiency Video Coding (HEVC), to encode the texture image of the texture image stack 325 and the alpha image of the alpha image stack 330 to create an exemplary encoded texture video frame 335 and an exemplary encoded alpha video frame 340. Next, the MPI stack encoder 310 stores the encoded texture video frame 335 and the encoded alpha video frame 340 and / or transmits the encoded texture video frame 335 and the encoded alpha video frame 340 to the target platform 315.

[0032] In the example shown in FIG. 3, the MPI stack decoder 320 of the target platform 315 is associated with the video encoder utilized by the MPI stack encoder 310 and uses an appropriate video decoder, such as a decoder implementing HEVC decoding, to decode the received or otherwise accessed and encoded texture video frame 335 and encoded alpha video frame 340. The result of such video decoding is an exemplary decoded texture image stack 345 corresponding to the texture image stack 325 and an associated exemplary decoded alpha image stack 350 corresponding to the alpha image stack 330, where the decoded texture image 345 is formatted in the YUV format. In the example shown, the MPI stack decoder 320 converts the decoded texture image stack 345 and the associated decoded alpha image stack 350 into an exemplary decoded MPI stack 355 corresponding to the MPI stacks 130 and 135. The decoded MPI stack 355 includes a texture image in which the pixels are formatted in the RGB format. As described above, the MPI stack renderer 200 renders the decoded MPI stack to output an exemplary target image 360 corresponding to the desired target viewpoint.

[0033] To implement MPI compression in accordance with the teachings of the present disclosure, a block diagram of an exemplary video encoding and decoding system 400 is shown in FIG. 4. The illustrated example of the video encoding and decoding system 400, also referred to as video system 400, includes an exemplary source platform 405 that includes an MPI stack generator 100 and an exemplary MPI stack encoder 410, and an exemplary target platform 415 that includes an exemplary MPI stack decoder 420 and an MPI renderer 200. The source platform 405 can be implemented by any computing platform / device such as one or more servers, personal computers, workstations, mobile devices (e.g., tablets such as cellular phones, smartphones, iPads (trademark)), PDAs, Internet appliances, cameras, etc., but is not limited thereto. In some examples, the source platform 405 is implemented by the exemplary processor platform 1900 of FIG. 19 as will be described in more detail below. Similarly, the target platform 415 can be implemented by any computing platform / device such as one or more servers, personal computers, workstations, mobile devices (e.g., tablets such as cellular phones, smartphones, iPads (trademark)), PDAs, Internet appliances, DVD players, CD players, digital video recorders, Blu-ray players, game consoles, personal video recorders, set-top boxes, etc., but is not limited thereto. In some examples, the target platform 415 is implemented by the exemplary processor platform 2000 of FIG. 20 as will be described in more detail below.

[0034] In the example shown in FIG. 4, the MPI stack corresponding to the input image frame is not directly encoded and transmitted. Instead, a single composite texture image and / or a single composite alpha image that captures much of the useful information from the MPI stack is generated, stored / transmitted by the source platform 405. At the target platform 415, the single composite texture image and the single composite alpha image are then used to regenerate the MPI stack that is employed for image rendering.

[0035] More specifically, in the example shown in FIG. 4, the MPI stack generator 100 of the source platform 405 generates respective MPI stacks, such as MPI stacks 130 and 135, for each input video frame encoded and stored / transmitted by the source platform 405 as described above. The MPI stack encoder 410 in the example shown converts each of the texture image stacks 140 / 160 into a corresponding exemplary single composite texture image 430, and / or each of the alpha image stacks 150 / 170 into a corresponding exemplary single composite alpha image 435, and includes an exemplary MPI compressed image encoder 425 for compressing each of the MPI stacks 130 / 135. In some examples, the MPI compressed image encoder 425 performs compression of both the texture image stacks 140 / 160 and the alpha image stacks 150 / 170 to form a single composite texture image 430 and a single composite alpha image 435, respectively. However, in some examples, the MPI compressed image encoder 425 performs compression of either the texture image stacks 140 / 160 to form a single composite texture image 430 or the alpha image stacks 150 / 170 to form a single composite alpha image 435, and not both. Thus, the MPI compressed image encoder 425 is an example of means for encoding an MPI stack (e.g., such as MPI stacks 130 / 135) in a manner that includes one or both of: (i) converting the texture image of the MPI stack (e.g., such as texture image stacks 140 / 160) into a single composite texture image (e.g., such as single composite texture image 430) to generate a compressed multi-plane image stack, and / or (ii) converting the alpha image of the MPI stack (e.g., such as alpha image stacks 150 / 170) into a single composite alpha image (e.g., such as single composite alpha image 435) to generate a compressed multi-plane image stack.

[0036] The illustrated MPI stack encoder 410 encodes a single composite texture image 430 and a single composite alpha image 435 using any video encoder, such as an encoder implementing HEVC, to create an exemplary encoded texture video frame 440 and an exemplary encoded alpha video frame 445. Next, the MPI stack encoder 410 stores the encoded texture video frame 440 and the encoded alpha video frame 445 and / or transmits the encoded texture video frame 440 and the encoded alpha video frame 445 to the target platform 415.

[0037] In the example shown in FIG. 4, the MPI stack decoder 420 of the target platform 415 is associated with the video encoder utilized by the MPI stack encoder 410 and uses a suitable video decoder, such as a decoder implementing HEVC decoding, to decode the received or otherwise accessed and encoded texture video frame 440 and encoded alpha video frame 445. The results of such video decoding are an exemplary decoded composite texture image 450 corresponding to the composite texture image 430 and an associated exemplary decoded composite alpha image 455 corresponding to the composite alpha image 435. In the example shown, the MPI stack decoder 420 includes an exemplary MPI compressed image decoder 460 for extending the decoded composite texture image 450 by converting the decoded composite texture image 450 into respective decoded texture image stacks 465 for inclusion in the corresponding decoded MPI stack. Additionally or alternatively, the MPI compressed image decoder 460 extends the decoded composite alpha image 455 by converting the decoded composite alpha image 455 into respective decoded alpha image stacks 470 for inclusion in the corresponding decoded MPI stack. Thus, the MPI compressed image encoder 425 is an example of means for decoding the compressed MPI stack in a manner that includes one or both of (i) unwrapping a single composite texture image (such as composite texture image 450) to obtain the uncompressed texture images of an uncompressed MPI stack (such as decoded texture image stack 465), and / or (ii) unwrapping a single composite alpha image (such as composite alpha image 455) to obtain the uncompressed alpha images of an uncompressed MPI stack (such as decoded alpha image stack 470). As described above, the MPI stack renderer 200 renders the decoded MPI stack to output an exemplary target image 475 corresponding to a desired target viewpoint.

[0038] In some examples, a single synthetic texture frame is obtained by an MPI compression image encoder 425 from the texture components of a stack of texture frames in the MPI stack in a way that captures sufficient texture information for use by the MPI stack renderer 200. In other examples, a single synthetic texture frame is obtained directly by the MPI compression image encoder 425 from the original camera image. In either type of example, this single synthetic texture frame is replicated N (e.g., N = 32) times, and the replicated texture frames are employed, along with N (e.g., N = 32) alpha planes generated by a machine learning model, in place of the stack of actual texture frames included in the original MPI stack such that a target image is rendered on the target platform 415.

[0039] In some examples, a single synthetic alpha image is obtained by an MPI compression image encoder 425 from the alpha components of a stack of alpha frames in the MPI stack in a way that captures sufficient alpha (transparency) information for use by the MPI stack renderer 200. In other examples, a single synthetic alpha image is obtained by the MPI compression image encoder 425 from a depth map acquired from a source camera associated with the input MPI stack. In either type of example, the target image is rendered on the target platform 415 by using the single synthetic alpha image to generate an approximation of the original stack of N (e.g., N = 32) alpha frames included in the original MPI stack.

[0040] In some examples, a single synthetic texture image and a single synthetic alpha image may be used alone or in combination. For example, a single texture frame from an MPI stack and a set of N (e.g., N = 32) alpha planes may be stored / transmitted, a set of N (e.g., N = 32) texture planes and a single synthetic alpha image may be stored / transmitted, or a single synthetic texture image and a single synthetic alpha image may be stored / transmitted. In some examples, the single synthetic texture image can be projected to the target viewpoint by directly using the single synthetic texture image and the single synthetic alpha image and interpreting the single synthetic alpha image as a coarse depth map.

[0041] A block diagram of an exemplary implementation of the MPI stack encoder 410 of FIG. 4 is shown in FIG. 5. The exemplary MPI stack encoder 410 of FIG. 5 has an exemplary MPI compressed image encoder 425 and an exemplary data interface 505. The exemplary MPI compressed image encoder 425 of FIG. 5 has an exemplary texture stack compressor 510 and an exemplary alpha stack compressor 515.

[0042] In the example shown in FIG. 5, the data interface 505 accesses the input MPI stack corresponding to the source camera viewpoint. As described above, the input MPI stack includes an exemplary stack of texture images / planes such as the input texture image stack 140 and a corresponding exemplary stack of alpha images / planes such as the input alpha image stack 150. Also, as described above, each one of the alpha images in the input alpha image stack 150 includes a pixel value representing the transparency of the corresponding pixel in each one of the paired texture images in the input texture image stack 140. Thus, the data interface 505 is an example of means for accessing the input multi-plane image stack corresponding to the source camera viewpoint.

[0043] In the example shown in FIG. 5, the MPI compressed image encoder 425 includes a texture stack compressor 510 for converting the texture images of the input texture image stack 140 into a single composite texture image such as a single composite texture image 430. In the example shown in FIG. 5, the MPI compressed image encoder 425 includes an alpha stack compressor 515 for converting the alpha images of the input alpha image stack 150 into a single composite alpha image such as a single composite alpha image 435. In the example shown, the data interface 505 outputs the single composite texture image 430 and the single composite alpha image 435 as a compressed MPI stack corresponding to the input MPI stack. Thus, the data interface 505 is an example of means for outputting a compressed MPI stack.

[0044] A block diagram of an exemplary implementation of the MPI stack decoder 420 of FIG. 4 is shown in FIG. 6. The exemplary MPI stack decoder 420 of FIG. 6 has an exemplary MPI compressed image decoder 460 and an exemplary data interface 605. The exemplary MPI compressed image decoder 460 of FIG. 6 has an exemplary texture stack expander 610 and an exemplary alpha stack expander 615.

[0045] In the example shown in FIG. 6, the data interface 605 accesses an input compressed multi-plane image stack corresponding to the source camera view. As described above, in connection with FIG. 5, the compressed multi-plane image stack includes (i) a single composite texture image such as the composite texture image 450, which is expanded to represent the uncompressed texture images included in the decoded texture image stack, such as the decoded texture image stack 465, and / or (ii) a single composite alpha image such as the composite alpha image 455, which is expanded to represent the uncompressed alpha images included in the decoded alpha image stack, such as the decoded alpha image stack 470. As described above, each one of the uncompressed alpha images includes a pixel value representing the transparency of the corresponding pixel in each one of the corresponding pairs of the uncompressed texture images. Thus, the data interface 605 is an example of means for accessing a compressed multi-plane image stack corresponding to the source camera view.

[0046] In the example shown in FIG. 6, the MPI compression image decoder 460 includes a texture stack expander 610 for expanding a single composite texture image 450 in order to obtain the uncompressed texture images to be included in the decoded texture image stack 465. In the example shown in FIG. 5, the MPI compression image decoder 460 includes an alpha stack expander 615 for expanding a single composite alpha image 455 in order to obtain the uncompressed alpha images to be included in the decoded alpha image stack 470. In the example shown, the data interface 605 outputs a decoded (uncompressed) MPI stack including the decoded texture image stack 465 and the decoded alpha image stack 470. Thus, the data interface 605 is an exemplary means for outputting an uncompressed MIP stack including uncompressed texture images (e.g., the uncompressed texture images of the decoded texture image stack 465) and uncompressed alpha images (e.g., the uncompressed alpha images included in the decoded alpha image stack 470).

[0047] Returning to FIG. 5, in some examples, texture stack compressor 510 converts the texture images of input texture image stack 140 into a single composite texture image 430 in source platform 405 based on the observation that for each pixel position, the alpha values are large only for a few alpha planes of input alpha image stack 150 corresponding to input texture image stack 140. Thus, for that pixel position, the texture values of the texture planes corresponding to the few alpha planes with large alpha values will have the most influence on the rendered target image. To utilize this observation, in some examples, texture stack compressor 510 generates a single composite texture image 430 to have a pixel value that is the alpha-weighted average of the texture values across the texture images included in input texture image stack 140.

[0048] Mathematically, the foregoing exemplary operation of texture stack compressor 510 can be expressed as follows. For pixel position i, the texture pixel value p of single composite texture image 430 i,j is defined by Equation 1.

Equation

[0049] FIG. 7 shows an exemplary MPI texture stack compression operation 700 performed by a texture stack compressor 510 included in the MPI stack encoder 410 of FIG. 5. In the exemplary operation 700 shown, the texture stack compressor 510 processes an input texture image stack 140 and an input alpha image stack 150 of an input MPI stack according to Equation 1 to generate a single composite texture image 430 having an alpha weighted average of texture values across the input texture image stack 140.

[0050] Returning to FIG. 6, the texture stack expander 610 expands a single composite texture image 450 on a target platform 415 by replicating the single composite texture image 450 N (e.g., N = 32) times to obtain non-compressed texture images to include in the non-compressed texture image stack 465 of the decoded MPI stack. FIG. 8 shows a corresponding exemplary MPI texture stack expansion operation 800 performed by the texture stack expander 610 of the MPI stack decoder 420. In the exemplary operation 800 shown, the texture stack expander 610 replicates a single composite texture image 450 to obtain a non-compressed texture image stack 465. In the example shown, the corresponding alpha image stack obtained on the target platform can be the non-compressed alpha image stack 470 generated by the alpha stack expander 615 or the original input alpha image stack 150.

[0051] As can be seen from the above example, the composite texture images 430 / 450 may be similar to the original texture image frames from the source camera. Thus, in some examples, the texture stack compressor 510 included in the MPI stack encoder 410 uses the original texture image frames from the source camera as the composite texture images 430 / 450.

[0052] 5, in some examples, the alpha stack compressor 515 converts the alpha images of the input alpha image stack 150 at the source platform 405 into a single composite alpha image 435 based on the observation that for each pixel location, only a small number of alpha planes in the input alpha image stack 150 have large alpha values. To take advantage of this observation, in some examples, the alpha stack compressor 515 generates the single composite alpha image 435, also referred to as an alpha map, to encode a subset of the alpha planes with large alpha values and their alpha values.

[0053] In some such examples, the alpha stack compressor 515 generates a single composite alpha image 435 (alpha map) to capture the location and size of the alpha plane with the largest alpha value and that of the neighboring alpha plane with the next largest alpha value. Mathematically, the alpha plane with the largest alpha value at pixel location i is denoted by p i,m Let the corresponding alpha value be a i,m The neighboring plane with the next largest alpha value is p i,m +1 or p i,m -1, and the alpha value is a i,2 The bias value b at pixel position i i is defined by Equation 2.

number

[0054] For example, in an alpha plane where one pixel position i corresponds to an alpha plane index = 10 and has a maximum alpha value of 200, and a second largest alpha value of 60 in an alpha plane corresponding to an alpha plane index = 9, the alpha plane index p of the plane with the largest alpha value i,m is 10. Since plane 9 is smaller than plane 10, the bias factor has a negative sign. The bias value b i is a value of 2 representing 8 * 60 / 260 (where 8 corresponds to 8-bit precision). Thus, the composite alpha value d of pixel position i in this example i is 8 * 10 + (-2) = 78.

[0055] In some examples, the positions of some pixels do not have any significant alpha values across the entire plane. In some examples, some pixels are even completely transparent across all alpha planes. A set of pixels that goes against such intuition can be an artifact of the machine learning model used to generate the MPI stack. These pixels, although somewhat infrequent, should be maintained to indicate a low prediction probability in the field of view. Considering these pixels, pixel values 0 - 7 are used to map the pixel values in these ranges. For example, a composite alpha value d of 0 can be used to represent the position of a completely transparent pixel. In some examples, composite alpha values from 1 - 7 are used to indicate the range of the maximum value across two planes such that the higher the composite alpha value, the higher the maximum value for the two largest planes at that pixel position. i can be used. In some examples, composite alpha values from 1 - 7 are used to indicate the range of the maximum value across two planes such that the higher the composite alpha value, the higher the maximum value for the two largest planes at that pixel position.

[0056] In some examples, the alpha stack compressor 515 filters the resulting single composite alpha image 435 (alpha map) with a smoothing filter, at the associated cost of a slight reduction in accuracy, to reduce the bitrate for transmission.

[0057] 6, in some examples, the alpha stack decompressor 615 expands a single composite alpha image 455 (alpha map) in the target platform 415 as follows: Assuming that a single composite alpha image 455 is generated based on Equations 2-4 as described above, the alpha stack decompressor 615 uses the single composite alpha image 455 (alpha map) to approximately reconstruct N (e.g., N=32) alpha images in the uncompressed alpha image stack 470 as follows: The composite alpha value d for pixel location i of the composite alpha image 435 (alpha map) defined in Equation 4 above is i Given , the alpha stack decompressor 615 calculates the following values as given in Equations 5 and 6:

number

number

number

number

number

[0058] Then, for pixel location i, the alpha stack decompressor 615 calculates the following values given by Equations 7 and 8:

number

number

number

number

number

[0059] The exemplary implementation of the alpha stack compressor 515 and alpha stack decompressor 615 based on Equations 2-8 corresponds to a special case of the following exemplary implementation: For pixel location i, the value of the composite alpha image (alpha map) is defined by Equation 9.

number

number

number

[0060] In some examples, the depth map from the source camera can be used by the encoder-side alpha stack compressor 515 to generate a single composite alpha image 435 (alpha map) to replace the input alpha image stack 150. In a first exemplary implementation based on a depth map from a source camera, illustrated by exemplary operations 900 in FIG. 9 , assume that N=32 alpha planes are uniformly spaced in the numerical range [0...65535] corresponding to the 16-bit depth map, with a gap of I=65535 / 31 between each plane. A pixel having depth d in the 16-bit depth map (represented by reference numeral 905 in FIG. 9 ) will be located between alpha plane j (represented by reference numeral 910 in FIG. 9 ) and alpha plane j+1 (represented by reference numeral 915 in FIG. 9 ), where the value of j is given by Equation 12 below:

number

number

[0061] In a second exemplary implementation based on the depth map from the source camera, the alpha stack compressor 515 transmits the depth map to the decode side. The alpha stack expander 615 on the decode side initially sets the alpha values of the stack of N = 32 non-compressed alpha planes to zero. Referring to FIG. 9 and the description of the first exemplary implementation above, a pixel having a depth d (represented by reference numeral 905 in FIG. 9) in a 16-bit depth map is located between alpha plane j (represented by reference numeral 910 in FIG. 9) and alpha plane j + 1 (represented by reference numeral 915 in FIG. 9), and the value of j will be given by Equation 12. The alpha stack expander 615 on the decode side calculates the values d j and d j+1 according to Equations 14 and 15. d j = d - j * I Equation 14 d j+1 = I - d j Equation 15 Then, the alpha stack expander 615 sets the value of pixel i in alpha plane j indicated by a i,j and the value of pixel i in alpha plane j + 1 indicated by a i,j+1 according to the following Equations 16 and 17.

Equation

[0062] In a third exemplary implementation based on the depth map from the source camera, the alpha stack compressor 515 adds a preprocessing stage to the first exemplary implementation described above. In this example, the alpha stack compressor 515 applies the k-means method with k = 32 to the 16-bit depth map from the camera and generates an alpha plane at the position of the depth range that best matches the objects in the scene. The depth of the alpha plane (corresponding to the position where the centroid generated by the k-means method is found) is transmitted as metadata to the target platform (decoding side). This allows the true depth d to be approximated more accurately using the encoding technique used in the first exemplary implementation described above.

[0063] In a fourth exemplary implementation based on the depth map from the source camera, the alpha stack compressor 515 transmits the original 16-bit depth map to the decoder. In some examples, the alpha stack compressor 515 reduces the accuracy of the depth map to use fewer bits, such as 10 bits or some other value. At the decoder, the depth map is used to directly warp (e.g., reprojection) the texture frame from the source camera to the frame of the target view. In this fourth exemplary implementation, the use of the MPI stack can be removed, and as a result, the use of the machine learning model described above can be avoided.

[0064] 4-6, one or more of the elements, processes, and / or devices shown in FIGURES 4-6 may be combined, divided, rearranged, omitted, eliminated, and / or implemented in any other manner. Additionally, the exemplary MPI stack generator 100, the exemplary MPI renderer 200, the exemplary source platform 405, the exemplary MPI stack encoder 410, the exemplary target platform 415, the exemplary MPI stack decoder 420, the exemplary MPI compressed image encoder 425, the exemplary MPI compressed image decoder 460, the exemplary data interface 505, the exemplary texture stack compressor 510, the exemplary alpha stack compressor 515, the exemplary data interface 605, the exemplary texture stack decompressor 610, the exemplary alpha stack decompressor 615, and / or more generally, the exemplary video encoding and decoding system 400 of FIGURES 4-6 may be implemented by hardware, software, firmware, and / or any combination of hardware, software, and / or firmware.Accordingly, for example, any one of the exemplary MPI stack generator 100, the exemplary MPI renderer 200, the exemplary source platform 405, the exemplary MPI stack encoder 410, the exemplary target platform 415, the exemplary MPI stack decoder 420, the exemplary MPI compressed image encoder 425, the exemplary MPI compressed image decoder 460, the exemplary data interface 505, the exemplary texture stack compressor 510, the exemplary alpha stack compressor 515, the exemplary data interface 605, the exemplary texture stack expander 610, the exemplary alpha stack expander 615, and / or, more generally, the exemplary video encoding and decoding system 400 may be implemented by one or more analog or digital circuits, logic circuits, programmable processors, programmable controllers, graphics processing units (GPUs), digital signal processors (DSPs), application specific integrated circuits (ASICs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs) and / or field programmable logic devices (FPLDs).If any of the apparatus or system claims of this patent are read to cover a purely software and / or firmware implementation, then at least one of the exemplary MPI stack generator 100, the exemplary MPI renderer 200, the exemplary source platform 405, the exemplary MPI stack encoder 410, the exemplary target platform 415, the exemplary MPI stack decoder 420, the exemplary MPI compressed image encoder 425, the exemplary MPI compressed image decoder 460, the exemplary data interface 505, the exemplary texture stack compressor 510, the exemplary alpha stack compressor 515, the exemplary data interface 605, the exemplary texture stack decompressor 610, the exemplary alpha stack decompressor 615, and / or more generally, the exemplary video encoding and decoding system 400 is expressly defined herein to include a non-transitory computer-readable storage device or storage disk containing software and / or firmware, such as a memory, a digital versatile disk (DVD), a compact disk (CD), a Blu-ray disk, or the like. Further still, the exemplary video encoding and decoding system 400 may include one or more elements, processes, and / or devices in addition to or instead of those illustrated in Figures 4-6, and / or may include more than one of any or all of the illustrated elements, processes, and devices. As used herein, the term "in communication," including variations thereof, encompasses direct communication and / or indirect communication through one or more intervening components, and does not require direct physical (e.g., hardwired) communication and / or constant communication, but rather additionally includes selective communication at regular intervals, scheduled intervals, aperiodic intervals, and / or one-time events.

[0065] Flowcharts depicting example hardware logic, machine-readable instructions, hardware-implemented state machines, and / or any combination thereof for implementing the MPI stack encoder 410 are shown in FIGS. 10-14. In these examples, the machine-readable instructions may be one or more executable programs or portions of executable programs for execution by a computer processor and / or processor circuitry, such as the processor 1912 shown in the example processor platform 1900 described below in connection with FIG. 19. The one or more programs, or portions thereof, may be embodied in software stored on a non-transitory computer-readable storage medium, such as a CD-ROM, floppy disk, hard drive, DVD, Blu-ray Disc™, or memory associated with the processor 1912, although the one or more programs, in their entirety, and / or portions thereof, may alternatively be executed by a device other than the processor 1912 and / or embodied in firmware or dedicated hardware. Furthermore, although the example programs are described with reference to the flowcharts shown in FIGS. 10-14, many other methods of implementing the example MPI stack encoder 410 may alternatively be used. 10-14, the order of execution of the blocks may be changed, and / or some of the described blocks may be modified, removed, combined, and / or fragmented into multiple blocks. Additionally or alternatively, any or all of these blocks may be implemented by one or more hardware circuits (e.g., discrete and / or integrated analog and / or digital circuits, FPGAs, ASICs, comparators, operational amplifiers (op-amps), logic circuits, etc.) configured to perform the corresponding operations without executing software or firmware. The processor circuitry may be distributed locally across different network locations and / or one or more devices (e.g., a multi-core processor on a single machine, multiple processors distributed across a server rack, etc.).

[0066] Flowcharts depicting example hardware logic, machine-readable instructions, hardware-implemented state machines, and / or any combination thereof for implementing the MPI stack decoder 420 are shown in FIGS. 15-17. In these examples, the machine-readable instructions may be one or more executable programs or portions of executable programs for execution by a computer processor and / or processor circuitry, such as the processor 2012 shown in the example processor platform 2000 described below in connection with FIG. 20. The one or more programs, or portions thereof, may be embodied in software stored on a non-transitory computer-readable storage medium, such as a CD-ROM, floppy disk, hard drive, DVD, Blu-ray disk, or memory associated with the processor 2012, although the one or more programs, in their entirety, and / or portions thereof, may alternatively be executed by a device other than the processor 2012 and / or embodied in firmware or dedicated hardware. Furthermore, although the example programs have been described with reference to the flowcharts shown in FIGS. 15-17, many other methods of implementing the example MPI stack decoder 420 may alternatively be used. 15-17, the order of execution of the blocks may be changed, and / or some of the described blocks may be modified, removed, combined, and / or fragmented into multiple blocks. Additionally or alternatively, any or all of these blocks may be implemented by one or more hardware circuits (e.g., discrete and / or integrated analog and / or digital circuits, FPGAs, ASICs, comparators, operational amplifiers (op-amps), logic circuits, etc.) configured to perform the corresponding operations without executing software or firmware. The processor circuitry may be distributed locally across different network locations and / or one or more devices (e.g., a multi-core processor on a single machine, multiple processors distributed across a server rack, etc.).

[0067] The machine-readable instructions described in this specification may be stored in one or more of a compressed format, an encrypted format, a fragmentation format, a compilation format, an executable format, a package format, etc. The machine-readable instructions described herein may be stored as data or data structures (e.g., portions of instructions, code, representations of code, etc.) that can be used to create, manufacture, and / or generate machine-executable instructions. For example, the machine-readable instructions may be fragmented and stored in one or more storage devices and / or computing devices (e.g., servers) located at the same or different locations of a network or a collection of networks (e.g., within a cloud, within an edge device, etc.). The machine-readable instructions may require one or more of installation, modification, adaptation, update, combination, supplementation, configuration, decryption, expansion, unpacking, distribution, rearrangement, compilation, etc. to be directly readable, interpretable, and / or executable by a computing device and / or other machine. For example, the machine-readable instructions may be stored in multiple portions that are individually compressed, encrypted, and stored on separate computing devices. These portions, when decoded, expanded, and combined, form a set of executable instructions that implement one or more functions that together form a program such as that described herein.

[0068] In another example, the machine-readable instructions can be stored in a state readable by a processor circuit, but additional libraries (e.g., Dynamic Link Libraries (DLLs)), software development kits (SDKs), application programming interfaces (APIs), etc. may be required to execute the instructions on a particular computing device or other device. In another example, the machine-readable instructions may need to be configured (e.g., stored settings, data inputs, recorded network addresses, etc.) before the machine-readable instructions and / or corresponding programs can be executed in whole or in part. Thus, as used herein, a machine-readable medium can include machine-readable instructions and / or programs regardless of the particular format or state of the machine-readable instructions and / or programs when stored, or while in a stationary or moving state.

[0069] The machine-readable instructions described herein may be represented by any past, present, or future instruction language, scripting language, programming language, etc. For example, the machine-readable instructions may be represented using any of the following languages, namely, C, C++, Java®, C#, Perl, Python, JavaScript®, HyperText Markup Language (HTML), Structured Query Language (SQL), Swift, etc.

[0070] As described above, the exemplary processes of FIGS. 10-17 may be implemented using executable instructions (e.g., computer and / or machine-readable instructions) stored on a non-transitory computer and / or machine-readable medium such as a hard disk drive, flash memory, read-only memory, compact disk, digital versatile disk, cache, random access memory, and / or any other storage device or storage disk where information is stored for any period of time (e.g., for a long period of time, permanently, for a short time, during a temporary buffer, and / or while caching information). As used herein, the term non-transitory computer-readable medium is explicitly defined to include any type of computer-readable storage device and / or storage disk, and to exclude propagated signals and to exclude transmission media. Also, as used herein, the terms "computer-readable" and "machine-readable" are considered equivalent unless otherwise noted.

[0071] The terms "including" and "comprising" (and all their forms and tenses) are used herein as open-ended terms. Thus, whenever a claim uses any form of "include" or "comprise" (e.g., comprises, includes, comprising, including, having, etc.) as a preamble or within any kind of claim recitation, it is to be understood that additional elements, terms, etc. may be present without departing from the scope of the corresponding claim or recitation. As used herein, when the term "at least" is used as a transitional term, e.g., in the preamble of a claim, this language is also open-ended in the same way that the terms "comprising" and "including" are open-ended. When the term "and / or" is used in the form, for example, A, B, and / or C, it refers to any combination or subset of A, B, and C, such as, for example, (1) A only, (2) B only, (3) C only, (4) A and B, (5) A and C, (6) B and C, and (7) A, B, and C. As used herein, in the context of describing structures, components, items, objects, and / or things, the phrase "at least one of A and B" is intended to mean implementations that include any of (1) at least one A, (2) at least one B, and (3) at least one A and at least one B. Similarly, as used herein, in the context of describing structures, components, items, objects, and / or things, the phrase "at least one of A or B" is intended to mean implementations that include any of (1) at least one A, (2) at least one B, and (3) at least one A and at least one B.As used herein, in the context of describing the performance or execution of a process, instruction, action, activity, and / or step, the phrase "at least one of A and B" is intended to mean an implementation example that includes any one of (1) at least one A, (2) at least one B, and (3) at least one A and at least one B. Similarly, as used herein, in the context of describing the performance or execution of a process, instruction, action, activity, and / or step, the phrase "at least one of A or B" is intended to mean an implementation example that includes any one of (1) at least one A, (2) at least one B, and (3) at least one A and at least one B.

[0072] As used herein, a single reference (e.g., "a", "an", "first", "second", etc.) does not exclude a plurality. As used herein, the term "an" entity refers to one or more of that entity. The terms "a" (or "an"), "one or more", and "at least one" may be used interchangeably herein. Further, although multiple means, elements, or method actions are individually recited, they may be implemented, for example, by a single unit or processor. Further, individual features may be included in different examples or claims, but these features may, in some cases, be combined, and inclusion in different examples or claims does not imply that combinations of these features are not feasible and / or not advantageous.

[0073] An exemplary program 1000 that may be executed to implement the MPI stack encoder 410 of FIG. 4 and / or FIG. 5 is represented by the flowchart shown in FIG. 10. Referring to the above-described drawings and the associated written description, the exemplary program 1000 of FIG. 10 begins execution at block 1005, where the MPI stack encoder 410 determines whether the input texture image stack 140 included in the generated MPI stack 135 corresponding to the source camera view is compressed. If the input texture image stack 140 is compressed (block 1005), at block 1010, the MPI compression image encoder 425 of the MPI stack encoder 410 converts the input texture image stack 140 into a single composite texture image 430, as described above. Exemplary programs implementing the processing at block 1010 are shown in FIGS. 11 and 12 and are described in further detail below. At block 1015, the MPI compression image encoder 425 outputs a single composite texture image 430. Thereafter, control proceeds to block 1020. However, if the input texture image stack 140 is not compressed (block 1005), at block 1025, the MPI stack encoder 410 outputs the texture image stack 140. Thereafter, control proceeds to block 1020.

[0074] In block 1020, the MPI stack encoder 410 determines whether the input alpha image stack 150 included in the generated MPI stack 135 corresponding to the source camera view is compressed. If the input alpha image stack 150 is compressed (block 1020), in block 1030, the MPI compressed image encoder 425 of the MPI stack encoder 410 converts the input alpha image stack 150 into a single composite alpha image 435 as described above. Exemplary programs implementing the processing in block 1030 are shown in FIGS. 13 and 14 and are further described in detail below. In block 1035, the MPI compressed image encoder 425 outputs a single composite alpha image 435. Then, the control proceeds to block 1040. However, if the input alpha image stack 150 is not compressed (block 1020), in block 1045, the MPI stack encoder 410 outputs the alpha image stack 150. Then, the control proceeds to block 1040.

[0075] In block 1040, the MPI stack encoder 410 determines whether other MPI stacks 135 corresponding to other source camera views are to be processed. If other MPI stacks 135 are to be processed, the control returns to block 1005. Otherwise, the execution of the exemplary program 1000 ends.

[0076] An exemplary program 1010P1 that may be executed to implement the texture stack compressor 510 of the MPI compression image encoder 425 and / or the processing at block 1010 of FIG. 10 is shown in FIG. 11. With reference to the above-mentioned figures and associated written description, the exemplary program 1010P1 of FIG. 11 begins execution at block 1105, where the texture stack compressor 510 accesses the input texture image stack 140 from the data interface 505. At block 1110, the texture stack compressor 510 accesses the input alpha image stack 150 from the data interface 505. At block 1115, the texture stack compressor 510 begins processing each texture image in the texture image stack 140. For example, at block 1120, the texture stack compressor 510 weights the pixel values of a given texture image by the pixel values of the corresponding alpha image, as described above. For example, the processing at block 1120 may involve weighting the numerator term t in Equation 1 above. i,j α i,j At block 1125, the texture stack compressor 510 continues processing the texture images until all texture images in the texture image stack 140 have been processed.

[0077] At block 1130, the texture stack compressor 510 begins iterating through each pixel location i to determine a pixel value for the single composite texture image 430. For example, at block 1135, the texture stack compressor 510 uses the weighted texture images determined at block 1120 to determine an alpha-weighted average pixel value for pixel location i of the single composite texture image 430, as described above. For example, the processing at block 1120 begins when the texture stack compressor 510 iterates through the output term p i,jIt can correspond to calculating. In block 1140, the texture stack compressor 510 continues to process pixel positions until all pixel positions of the single synthesized texture image 430 are evaluated. In block 1145, the texture stack compressor 510 outputs a single synthesized texture image 430 via the data interface 505, where each pixel position i is set to the corresponding alpha-weighted average pixel value p i,j determined in block 1135. Thereafter, the execution of the exemplary program 1010P1 ends.

[0078] An exemplary program 1010P2 that can be executed to implement the texture stack compressor 510 of the MPI compressed image encoder 425 and / or the processing in block 1010 of FIG. 10 is shown in FIG. 12. Referring to the above drawings and the associated written description, the exemplary program 1010P2 of FIG. 12 starts execution at block 1205, where the texture stack compressor 510 accesses, via the data interface 505, the source camera texture image associated with the source camera viewpoint corresponding to the input texture image stack 140. In block 1210, the texture stack compressor 510 outputs a single synthesized texture image 430 via the data interface 505, where each pixel position i is set to the corresponding pixel value of the source camera texture image. Thereafter, the execution of the exemplary program 1010P2 ends.

[0079] An exemplary program 1030P1 that can be executed to implement the alpha stack compressor 515 of the MPI compressed image encoder 425 and / or the processing at block 1030 of FIG. 10 is shown in FIG. 13. Referring to the above-described drawings and the associated written description, the exemplary program 1030P1 of FIG. 13 begins execution at block 1305, where the alpha stack compressor 515 accesses the input alpha image stack 150 from the data interface 505. At block 1310, the alpha stack compressor 515 begins processing for each pixel position i to determine pixel values for a single composite alpha image 435. For example, at block 1315, the alpha stack compressor 515, as described above, at a given pixel position i, determines a first alpha plane index value (e.g., p i,m ) that identifies the alpha plane of the input alpha image stack 150 having the largest alpha value (e.g., a i,m ). At block 1320, the alpha stack compressor 515, as described above, at a given pixel position i, determines a second alpha plane index value (e.g., p i,2 + 1 or p i,m - 1) that identifies the neighboring alpha plane having the next largest alpha value (e.g., a i,m ). At block 1325, the alpha stack compressor 515, as described above, based on the largest alpha value (e.g., a i,m ) and the next largest alpha value (e.g., a i,2 ), determines a bias value (e.g., b i ) for the given pixel position i. For example, at block 1325, the alpha stack compressor 515 may determine the bias value b i according to Equation 2 as described above. At block 1330, the alpha stack compressor 515, as described above, to determine a composite alpha value (e.g., d i ) for the given pixel position i, uses the first alpha plane index value (e.g., p i,m ) and the bias value (e.g., b i) are combined. For example, in block 1330, the alpha stack compressor 515, as described above, determines the composite alpha value d for a given pixel position i according to Equation 3. i may be determined. In block 1335, the alpha stack compressor 515 continues processing pixel positions until all pixel positions of the single composite alpha image 435 have been evaluated. In block 1340, the alpha stack compressor 515 outputs a single composite alpha image 435 via the data interface 505, where each pixel position i is set to the corresponding composite alpha value d i determined in block 1330. Thereafter, the execution of the exemplary program 1030P1 ends.

[0080] An exemplary program 1030P2 that can be executed to implement the alpha stack compressor 515 of the MPI compressed image encoder 425 and / or the processing in block 1030 of FIG. 10 is shown in FIG. 14. Referring to the above drawings and the associated written description, the exemplary program 1030P2 of FIG. 14 starts execution in block 1405, where the alpha stack compressor 515 accesses the source camera depth image associated with the source camera viewpoint corresponding to the input alpha image stack 150 via the data interface 505. In block 1410, the alpha stack compressor 515 starts processing each pixel position i to determine the pixel value for the single composite alpha image 435. For example, in block 1415, the alpha stack compressor 515 determines the alpha plane index value (e.g., j) based on the depth image value (e.g., d) at the given pixel position i, as described above. For example, in block 1415, the alpha stack compressor 515 may determine the alpha plane index value j according to Equation 12 at the given pixel position i, as described above. In block 1420, the alpha stack compressor 515, as described above, based on the alpha plane index value (e.g., j) and the depth image value (e.g., d) at the given pixel position i, determines the bias value (e.g., d for the given pixel position ij For example, in block 1420, the alpha stack compressor 515 determines the value of the equation d j = dj*I according to the bias value d j In block 1425, the alpha stack compressor 515 may determine the composite alpha value (e.g., d i ) to determine the alpha plane index value (e.g., j) and bias value (e.g., d i For example, in block 1425, the alpha stack compressor 515 combines the resulting alpha value d for a given pixel location i according to Equation 13, as described above. i At block 1430, the alpha stack compressor 515 continues processing pixel positions until all pixel positions of the single composite alpha image 435 have been evaluated. At block 1435, the alpha stack compressor 515 outputs the single composite alpha image 435 via the data interface 505, where each pixel position i has a corresponding composite alpha value d determined at block 1425. i The exemplary program 1030P1 then finishes execution.

[0081] An exemplary program 1500 that may be executed to implement the MPI stack decoder 420 of FIG. 4 and / or FIG. 6 is represented by the flowchart shown in FIG. 15. Referring to the above-described drawings and the associated written description, the exemplary program 1500 of FIG. 15 begins execution at block 1505, where the MPI stack decoder 420 determines whether the composite texture image 450 corresponding to the source camera view is to be extended. If there is a composite texture image 450 to be extended (block 1505), at block 1510, the MPI compressed image decoder 460 of the MPI stack decoder 420 extends the composite texture image 450 in order to obtain the uncompressed texture image stack 465, as described above. An exemplary program for implementing the processing at block 1510 is shown in FIG. 16 and is described in more detail below. At block 1015, the MPI compressed image decoder 460 outputs the uncompressed texture image stack 465. Thereafter, control proceeds to block 1520. However, if there is no composite texture image 450 to be extended (block 1505), at block 1525, the MPI stack decoder 420 outputs the received texture image stack (since there is no texture stack compression on the source platform 405). Thereafter, control proceeds to block 1520.

[0082] In block 1520, the MPI stack decoder 420 determines whether the synthesized alpha image 455 corresponding to the source camera view is to be extended. If there is a synthesized alpha image 455 to be extended (block 1520), in block 1530, the MPI compressed image decoder 460 of the MPI stack decoder 420 extends the synthesized alpha image 455 in order to obtain the uncompressed alpha image stack 470 as described above. An exemplary program for implementing the process in block 1530 is shown in FIG. 17 and will be described in more detail below. In block 1535, the MPI compressed image decoder 460 outputs the uncompressed alpha image stack 470. Then, the control proceeds to block 1540. However, if there is no synthesized alpha image 455 to be extended (block 1520), in block 1545, the MPI stack decoder 420 outputs the received alpha image stack (since there is no alpha stack compression in the source platform 405). Then, the control proceeds to block 1520.

[0083] In block 1540, the MPI stack decoder 420 determines whether an MPI extension is to be performed for other source camera views. If other source camera views receive an MPI extension, the control returns to block 1505. Otherwise, the execution of the exemplary program 1500 ends.

[0084] An exemplary program 1510P that may be executed to implement the processing in the texture stack expander 610 of the MPI compressed image decoder 460 and / or block 1510 of FIG. 15 is shown in FIG. 16. Referring to the above drawings and the associated written description, the exemplary program 1510P of FIG. 16 starts execution at block 1605, where the texture stack expander 610 accesses the composite texture image 450 via the data interface 605. At block 1610, the texture stack expander 610 duplicates the composite texture image 450 to form the uncompressed texture image stack 465 as described above. At block 1615, the texture stack expander 610 outputs the uncompressed texture image stack 465 via the data interface 605. Thereafter, the execution of the exemplary program 1510P ends.

[0085] An exemplary program 1530P that may be executed to implement the processing in the alpha stack expander 615 of the MPI compressed image decoder 460 and / or block 1530 of FIG. 15 is shown in FIG. 17. Referring to the above drawings and the associated written description, the exemplary program 1530P of FIG. 17 starts execution at block 1705, where the alpha stack expander 615 accesses the composite alpha image 455 via the data interface 605. At block 1710, the alpha stack expander 615 starts processing the respective pixel position i of the composite alpha image 455 to determine the uncompressed alpha image stack 470. For example, at block 1715, the alpha stack expander 615 determines, as described above, a first alpha plane index value (e.g., i ) for a given pixel position i based on the composite alpha value (e.g., d [Number] ). For example, at block 1715, the alpha stack expander 615 determines the first alpha plane index value

Number

Number

Number

Number

Number

Number

Number

Number

Number

Number

Number

[0086] In block 1730, the alpha stack expander 615, as described above, the bias value for pixel position i (

Number

Number

Number

Number

Number

Number

Number

Number

Number

Number

Number

[0087] FIG. 18 shows an exemplary procedure 1800 executed by the exemplary video encoding and decoding system 400 of FIG. 4 to render a target image using a single composite alpha image 435 / 455 and a single composite texture image 430 / 450 determined by the MPI compression image encoder 425 of the MPI stack encoder 410 as described above. In the exemplary procedure 1800 shown, as an exemplary coarse depth map, the MPI stack decoder 420 decodes a single composite alpha image 435 / 455 (also referred to as an alpha map or a compressed alpha map), and an exemplary obtained depth map 1820 is created from the exemplary coarse depth map. The obtained depth map 1820 is used to directly project the single composite texture image 430 / 450 onto the rendered target image 1815 at the target viewpoint.

[0088] In the example shown, the obtained depth map 1820 is determined as follows. First, the composite alpha image 435 / 455 (e.g., an alpha map or a compressed alpha map) is treated as a coarse 8-bit depth map. To obtain a 10-bit or 16-bit depth map, the MPI stack decoder 420 multiplies the values of the composite alpha image 435 / 455 (e.g., an alpha map or a compressed alpha map) by 4 or 256 respectively. From an 8-bit depth map having a pixel value d, the MPI stack decoder 420 determines the raw depth z according to Equation 18.

Equation

[0089] In exemplary procedure 1800, the raw depth is used to project each pixel into 3D space using the camera's intrinsic properties (focal length, principal point) and extrinsic properties (position of the camera center relative to world coordinates). For example, in block 1825, the pixel can be projected directly onto the image of the target field of view (which can involve less computation), and / or in block 1830, the raw depth can be used to create a 3D triangular mesh that is projected onto the target and rasterized (which can involve more computation).

[0090] In block 1835, the images obtained by projecting multiple cameras are then blended to generate the final rendered target image 1815. Blending uses a weighting function that depends on the distance from the source to the target camera center.

[0091] FIG. 19 is a block diagram of an exemplary processor platform 1900 configured to execute the instructions of FIGS. 10 - 14 to implement the MPI stack encoder 410 of FIGS. 4 and / or 5. Processor platform 1900 can be, for example, a server, personal computer, workstation, self - learning machine (e.g., neural network), mobile device (e.g., mobile phone, smartphone, tablet such as iPad (trademark)), personal digital assistant (PDA), Internet appliance, DVD player, CD player, digital video recorder, Blu - ray player, game console, personal video recorder, set - top box, digital camera, headset or other wearable device, or any other type of computing device.

[0092] The processor platform 1900 of the example shown includes a processor 1912. The processor 1912 of the example shown is hardware. For example, the processor 1912 may be implemented by one or more integrated circuits, logic circuits, microprocessors, GPUs, DSPs, or controllers from any desired family or manufacturer. The hardware processor 1912 can be a semiconductor-based (e.g., silicon-based) device. In this example, the processor 1912 implements an MPI compressed image encoder 425, a data interface 505, a texture stack compressor 510, and an alpha stack compressor 515.

[0093] The processor 1912 of the example shown includes local memory 1913 (e.g., a cache). The processor 1912 of the example shown communicates via a link 1918 with a main memory that includes volatile memory 1914 and non-volatile memory 1916. The link 1918 may be implemented by a bus, one or more point-to-point connections, etc., or a combination thereof. The volatile memory 1914 may be implemented by synchronous dynamic random access memory (SDRAM), dynamic random access memory (DRAM), Rambus (registered trademark) dynamic random access memory (RDRAM (registered trademark)), and / or any other type of random access memory device. The non-volatile memory 1916 may be implemented by flash memory and / or any other desired type of memory device. Access to the main memories 1914, 1916 is controlled by a memory controller.

[0094] The processor platform 1900 of the example shown also includes an interface circuit 1920. The interface circuit 1920 may be implemented by any type of interface standard such as an Ethernet (registered trademark) interface, a universal serial bus (USB), a Bluetooth (registered trademark) interface, a near field communication (NFC) interface, and / or a PCI Express interface.

[0095] In the example shown, one or more input devices 1922 are connected to the interface circuit 1920. The input devices 1922 allow a user to input data and / or commands to the processor 1912. The input devices may be implemented, for example, by an audio sensor, a microphone, a camera (still or video), a keyboard, a button, a mouse, a touch screen, a track pad, a track ball, a track bar (such as an isopoint), a voice recognition system, and / or any other human-machine interface. Also, with many systems such as the processor platform 1900, a user may be able to control a computer system and provide data to the computer using, but not limited to, physical movements such as hand or body movements, facial expressions, and facial recognition.

[0096] One or more output devices 1924 are also connected to the interface circuit 1920 of the example shown. The output devices 1924 can be implemented, for example, by a display device (such as a light emitting diode (LED), an organic light emitting diode (OLED), a liquid crystal display (LCD), a cathode ray tube display (CRT), an in-place switching (IPS) display, a touch screen, etc.), a tactile output device, a printer and / or a speaker. Thus, the interface circuit 1920 of the example shown typically includes a graphics driver card, a graphics driver chip, and / or a graphics driver processor.

[0097] The illustrated example of interface circuit 1920 also includes communication devices such as a transmitter, receiver, transceiver, modem, residential gateway, wireless access point, and / or network interface that facilitate data exchange with an external machine (e.g., any type of computing device) via network 1926. The communication may be via, for example, an Ethernet connection, a digital subscriber line (DSL) connection, a telephone line connection, a coaxial cable system, a satellite system, a line-of-sight wireless system, a cellular telephone system, etc.

[0098] The illustrated example of processor platform 1900 also includes one or more mass storage devices 1928 for storing software and / or data. Examples of such mass storage devices 1928 include floppy disk drives, hard disk drives, compact disk drives, Blu-ray (registered trademark) disk drives, redundant arrays of independent disks (RAID) systems, and digital versatile disk (DVD) drives.

[0099] Machine-executable instructions 1932 corresponding to the instructions of FIGS. 10-14 may be stored on a mass storage device 1928, volatile memory 1914, non-volatile memory 1916, local memory 1913, and / or a removable non-transitory computer-readable storage medium such as a CD or DVD 1936.

[0100] FIG. 20 is a block diagram of an exemplary processor platform 2000 configured to execute the instructions of FIGS. 15 - 17 to implement the MPI stack decoder 420 of FIG. 4 and / or FIG. 6. The processor platform 2000 can be, for example, a server, a personal computer, a workstation, a self - learning machine (e.g., a neural network), a mobile device (e.g., a mobile phone, a smartphone, a tablet such as an iPad (trademark)), a PDA, an Internet device, a DVD player, a CD player, a digital video recorder, a Blu - ray player, a game console, a personal video recorder, a set - top box, a digital camera, a headset or other wearable device, or any other type of computing device.

[0101] The illustrated example of the processor platform 2000 includes a processor 2012. The illustrated example of the processor 2012 is hardware. For example, the processor 2012 may be implemented by one or more integrated circuits, logic circuits, microprocessors, GPUs, DSPs, or controllers from any desired family or manufacturer. The hardware processor 2012 can be a semiconductor - based (e.g., silicon - based) device. In this example, the processor 2012 implements an MPI compressed image decoder 460, a data interface 605, a texture stack extender 610, and an alpha stack extender 615.

[0102] The processor 2012 in the illustrated example includes local memory 2013 (e.g., cache). The processor 2012 in the illustrated example communicates, via link 2018, with main memory that includes volatile memory 2014 and non-volatile memory 2016. Link 2018 may be implemented by, for example, a bus, one or more point-to-point connections, or a combination thereof. Volatile memory 2014 may be implemented by SDRAM, DRAM, RDRAM®, and / or any other type of random access memory device. Non-volatile memory 2016 may be implemented by flash memory and / or any other desired type of memory device. Access to main memories 2014, 2016 is controlled by a memory controller.

[0103] The processor platform 2000 in the illustrated example also includes interface circuit 2020. Interface circuit 2020 may be implemented by any type of interface standard, such as an Ethernet interface, USB, Bluetooth® interface, NFC interface, and / or a PCI Express interface.

[0104] In the illustrated example, one or more input devices 2022 are connected to interface circuit 2020. Input devices 2022 allow a user to input data and / or commands to processor 2012. Input devices may be implemented by, for example, audio sensors, microphones, cameras (still or video), keyboards, buttons, mice, touchscreens, trackpads, trackballs, trackbars (such as isopoints), voice recognition systems, and / or any other human machine interface. Also, with many systems such as processor platform 2000, a user may be able to control a computer system and provide data to the computer using, but not limited to, physical movements such as hand or body movements, facial expressions, and face recognition.

[0105] One or more output devices 2024 are also connected to the illustrated example of the interface circuit 2020. The output device 2024 can be implemented, for example, by a display device (e.g., LED, OLED, LCD, CRT display, IP display, touch screen, etc.), a tactile output device, a printer, and / or a speaker. Thus, the illustrated example of the interface circuit 2020 typically includes a graphics driver card, a graphics driver chip, and / or a graphics driver processor.

[0106] The illustrated example of the interface circuit 2020 also includes communication devices such as a transmitter, a receiver, a transceiver, a modem, a residential gateway, a wireless access point, and / or a network interface that facilitate data exchange with an external machine (e.g., any type of computing device) via the network 2026. The communication can be, for example, via an Ethernet connection, a DSL connection, a telephone line connection, a coaxial cable system, a satellite system, a line-of-sight wireless system, a cellular telephone system, etc.

[0107] The illustrated example of the processor platform 2000 also includes one or more mass storage devices 2028 for storing software and / or data. Examples of such mass storage devices 2028 include a floppy disk drive, a hard disk drive, a compact disk drive, a Blu-ray (registered trademark) disk drive, a RAID system, and a DVD drive.

[0108] Machine-executable instructions 2032 corresponding to the instructions of FIGS. 10-14 may be stored on a mass storage device 2028, volatile memory 2014, non-volatile memory 2016, local memory 2013, and / or a removable non-transitory computer-readable storage medium such as a CD or DVD 2036.

[0109] FIG. 21 shows a block diagram of an exemplary software distribution platform 2105 for distributing software such as exemplary computer-readable instructions 1932 and / or 2032 of FIG. 19 and / or FIG. 20 to a third party. The exemplary software distribution platform 2105 can be implemented by any computer server, data facility, cloud service, etc. that can store and transmit software to other computing devices. The third party may be a customer of an entity that owns and / or operates the software distribution platform. For example, the entity that owns and / or operates the software distribution platform may be a developer, seller, and / or licensor of software such as exemplary computer-readable instructions 1932 and / or 2032 of FIG. 19 and / or FIG. 20. The third party may be a consumer, user, retailer, OEM, etc. that purchases and / or licenses the software for use and / or resale and / or sublicense. In the example shown, the software distribution platform 2105 includes one or more servers and one or more storage devices. The storage device may store computer-readable instructions 1932 and / or 2032 that may correspond to the exemplary computer-readable instructions of FIGS. 10-14 and FIGS. 15-17 as described above. One or more servers of the exemplary software distribution platform 2105 communicate with a network 2110 that may correspond to any one or more of the Internet and / or the exemplary networks 1926 and / or 2026 described above. In some examples, one or more servers respond to requests to transmit software to the requesting side as part of a commercial transaction. Payments for the transfer, sale, and / or license of the software may be processed by one or more servers of the software distribution platform and / or via a payment entity of the third party.The server enables the purchaser and / or licensee to download computer-readable instructions 1932 and / or 2032 from the software distribution platform 2105. For example, software corresponding to the exemplary computer-readable instructions of FIGS. 10-14 may be downloaded to an exemplary processor platform 1900 that executes the computer-readable instructions 1932 to implement the MPI stack encoder 410. Additionally or alternatively, software corresponding to the exemplary computer-readable instructions of FIGS. 15-17 may be downloaded to an exemplary processor platform 2000 that executes the computer-readable instructions 2032 to implement the MPI stack decoder 420. In some examples, one or more servers of the software distribution platform 2105 periodically provide, transmit, and / or enforce updates to the software (e.g., the exemplary computer-readable instructions 1932 and / or 2032 of FIGS. 19 and / or 20) to ensure that improvements, patches, updates, etc. are distributed and applied to the software on the end-user device.

[0110] From the foregoing, it will be understood that exemplary methods, apparatuses, and articles of manufacture for implementing multi-plane image compression are disclosed. The disclosed methods, apparatuses, and articles of manufacture improve the efficiency of computing devices by reducing the amount of MPI data transferred to a target platform for image rendering while maintaining the quality of the rendered image. Thus, the exemplary MPI compression techniques disclosed herein enable low-bandwidth transmission to a target or rendering node relative to the transmission bandwidth required by other (e.g., conventional) techniques. Accordingly, the disclosed exemplary MPI compression techniques can enable the execution of virtual reality and immersive media applications on platforms with limited resources, but can also be used in any number of image processing systems and applications. The disclosed methods, apparatuses, and articles of manufacture are thus directed to one or more improvements in the functionality of a computer.

[0111] The foregoing disclosure provides exemplary solutions for implementing multi-plane image compression. Further examples are disclosed herein, including, for example, apparatuses for implementing multi-plane image compression, at least one non-transitory computer-readable medium including instructions that, when executed, cause at least one processor to implement multi-plane image compression, and associated methods. The disclosed examples can be implemented individually and / or in one or more combinations.

[0112] Example 1 is an apparatus for compressing a multi-plane image stack, the apparatus comprising: an interface for accessing an input multi-plane image stack corresponding to a source camera viewpoint, the input multi-plane image stack having a plurality of texture images and a corresponding plurality of alpha images, the alpha images of the plurality of alpha images including pixel values representing the transparency of corresponding pixels in each of the plurality of texture images; and a compressed image encoder for performing at least one of (i) converting the plurality of texture images into a single composite texture image or (ii) converting the plurality of alpha images into a single composite alpha image to generate the compressed multi-plane image stack, the interface outputting the compressed multi-plane image stack.

[0113] Example 2 includes the apparatus of Example 1, wherein the compressed image encoder combines the plurality of texture images based on the plurality of alpha images to convert the plurality of texture images into the single composite texture image.

[0114] Example 3 includes the apparatus of Example 2, wherein, to combine the plurality of texture images based on the plurality of alpha images, the compressed image encoder weights pixel values of a texture image of the plurality of texture images with pixel values of each of the plurality of alpha images corresponding to the texture image of the plurality of texture images to determine a plurality of alpha-weighted texture images.

[0115] Example 4 includes the apparatus of example 3, wherein to combine the multiple texture images based on the multiple alpha images, the compression image encoder averages the alpha-weighted texture images to determine the single composite texture image, the single composite texture image including alpha-weighted pixel values at each pixel location of the single composite texture image.

[0116] Example 5 includes the apparatus of Example 1, wherein the compressed image encoder replaces the plurality of texture images with source camera images associated with the source camera viewpoints corresponding to the input multi-plane image stack to convert the plurality of texture images to the single composite texture image.

[0117] Example 6 includes the apparatus according to Example 1, in which, in order to convert the plurality of alpha images into the single composite alpha image, the compression image encoder identifies a first alpha plane index value corresponding to a first one of the alpha images having the largest alpha value among the alpha images at a first pixel position, identifies a first alpha plane index value corresponding to an alpha image in the vicinity of the first one of the alpha images having the next largest alpha value among the alpha images at the first pixel position, determines a bias value for the first pixel position, the bias value being based on the largest alpha value and the next largest alpha value at the first pixel position, and combines the first alpha plane index value and the bias value to determine a composite alpha value to be included in the single composite alpha image at the first pixel position.

[0118] Example 7 includes the apparatus according to Example 1, in which, in order to convert the plurality of alpha images into the single composite alpha image, the compression image encoder identifies an alpha plane index value corresponding to a value of a source camera depth image at a first pixel position, the source camera depth image being associated with the source camera viewpoint corresponding to the input multi-plane image stack, determines a bias value for the first pixel position, the bias value being based on the alpha plane index value for the first pixel position and the source camera depth image at the first pixel position, and combines the alpha plane index value and the bias value to determine a composite alpha value to be included in the single composite alpha image at the first pixel position.

[0119] Example 8 includes the apparatus according to Example 1, in which the compression image encoder (i) converts the plurality of texture images into the single composite texture image to generate a compressed multi-plane image stack, and (ii) converts the plurality of alpha images into the single composite alpha image to generate the compressed multi-plane image stack.

[0120] Example 9 is at least one non-transitory computer-readable medium comprising computer-readable instructions that, when executed, cause at least one processor to access at least an input multi-plane image stack corresponding to a source camera viewpoint, the input multi-plane image stack having a plurality of texture images and corresponding alpha images, the alpha images of the plurality of alpha images including pixel values representing the transparency of corresponding pixels in each of the plurality of texture images, and causing the at least one processor to (i) convert the plurality of texture images into a single composite texture image or (ii) convert the plurality of alpha images into a single composite alpha image, in order to generate a compressed multi-plane image stack, and output the compressed multi-plane image stack, including at least one non-transitory computer-readable medium.

[0121] Example 10 includes the at least one non-transitory computer-readable medium according to Example 9, wherein the instructions cause the at least one processor to combine the plurality of texture images based on the plurality of alpha images to convert the plurality of texture images into the single composite texture image.

[0122] Example 11 includes the at least one non-transitory computer-readable medium according to Example 10, wherein, in order to combine the plurality of texture images based on the plurality of alpha images, the instructions cause the at least one processor to weight pixel values of the texture images of the plurality of texture images with respective pixel values of the plurality of alpha images corresponding to the texture images of the plurality of texture images to determine a plurality of alpha-weighted texture images.

[0123] Example 12 includes at least one non-transitory computer-readable medium of Example 11, wherein, to combine the multiple texture images based on the multiple alpha images, the instructions cause the at least one processor to average the alpha-weighted texture images to determine the single composite texture image, the single composite texture image including alpha-weighted pixel values at each pixel location of the single composite texture image.

[0124] Example 13 includes at least one non-transitory computer-readable medium of Example 9, wherein the instructions cause the at least one processor to replace the multiple texture images with source camera images associated with the source camera viewpoint corresponding to the input multi-plane image stack to convert the multiple texture images into the single composite texture image.

[0125] Example 14 includes the at least one non-transitory computer-readable medium of Example 9, wherein, to convert the multiple alpha images to the single composite alpha image, the instructions cause the at least one processor to: identify a first alpha plane index value corresponding to a first one of the alpha images at a first pixel location having a largest alpha value; identify a first alpha plane index value corresponding to an alpha image neighboring the first one of the alpha images at the first pixel location having a next largest alpha value; determine a bias value for the first pixel location, the bias value being based on the largest alpha value and the next largest alpha value at the first pixel location; and combine the first alpha plane index value and the bias value to determine a composite alpha value to include in the single composite alpha image at the first pixel location.

[0126] Example 15, to convert the plurality of alpha images into the single composite alpha image, the instructions cause the at least one processor to identify an alpha plane index value corresponding to a value of a source camera depth image at a first pixel position, the source camera depth image being associated with the source camera view corresponding to the input multi-plane image stack, determine a bias value for the first pixel position, the bias value being based on the alpha plane index value for the first pixel position and the source camera depth image at the first pixel position, combine the alpha plane index value and the bias value to determine a composite alpha value to include in the single composite alpha image at the first pixel position, and include at least one non-transitory computer-readable medium as described in Example 9.

[0127] Example 16, the instructions cause the at least one processor to (i) convert the plurality of texture images into the single composite texture image to generate a compressed multi-plane image stack, and (ii) convert the plurality of alpha images into the single composite alpha image to generate the compressed multi-plane image stack, and include at least one non-transitory computer-readable medium as described in Example 9.

[0128] Example 17 is a method for compressing a multi-plane image stack, the method comprising accessing an input multi-plane image stack corresponding to a source camera view, the input multi-plane image stack having a plurality of texture images and corresponding plurality of alpha images, the alpha images of the plurality of alpha images including pixel values representing the transparency of corresponding pixels in each of the plurality of texture images; at least one of (i) converting the plurality of texture images into a single composite texture image to generate a compressed multi-plane image stack, or (ii) converting the plurality of alpha images into a single composite alpha image to generate the compressed multi-plane image stack; and outputting the compressed multi-plane image stack.

[0129] Example 18 includes the method according to Example 17, wherein the step of converting the plurality of texture images includes a step of combining the plurality of texture images based on the plurality of alpha images in order to convert the plurality of texture images into the single composite texture image.

[0130] Example 19 includes the method according to Example 18, wherein the step of combining the plurality of texture images includes a step of weighting some of the pixel values of the texture images with respective pixel values of some of the alpha images corresponding to the some of the texture images in order to determine a plurality of alpha-weighted texture images.

[0131] Example 20 includes the method according to Example 19, wherein the step of combining the plurality of texture images includes a step of averaging the alpha-weighted texture images in order to determine the single composite texture image, and the single composite texture image includes alpha-weighted pixel values at respective pixel positions of the single composite texture image.

[0132] Example 21 includes the method according to Example 17, wherein the step of converting the plurality of texture images includes a step of replacing the plurality of texture images with source camera images associated with a source camera viewpoint corresponding to the input multi-plane image stack in order to convert the plurality of texture images into the single composite texture image.

[0133] Example 22 includes the method according to Example 17, wherein the step of converting the plurality of alpha images includes: identifying a first alpha plane index value corresponding to a first one of the alpha images having the largest alpha value among the alpha images at a first pixel position; identifying a first alpha plane index value corresponding to an alpha image in the vicinity of the first one of the alpha images having the next largest alpha value among the alpha images at the first pixel position; determining a bias value for the first pixel position, the bias value being based on the largest alpha value and the next largest alpha value at the first pixel position; and combining the first alpha plane index value and the bias value to determine a composite alpha value to be included in the single composite alpha image at the first pixel position.

[0134] Example 23 includes the method according to Example 17, wherein the step of converting the plurality of alpha images includes: identifying an alpha plane index value corresponding to a value of a source camera depth image at a first pixel position, the source camera depth image being associated with a source camera viewpoint corresponding to the input multi-plane image stack; determining a bias value for the first pixel position, the bias value being based on the alpha plane index value for the first pixel position and the source camera depth image at the first pixel position; and combining the alpha plane index value and the bias value to determine a composite alpha value to be included in the single composite alpha image at the first pixel position.

[0135] Example 24 includes the method according to Example 17, wherein the method comprises: (i) converting the plurality of texture images into the single composite texture image to generate a compressed multi-plane image stack; and (ii) converting the plurality of alpha images into the single composite alpha image to generate the compressed multi-plane image stack.

[0136] Example 25 is an apparatus for compressing a multi-plane image stack, the apparatus comprising at least one memory, computer-readable instructions, and at least one processor for executing the computer-readable instructions, the at least one processor at least accessing an input multi-plane image stack corresponding to a source camera viewpoint, the input multi-plane image stack having a plurality of texture images and a plurality of corresponding alpha images, the alpha images of the plurality of alpha images including pixel values representing the transparency of corresponding pixels in each of the plurality of texture images, and performing at least one of (i) converting the plurality of texture images into a single composite texture image to generate a compressed multi-plane image stack, or (ii) converting the plurality of alpha images into a single composite alpha image to generate the compressed multi-plane image stack, and outputting the compressed multi-plane image stack.

[0137] Example 26 includes the apparatus of Example 25, wherein the at least one processor combines the plurality of texture images based on the plurality of alpha images to convert the plurality of texture images into the single composite texture image.

[0138] Example 27 includes the apparatus of Example 26, wherein, to combine the plurality of texture images based on the plurality of alpha images, the at least one processor weights pixel values of the texture images of the plurality of texture images with respective pixel values of the plurality of alpha images corresponding to the texture images of the plurality of texture images to determine a plurality of alpha-weighted texture images.

[0139] Example 28 includes the apparatus of example 27, wherein to combine the multiple texture images based on the multiple alpha images, the at least one processor averages the alpha-weighted texture images to determine the single composite texture image, the single composite texture image including alpha-weighted pixel values at each pixel location of the single composite texture image.

[0140] Example 29 includes the apparatus of Example 25, wherein the at least one processor replaces the multiple texture images with source camera images associated with the source camera viewpoints corresponding to the input multi-plane image stack to convert the multiple texture images into the single composite texture image.

[0141] Example 30 includes the apparatus of Example 25, wherein, to convert the multiple alpha images to the single composite alpha image, the at least one processor identifies a first alpha plane index value corresponding to a first one of the alpha images having a largest alpha value at a first pixel location, identifies a first alpha plane index value corresponding to an alpha image neighboring the first one of the alpha images having a next largest alpha value at the first pixel location, and determines a bias value for the first pixel location, the bias value being based on the largest alpha value and the next largest alpha value at the first pixel location, and combines the first alpha plane index value with the bias value to determine a composite alpha value to include in the single composite alpha image at the first pixel location.

[0142] Example 31 includes the apparatus of Example 25, wherein, to convert the multiple alpha images into the single composite alpha image, the at least one processor identifies an alpha plane index value corresponding to a value in a source camera depth image at a first pixel location, the source camera depth image being associated with the source camera viewpoint that corresponds to the input multi-plane image stack; determines a bias value for the first pixel location, the bias value being based on the alpha plane index value for the first pixel location and the source camera depth image at the first pixel location; and combines the alpha plane index value and the bias value to determine a composite alpha value to include in the single composite alpha image at the first pixel location.

[0143] Example 32 includes the apparatus of Example 25, wherein the at least one processor (i) converts the plurality of texture images into the single composite texture image to generate a compressed multi-plane image stack, and (ii) converts the plurality of alpha images into the single composite alpha image to generate the compressed multi-plane image stack.

[0144] Example 33 includes an apparatus for compressing a multi-planar image stack, the apparatus comprising: means for accessing an input multi-planar image stack corresponding to a source camera viewpoint, the input multi-planar image stack having a plurality of texture images and a corresponding plurality of alpha images, the alpha images of the plurality of alpha images including pixel values representing transparency of corresponding pixels in each of the plurality of texture images; and means for encoding the input multi-planar image stack such that at least one of (i) converting the plurality of texture images into a single composite texture image to generate a compressed multi-planar image stack, or (ii) converting the plurality of alpha images into a single composite alpha image to generate the compressed multi-planar image stack.

[0145] Example 34 includes the apparatus according to Example 33, wherein the means for encoding combines the plurality of texture images based on the plurality of alpha images to convert the plurality of texture images into the single composite texture image.

[0146] Example 35 includes the apparatus according to Example 34, wherein the means for encoding weights the pixel values of the texture images of the plurality of texture images with respective pixel values of the plurality of alpha images corresponding to the texture images of the plurality of texture images to determine a plurality of alpha-weighted texture images in order to combine the plurality of texture images based on the plurality of alpha images.

[0147] Example 36 includes the apparatus according to Example 35, wherein the means for encoding averages the alpha-weighted texture images to determine the single composite texture image, and the single composite texture image includes alpha-weighted pixel values at respective pixel positions of the single composite texture image.

[0148] Example 37 includes the apparatus according to Example 33, wherein the means for encoding replaces the plurality of texture images with source camera images associated with the source camera viewpoints corresponding to the input multi-plane image stack to convert the plurality of texture images into the single composite texture image.

[0149] Example 38 includes the apparatus according to Example 33, wherein, to convert the plurality of alpha images into the single composite alpha image, the means for encoding identifies a first alpha plane index value corresponding to a first one of the alpha images having the largest alpha value among the alpha images at a first pixel position, identifies a first alpha plane index value corresponding to an alpha image in the vicinity of the first one of the alpha images having the next largest alpha value among the alpha images at the first pixel position, determines a bias value for the first pixel position, the bias value being based on the largest alpha value and the next largest alpha value at the first pixel position, and combines the first alpha plane index value and the bias value to determine a composite alpha value to be included in the single composite alpha image at the first pixel position.

[0150] Example 39 includes the apparatus according to Example 33, wherein, to convert the plurality of alpha images into the single composite alpha image, the means for encoding identifies an alpha plane index value corresponding to a value of a source camera depth image at a first pixel position, the source camera depth image being associated with a source camera viewpoint corresponding to the input multi-plane image stack, determines a bias value for the first pixel position, the bias value being based on the alpha plane index value for the first pixel position and the source camera depth image at the first pixel position, and combines the alpha plane index value and the bias value to determine a composite alpha value to be included in the single composite alpha image at the first pixel position.

[0151] Example 40 includes the apparatus according to Example 33, wherein the means for encoding (i) converts the plurality of texture images into the single composite texture image to generate a compressed multi-plane image stack and (ii) converts the plurality of alpha images into the single composite alpha image to generate the compressed multi-plane image stack.

[0152] Example 41 is an apparatus for expanding a compressed multi-plane image stack, the apparatus comprising: an interface for accessing the compressed multi-plane image stack, the compressed multi-plane image stack corresponding to a source camera viewpoint, the compressed multi-plane image stack having at least one of (i) a single composite texture image representing a plurality of uncompressed texture images, or (ii) a single composite alpha image representing a plurality of uncompressed alpha images, some of the uncompressed alpha images including pixel values representing the transparency of corresponding pixels in some of the uncompressed texture images; and a compressed image decoder for (i) decompressing the single composite texture image to obtain the plurality of uncompressed texture images, or (ii) decompressing the single composite alpha image to obtain the plurality of uncompressed alpha images, the interface outputting an uncompressed multi-plane image stack having the plurality of uncompressed texture images and the plurality of uncompressed alpha images.

[0153]

[0154] Example 42 includes the apparatus of Example 41, wherein the compressed image decoder duplicates the single composite texture image to decompress the single composite texture image to obtain the plurality of uncompressed texture images.

[0155] ​Example 44 includes the apparatus according to Example 43, wherein the compressed image decoder determines a first alpha plane index value for the first pixel position based on the value of the single composite alpha image at the first pixel position, determines a bias value for the first pixel position based on the first alpha plane index value and the value of the single composite alpha image at the first pixel position, determines a second alpha plane index value for the first pixel position based on the bias value for the first pixel position, determines the second alpha value based on the bias value for the first pixel position, and determines the first alpha value based on the second alpha value.

[0156] Example 45 includes the apparatus according to Example 41, wherein the compressed image decoder (i) unfolds the single composite texture image to obtain the plurality of uncompressed texture images, and (ii) unfolds the single composite alpha image to obtain the plurality of uncompressed alpha images.

[0157] Example 46 is at least one non-transitory computer-readable medium including computer-readable instructions that, when executed, cause at least one processor to access at least a compressed multi-plane image stack, the compressed multi-plane image stack corresponding to a source camera viewpoint, the compressed multi-plane image stack having at least one of (i) a single composite texture image representing a plurality of uncompressed texture images or (ii) a single composite alpha image representing a plurality of uncompressed alpha images, some of the uncompressed alpha images including pixel values representing the transparency of corresponding pixels in some of the uncompressed texture images, causing at least one of (i) expanding the single composite texture image to obtain the plurality of uncompressed texture images or (ii) expanding the single composite alpha image to obtain the plurality of uncompressed alpha images, and outputting an uncompressed multi-plane image stack having the plurality of uncompressed texture images and the plurality of uncompressed alpha images.

[0158] Example 47 includes the at least one non-transitory computer-readable medium of Example 46, wherein the instructions cause the at least one processor to duplicate the single composite texture image to expand the single composite texture image to obtain the plurality of uncompressed texture images.

[0159] Example 48 includes the at least one non-transitory computer-readable medium of Example 46, wherein, to expand the single composite alpha image, the instructions cause the at least one processor to set a first pixel position of a first one of the uncompressed alpha images to a first alpha value based on a value of the single composite alpha image at the first pixel position, set a first pixel position of a second one of the uncompressed alpha images to a second alpha value based on the value of the single composite alpha image at the first pixel position, and set alpha values at the first pixel position of the remaining ones of the uncompressed alpha images to zero.

[0160] Example 49 includes at least one non-transitory computer-readable medium as described in Example 48, wherein the instructions cause the at least one processor to determine a first alpha plane index value for the first pixel position based on the value of the single composite alpha image at the first pixel position, determine a bias value for the first pixel position based on the first alpha plane index value and the value of the single composite alpha image at the first pixel position, determine a second alpha plane index value for the first pixel position based on the bias value for the first pixel position, determine a second alpha value based on the bias value for the first pixel position, and determine the first alpha value based on the second alpha value.

[0161] Example 50 includes at least one non-transitory computer-readable medium as described in Example 46, wherein the instructions cause the at least one processor to (i) unroll the single composite texture image to obtain the plurality of non-compressed texture images and (ii) unroll the single composite alpha image to obtain the plurality of non-compressed alpha images.

[0162] Example 51 includes an apparatus for a method for decompressing a compressed multi-plane image stack, the method comprising: accessing the compressed multi-plane image stack, the compressed multi-plane image stack corresponding to a source camera viewpoint, the compressed multi-plane image stack having at least one of (i) a single composite texture image representing a plurality of uncompressed texture images, or (ii) a single composite alpha image representing a plurality of uncompressed alpha images, some of the uncompressed alpha images including pixel values representing transparency of corresponding pixels in each of some of the uncompressed texture images; and at least one of (i) decompressing the single composite texture image to obtain the plurality of uncompressed texture images, or (ii) decompressing the single composite alpha image to obtain the plurality of uncompressed alpha images; and outputting an uncompressed multi-plane image stack having the plurality of uncompressed texture images and the plurality of uncompressed alpha images.

[0163] Example 52 includes the method of example 51, wherein developing the single synthetic texture image comprises replicating the single synthetic texture image.

[0164] Example 53 includes the method of example 51, in which expanding the single composite alpha image includes setting a first pixel location of a first one of the uncompressed alpha images to a first alpha value based on a value of the single composite alpha image at the first pixel location, setting the first pixel location of a second one of the uncompressed alpha images to a second alpha value based on a value of the single composite alpha image at the first pixel location, and setting alpha values at the first pixel locations of the remaining ones of the uncompressed alpha images to zero.

[0165] Example 54 includes the method according to Example 53, wherein the step of expanding the single synthesized alpha image includes determining a first alpha plane index value for the first pixel position based on the value of the single synthesized alpha image at the first pixel position; determining a bias value for the first pixel position based on the first alpha plane index value and the value of the single synthesized alpha image at the first pixel position; determining a second alpha plane index value for the first pixel position based on the bias value for the first pixel position; determining a second alpha value based on the bias value for the first pixel position; and determining the first alpha value based on the second alpha value.

[0166] Example 55 includes the method according to Example 51, wherein the method comprises: (i) expanding the single synthesized texture image to obtain the plurality of uncompressed texture images; and (ii) expanding the single synthesized alpha image to obtain the plurality of uncompressed alpha images.

[0167] Example 56 is an apparatus for expanding a compressed multi-plane image stack, the apparatus comprising at least one memory, computer-readable instructions, and at least one processor for executing the computer-readable instructions, the at least one processor at least accessing a compressed multi-plane image stack, the compressed multi-plane image stack corresponding to a source camera viewpoint, the compressed multi-plane image stack having at least one of (i) a single composite texture image representing a plurality of uncompressed texture images, or (ii) a single composite alpha image representing a plurality of uncompressed alpha images, some of the uncompressed alpha images including pixel values representing the transparency of corresponding pixels in some of the uncompressed texture images, and performing at least one of (i) expanding the single composite texture image to obtain the plurality of uncompressed texture images, or (ii) expanding the single composite alpha image to obtain the plurality of uncompressed alpha images, and outputting an uncompressed multi-plane image stack having the plurality of uncompressed texture images and the plurality of uncompressed alpha images.

[0168] Example 57 includes the apparatus of Example 56, wherein the at least one processor replicates the single composite texture image to expand the single composite texture image to obtain the plurality of uncompressed texture images.

[0169] Example 58 includes the apparatus of Example 56, wherein, to expand the single composite alpha image, the at least one processor sets the first pixel position of the first one of the uncompressed alpha images to a first alpha value based on the value of the single composite alpha image at the first pixel position, sets the first pixel position of the second one of the uncompressed alpha images to a second alpha value based on the value of the single composite alpha image at the first pixel position, and sets the alpha value at the first pixel position of the remaining ones of the uncompressed alpha images to zero.

[0170] Example 59 includes the apparatus according to Example 58, wherein the at least one processor determines a first alpha plane index value for the first pixel position based on the value of the single composite alpha image at the first pixel position, determines a bias value for the first pixel position based on the first alpha plane index value and the value of the single composite alpha image at the first pixel position, determines a second alpha plane index value for the first pixel position based on the bias value for the first pixel position, determines a second alpha value based on the bias value for the first pixel position, and determines the first alpha value based on the second alpha value.

[0171] Example 60 includes the apparatus according to Example 56, wherein the at least one processor (i) unfolds the single composite texture image to obtain the plurality of uncompressed texture images, and (ii) unfolds the single composite alpha image to obtain the plurality of uncompressed alpha images.

[0172] Example 61 is an apparatus for decompressing a compressed multi-plane image stack, the apparatus comprising means for accessing the compressed multi-plane image stack, the compressed multi-plane image stack corresponding to a source camera view point and having at least one of (i) a single composite texture image representing a plurality of uncompressed texture images or (ii) a single composite alpha image representing a plurality of uncompressed alpha images, some of the uncompressed alpha images including pixel values representing the transparency of corresponding pixels in some of the uncompressed texture images; and means for decoding the compressed multi-plane image stack to perform at least one of (i) unfolding the single composite texture image to obtain the plurality of uncompressed texture images or (ii) unfolding the single composite alpha image to obtain the plurality of uncompressed alpha images.

[0173] Example 62 includes the apparatus according to Example 61, wherein the means for decoding replicates the single composite texture image so as to expand the single composite texture image in order to obtain the plurality of uncompressed texture images.

[0174] Example 63 includes the apparatus according to Example 61, wherein the means for decoding, in order to expand the single composite alpha image, sets the first pixel position of the first one of the uncompressed alpha images to a first alpha value based on the value of the single composite alpha image at the first pixel position, sets the first pixel position of the second one of the uncompressed alpha images to a second alpha value based on the value of the single composite alpha image at the first pixel position, and sets the alpha value at the first pixel position of the remaining ones of the uncompressed alpha images to zero.

[0175] Example 64 includes the apparatus according to Example 63, wherein the means for decoding determines a first alpha plane index value for the first pixel position based on the value of the single composite alpha image at the first pixel position, determines a bias value for the first pixel position based on the first alpha plane index value and the value of the single composite alpha image at the first pixel position, determines a second alpha plane index value for the first pixel position based on the bias value for the first pixel position, determines the second alpha value based on the bias value for the first pixel position, and determines the first alpha value based on the second alpha value.

[0176] Example 65 includes the apparatus according to Example 61, wherein the means for decoding (i) expands the single composite texture image in order to obtain the plurality of uncompressed texture images and (ii) expands the single composite alpha image in order to obtain the plurality of uncompressed alpha images.

[0177] Specific exemplary methods, apparatuses, and products have been disclosed herein, but the scope of this patent is not so limited. Rather, conversely, this patent extends to all methods, apparatuses, and products fairly falling within the scope of the claims of this patent.

[0178] The following claims are hereby incorporated by reference into the embodiments for carrying out the invention, and each claim stands on its own as a separate embodiment of this disclosure. [Other possible items] [Item 1] An apparatus for compressing a multi-plane image stack, the apparatus comprising: An interface for accessing an input multi-plane image stack corresponding to a source camera viewpoint, the input multi-plane image stack having a plurality of texture images and corresponding plurality of alpha images, wherein the alpha image of the plurality of alpha images includes a pixel value representing the transparency of a corresponding pixel in each of the plurality of texture images; A compression image encoder that performs at least one of (i) converting the plurality of texture images into a single composite texture image or (ii) converting the plurality of alpha images into a single composite alpha image to generate the compressed multi-plane image stack, wherein the interface outputs the compressed multi-plane image stack; An apparatus comprising. [Item 2] The apparatus according to item 1, wherein the compression image encoder combines the plurality of texture images based on the plurality of alpha images to convert the plurality of texture images into the single composite texture image. [Item 3] To combine the plurality of texture images based on the plurality of alpha images, the compression image encoder weights the pixel values of the texture images of the plurality of texture images with the respective pixel values of the plurality of alpha images corresponding to the texture images of the texture images to determine a plurality of alpha-weighted texture images. The apparatus according to item 2. [Item 4] To combine the plurality of texture images based on the plurality of alpha images, the compression image encoder averages the alpha-weighted texture images to determine the single composite texture image. The single composite texture image includes alpha-weighted pixel values at respective pixel positions of the single composite texture image. The apparatus according to item 3. [Item 5] To convert the plurality of texture images into the single composite texture image, the compression image encoder replaces the plurality of texture images with source camera images associated with the source camera viewpoints corresponding to the input multi-plane image stack. The apparatus according to item 1. [Item 6] To convert the plurality of alpha images into the single composite alpha image, the compression image encoder identifies a first alpha plane index value corresponding to a first one of the alpha images having the largest alpha value among the alpha images at a first pixel position, identifies a first alpha plane index value corresponding to an alpha image in the vicinity of the first one of the alpha images having the next largest alpha value among the alpha images at the first pixel position, determines a bias value for the first pixel position, the bias value being based on the largest alpha value and the next largest alpha value at the first pixel position, The apparatus according to item 1, wherein the first alpha plane index value and the bias value are combined to determine a composite alpha value to be included in the single composite alpha image at the first pixel position. [Item 7] To convert the plurality of alpha images into the single composite alpha image, the compression image encoder identifies an alpha plane index value corresponding to a value of a source camera depth image at a first pixel position, the source camera depth image being associated with the source camera viewpoint corresponding to the input multi-plane image stack, determines a bias value for the first pixel position, the bias value being based on the alpha plane index value for the first pixel position and the source camera depth image at the first pixel position, The apparatus according to item 1, wherein the alpha plane index value and the bias value are combined to determine a composite alpha value to be included in the single composite alpha image at the first pixel position. [Item 8] The compression image encoder according to item 1, wherein (i) the plurality of texture images are converted into the single composite texture image to generate a compressed multi-plane image stack, and (ii) the plurality of alpha images are converted into the single composite alpha image to generate the compressed multi-plane image stack. [Item 9] At least one non-transitory computer-readable medium comprising computer-readable instructions that, when executed, cause at least one processor to at least access an input multi-plane image stack corresponding to a source camera viewpoint, the input multi-plane image stack having a plurality of texture images and a corresponding plurality of alpha images, the alpha images of the plurality of alpha images including pixel values representing the transparency of corresponding pixels in each of the plurality of texture images, (i) To generate a compressed multi-plane image stack, converting the plurality of texture images into a single composite texture image, or (ii) to generate the compressed multi-plane image stack, causing at least one of the plurality of alpha images to be converted into a single composite alpha image, At least one non-transitory computer-readable medium that causes the compressed multi-plane image stack to be output. [Item 10] The instruction causes the at least one processor to combine the plurality of texture images based on the plurality of alpha images to convert the plurality of texture images into the single composite texture image, the at least one non-transitory computer-readable medium according to item 9. [Item 11] To combine the plurality of texture images based on the plurality of alpha images, the instruction causes the at least one processor to weight the pixel values of the texture images of the plurality of texture images with the respective pixel values of the plurality of alpha images corresponding to the texture images of the plurality of texture images to determine a plurality of alpha-weighted texture images, the at least one non-transitory computer-readable medium according to item 10. [Item 12] To combine the plurality of texture images based on the plurality of alpha images, the instruction causes the at least one processor to average the alpha-weighted texture images to determine the single composite texture image, the single composite texture image including the alpha-weighted pixel values at respective pixel positions of the single composite texture image, the at least one non-transitory computer-readable medium according to item 11. [Item 13] The command causes the at least one processor to replace the plurality of texture images with source camera images associated with the source camera viewpoint associated with the input multi-plane image stack in order to convert the plurality of texture images into the single composite texture image, the at least one non-transitory computer-readable medium of item 9. [Item 14] To convert the plurality of alpha images into the single composite alpha image, the command causes the at least one processor to identify a first alpha plane index value corresponding to a first one of the alpha images having the largest alpha value among the alpha images at a first pixel position, identify a first alpha plane index value corresponding to an alpha image in the vicinity of the first one of the alpha images having the next largest alpha value among the alpha images at the first pixel position, determine a bias value for the first pixel position, the bias value being based on the largest alpha value and the next largest alpha value at the first pixel position, combine the first alpha plane index value and the bias value to determine a composite alpha value to include in the single composite alpha image at the first pixel position, the at least one non-transitory computer-readable medium of item 9. [Item 15] To convert the plurality of alpha images into the single composite alpha image, the command causes the at least one processor to identify an alpha plane index value corresponding to the value of the source camera depth image at a first pixel position, the source camera depth image being associated with the source camera viewpoint associated with the input multi-plane image stack, determine a bias value for the first pixel position, the bias value being based on the alpha plane index value for the first pixel position and the source camera depth image at the first pixel position, 10. The at least one non-transitory computer-readable medium of claim 9, wherein the alpha plane index value and the bias value are combined to determine a composite alpha value to include in the single composite alpha image at the first pixel location. [Item 16] 10. The at least one non-transitory computer-readable medium of claim 9, wherein the instructions cause the at least one processor to (i) transform the multiple texture images into the single composite texture image to generate a compressed multi-plane image stack, and (ii) transform the multiple alpha images into the single composite alpha image to generate the compressed multi-plane image stack. [Item 17] 1. A method for compressing a multi-planar image stack, the method comprising: accessing an input multi-planar image stack corresponding to a source camera viewpoint, the input multi-planar image stack having a plurality of texture images and a corresponding plurality of alpha images, each of the alpha images including pixel values representing a transparency of a corresponding pixel in each of the plurality of texture images; at least one of (i) converting the plurality of texture images into a single composite texture image to generate a compressed multi-planar image stack, or (ii) converting the plurality of alpha images into a single composite alpha image to generate the compressed multi-planar image stack; outputting the compressed multi-plane image stack; A method comprising: [Item 18] Item 18. The method of item 17, wherein transforming the plurality of texture images comprises combining the plurality of texture images based on the plurality of alpha images to transform the plurality of texture images into the single composite texture image. [Item 19] The step of combining the plurality of texture images includes weighting pixel values of the texture images of the plurality of texture images with respective pixel values of the plurality of alpha images corresponding to the texture images of the plurality of texture images to determine a plurality of alpha-weighted texture images, according to the method of item 18. [Item 20] The step of combining the plurality of texture images includes averaging the alpha-weighted texture images to determine the single composite texture image, and the single composite texture image includes alpha-weighted pixel values at respective pixel positions of the single composite texture image, according to the method of item 19. [Item 21] The step of converting the plurality of texture images includes replacing the plurality of texture images with source camera images associated with a source camera viewpoint corresponding to the input multi-plane image stack to convert the plurality of texture images into the single composite texture image, according to the method of item 17. [Item 22] The step of converting the plurality of alpha images includes identifying a first alpha plane index value corresponding to a first one of the alpha images having the largest alpha value among the alpha images at a first pixel position; identifying a first alpha plane index value corresponding to an alpha image in the vicinity of the first one of the alpha images having the next largest alpha value among the alpha images at the first pixel position; determining a bias value for the first pixel position, the bias value being based on the largest alpha value and the next largest alpha value at the first pixel position; combining the first alpha plane index value and the bias value to determine a composite alpha value to be included in the single composite alpha image at the first pixel position; and The method according to item 17, having [Item 23] The step of converting the plurality of alpha images into the single composite alpha image is identifying an alpha plane index value corresponding to a value of a source camera depth image at a first pixel position, the source camera depth image being associated with a source camera viewpoint corresponding to the input multi-plane image stack, determining a bias value for the first pixel position, the bias value being based on the alpha plane index value for the first pixel position and the source camera depth image at the first pixel position, combining the alpha plane index value and the bias value to determine a composite alpha value to include in the single composite alpha image at the first pixel position The method according to item 17, having [Item 24] The method includes (i) converting the plurality of texture images into the single composite texture image to generate a compressed multi-plane image stack, and (ii) converting the plurality of alpha images into the single composite alpha image to generate the compressed multi-plane image stack, the method according to item 17. [Item 25] An apparatus for compressing a multi-plane image stack, the apparatus comprising at least one memory, computer-readable instructions, and at least one processor for executing the computer-readable instructions, the at least one processor at least accesses an input multi-plane image stack corresponding to a source camera viewpoint, the input multi-plane image stack having a plurality of texture images and a plurality of corresponding alpha images, the alpha images of the plurality of alpha images including pixel values representing the transparency of corresponding pixels in each of the plurality of texture images, (i) Converting the plurality of texture images into a single composite texture image to generate a compressed multi-plane image stack, or (ii) performing at least one of converting the plurality of alpha images into a single composite alpha image to generate the compressed multi-plane image stack, Outputting the compressed multi-plane image stack Device. [Item 26] The device according to item 25, wherein the at least one processor combines the plurality of texture images based on the plurality of alpha images to convert the plurality of texture images into the single composite texture image. [Item 27] The device according to item 26, wherein the at least one processor weights pixel values of the texture images of the plurality of texture images with respective pixel values of the plurality of alpha images corresponding to the texture images of the plurality of texture images to determine a plurality of alpha-weighted texture images in order to combine the plurality of texture images based on the plurality of alpha images. [Item 28] The device according to item 27, wherein the at least one processor averages the alpha-weighted texture images to determine the single composite texture image, and the single composite texture image includes alpha-weighted pixel values at respective pixel positions of the single composite texture image, in order to combine the plurality of texture images based on the plurality of alpha images. [Item 29] The device according to item 25, wherein the at least one processor replaces the plurality of texture images with source camera images associated with the source camera viewpoints corresponding to the input multi-plane image stack to convert the plurality of texture images into the single composite texture image. [Item 30] To convert the plurality of alpha images into the single composite alpha image, the at least one processor Identifying a first alpha plane index value corresponding to a first one of the alpha images having a largest alpha value at a first pixel location; identifying a first alpha plane index value corresponding to an alpha image at the first pixel location that is adjacent to the first one of the alpha images having a next-largest alpha value; determining a bias value for the first pixel location, the bias value being based on the largest alpha value and the next largest alpha value at the first pixel location; Item 26. The apparatus of item 25, wherein the first alpha plane index value and the bias value are combined to determine a composite alpha value to include in the single composite alpha image at the first pixel location. [Item 31] To convert the multiple alpha images into the single composite alpha image, the at least one processor: identifying an alpha plane index value corresponding to a value of a source camera depth image at a first pixel location, the source camera depth image being associated with the source camera viewpoint corresponding to the input multi-planar image stack; determining a bias value for the first pixel location, the bias value being based on the alpha plane index value for the first pixel location and the source camera depth image at the first pixel location; Item 26. The apparatus of item 25, wherein the alpha plane index value and the bias value are combined to determine a composite alpha value to include in the single composite alpha image at the first pixel location. [Item 32] 26. The apparatus of claim 25, wherein the at least one processor (i) converts the plurality of texture images into the single composite texture image to generate a compressed multi-plane image stack, and (ii) converts the plurality of alpha images into the single composite alpha image to generate the compressed multi-plane image stack. [Item 33] 1. An apparatus for compressing a multi-planar image stack, the apparatus comprising: means for accessing an input multi-planar image stack corresponding to a source camera viewpoint, the input multi-planar image stack having a plurality of texture images and a corresponding plurality of alpha images, each of the alpha images including pixel values representing a transparency of a corresponding pixel in each of the plurality of texture images; means for encoding the input multi-plane image stack, the means for encoding at least one of: (i) converting the plurality of texture images into a single composite texture image to generate a compressed multi-plane image stack; or (ii) converting the plurality of alpha images into a single composite alpha image to generate the compressed multi-plane image stack; An apparatus comprising: [Item 34] Item 34. The apparatus of item 33, wherein the means for encoding combines the multiple texture images based on the multiple alpha images to transform the multiple texture images into the single composite texture image. [Item 35] Item 35. The apparatus of item 34, wherein to combine the plurality of texture images based on the plurality of alpha images, the encoding means weights pixel values of texture images of the plurality of texture images with pixel values of each of the plurality of alpha images corresponding to the texture images of the plurality of texture images to determine a plurality of alpha-weighted texture images. [Item 36] Item 36. The apparatus of item 35, wherein to combine the multiple texture images based on the multiple alpha images, the means for encoding averages the alpha-weighted texture images to determine the single composite texture image, the single composite texture image including alpha-weighted pixel values at each pixel location of the single composite texture image. [Item 37] The means for encoding replaces the plurality of texture images with source camera images associated with the source camera viewpoints associated with the input multi-plane image stack in order to convert the plurality of texture images into the single composite texture image, the apparatus according to item 33. [Item 38] In order to convert the plurality of alpha images into the single composite alpha image, the means for encoding identifies a first alpha plane index value corresponding to a first one of the alpha images having the largest alpha value among the alpha images at a first pixel position, identifies a first alpha plane index value corresponding to an alpha image in the vicinity of the first one of the alpha images having the next largest alpha value among the alpha images at the first pixel position, determines a bias value for the first pixel position, the bias value being based on the largest alpha value and the next largest alpha value at the first pixel position, combines the first alpha plane index value and the bias value to determine a composite alpha value to be included in the single composite alpha image at the first pixel position, the apparatus according to item 33. [Item 39] In order to convert the plurality of alpha images into the single composite alpha image, the means for encoding identifies an alpha plane index value corresponding to the value of the source camera depth image at a first pixel position, the source camera depth image being associated with the source camera viewpoints associated with the input multi-plane image stack, determines a bias value for the first pixel position, the bias value being based on the alpha plane index value for the first pixel position and the source camera depth image at the first pixel position, The apparatus according to item 33, which combines the alpha plane index value and the bias value to determine a composite alpha value to be included in the single composite alpha image at the first pixel position. [Item 40] The means for encoding: (i) converts the plurality of texture images into the single composite texture image to generate a compressed multi-plane image stack; (ii) converts the plurality of alpha images into the single composite alpha image to generate the compressed multi-plane image stack. The apparatus according to item 33. [Item 41] An apparatus for decompressing a compressed multi-plane image stack, the apparatus comprising: An interface for accessing the compressed multi-plane image stack, the compressed multi-plane image stack corresponding to a source camera viewpoint, the compressed multi-plane image stack having at least one of: (i) a single composite texture image representing a plurality of uncompressed texture images; or (ii) a single composite alpha image representing a plurality of uncompressed alpha images, some of the uncompressed alpha images including pixel values representing the transparency of corresponding pixels in some of the uncompressed texture images; and A compressed image decoder for: (i) decompressing the single composite texture image to obtain the plurality of uncompressed texture images; or (ii) decompressing the single composite alpha image to obtain the plurality of uncompressed alpha images, the interface outputting an uncompressed multi-plane image stack having the plurality of uncompressed texture images and the plurality of uncompressed alpha images. The compressed image decoder performs at least one of the above. An apparatus comprising. [Item 42] The apparatus according to item 41, wherein the compressed image decoder duplicates the single composite texture image to decompress the single composite texture image to obtain the plurality of uncompressed texture images. [Item 43] To expand the single synthesized alpha image, the compressed image decoder sets the first pixel position of the first of the uncompressed alpha images to a first alpha value based on the value of the single synthesized alpha image at the first pixel position, sets the first pixel position of the second of the uncompressed alpha images to a second alpha value based on the value of the single synthesized alpha image at the first pixel position, and sets the alpha value at the first pixel position of the remaining ones of the uncompressed alpha images to zero, the apparatus according to item 41. [Item 44] The compressed image decoder determines a first alpha plane index value for the first pixel position based on the value of the single synthesized alpha image at the first pixel position, determines a bias value for the first pixel position based on the first alpha plane index value and the value of the single synthesized alpha image at the first pixel position, determines a second alpha plane index value for the first pixel position based on the bias value for the first pixel position, determines the second alpha value based on the bias value for the first pixel position, and determines the first alpha value based on the second alpha value, the apparatus according to item 43. [Item 45] The compressed image decoder does (i) expand the single synthesized texture image to obtain the plurality of uncompressed texture images and (ii) expand the single synthesized alpha image to obtain the plurality of uncompressed alpha images, the apparatus according to item 41. [Item 46] At least one non-transitory computer-readable medium including computer-readable instructions that, when executed, cause at least one processor to at least accessing a compressed multi-planar image stack, the compressed multi-planar image stack corresponding to a source camera viewpoint, the compressed multi-planar image stack having at least one of (i) a single composite texture image representing a plurality of uncompressed texture images, or (ii) a single composite alpha image representing a plurality of uncompressed alpha images, some of the uncompressed alpha images including pixel values representing transparency of corresponding pixels in each of some of the uncompressed texture images; causing at least one of (i) decompressing the single composite texture image to obtain the plurality of uncompressed texture images, or (ii) decompressing the single composite alpha image to obtain the plurality of uncompressed alpha images; At least one non-transitory computer-readable medium that outputs an uncompressed multi-plane image stack having the plurality of uncompressed texture images and the plurality of uncompressed alpha images. [Item 47] Item 47. The at least one non-transitory computer-readable medium of item 46, wherein the instructions cause the at least one processor to replicate the single synthetic texture image to uncompress the single synthetic texture image to obtain the multiple uncompressed texture images. [Item 48] To develop the single composite alpha image, the instructions cause the at least one processor to: setting a first pixel location of a first one of the uncompressed alpha images to a first alpha value based on the value of the single composite alpha image at the first pixel location; setting the first pixel location of a second one of the uncompressed alpha images to a second alpha value based on the value of the single composite alpha image at the first pixel location; Item 47. The at least one non-transitory computer-readable medium of item 46, causing the alpha value at the first pixel location of the remainder of the uncompressed alpha images to be set to zero. [Item 49] The command causes the at least one processor to determine a first alpha plane index value for the first pixel position based on the value of the single composite alpha image at the first pixel position, determine a bias value for the first pixel position based on the first alpha plane index value and the value of the single composite alpha image at the first pixel position, determine a second alpha plane index value for the first pixel position based on the bias value for the first pixel position, determine a second alpha value based on the bias value for the first pixel position, The at least one non - transitory computer - readable medium according to item 48, which causes the first alpha value to be determined based on the second alpha value. [Item 50] The command causes the at least one processor to (i) unroll the single composite texture image to obtain the plurality of uncompressed texture images, and (ii) unroll the single composite alpha image to obtain the plurality of uncompressed alpha images, the at least one non - transitory computer - readable medium according to item 46. [Item 51] A method for decompressing a compressed multi - plane image stack, the method comprising: accessing the compressed multi - plane image stack, the compressed multi - plane image stack corresponding to a source camera viewpoint, the compressed multi - plane image stack having at least one of (i) a single composite texture image representing a plurality of uncompressed texture images, or (ii) a single composite alpha image representing a plurality of uncompressed alpha images, some of the uncompressed alpha images including pixel values representing the transparency of corresponding pixels in some of the uncompressed texture images; (i) a step of unfolding the single composite texture image to obtain the plurality of uncompressed texture images, or (ii) a step of unfolding the single composite alpha image to obtain the plurality of uncompressed alpha images, at least one of which is a step; a step of outputting an uncompressed multi-plane image stack having the plurality of uncompressed texture images and the plurality of uncompressed alpha images; An apparatus comprising: [Item 52] The method according to item 51, wherein the step of unfolding the single composite texture image includes a step of duplicating the single composite texture image. [Item 53] The step of unfolding the single composite alpha image includes: a step of setting a first pixel position of a first one of the uncompressed alpha images to a first alpha value based on a value of the single composite alpha image at the first pixel position; a step of setting the first pixel position of a second one of the uncompressed alpha images to a second alpha value based on the value of the single composite alpha image at the first pixel position; The method according to item 51, further comprising a step of setting an alpha value at the first pixel position of the remaining ones of the uncompressed alpha images to zero. [Item 54] The step of unfolding the single composite alpha image includes: a step of determining a first alpha plane index value for the first pixel position based on the value of the single composite alpha image at the first pixel position; a step of determining a bias value for the first pixel position based on the first alpha plane index value and the value of the single composite alpha image at the first pixel position; a step of determining a second alpha plane index value for the first pixel position based on the bias value for the first pixel position; a step of determining the second alpha value based on the bias value for the first pixel position; The method according to item 53, comprising the step of determining the first alpha value based on the second alpha value. [Item 55] The method according to item 51, comprising: (i) a step of unwrapping the single composite texture image to obtain the plurality of uncompressed texture images; and (ii) a step of unwrapping the single composite alpha image to obtain the plurality of uncompressed alpha images. [Item 56] An apparatus for expanding a compressed multi-plane image stack, the apparatus comprising: at least one memory; computer-readable instructions; at least one processor for executing the computer-readable instructions, the at least one processor being at least accessing a compressed multi-plane image stack, the compressed multi-plane image stack corresponding to a source camera viewpoint, the compressed multi-plane image stack having at least one of (i) a single composite texture image representing a plurality of uncompressed texture images, or (ii) a single composite alpha image representing a plurality of uncompressed alpha images, some of the uncompressed alpha images including pixel values representing the transparency of corresponding pixels in some of the uncompressed texture images, performing at least one of (i) unwrapping the single composite texture image to obtain the plurality of uncompressed texture images, or (ii) unwrapping the single composite alpha image to obtain the plurality of uncompressed alpha images, outputting an uncompressed multi-plane image stack having the plurality of uncompressed texture images and the plurality of uncompressed alpha images Apparatus. [Item 57] The apparatus according to item 56, wherein the at least one processor duplicates the single composite texture image to unwrap the single composite texture image to obtain the plurality of uncompressed texture images. [Item 58] To expand the single synthesized alpha image, the at least one processor sets the first pixel position of the first of the uncompressed alpha images to a first alpha value based on the value of the single synthesized alpha image at the first pixel position, sets the first pixel position of the second of the uncompressed alpha images to a second alpha value based on the value of the single synthesized alpha image at the first pixel position, and sets the alpha value at the first pixel position of the remaining ones of the uncompressed alpha images to zero, the apparatus according to item 56. [Item 59] The at least one processor determines a first alpha plane index value for the first pixel position based on the value of the single synthesized alpha image at the first pixel position, determines a bias value for the first pixel position based on the first alpha plane index value and the value of the single synthesized alpha image at the first pixel position, determines a second alpha plane index value for the first pixel position based on the bias value for the first pixel position, determines the second alpha value based on the bias value for the first pixel position, and determines the first alpha value based on the second alpha value, the apparatus according to item 58. [Item 60] The at least one processor does (i) expand the single synthesized texture image to obtain the plurality of uncompressed texture images and (ii) expand the single synthesized alpha image to obtain the plurality of uncompressed alpha images, the apparatus according to item 56. [Item 61] An apparatus for decompressing a compressed multi-plane image stack, the apparatus an interface for accessing the compressed multi-planar image stack, the compressed multi-planar image stack corresponding to a source camera viewpoint, the compressed multi-planar image stack having at least one of (i) a single composite texture image representing a plurality of uncompressed texture images, or (ii) a single composite alpha image representing a plurality of uncompressed alpha images, some of the uncompressed alpha images including pixel values representing transparency of corresponding pixels in each of some of the uncompressed texture images; means for decoding the compressed multi-plane image stack, the means for decoding performing at least one of: (i) decompressing the single composite texture image to obtain the plurality of uncompressed texture images; or (ii) decompressing the single composite alpha image to obtain the plurality of uncompressed alpha images; An apparatus comprising: [Item 62] Item 62. The apparatus of item 61, wherein the means for decoding replicates the single synthetic texture image to expand the single synthetic texture image to obtain the multiple uncompressed texture images. [Item 63] To develop the single composite alpha image, the means for decoding comprises: setting a first pixel location of a first one of the uncompressed alpha images to a first alpha value based on the value of the single composite alpha image at the first pixel location; setting the first pixel location of a second one of the uncompressed alpha images to a second alpha value based on the value of the single composite alpha image at the first pixel location; Item 62. The apparatus of item 61, wherein the alpha value at the first pixel location of the remainder of the uncompressed alpha images is set to zero. [Item 64] The means for decoding comprises: Based on the value of the single composite alpha image at the first pixel position, determine a first alpha plane index value for the first pixel position. Based on the first alpha plane index value and the value of the single composite alpha image at the first pixel position, determine a bias value for the first pixel position. Based on the bias value for the first pixel position, determine a second alpha plane index value for the first pixel position. Based on the bias value for the first pixel position, determine the second alpha value. The apparatus according to item 63, wherein the first alpha value is determined based on the second alpha value. [Item 65] The means for decoding is to (i) expand the single composite texture image to obtain the plurality of uncompressed texture images and (ii) expand the single composite alpha image to obtain the plurality of uncompressed alpha images, according to the apparatus of item 61.

Claims

1. An apparatus for compressing a multi-plane image stack, the apparatus comprising: an interface for accessing an input multi-plane image stack corresponding to a source camera viewpoint, the input multi-plane image stack having a plurality of texture images and corresponding plural alpha images, the alpha images of the plurality of alpha images including pixel values representing the transparency of corresponding pixels in each of the plurality of texture images; a compression image encoder that performs at least one of (i) converting the plurality of texture images into a single composite texture image or (ii) converting the plurality of alpha images into a single composite alpha image to generate a compressed multi-plane image stack, the interface outputting the compressed multi-plane image stack; An apparatus comprising.

2. The apparatus according to claim 1, wherein the compression image encoder combines the plurality of texture images based on the plurality of alpha images to convert the plurality of texture images into the single composite texture image.

3. The apparatus according to claim 2, wherein, to combine the plurality of texture images based on the plurality of alpha images, the compression image encoder weights pixel values of the texture images of the plurality of texture images with respective pixel values of the plurality of alpha images corresponding to the texture images of the plurality of texture images to determine a plurality of alpha-weighted texture images.

4. The apparatus according to claim 3, wherein, to combine the plurality of texture images based on the plurality of alpha images, the compression image encoder averages the plurality of alpha-weighted texture images to determine the single composite texture image, the single composite texture image including alpha-weighted pixel values at respective pixel positions of the single composite texture image.

5. The apparatus according to claim 1, wherein the compressed image encoder replaces the plurality of texture images with source camera images associated with the source camera viewpoints corresponding to the input multi-plane image stack in order to convert the plurality of texture images into the single composite texture image.

6. In order to convert the plurality of alpha images into the single composite alpha image, the compressed image encoder identifies a first alpha plane index value corresponding to a first one of the alpha images having the largest alpha value among the alpha images at a first pixel position, identifies a first alpha plane index value corresponding to an alpha image in the vicinity of the first one of the alpha images having the next largest alpha value among the alpha images at the first pixel position, determines a bias value for the first pixel position, the bias value being based on the largest alpha value and the next largest alpha value at the first pixel position, combines the first alpha plane index value and the bias value to determine a composite alpha value to be included in the single composite alpha image at the first pixel position, the apparatus according to any one of claims 1 to 5.

7. In order to convert the plurality of alpha images into the single composite alpha image, the compressed image encoder identifies an alpha plane index value corresponding to a value of a source camera depth image at a first pixel position, the source camera depth image being associated with the source camera viewpoint corresponding to the input multi-plane image stack, determines a bias value for the first pixel position, the bias value being based on the alpha plane index value for the first pixel position and the source camera depth image at the first pixel position, combines the alpha plane index value and the bias value to determine a composite alpha value to be included in the single composite alpha image at the first pixel position, the apparatus according to any one of claims 1 to 5.

8. The compression image encoder according to any one of claims 1 to 7, which (i) converts the plurality of texture images into the single composite texture image to generate the compressed multi-plane image stack, and (ii) converts the plurality of alpha images into the single composite alpha image to generate the compressed multi-plane image stack.

9. A procedure for a processor to access an input multi-plane image stack corresponding to a source camera viewpoint, the input multi-plane image stack having a plurality of texture images and a plurality of corresponding alpha images, and the alpha images of the plurality of alpha images including pixel values representing the transparency of corresponding pixels in each of the plurality of texture images, a procedure of performing at least one of (i) converting the plurality of texture images into a single composite texture image to generate the compressed multi-plane image stack, or (ii) converting the plurality of alpha images into a single composite alpha image to generate the compressed multi-plane image stack, and a procedure of outputting the compressed multi-plane image stack A computer program for causing the above to be executed.

10. The computer program according to claim 9, which causes the processor to execute a procedure of combining the plurality of texture images based on the plurality of alpha images to convert the plurality of texture images into the single composite texture image.

11. The computer program according to claim 10, which causes the processor to execute a procedure of weighting pixel values of the texture images of the plurality of texture images with respective pixel values of the plurality of alpha images corresponding to the texture images of the plurality of texture images to determine a plurality of alpha-weighted texture images, in order to combine the plurality of texture images based on the plurality of alpha images.

12. To combine the plurality of texture images based on the plurality of alpha images, the processor is caused to execute a procedure of averaging the plurality of alpha-weighted texture images to determine the single composite texture image, wherein the single composite texture image includes alpha-weighted pixel values at respective pixel positions of the single composite texture image. The computer program according to claim 11.

13. To cause the processor to execute a procedure of replacing the plurality of texture images with source camera images associated with the source camera viewpoints associated with the input multi-plane image stack in order to convert the plurality of texture images into the single composite texture image. The computer program according to claim 9.

14. To convert the plurality of alpha images into the single composite alpha image, the processor A procedure for identifying a first alpha plane index value corresponding to a first one of the alpha images having the largest alpha value among the alpha images at a first pixel position; A procedure for identifying a first alpha plane index value corresponding to an alpha image in the vicinity of the first one of the alpha images having the next largest alpha value among the alpha images at the first pixel position; A procedure for determining a bias value for the first pixel position, the bias value being based on the largest alpha value and the next largest alpha value at the first pixel position; A procedure for combining the first alpha plane index value and the bias value to determine a composite alpha value to be included in the single composite alpha image at the first pixel position. The computer program according to any one of claims 9 to 13.

15. To convert the plurality of alpha images into the single composite alpha image, the processor A procedure for identifying an alpha plane index value corresponding to a value of a source camera depth image at a first pixel position, the source camera depth image being associated with the source camera viewpoint corresponding to the input multi-plane image stack. A procedure for determining a bias value for the first pixel position, wherein the bias value is based on the alpha plane index value for the first pixel position and the source camera depth image at the first pixel position, the procedure and, The computer program according to any one of claims 9 to 13, which causes the alpha plane index value and the bias value to be combined to execute a procedure for determining a composite alpha value to be included in the single composite alpha image at the first pixel position.

16. The computer program according to any one of claims 9 to 15, which causes the processor to execute a procedure of (i) converting the plurality of texture images into the single composite texture image and (ii) converting the plurality of alpha images into the single composite alpha image in order to generate the compressed multi-plane image stack.

17. A computer-readable storage medium storing the computer program according to any one of claims 9 to 16.

18. A method for compressing a multi-plane image stack, the method comprising: Accessing an input multi-plane image stack corresponding to a source camera viewpoint, the input multi-plane image stack having a plurality of texture images and a corresponding plurality of alpha images, the alpha images of the plurality of alpha images including pixel values representing the transparency of corresponding pixels in each of the plurality of texture images; At least one of (i) converting the plurality of texture images into a single composite texture image or (ii) converting the plurality of alpha images into a single composite alpha image to generate the compressed multi-plane image stack; and Outputting the compressed multi-plane image stack. A method comprising.

19. The method according to claim 18, wherein the step of converting the plurality of texture images includes combining the plurality of texture images based on the plurality of alpha images to convert the plurality of texture images into the single composite texture image.

20. The step of combining the plurality of texture images includes weighting pixel values of the texture images of the plurality of texture images with respective pixel values of the plurality of alpha images corresponding to the texture images of the plurality of texture images to determine a plurality of alpha-weighted texture images. The method according to claim 19.

21. The step of combining the plurality of texture images includes averaging the plurality of alpha-weighted texture images to determine the single composite texture image, and the single composite texture image includes alpha-weighted pixel values at respective pixel positions of the single composite texture image. The method according to claim 20.

22. The step of converting the plurality of texture images includes replacing the plurality of texture images with source camera images associated with a source camera viewpoint associated with the input multi-plane image stack to convert the plurality of texture images into the single composite texture image. The method according to claim 18.

23. The step of converting the plurality of alpha images includes identifying a first alpha plane index value corresponding to a first one of the alpha images having the largest alpha value among the alpha images at a first pixel position; identifying a first alpha plane index value corresponding to an alpha image in the vicinity of the first one of the alpha images having the next largest alpha value among the alpha images at the first pixel position; determining a bias value for the first pixel position, the bias value being based on the largest alpha value and the next largest alpha value at the first pixel position; combining the first alpha plane index value and the bias value to determine a composite alpha value to be included in the single composite alpha image at the first pixel position The method according to any one of claims 18 to 22.

24. The step of converting the plurality of alpha images includes Identifying an alpha plane index value corresponding to a value of a source camera depth image at a first pixel position, wherein the source camera depth image is associated with the source camera viewpoint corresponding to the input multi-plane image stack; Determining a bias value for the first pixel position, the bias value being based on the alpha plane index value for the first pixel position and the source camera depth image at the first pixel position; Combining the alpha plane index value and the bias value to determine a composite alpha value to be included in the single composite alpha image at the first pixel position; The method according to any one of claims 18 to 22, comprising:

25. The method further comprises: (i) converting the plurality of texture images into a single composite texture image to generate the compressed multi-plane image stack; and (ii) converting the plurality of alpha images into a single composite alpha image to generate the compressed multi-plane image stack. The method according to any one of claims 18 to 24.

26. An apparatus for compressing a multi-plane image stack, the apparatus comprising: Means for accessing an input multi-plane image stack corresponding to a source camera viewpoint, the input multi-plane image stack having a plurality of texture images and a corresponding plurality of alpha images, the alpha images of the plurality of alpha images including pixel values representing the transparency of corresponding pixels in each of the plurality of texture images; Means for encoding the input multi-plane image stack, the means for encoding performing at least one of: (i) converting the plurality of texture images into a single composite texture image to generate the compressed multi-plane image stack; or (ii) converting the plurality of alpha images into a single composite alpha image to generate the compressed multi-plane image stack; An apparatus comprising:

Images