Multi-plane image compression

Abstract

Example methods, apparatus, systems, and articles of manufacture (e.g., physical storage media) to implement multi-plane image (MPI) compression are disclosed. An example apparatus disclosed herein includes an interface to access an input multi-plane image stack corresponding to a source camera viewpoint, the input multi-plane image stack including a plurality of texture images and a corresponding plurality of alpha images, each of the alpha images includes a pixel value representative of a transparency of a respective pixel in each of the texture images. The disclosed example apparatus also includes a compressed image encoder to at least one of (i) convert the plurality of texture images to a single composite texture image to generate a compressed multi-plane image stack, or (ii) convert the plurality of alpha images to a single composite alpha image to generate a compressed multi-plane image stack. In some disclosed examples, an interface outputs a compressed multi-plane image stack.

Description

[0001] Related applications

[0002] This patent claims the benefit of U.S. Provisional Application No. 63 / 041,589, filed June 19, 2020, entitled “COMPRESSION OF MULTIPLANEIMAGES FOR PARALLAX-ENABLEDVIDEO RENDERING”. Priority is claimed to U.S. Provisional Application No. 63 / 041,589. The entirety of U.S. Provisional Application No. 63 / 041,589 is hereby incorporated by reference. Technical Field

[0003] This disclosure relates generally to image compression, and more specifically to multiplanar image compression. Background Technology

[0004] 3D image rendering techniques sometimes use multi-plane images (MPI), which form a stack of semi-transparent image planes to represent different depths of a 3D scene. Each plane of the MPI stack includes a texture image and an alpha image. The texture image provides texture pixel values ​​(e.g., red-blue-green, or RGB values), while the alpha image includes alpha pixel values ​​that indicate the transparency of individual texture pixels. If the alpha value is large, the texture pixel is opaque, and the background is not visible. If the alpha value is small, the texture pixel is transparent, and the background is visible. In some examples, the MPI stack is generated on one platform (e.g., where the source camera is deployed) and then transferred to another platform (e.g., a target / client platform) where a 3D image is rendered corresponding to the desired viewpoint. Attached Figure Description

[0005] Figure 1 This is a block diagram of an example MPI stack generator used to generate an MPI stack from a source image.

[0006] Figure 2 This is a block diagram of a sample MPI renderer used to render a target image from one or more MPI stacks.

[0007] Figure 3 This is a block diagram of an example video encoding and decoding system, which includes... Figure 1 MPI stack generator and Figure 2 The MPI renderer.

[0008] Figure 4This is a block diagram of an example video encoding and decoding system, which includes an example MPI stack encoder and an example MPI stack decoder to implement MPI compression in accordance with the teachings of this disclosure.

[0009] Figure 5 yes Figure 4 A block diagram of an example implementation of an MPI stack encoder.

[0010] Figure 6 yes Figure 4 A block diagram of an example implementation of an MPI stack decoder.

[0011] Figure 7 The diagram illustrates the process of... Figure 4 and / or Figure 5 The example MPI texture stack compression operation performed by the MPI stack encoder.

[0012] Figure 8 The diagram illustrates the process of... Figure 4 and / or Figure 6 The example MPI texture stack decompression operation performed by the MPI stack decoder.

[0013] Figure 9 The diagram illustrates the process of... Figure 4 and / or Figure 5 The example MPI alpha stack compression operation performed by the MPI stack encoder.

[0014] Figures 10-14 It represents something that can be executed and implemented. Figure 4 and / or Figure 5 A flowchart of example computer-readable instructions for an MPI stack encoder.

[0015] Figures 15-17 It represents something that can be executed and implemented. Figure 4 and / or Figure 6 A flowchart of example computer-readable instructions for the MPI stack decoder.

[0016] Figure 18 The diagram illustrates the process of... Figure 4 The example video encoding and decoding system performs an example process to render a target image using a single composite alpha image and a single composite texture image.

[0017] Figure 19 It is constructed to execute Figures 10-14 Example machine-readable instructions to implement Figure 4 and / or Figure 5 A block diagram of an example processor platform for an MPI stack encoder.

[0018] Figure 20 It is constructed to execute Figures 15-17Example machine-readable instructions to implement Figure 4 and / or Figure 6 A block diagram of an example processor platform for an MPI stack decoder.

[0019] Figure 21 This is a block diagram of an example software distribution platform used to distribute software (e.g., with...) Figures 10-17 The software corresponding to the example computer-readable instructions is distributed to client devices, such as consumers (e.g., for licensing, selling and / or using), retailers (e.g., for selling, reselling, licensing and / or sublicensing), and / or original equipment manufacturers (OEMs) (e.g., for inclusion in products to be distributed to, for example, retailers and / or direct-purchase customers).

[0020] The accompanying drawings are not to scale. Generally, the same reference numerals will be used throughout the drawings and the accompanying written description to refer to the same or similar parts, elements, etc. As used herein, references to connections (e.g., attachment, coupling, joining, and joining) may include intermediate members between the elements mentioned by that connection and / or relative movement between these elements, unless otherwise indicated. Therefore, a reference to a connection does not necessarily imply that two elements are directly connected and / or have a fixed relationship with each other. As used herein, the statement that any part is in “contact” with another part is defined to mean that there is no intermediate member between the two parts.

[0021] Unless otherwise specifically stated, this document uses descriptive terms such as “first,” “second,” “third,” etc., without indicating or otherwise specifying any priority, physical order, arrangement in a list, and / or any sorting, but merely as labels and / or arbitrary names to distinguish elements for ease of understanding of the disclosed examples. In some examples, the descriptive term “first” may be used in the detailed description to refer to an element, while the same element may be referred to in the claims with different descriptive terms, such as “second” or “third.” In such cases, it should be understood that such descriptive terms are only used to explicitly identify those elements that may, for example, share the same name in other cases. As used herein, “approximately” and “about” refer to dimensions that may be imprecise due to manufacturing tolerances and / or other real-world imperfections. As used herein, “substantially real-time” refers to something that occurs in a near-instantaneous manner, acknowledging that delays in computation time, transmission, etc., may exist in the real world. Thus, unless otherwise specified, “substantially real-time” means real-time + / - 1 second. Detailed Implementation

[0022] Example methods, apparatuses, systems, and articles of manufacture (e.g., physical storage media) for implementing multi-plane image (MPI) compression are disclosed. In general, MPI systems use machine learning to create a volumetric representation of a scene as a stack of semi-transparent images or planes containing textures derived from the original camera view. When these planes are stacked on top of each other, the original camera view is recovered. These stacks can be used to render the scene that will be visible from a new viewpoint, achieving visual quality superior to many other viewpoint interpolation techniques.

[0023] As described above, the MPI stack comprises a stack of texture images (also referred to herein as a texture image plane or texture plane) and a stack of alpha images (also referred to herein as an alpha image plane or alpha plane). The texture images and alpha images together represent different depths of the source camera view. A texture image for a given depth provides texture pixel values ​​(e.g., red-blue-green, or RGB values), while the alpha image includes alpha pixel values ​​that indicate the opacity of individual texture pixels, where an increase in alpha value indicates an increase in opacity (or a decrease in opacity), and a decrease in alpha value indicates a decrease in opacity (or an increase in opacity). In some examples, the MPI stack is generated on the source platform (e.g., where the camera is deployed) and then transferred to the target platform (e.g., a client platform) where a 3D image is rendered corresponding to the desired viewpoint.

[0024] The example MPI compression technique disclosed herein reduces the amount of MPI data to be transmitted to the target platform for image rendering while maintaining the quality of the rendered image. Therefore, compared to the transmission bandwidth required by other (e.g., conventional) techniques, the example MPI compression technique disclosed herein allows for lower bandwidth transmission to the target or rendering node. Thus, the disclosed example MPI compression technique enables the execution of virtual reality and immersive media applications on resource-constrained platforms, but can also be used in any number of image processing systems and applications.

[0025] In some examples, the disclosed example MPI compression technique does not transmit a stack of N textures and alpha planes (e.g., where N = 32 or some other value), but instead generates a single composite texture frame and / or a single composite alpha frame (also referred to as an alpha map frame) derived from the MPI stack, which together contain sufficient information for high-quality rendering of the MPI image rendering system by providing stack information. In some examples, the single composite alpha frame and the single composite texture are generated from the MPI stack by the disclosed example encoder on the source (e.g., transmitting) platform. On the target (e.g., receiving) platform, the disclosed example decoder uses the single composite alpha frame and the single composite texture frame to reconstruct the stack of N planes of the MPI stack for use by the MPI renderer to render the target image.

[0026] It will become clear that, compared to a system transmitting all N textures and alpha planes, the disclosed example MPI compression technique reduces the bit rate and pixel rate used to transmit MPI data (e.g., by a factor of N, such as N=32 or some other value). This, in turn, reduces the bandwidth and computational burden at the target (e.g., client) image renderer. Thus, the disclosed example technique enables video systems to deliver immersive media experiences on resource-constrained platforms, including, for example, tablets, laptops, smartphones, and other such devices that may be constrained in terms of transmission and / or computational bandwidth.

[0027] Switch to the attached image, in Figure 1 A block diagram of an example MPI stack generator 100 used to generate MPI stacks from source images is illustrated. One or more machine learning models are implemented in the MPI stack generator 100 to generate MPI stacks from multiple example source camera image frames 110 and 120. Any suitable machine learning model can be implemented by the MPI stack generator 100. In some examples, a local light field fusion (LLFF) model can be used as the machine learning model implemented by the MPI stack generator 100. In some examples, the LLF model uses images from several adjacent cameras as samples of the light field. These samples are used to generate stacks of MPI planes, which can be used to expand the acceptable viewing range in which a target (e.g., a virtual) camera can be positioned. This approach also allows for the capture and reproduction of viewpoint-dependent lighting effects such as specular highlights, refraction, and reflection. Figure 1 In the illustrated example, the MPI stack generator 100 implements a trained deep learning LLF model that generates example MPI stacks 130 and 135 from source camera image frames 110 and 120, corresponding to the respective neighboring camera viewpoints associated with the source camera that provided the source camera image frames 110 and 120.

[0028] exist Figure 1 In the illustrated example, the generated MPI stacks, such as MPI stacks 130 and 135, represent several planes N (e.g., N = 32 or some other value), which are placed in the view frustum of the corresponding source camera such that the reciprocal of the plane's depth is in the range [1 / d]. max ...1 / d min The inner uniform interval is d min and d max These are the minimum and maximum depths of the view frustum relative to the camera center. For example... Figure 1 As shown in the example, the generated MPI stack 130 includes an example texture image stack 140 associated with the frustum (or field of view) of the first camera corresponding to source image frame 110, and an example alpha stack 150 associated with the frustum (or field of view) of the first camera corresponding to source image frame 110. Similarly, the generated MPI stack 135 includes an example texture image stack 160 associated with the frustum (or field of view) of the second camera corresponding to source image frame 120, and an example alpha stack 170 associated with the frustum (or field of view) of the second camera corresponding to source image frame 115. In the following disclosure, processing for N = 32 planes will be described, but any number of planes can actually be used.

[0029] To render a scene from the desired viewpoint of the target virtual camera, the MPI stack is warped to the virtual camera's coordinate system, and planes are composited from back to front, where alpha values ​​are used as weighting factors for texture values ​​applied to each texture plane. Therefore, MPI-based techniques allow for view composition with less computation than traditional 3D mesh projection, but at the cost of quality degradation as the virtual camera moves away from the source camera. Furthermore, a 32-plane MPI stack with 1920x1080 resolution and 8-bit red, green, blue, and alpha (RGBA) samples per pixel comprises 0.265 gigabits (GB) of data. When traditional MPI-based techniques are used to encode and render video sequences at 30 frames per second (fps), 63.7 Gb / s of bandwidth is utilized for a single camera source, and correspondingly higher bandwidth is utilized for multiple camera inputs. This bandwidth utilization may be large enough to render MPI-based techniques impractical for some applications. High pixel rates can also impose a significant computational burden on the source (e.g., sending) platform encoder and the target (e.g., client) platform decoder / renderer, which may make traditional MPI-based techniques unsuitable for some client-side video applications.

[0030] exist Figure 2The diagram illustrates a block diagram of an example MPI renderer 200, which is used to render an example target image 205 from one or more MPI stacks. Figure 2 In the illustrated example, the MPI renderer 200 is implemented on the target platform and receives the above contact information. Figure 1 MPI stacks 130 and 135 are described. As described above, MPI stack 130 includes a texture image stack 140 and an alpha image stack 150, and MPI stack 135 includes a texture image stack 160 and an alpha image stack 170. The illustrated example MPI renderer 200 weights texture images 140 and 160 using the respective alpha images 150 and 170, and then blends the weighted images to create a rendered image 205 corresponding to the target viewpoint. In some examples, the blending performed by the MPI renderer 200 includes warping texture image stacks 140 and 160, as well as alpha image stacks 150 and 170, to the target camera's coordinate system.

[0031] exist Figure 3 The diagram shows a block diagram of an example video encoding and decoding system 300, which includes... Figure 1 MPI stack generator 100 and Figure 2 The MPI renderer 200 is illustrated. The video encoding and decoding system 300, also referred to as video system 300, includes an example source platform 305 and an example target platform 315. The example source platform 305 includes an MPI stack generator 100 and an example MPI stack encoder 310, while the example target platform 315 includes an example MPI stack decoder 320 and an MPI renderer 200. The source platform 305 can be implemented by any computing platform / device, such as, but not limited to, one or more servers, personal computers, workstations, mobile devices (e.g., cellular phones, smartphones, tablet devices such as iPads). TM Examples of applications include personal digital assistants (PDAs), internet-connected appliances, cameras, and more. Similarly, target platform 315 can be implemented by any computing platform / device, such as, but not limited to, one or more servers, personal computers, workstations, mobile devices (e.g., cellular phones, smartphones, tablets such as iPads). TM PDAs, internet-connected home appliances, DVD players, CD players, digital video recorders, Blu-ray players, game consoles, personal video recorders, set-top boxes, etc.

[0032] exist Figure 3In the illustrated example, the MPI stack generator 100 of the source platform 305, as described above, generates a separate MPI stack, such as MPI stacks 130 and 135, for each input video frame to be encoded and stored / transmitted by the source platform 305. The generated MPI stacks 130 and 135 have separate texture stacks 140 and 160, where pixels are represented in RGB format. The MPI stack encoder 310 of the illustrated example converts MPI stacks 130 and 135 into separate example texture image stacks 325 and separate example alpha image stacks 330, where the pixels of texture image 325 are converted to a luma and chroma format, such as YUV, while the pixels of alpha image 330 remain unchanged. The MPI stack encoder 310 uses any video encoder, such as an encoder implementing high efficiency video encoding (HEVC), to encode the texture image of texture image stack 325 and the alpha image of alpha image stack 330 to create example encoded texture video frame 335 and example encoded alpha video frame 340. Then the MPI stack encoder 310 stores the encoded texture video frame 335 and the encoded alpha video frame 340 and / or transmits the encoded texture video frame 335 and the encoded alpha video frame 340 to the target platform 315.

[0033] exist Figure 3 In the illustrated example, the MPI stack decoder 320 of the target platform 315 uses an appropriate video decoder associated with the video encoder utilized by the MPI stack encoder 310, such as a decoder implementing HEVC decoding, to decode the received or otherwise accessed encoded texture video frame 335 and encoded alpha video frame 340. The result of this video decoding is an example decoded texture image stack 345 corresponding to the texture image stack 325, and an associated example decoded alpha image stack 350 corresponding to the alpha image stack 330, wherein the decoded texture image 345 is formatted in YUV format. In the illustrated example, the MPI stack decoder 320 transforms the decoded texture image stack 345 and the associated decoded alpha image stack 350 into an example decoded MPI stack 355 corresponding to MPI stacks 130 and 135. The decoded MPI stack 355 comprises a texture image with pixels formatted in RGB format. As described above, the MPI stack renderer 200 renders the decoded MPI stack to output an example target image 360 ​​corresponding to the desired target viewpoint.

[0034] exist Figure 4The diagram illustrates a block diagram of an example video encoding and decoding system 400 that implements MPI compression according to the teachings of this disclosure. The illustrated example video encoding and decoding system 400, also referred to as video system 400, includes an example source platform 405 and an example target platform 415. The example source platform 405 includes an MPI stack generator 100 and an example MPI stack encoder 410, while the example target platform 415 includes an example MPI stack decoder 420 and an MPI renderer 200. The source platform 405 can be implemented by any computing platform / device, such as, but not limited to, one or more servers, personal computers, workstations, mobile devices (e.g., cellular phones, smartphones, tablet devices such as iPads). TM PDAs, internet-connected appliances, cameras, etc. In some examples, the source platform 405 is... Figure 19 The example processor platform 1900 is implemented, which is described in further detail below. Similarly, the target platform 415 can be implemented by any computing platform / device, such as, but not limited to, one or more servers, personal computers, workstations, mobile devices (e.g., cellular phones, smartphones, tablet devices such as iPads). TM PDAs, internet-connected home appliances, DVD players, CD players, digital video recorders, Blu-ray players, game consoles, personal video recorders, set-top boxes, etc. In some examples, target platform 415 is... Figure 20 The example processor platform 2000 is implemented, which is described in further detail below.

[0035] exist Figure 4 In the illustrated example, the MPI stack corresponding to the input image frame is not directly encoded and transmitted. Instead, a single composite texture image and / or a single composite alpha image, which captures much useful information from the MPI stack, is generated and stored / transmitted by the source platform 405. On the target platform 415, the single composite texture image and the single composite alpha image are used to regenerate the MPI stack, which is then used for image rendering.

[0036] More specifically, in Figure 4In the illustrated example, the MPI stack generator 100 of the source platform 405, as described above, generates a respective MPI stack, such as MPI stacks 130 and 135, for each input video frame to be encoded and stored / transmitted by the source platform 405. The illustrated example's MPI stack encoder 410 includes an example MPI compression image encoder 425 to compress each of the MPI stacks 130 / 135 by converting each of the texture image stacks 140 / 160 into a corresponding example single composite texture image 430, and / or by converting each of the alpha image stacks 150 / 170 into a corresponding example single composite alpha image 435. In some examples, the MPI compression image encoder 425 performs compression on both the texture image stacks 140 / 160 and the alpha image stacks 150 / 170 to form a respective single composite texture image 430 and a single composite alpha image 435. However, in some examples, the MPI compression image encoder 425 performs compression on the texture image stacks 140 / 160 to form individual composite texture images 430 or on the alpha image stacks 150 / 170 to form individual composite alpha images 435, but not on both. Therefore, the MPI compression image encoder 425 is an example of an apparatus for encoding an MPI stack (e.g., MPI stack 130 / 135) in a manner involving one or both of the following: (i) converting the texture images of the MPI stack (e.g., texture image stack 140 / 160) into individual composite texture images (e.g., individual composite texture image 430) to generate a compressed multiplane image stack, and / or (ii) converting the alpha images of the MPI stack (e.g., alpha image stack 150 / 170) into individual composite alpha images (e.g., individual composite alpha image 435) to generate a compressed multiplane image stack.

[0037] The illustrated MPI stack encoder 410 uses any video encoder, such as an encoder implementing HEVC, to encode a single composite texture image 430 and a single composite alpha image 435 to create an example encoded texture video frame 440 and an example encoded alpha video frame 445. The MPI stack encoder 410 then stores the encoded texture video frame 440 and the encoded alpha video frame 445 and / or transmits the encoded texture video frame 440 and the encoded alpha video frame 445 to a target platform 415.

[0038] exist Figure 4In the illustrated example, the MPI stack decoder 420 of the target platform 415 uses an appropriate video decoder associated with the video encoder utilized by the MPI stack encoder 410, such as a decoder implementing HEVC decoding, to decode the received or otherwise accessed encoded texture video frame 440 and encoded alpha video frame 445. The result of this video decoding is an example decoded composite texture image 450 corresponding to the composite texture image 430, and an associated example decoded composite alpha image 455 corresponding to the composite alpha image 435. In the illustrated example, the MPI stack decoder 420 includes an example MPI compressed image decoder 460 to decompress the decoded composite texture image 450 by converting it into a respective decoded texture image stack 465 for inclusion in the corresponding decoded MPI stack. Additionally or alternatively, the MPI compressed image decoder 460 decompresses the decoded composite alpha image 455 by converting it into a respective decoded alpha image stack 470 for inclusion in the corresponding decoded MPI stack. Therefore, the MPI compressed image decoder 460 is an example of an apparatus for decoding a compressed MPI stack in a manner that involves one or both of the following: (i) decompressing a single composite texture image (e.g., composite texture image 450) to obtain an uncompressed texture image of an uncompressed MPI stack (e.g., decoded texture image stack 465), and / or (ii) decompressing a single composite alpha image (e.g., composite alpha image 455) to obtain an uncompressed alpha image of an uncompressed MPI stack (e.g., decoded alpha image stack 470). As described above, the MPI stack renderer 200 renders the decoded MPI stack to output an example target image 475 corresponding to the desired target viewpoint.

[0039] In some examples, a single composite texture frame is derived by the MPI compressed image encoder 425 from the texture components of a stack of texture frames in the MPI stack, in a manner sufficient to capture enough texture information for use by the MPI stack renderer 200. In other examples, a single composite texture frame is derived directly from the raw camera image by the MPI compressed image encoder 425. In either type of example, the target image is rendered on the target platform 415 by copying the single composite texture frame N (e.g., N = 32) times and using the copied texture frames to replace the actual stack of texture frames included in the original MPI stack, along with N (e.g., N = 32) alpha planes generated by a machine learning model.

[0040] In some examples, a single composite alpha image is derived by the MPI compressed image encoder 425 from the alpha components of the stack of alpha frames in the MPI stack, in a manner sufficient to capture enough alpha (transparency) information for use by the MPI stack renderer 200. In other examples, the single composite alpha image is derived by the MPI compressed image encoder 425 from a depth map obtained from a source camera associated with the input MPI stack. In either type of example, the target image is rendered on the target platform 415 by using the single composite alpha image to generate an approximation of the original N (e.g., N = 32) stack of alpha frames included in the original MPI stack.

[0041] In some examples, a single composite texture image and a single composite alpha image can be used individually or in combination. For example, a single texture frame and a set of N (e.g., N=32) alpha planes from the MPI stack can be stored / transmitted, or a set of N (e.g., N=32) texture planes and a single composite alpha image can be stored / transmitted, or a single composite texture image and a single composite alpha image can be stored / transmitted. In some examples, a single composite texture image and a single composite alpha image can be used directly to project a single composite texture image onto a target viewpoint by interpreting the single composite alpha image as a coarse depth map.

[0042] exist Figure 5 The Chinese illustration shows Figure 4 A block diagram of an example implementation of the MPI stack encoder 410. Figure 5 The example MPI stack encoder 410 includes an example MPI compressed image encoder 425 and an example data interface 505. Figure 5 The example MPI compressed image encoder 425 includes an example texture stack compressor 510 and an example alpha stack compressor 515.

[0043] exist Figure 5 In the illustrated example, data interface 505 is used to access the input MPI stack corresponding to the source camera viewpoint. As described above, the input MPI stack includes an example stack of texture images / planes, such as input texture image stack 140, and a corresponding example stack of alpha images / planes, such as input alpha image stack 150. As described above, each alpha image in input alpha image stack 150 includes a pixel value representing the transparency of a corresponding pixel in each paired texture image in input texture image stack 140. Therefore, data interface 505 is an example of a means for accessing an input multi-plane image stack corresponding to the source camera viewpoint.

[0044] exist Figure 5In the illustrated example, the MPI compressed image encoder 425 includes a texture stack compressor 510 for converting the texture image of the input texture image stack 140 into a single composite texture image, such as a single composite texture image 430. Figure 5 In the illustrated example, the MPI compressed image encoder 425 includes an alpha stack compressor 515 for converting the alpha image of the input alpha image stack 150 into a single composite alpha image, such as a single composite alpha image 435. In the illustrated example, the data interface 505 outputs the single composite texture image 430 and the single composite alpha image 435 as a compressed MPI stack corresponding to the input MPI stack. Therefore, the data interface 505 is an example of a device for outputting a compressed MPI stack.

[0045] exist Figure 6 The Chinese illustration shows Figure 4 A block diagram of an example implementation of the MPI stack decoder 420. Figure 6 The example MPI stack decoder 420 includes an example MPI compressed image decoder 460 and an example data interface 605. Figure 6 The example MPI compressed image decoder 460 includes an example texture stack decompressor 610 and an example alpha stack decompressor 615.

[0046] exist Figure 6 In the illustrated example, data interface 605 is used to access the input compressed multiplanar image stack, which corresponds to the source camera viewpoint. (See above for contact information.) Figure 5 The described compressed multiplane image stack includes (i) a single composite texture image, such as composite texture image 450, which will be decompressed to represent an uncompressed texture image to be included in a decoded texture image stack (e.g., decoded texture image stack 465), and / or (ii) a single composite alpha image, such as single composite alpha image 455, which will be decompressed to represent an uncompressed alpha image to be included in a decoded alpha image stack (e.g., decoded alpha image stack 470). As described above, each uncompressed alpha image includes a pixel value representing the transparency of a corresponding pixel in each paired uncompressed texture image. Thus, data interface 605 is an example of a means for accessing a compressed multiplane image stack corresponding to a source camera viewpoint.

[0047] exist Figure 6 In the illustrated example, the MPI compressed image decoder 460 includes a texture stack decompressor 610 to decompress a single composite texture image 450 to obtain an uncompressed texture image to be included in the decoded texture image stack 465. Figure 5In the illustrated example, the MPI compressed image decoder 460 includes an alpha stack decompressor 615 to decompress a single composite alpha image 455 to obtain an uncompressed alpha image to be included in the decoded alpha image stack 470. In the illustrated example, the data interface 605 outputs a decoded (uncompressed) MPI stack, which includes a decoded texture image stack 465 and a decoded alpha image stack 470. Therefore, the data interface 605 is an example of a device for outputting an uncompressed MIP stack including an uncompressed texture image (e.g., an uncompressed texture image such as the decoded texture image stack 465) and an uncompressed alpha image (e.g., an uncompressed alpha image included in the decoded alpha image stack 470).

[0048] Return to Figure 5 In some examples, the texture stack compressor 510 converts the texture images of the input texture image stack 140 into a single composite texture image 430 at the source platform 405, based on the observation that for each pixel location, only a few alpha planes in the input alpha image stack 150 corresponding to the input texture image stack 140 have large alpha values. Therefore, for that pixel location, the texture value of the texture plane corresponding to the few alpha planes with large alpha values ​​will have the greatest influence in the rendered target image. To take advantage of this observation, in some examples, the texture stack compressor 510 generates a single composite texture image 430 with pixel values ​​that are an alpha-weighted average of the texture values ​​on the texture images included in the input texture image stack 140.

[0049] Mathematically, the aforementioned example operation of the texture stack compressor 510 can be expressed as follows. For pixel position i, the texture pixel value p in a single composite texture image 430 i,j Defined by Formula 1, the formula is:

[0050]

[0051] In Formula 1, j = [0…31] is the index on the texture image (plane) in the input texture image stack 140, and t i,j αi,j is the texture (e.g., R,G,B) component at pixel position i in plane j, αi,j is the alpha value at pixel position i in plane j, and the round() operation converts the fractional parameter to the nearest integer.

[0052] Figure 7 The diagram illustrates the process of... Figure 5The MPI stack encoder 410 includes a texture stack compressor 510 that performs an example MPI texture stack compression operation 700. In the illustrated example operation 700, the texture stack compressor 510 processes the input texture image stack 140 and the input alpha image stack 150 of the input MPI stack according to Equation 1 to generate a single composite texture image 430, which is an alpha-weighted average of the texture values ​​on the input texture image stack 140.

[0053] Return to Figure 6 The texture stack decompressor 610 decompresses the single composite texture image 450 at the target platform 415 by copying the single composite texture image 450 N (e.g., N=32) times to obtain an uncompressed texture image, which is then included in the uncompressed texture image stack 465 of the decoded MPI stack. Figure 8 The illustration depicts a corresponding example MPI texture stack decompression operation 800 performed by the texture stack decompressor 610 of the MPI stack decoder 420. In the illustrated example operation 800, the texture stack decompressor 610 copies a single composite texture image 450 to obtain an uncompressed texture image stack 465. In the illustrated example, the corresponding alpha image stack obtained at the target platform can be either an uncompressed alpha image stack 470 generated by the alpha stack decompressor 615 or the original input alpha image stack 150.

[0054] As can be seen from the previous examples, the composite texture images 430 / 450 can resemble the original texture image frames from the source camera. Therefore, in some examples, the texture stack compressor 510 included in the MPI stack encoder 410 uses the original texture image frames from the source camera as the composite texture images 430 / 450.

[0055] Return to Figure 5 In some examples, the alpha stack compressor 515 converts the alpha image of the input alpha image stack 150 into a single composite alpha image 435 at the source platform 405, based on the observation that for each pixel location, only a few alpha planes in the input alpha image stack 150 have large alpha values. To take advantage of this observation, in some examples, the alpha stack compressor 515 generates a single composite alpha image 435, also referred to as an alpha map, to encode a subset of alpha planes with large alpha values ​​and these alpha values.

[0056] In some such examples, the alpha stack compressor 515 generates a single composite alpha image 435 (alpha map) to capture the location and amplitude of the alpha plane with the largest alpha value, as well as the location and amplitude of the neighboring alpha planes with the second largest alpha value. Mathematically, the alpha plane with the largest alpha value at pixel location i is designated as p. i,m And the corresponding alpha value is specified as a. i,m The nearest neighbor plane with the second largest alpha value is p. i,m +1 or p i,m -1, alpha value is a i,2 The offset value b at pixel position i i Defined by Formula 2, the formula is:

[0057]

[0058] In Equation 2, if the neighboring plane with the second largest alpha value is p m If +1 is added, a positive sign is used; otherwise, the term is negative, meaning the nearest plane with the second-largest alpha value is p. m -1. Then, the composite alpha value d at pixel position i in the composite alpha image 435 (alpha map) is... i Equation 3 gives the following formula:

[0059] d i =8*p i,m +b i

[0060] Formula 3

[0061] For example, if a pixel location i has the largest alpha value of 200 in the alpha plane corresponding to alpha plane index = 10, and the second largest alpha value of 60 in the alpha plane corresponding to alpha plane index = 9, then the alpha plane index p of the plane with the largest alpha value is... i,m It will be 10. The bias factor has a negative sign because plane 9 is smaller than plane 10. Bias value b i The value will be 2, representing 8*60 / 260 (where 8 corresponds to 8 bits of precision). Therefore, in this example, the resulting composite alpha value d at pixel position i is... i It will be 8*10+(-2)=78.

[0062] In some examples, some pixel locations have no significant alpha value across the entire plane. In some examples, some pixels are even completely transparent across the entire alpha plane. This counterintuitive set of pixels may be an artifact of the machine learning model used to generate the MPI stack. These pixels, though some may appear infrequently, should be maintained to indicate low prediction probabilities between views. To account for these pixels, pixel values ​​from 0 to 7 are used to map these ranges of pixel values. For example, a composite alpha value of 0 is d. i It can be used to represent the location of a completely transparent pixel. In some examples, a composite alpha value from 1 to 7 is used to represent the range of maximum values ​​across two planes, so that the higher the composite alpha value, the higher the maximum value of the two maximum planes at that pixel location.

[0063] In some examples, the alpha stack compressor 515 filters the resulting single composite alpha image 435 (alpha spectrum) with a smoothing filter to reduce the bit rate of transmission, at the cost of a slight decrease in accuracy.

[0064] Return to Figure 6 In some examples, the alpha stack decompressor 615 decompresses a single composite alpha image 455 (alpha map) at the target platform 415, as follows. Assuming the single composite alpha image 455 is generated based on Equations 2-4 as described above, the alpha stack decompressor 615 uses the single composite alpha image 455 (alpha map) to approximately reconstruct N (e.g., N = 32) alpha images of the uncompressed alpha image stack 470, as follows. Given the composite alpha value d at pixel position i in the composite alpha image 435 (alpha map) defined by Equation 4 above. i The Alpha Stack Decompressor 615 calculates the following values ​​given by formulas 5 and 6, which are:

[0065]

[0066] if Yes, it is positive. The Alpha stack decompressor 615 sets the decoding neighbor plane of pixel position i to... Otherwise, the neighbor plane is set to

[0067] Next, for pixel position i, the Alpha stack decompressor 615 calculates the following values ​​given by formulas 7 and 8, which are:

[0068]

[0069] The Alpha Stack Decompressor 615 will decompress the Alpha plane. The alpha value at pixel position i in the image is set to And set the alpha value at pixel position i in the neighboring plane to As defined above. The alpha stack decompressor 615 sets the alpha value at pixel position i in the other alpha images of the uncompressed alpha image stack 470 to zero.

[0070] The alpha stack compressor 515 and alpha stack decompressor 615, based on the example implementations of Equations 2-8, correspond to special cases of the following example implementations. For pixel position i, the value in the composite alpha image (alpha spectrum) is defined by Equation 9, which is:

[0071]

[0072] In Equation 9, j = [0…31] is the index on the alpha image (plane) in the input alpha image stack 150, and α i,j It is the alpha value at pixel position i in alpha plane j. When only two neighboring planes j and j+1 have non-zero alpha values, Equation 9 simplifies to Equation 10, that is:

[0073]

[0074] In Formula 10, d i It has a fractional value. Because j is in the range [0…31], d i The value 8 can be used for scaling to allow the use of the full 8-bit range [0…255], which is then rounded to an integer, as given in Equation 11:

[0075]

[0076] On the decoder side, assuming α i,j +α i,j+1 =255, Formulas 5-8 above provide the reconstruction.

[0077] In some examples, the depth map from the source camera can be used by the alpha stack compressor 515 on the encoder side to generate a single composite alpha image 435 (alpha map) to replace the input alpha image stack 150. Figure 9 In the first example implementation based on a depth map from a source camera, illustrated in Example Operation 900, it is assumed that N = 32 alpha planes are uniformly spaced within the numerical range [0…65535] corresponding to a 16-bit depth map, with a gap of I = 65535 / 31 between each plane. The 16-bit depth map has a depth d (determined by…). Figure 9 The pixel labeled 905 in the diagram will be located in the alpha plane j (as indicated by the pixel label 905). Figure 9(represented by label 910 in the text) and the alpha plane j+1 (by...) Figure 9 The number 915 in the text indicates the range, where the value of j is given by formula 12, which is:

[0078]

[0079] In this example, the alpha stack compressor 515 calculates the composite alpha value at pixel position i according to formula 13, which is:

[0080]

[0081] In formula 13, d j =dj*I (where d j exist Figure 9 (represented by the number 920).

[0082] In a second example implementation based on a depth map from a source camera, an alpha stack compressor 515 transmits the depth map to the decoding side. The alpha stack decompressor 615 on the decoding side initially sets the alpha value of the stack of N = 32 uncompressed alpha planes to zero. (See reference...) Figure 9 As described above for the first example implementation, in a 16-bit depth map, there is a depth d (by...). Figure 9 The pixel labeled 905 in the diagram will be located in the alpha plane j (as indicated by the pixel label 905). Figure 9 (represented by label 910 in the text) and the alpha plane j+1 (by...) Figure 9 The reference 915 in the text indicates the range between, where the value of j is given by formula 12. The alpha stack decompressor 615 on the decoding side calculates the value d according to formulas 14 and 15. j and d j+1 These formulas are:

[0083] d j =dj*I

[0084] Formula 14

[0085] d j+1 =Id j

[0086] Formula 15

[0087] Then, the Alpha stack decompressor 615 sets the value of pixel i in the Alpha plane j according to formulas 16 and 17, which is determined by a i,j This represents, and the value of pixel i in the alpha plane j+1, which is determined by a i,j+1 These formulas are:

[0088]

[0089] In a third example implementation based on a depth map from the source camera, the alpha stack compressor 515 adds a preprocessing step to the first example implementation described above. In this example, the alpha stack compressor 515 applies k-means clustering, where k = 32, to the 16-bit depth map from the camera to generate an alpha plane at the location of the object in the best-matching scene within the depth range. The depth of the alpha plane (corresponding to finding the centroid generated by k-means clustering) is transmitted as metadata to the target platform (decoding side). This allows the encoding techniques used in the first example implementation to more accurately approximate the true depth d.

[0090] In the fourth example implementation based on the depth map from the source camera, the alpha stack compressor 515 transmits the raw 16-bit depth map to the decoder. In some examples, the alpha stack compressor 515 reduces the precision of the depth map to use fewer bits, such as 10 bits or some other value. At the decoder, the depth map is used to warp (e.g., reproject) the texture frame from the source camera directly to the target viewpoint's frame. This fourth example implementation avoids the use of an MPI stack, thus avoiding the machine learning model described above.

[0091] Although Figures 4-6 The diagram illustrates an example of implementing a video encoding and decoding system 400, but... Figures 4-6 One or more of the elements, processes, and / or devices shown may be combined, divided, rearranged, omitted, eliminated, and / or implemented in any other way. Additionally, the example MPI stack generator 100, example MPI renderer 200, example source platform 405, example MPI stack encoder 410, example target platform 415, example MPI stack decoder 420, example MPI compressed image encoder 425, example MPI compressed image decoder 460, example data interface 505, example texture stack compressor 510, example alpha stack compressor 515, example data interface 605, example texture stack decompressor 610, example alpha stack decompressor 615, and / or more generally, Figures 4-6The example video encoding and decoding system 400 can be implemented by hardware, software, firmware, and / or any combination of hardware, software, and / or firmware. Thus, for example, any of the following: example MPI stack generator 100, example MPI renderer 200, example source platform 405, example MPI stack encoder 410, example target platform 415, example MPI stack decoder 420, example MPI compressed image encoder 425, example MPI compressed image decoder 460, example data interface 505, example texture stack compressor 510, example alpha stack compressor 515, example data interface 605, example texture stack decompressor 610, example alpha stack decompressor 615, and / or more generally, example video encoding and decoding system 400, may be made of one or more analog or digital circuits, logic circuits, one or more programmable processors, one or more programmable controllers, one or more graphics processing units (GPUs), one or more digital signal processors (DSPs), one or more application-specific integrated circuits (ASICs), one or more programmable logic devices (PLDs), or field-programmable gate arrays (FPGAs). This is implemented using gate arrays (FPGAs) and / or one or more field-programmable logic devices (FPLDs). When any device or system claim in this patent covers a purely software and / or purely firmware implementation, at least one of the following: Example MPI stack generator 100, Example MPI renderer 200, Example source platform 405, Example MPI stack encoder 410, Example target platform 415, Example MPI stack decoder 420, Example MPI compressed image encoder 425, Example MPI compressed image decoder 460, Example data interface 505, Example texture stack compressor 510, Example alpha stack compressor 515, Example data interface 605, Example texture stack decompressor 610, Example alpha stack decompressor 615, and / or more generally, Example video encoding and decoding system 400, is hereby explicitly defined as including a non-transitory computer-readable storage device or disk containing such software and / or firmware, such as a memory, digital versatile disk (DVD), compact disk (CD), Blu-ray disc, etc. Furthermore, Example video encoding and decoding system 400 may include, in addition to... Figures 4-6Those other than or replacing those shown Figures 4-6 One or more of the elements, processes, and / or devices shown, and / or may include any or all of more than one of the elements, processes, and devices shown in the illustrations. As used herein, the phrase “communicating with”—including its variations—covers direct communication and / or indirect communication via one or more intermediate components, without requiring direct physical (e.g., wired) communication and / or continuous communication, but also including selective communication at periodic intervals, scheduled intervals, non-periodic intervals, and / or one-off events.

[0092] exist Figures 10-14 The diagram illustrates example hardware logic, machine-readable instructions, hardware implementation state machines, and / or any combination thereof for implementing an MPI stack encoder 410. In these examples, the machine-readable instructions may be one or more executable programs or portions thereof for execution by a computer processor and / or processor circuitry, such as those described below. Figure 19 The processor 1912 is shown in the example processor platform 1900 discussed in the discussion. The one or more programs, or portions thereof, may be embodied in storage on media such as CD-ROM, floppy disk, hard disk, DVD, or Blu-ray disc. TM Alternatively, the software may reside on a non-transitory computer-readable storage medium such as memory associated with the processor 1912, but one or more programs in whole and / or portions thereof may be executed by a device other than the processor 1912 and / or embodied in firmware or dedicated hardware. Additionally, although this is a reference... Figures 10-14 The flowcharts shown are used to describe one or more example programs, but many other methods of implementing the example MPI stack encoder 410 may be used alternatively. For example, refer to Figures 10-14 The execution order of the blocks shown in the flowchart can be changed, and / or some of the described blocks can be altered, eliminated, combined, and / or subdivided into multiple blocks. Additionally or alternatively, any or all blocks may be implemented by one or more hardware circuits (e.g., discrete and / or integrated analog and / or digital circuits, FPGAs, ASICs, comparators, operational amplifiers (op-amps), logic circuits, etc.) configured to perform the corresponding operations without executing software or firmware. Processor circuitry may be distributed across different network locations and / or local to one or more devices (e.g., a multi-core processor in a single machine, multiple processors distributed across a server rack, etc.).

[0093] exist Figures 15-17The diagram illustrates example hardware logic, machine-readable instructions, hardware implementation state machines, and / or any combination thereof for implementing an MPI stack decoder 420. In these examples, the machine-readable instructions may be one or more executable programs or portions thereof for execution by a computer processor and / or processor circuitry, such as those described below. Figure 20 The example processor platform 2000 illustrates the processor 2012. The program or its components may be stored on a medium such as a CD-ROM, floppy disk, hard disk drive, DVD, or Blu-ray disc. TM Alternatively, the software may be on a non-transitory computer-readable storage medium such as memory associated with the processor 2012, but one or more programs in whole and / or portions thereof may be executed by a device other than the processor 2012 and / or embodied in firmware or dedicated hardware. Additionally, although this is a reference... Figures 15-17 The flowcharts shown are used to describe one or more example programs, but many other methods of implementing the example MPI stack decoder 420 may be used instead. For example, refer to Figures 15-17 The execution order of the blocks shown in the flowchart can be changed, and / or some of the described blocks can be altered, eliminated, combined, and / or subdivided into multiple blocks. Additionally or alternatively, any or all blocks may be implemented by one or more hardware circuits (e.g., discrete and / or integrated analog and / or digital circuits, FPGAs, ASICs, comparators, operational amplifiers (op-amps), logic circuits, etc.) configured to perform the corresponding operations without executing software or firmware. Processor circuitry may be distributed across different network locations and / or local to one or more devices (e.g., a multi-core processor in a single machine, multiple processors distributed across a server rack, etc.).

[0094] The machine-readable instructions described herein can be stored in one or more formats, including compressed formats, encrypted formats, segmented formats, compiled formats, executable formats, packaged formats, etc. The machine-readable instructions described herein can be stored as data or data structures (e.g., portions of instructions, code, representations of code, etc.) that can be used to create, manufacture, and / or produce machine-executable instructions. For example, machine-readable instructions can be segmented and stored on one or more storage devices and / or computing devices (e.g., servers) located in the same or different locations within a network or set of networks (e.g., in the cloud, in edge devices, etc.). Machine-readable instructions can require installation, modification, adaptation, updating, combination, supplementation, configuration, decryption, decompression, unpacking, distribution, reassignment, compilation, etc., to make them directly readable, interpretable, and / or executable by computing devices and / or other machines. For example, machine-readable instructions can be stored as multiple parts that are individually compressed, encrypted, and stored on separate computing devices, wherein these parts, when decrypted, decompressed, and combined, form a set of executable instructions that implement one or more functions, which together form a program, such as that described herein.

[0095] In another example, machine-readable instructions may be stored in a state in which they can be read by processor circuitry, but require the addition of libraries (e.g., dynamic link libraries (DLLs)), software development kits (SDKs), application programming interfaces (APIs), etc., to execute these instructions on a specific computing device or other device. In another example, machine-readable instructions may need to be configured (e.g., storage settings, input data, recording network addresses, etc.) before they can be executed in whole or in part. Therefore, machine-readable media as used herein may include machine-readable instructions and / or (one or more) programs, regardless of the specific format or state of these machine-readable instructions and / or (one or more) programs at the time of storage or otherwise at rest or in transit.

[0096] The machine-readable instructions described in this article can be represented by any past, present, or future instruction language, scripting language, programming language, etc. For example, machine-readable instructions can be represented using any of the following languages: C, C++, Java, C#, Perl, Python, JavaScript, HyperText Markup Language (HTML), Structured Query Language (SQL), Swift, etc.

[0097] As described above, executable instructions (e.g., computer and / or machine-readable instructions) stored on a non-transitory computer and / or machine-readable medium can be used to implement... Figures 10-17 The example process, wherein the medium is, for example, a hard disk drive, flash memory, read-only memory, compact disk, digital multifunction disk, cache, random access memory, and / or any other storage device or disk in which information can be stored for any duration (e.g., long-term storage, permanent storage, temporary storage, temporary buffering, and / or caching for information). For the purposes of this document, the term non-transitory computer-readable medium is explicitly defined as including any type of computer-readable storage device and / or disk, and excludes propagating signals and transmission media. Furthermore, for the purposes of this document, the terms "computer-readable" and "machine-readable" are considered equivalent unless otherwise indicated.

[0098] "Comprising" and "including" (and all its forms and tenses) are used herein as introductory terms. Thus, whenever a claim uses any form of "comprising" or "including" (e.g., including, comprising, having, etc.) as a preamble or in any kind of claim statement, it is understood that additional elements, terms, etc., may exist without falling outside the scope of the corresponding claim or statement. As used herein, when the phrase "at least" is used as a transitional term in, for example, the preamble of a claim, it is introductory in the same way that the terms "comprising" and "including" are introductory. The term "and / or" when used, for example, in the form of, say, A, B, and / or C, refers to any combination or subset of A, B, and C, such as (1) A alone, (2) B alone, (3) C alone, (4) A and B, (5) A and C, (6) B and C, and (7) A and B and C. As used herein in the context of describing structures, components, items, objects, and / or things, the phrase “at least one of A and B” is intended to refer to an implementation that includes any one of the following: (1) at least one A, (2) at least one B, and (3) at least one A and at least one B. Similarly, as used herein in the context of describing structures, components, items, objects, and / or things, the phrase “at least one of A or B” is intended to refer to an implementation that includes any one of the following: (1) at least one A, (2) at least one B, and (3) at least one A and at least one B. As used herein in the context of describing the execution or operation of processes, instructions, actions, activities, and / or steps, the phrase “at least one of A and B” is intended to refer to an implementation that includes any one of the following: (1) at least one A, (2) at least one B, and (3) at least one A and at least one B. Similarly, for the purposes of this document in the context of describing the execution or operation of processes, instructions, actions, activities and / or steps, the phrase “at least one of A or B” is intended to refer to an implementation that includes any one of the following: (1) at least one A, (2) at least one B, and (3) at least one A and at least one B.

[0099] As used herein, singular references (e.g., “a,” “first,” “second,” etc.) do not exclude pluralism. For the purposes of this document, the term “a” refers to one or more of the same entity. The terms “a,” “one or more,” and “at least one” may be used interchangeably herein. Furthermore, although listed separately, multiple means, elements, or method actions may be implemented by, for example, a single unit or processor. Moreover, while individual features may be included in different examples or claims, they may be combined, and inclusion in different examples or claims does not imply that the combination of features is infeasible and / or not advantageous.

[0100] Can be executed to achieve Figure 4 and / or Figure 5 Example program 1000 of MPI stack encoder 410 by Figure 10 The flowchart shown is illustrated. Refer to the preceding figures and associated written description. Figure 10 Example program 1000 begins execution at block 1005, where MPI stack encoder 410 determines whether the input texture image stack 140 included in the generated MPI stack 135 corresponding to the source camera viewpoint should be compressed. If the input texture image stack 140 is to be compressed (block 1005), then at block 1010, the MPI compression image encoder 425 of MPI stack encoder 410 converts the input texture image stack 140 into a single composite texture image 430, as described above. Figure 11 and Figure 12 The diagram illustrates an example program where processing is implemented at block 1010, which is described in further detail below. At block 1015, the MPI compressed image encoder 425 outputs a single composite texture image 430. Control then continues to block 1020. However, if the input texture image stack 140 is not compressed (block 1005), then at block 1025, the MPI stack encoder 410 outputs the texture image stack 140. Control then continues to block 1020.

[0101] In block 1020, the MPI stack encoder 410 determines whether the input alpha image stack 150 included in the generated MPI stack 135 corresponding to the source camera viewpoint should be compressed. If the input alpha image stack 150 is to be compressed (block 1020), then in block 1030, the MPI compression image encoder 425 of the MPI stack encoder 410 converts the input alpha image stack 150 into a single composite alpha image 435, as described above. Figure 13 and Figure 14 The diagram illustrates an example program implementing the processing at block 1030, which is described in further detail below. At block 1035, the MPI compressed image encoder 425 outputs a single composite alpha image 435. Control then continues to block 1040. However, if the input alpha image stack 150 is not compressed (block 1020), then at block 1045, the MPI stack encoder 410 outputs the alpha image stack 150. Control then continues to block 1040.

[0102] In block 1040, the MPI stack encoder 410 determines whether to process other MPI stacks 135 corresponding to other source camera viewpoints. If other MPI stacks 135 are to be processed, control returns to block 1005. Otherwise, execution of example program 1000 ends.

[0103] exist Figure 11The diagram illustrates a texture stack compressor 510 that can be executed to implement an MPI compressed image encoder 425 and / or Figure 10 Example program 1010P1 for processing at block 1010. Refer to the preceding figures and associated written description, Figure 11 Example program 1010P1 begins execution at block 1105, where the texture stack compressor 510 accesses the input texture image stack 140 from the data interface 505. At block 1110, the texture stack compressor 510 accesses the input alpha image stack 150 from the data interface 505. At block 1115, the texture stack compressor 510 begins processing each texture image in the texture image stack 140. For example, at block 1120, the texture stack compressor 510 weights the pixel values ​​of the texture image by the pixel values ​​of the alpha image corresponding to the given texture image, as described above. For example, the processing at block 1120 can correspond to the texture stack compressor 510 calculating the numerator t of Equation 1 above. i,j α i,j In block 1125, texture stack compressor 510 continues to process texture images until all texture images in texture image stack 140 have been processed.

[0104] At block 1130, the texture stack compressor 510 begins iterating through each pixel position i to determine the pixel value of the single composite texture image 430. For example, at block 1135, the texture stack compressor 510 uses the weighted texture image determined at block 1120 to determine the alpha-weighted average pixel value of pixel position i of the single composite texture image 430, as described above. For example, the processing at block 1120 can correspond to the texture stack compressor 510 calculating the output term p of Equation 1 above. i,j In block 1140, the texture stack compressor 510 continues processing pixel positions until all pixel positions of the single composite texture image 430 have been evaluated. In block 1145, the texture stack compressor 510 outputs the single composite texture image 430 via data interface 505, where each pixel position i is set to the corresponding alpha-weighted average pixel value p determined at block 1135. i,j Then, the execution of example program 1010P1 ends.

[0105] exist Figure 12 The diagram illustrates a texture stack compressor 510 that can be executed to implement an MPI compressed image encoder 425 and / or Figure 10 Example program 1010P2 shows the processing at block 1010. Refer to the preceding figures and associated written description. Figure 12Example program 1010P2 begins execution at block 1205, where the texture stack compressor 510 accesses the source camera texture image associated with the source camera viewpoint corresponding to the input texture image stack 140 via data interface 505. At block 1210, the texture stack compressor 510 outputs a single composite texture image 430 via data interface 505, whose individual pixel positions i are set to the corresponding pixel values ​​of the source camera texture image. Then, execution of example program 1010P2 ends.

[0106] exist Figure 13 The diagram illustrates an alpha stack compressor 515 that can be executed to implement an MPI compressed image encoder 425 and / or Figure 10 Example program 1030P1 for processing at block 1030. Refer to the preceding figures and associated written description, Figure 13 Example program 1030P1 begins execution at block 1305, where the alpha stack compressor 515 accesses the input alpha image stack 150 from data interface 505. At block 1310, the alpha stack compressor 515 begins processing each pixel position i to determine the pixel value of the individual composite alpha image 435. For example, at block 1315, as described above, the alpha stack compressor 515 identifies the first alpha plane index value (e.g., p). i,m This index value identifies the pixel location i with the largest alpha value (e.g., a). i,m The input alpha image stack 150 has an alpha plane. In block 1320, as described above, the alpha stack compressor 515 identifies a second alpha plane index value (e.g., p). i,m +1 or p i,m -1), this index value identifies the second largest alpha value (e.g., a) at a given pixel location i. i,2 The nearest alpha plane. In block 1325, as described above, the alpha stack compressor 515 is based on the maximum alpha value (e.g., a). i,m ) and the second largest alpha value (e.g., a) i,2 Determine the bias value (e.g., b) for a given pixel position i. i For example, in block 1325, the Alpha stack compressor 515 can determine the bias value b according to Formula 2. i As described above. In block 1330, as described above, the Alpha stack compressor 515 combines the first Alpha plane index value (e.g., p). i,m ) and bias values ​​(e.g., b) i ) to determine the composite alpha value (e.g., d) for a given pixel position i. i For example, in block 1330, the alpha stack compressor 515 can determine the composite alpha value d for a given pixel position i according to Formula 3. iAs described above, in block 1335, the alpha stack compressor 515 continues processing pixel positions until all pixel positions of the single composite alpha image 435 have been evaluated. In block 1340, the alpha stack compressor 515 outputs the single composite alpha image 435 via data interface 505, where each pixel position i is set to the corresponding composite alpha value d determined at block 1330. i Then, the execution of example program 1030P1 ends.

[0107] exist Figure 14 The diagram illustrates an alpha stack compressor 515 that can be executed to implement an MPI compressed image encoder 425 and / or Figure 10 Example program 1030P2 shows the processing at block 1030. Refer to the preceding figures and associated written description. Figure 14 Example program 1030P2 begins execution at block 1405, where the alpha stack compressor 515 accesses the source camera depth image associated with the source camera viewpoint corresponding to the input alpha image stack 150 via data interface 505. At block 1410, the alpha stack compressor 515 begins processing each pixel location i to determine the pixel value of the single composite alpha image 435. For example, at block 1415, as described above, the alpha stack compressor 515 determines the alpha plane index value (e.g., j) based on the depth image value (e.g., d) at a given pixel location i. For example, at block 1415, the alpha stack compressor 515 can determine the alpha plane index value j at a given pixel location i according to Formula 12, as described above. At block 1420, as described above, the alpha stack compressor 515 determines the bias value (e.g., d) at a given pixel location i based on the alpha plane index value (e.g., j) and the depth image value (e.g., d) at a given pixel location i. j For example, in block 1420, the Alpha stack compressor 515 can be configured according to expression d. j =dj*I determines the bias value d j As described above. In block 1425, as described above, the alpha stack compressor 515 combines the alpha plane index value (e.g., j) and the bias value (e.g., d). j ) to determine the composite alpha value (e.g., d) for a given pixel position i. i For example, in block 1425, the alpha stack compressor 515 can determine the composite alpha value d for a given pixel position i according to formula 13. iAs described above, in block 1430, the alpha stack compressor 515 continues processing pixel positions until all pixel positions of the single composite alpha image 435 have been evaluated. In block 1435, the alpha stack compressor 515 outputs the single composite alpha image 435 via data interface 505, where each pixel position i is set to the corresponding composite alpha value d determined at block 1425. i Then, the execution of example program 1030P1 ends.

[0108] Can be executed to achieve Figure 4 and / or Figure 6 Example program 1500 for MPI stack decoder 420 by Figure 15 The flowchart shown is illustrated. Refer to the preceding figures and associated written description. Figure 15 Example program 1500 begins execution at block 1505, where the MPI stack decoder 420 determines whether to decompress the composite texture image 450 corresponding to the source camera viewpoint. If there is a composite texture image 450 to be decompressed (block 1505), then at block 1510, the MPI compressed image decoder 460 of the MPI stack decoder 420 decompresses the composite texture image 450 to obtain an uncompressed texture image stack 465, as described above. Figure 16 The diagram illustrates an example program implementing the processing at block 1510, which is described in further detail below. At block 1015, the MPI compressed image decoder 460 outputs an uncompressed texture image stack 465. Control then proceeds to block 1520. However, if there is no composite texture image 450 to decompress (block 1505), then at block 1525, the MPI stack decoder 420 outputs the received texture image stack (because there was no texture stack compression at the source platform 405). Control then proceeds to block 1520.

[0109] In block 1520, the MPI stack decoder 420 determines whether the composite alpha image 455 corresponding to the source camera viewpoint should be decompressed. If there is a composite alpha image 455 to be decompressed (block 1520), then in block 1530, the MPI compressed image decoder 460 of the MPI stack decoder 420 decompresses the composite alpha image 455 to obtain an uncompressed alpha image stack 470, as described above. Figure 17The diagram illustrates an example program implementing the processing at block 1530, which is described in further detail below. At block 1535, the MPI compressed image decoder 460 outputs the uncompressed alpha image stack 470. Control then proceeds to block 1540. However, if there is no composite alpha image 455 to decompress (block 1520), then at block 1545, the MPI stack decoder 420 outputs the received alpha image stack (because there was no alpha stack compression at the source platform 405). Control then proceeds to block 1520.

[0110] In block 1540, the MPI stack decoder 420 determines whether to perform MPI decompression on other source camera viewpoints. If other source camera viewpoints are to undergo MPI decompression, control returns to block 1505. Otherwise, execution of example program 1500 terminates.

[0111] exist Figure 16 The diagram illustrates a texture stack decompressor 610 that can be executed to implement an MPI compressed image decoder 460 and / or Figure 15 Example program 1510P handles the process at block 1510. Refer to the preceding figures and associated written description. Figure 16 Example program 1510P begins execution at block 1605, where texture stack decompressor 610 accesses composite texture image 450 via data interface 605. In block 1610, as described above, texture stack decompressor 610 copies composite texture image 450 to form an uncompressed texture image stack 465. In block 1615, texture stack decompressor 610 outputs the uncompressed texture image stack 465 via data interface 605. Then, execution of example program 1510P ends.

[0112] exist Figure 17 The diagram illustrates an alpha stack decompressor 615 that can be executed to implement an MPI compressed image decoder 460 and / or Figure 15 Example program 1530P handles block 1530. Refer to the preceding figures and associated written description. Figure 17 Example program 1530P begins execution at block 1705, where the alpha stack decompressor 615 accesses the composite alpha image 455 via data interface 605. At block 1710, the alpha stack decompressor 615 begins processing each pixel position i of the composite alpha image 455 to determine the uncompressed alpha image stack 470. For example, at block 1715, as described above, the alpha stack decompressor 615 bases its output on the composite alpha value (e.g., d) at pixel position i. i Determine the first alpha plane index value for a given pixel position i (e.g., For example, in block 1715, the Alpha stack decompressor 615 can determine the first Alpha plane index value according to Formula 5. As described above. In block 1720, as described above, the Alpha stack decompressor 615 is based on the first Alpha plane index value (e.g., ) and the composite alpha value at pixel position i (e.g., d i Determine the offset value at a given pixel position i. For example, in block 1720, the Alpha stack decompressor 615 can determine the bias value according to formula 6. As described above. In block 1725, as described above, the Alpha stack decompressor 615 is based on the bias value of pixel position i. Determine the second alpha plane index value for a given pixel position i (e.g., or For example, in block 1725, if the bias value... If it is positive, then the Alpha stack decompressor 615 can set the second Alpha plane index value to... And if the bias value If it is negative, then the second alpha plane index value can be set to... As stated above.

[0113] In block 1730, as described above, the Alpha stack decompressor 615 is based on the bias value of pixel position i. Determine the second alpha value for a given pixel position i (e.g., For example, in block 1730, the alpha stack decompressor 615 can determine the second alpha value according to formula 7. As described above. In block 1735, the alpha stack decompressor 615 is based on a second alpha value determined for pixel position i (e.g., Determine the first alpha value for a given pixel position i. For example, in block 1735, the alpha stack decompressor 615 can determine the first alpha value according to formula 8. As described above. In block 1740, the Alpha stack decompressor 615 will be indexed by the first Alpha plane index value (e.g., A given pixel position i in the uncompressed alpha image is set to have a first alpha value. As described above. In block 1745, the Alpha stack decompressor 615 will be indexed by the second Alpha plane index value (e.g., or The given pixel location i in the uncompressed alpha image is set to have a second alpha value (e.g., As described above. In block 1750, the alpha stack decompressor 615 sets the remaining uncompressed alpha image of the uncompressed alpha image stack 470 to zero at a given pixel position i, as described above. In block 1755, the alpha stack decompressor 615 continues processing pixel positions until all pixel positions of the composite alpha image 455 have been evaluated. In block 1760, the alpha stack decompressor 615 outputs the uncompressed alpha image stack 470 via data interface 605. Then, the execution of example program 1530P ends.

[0114] Figure 18 The diagram shows that it will be made by Figure 4 Example process 1800, performed by the example video encoding and decoding system 400, is used to render a target image using a single composite alpha image 435 / 455 and a single composite texture image 430 / 450 determined as described above by the MPI compressed image encoder 425 of the MPI stack encoder 410. In the illustrated example process 1800, the MPI stack decoder 420 uses the single composite alpha image 435 / 455 (also known as an alpha map or compressed alpha map) as an example coarse depth map, which is used to create an example derived depth map 1820. The derived depth map 1820 is used to render the target image 1815 by directly projecting the single composite texture image 430 / 450 onto the target viewpoint.

[0115] In the illustrated example, the derived depth map 1820 is determined as follows. First, the composite alpha image 435 / 455 (e.g., an alpha map or a compressed alpha map) is considered as a coarse 8-bit depth map. To obtain a 10-bit or 16-bit depth map, the MPI stack decoder 420 multiplies the value of the composite alpha image 435 / 455 (e.g., an alpha map or a compressed alpha map) by 4 or 256, respectively. From the 8-bit depth map with pixel value d, the MPI stack decoder 420 determines the original depth z according to Formula 18, which is:

[0116]

[0117] In example procedure 1800, the raw depth is used to project each pixel into 3D space using both intrinsic camera properties (focal length, principal point) and extrinsic properties (the position of the camera center relative to world coordinates). For example, in block 1825, pixels can be directly projected into the image of the target view (this may involve less computation), and / or in block 1830, the raw depth can be used to create a 3D triangular mesh that is projected onto the target and rasterized (this may involve more computation).

[0118] In block 1835, images obtained by projecting from multiple cameras are then blended to produce the final rendered target image 1815. This blending uses a weighting function that depends on the distance between the center of the target camera and the source.

[0119] Figure 19 It is constructed to execute Figures 10-14 Instructions to achieve Figure 4 and / or Figure 5 A block diagram of an example processor platform 1900 for an MPI stack encoder 410. The processor platform 1900 can be, for example, a server, personal computer, workstation, self-learning machine (e.g., neural network), mobile device (e.g., cellular phone, smartphone, such as iPad). TM Tablet devices, personal digital assistants (PDAs), internet-connected appliances, DVD players, CD players, digital video recorders, Blu-ray players, game consoles, personal video recorders, set-top boxes, digital cameras, headphones or other wearable devices, or any other type of computing device.

[0120] The illustrated processor platform 1900 includes a processor 1912. The illustrated processor 1912 is hardware. For example, the processor 1912 may be implemented by one or more integrated circuits, logic circuits, microprocessors, GPUs, DSPs, or controllers from any desired family or manufacturer. The hardware processor 1912 may be a semiconductor-based (e.g., silicon-based) device. In this example, the processor 1912 implements an MPI compressed image encoder 425, a data interface 505, a texture stack compressor 510, and an alpha stack compressor 515.

[0121] The illustrated processor 1912 includes local memory 1913 (e.g., cache). The illustrated processor 1912 communicates with main memory, including volatile memory 1914 and non-volatile memory 1916, via link 1918. Link 1918 may be implemented by a bus, one or more point-to-point connections, or a combination thereof. Volatile memory 1914 may be Synchronous Dynamic Random Access Memory (SDRAM), Dynamic Random Access Memory (DRAM), etc. Dynamic Random Access Memory (DRAM) Dynamic Random Access Memory, This can be implemented using flash memory and / or any other type of random access memory device. The non-volatile memory 1916 can be implemented using flash memory and / or any other desired type of memory device. Access to the main memory 1914 and 1916 is controlled by the memory controller.

[0122] The processor platform 1900 illustrated also includes interface circuitry 1920. Interface circuitry 1920 can be implemented using any type of interface standard, such as an Ethernet interface, Universal Serial Bus (USB), etc. Interfaces include near field communication (NFC) interfaces and / or PCI fast interfaces.

[0123] In the illustrated example, one or more input devices 1922 are connected to interface circuitry 1920. The input devices 1922 allow the user to input data and / or commands into processor 1912. These input devices may be implemented as, for example, audio sensors, microphones, cameras (still or video), keyboards, buttons, mice, touchscreens, touchpads, trackballs, trackbars (e.g., isopoints), speech recognition systems, and / or any other human-machine interface. Additionally, many systems, such as processor platform 1900, allow the user to control the computer system and provide data to the computer using physical gestures, such as, but not limited to, hand or body movements, facial expressions, and facial recognition.

[0124] One or more output devices 1924 are also connected to the interface circuitry 1920 illustrated in the figure. The output devices 1924 may be implemented, for example, by display devices (e.g., light-emitting diodes (LEDs), organic light-emitting diodes (OLEDs), liquid crystal displays (LCDs), cathode ray tube displays (CRTs), in-place switching (IPS) displays, touchscreens, etc.), haptic output devices, printers, and / or one or more speakers. The interface circuitry 1920 illustrated in the figure thus typically includes a graphics driver card, a graphics driver chip, and / or a graphics driver processor.

[0125] The interface circuit 1920 illustrated also includes communication devices, such as transmitters, receivers, transceivers, modems, residential gateways, wireless access points, and / or network interfaces, to facilitate data exchange with external machines (e.g., any kind of computing device) via network 1926. Communication can be conducted via, for example, Ethernet connections, digital subscriber line (DSL) connections, telephone line connections, coaxial cable systems, satellite systems, line-to-line wireless systems, cellular telephone systems, and so on.

[0126] The processor platform 1900 illustrated also includes one or more mass storage devices 1928 for storing software and / or data. Examples of such mass storage devices 1928 include floppy disk drives, hard disk drives, compact disk drives, Blu-ray disc drives, redundant array of independent disks (RAID) systems, and digital versatile disk (DVD) drives.

[0127] and Figures 10-14 The corresponding machine-executable instructions 1932 can be stored in mass storage devices 1928, volatile memory 1914, non-volatile memory 1916, local memory 1913, and / or on removable non-transitory computer-readable storage media such as CDs or DVDs 1936.

[0128] Figure 20 It is constructed to execute Figures 15-17 Instructions to achieve Figure 4 and / or Figure 6 A block diagram of an example processor platform 2000 with an MPI stack decoder 420. The processor platform 2000 can be, for example, a server, personal computer, workstation, self-learning machine (e.g., neural network), mobile device (e.g., cellular phone, smartphone, such as iPad). TM Tablet devices, PDAs, internet-connected appliances, DVD players, CD players, digital video recorders, Blu-ray players, game consoles, personal video recorders, set-top boxes, digital cameras, headphones or other wearable devices, or any other type of computing device.

[0129] The illustrated processor platform 2000 includes a processor 2012. The illustrated processor 2012 is hardware. For example, the processor 2012 may be implemented by one or more integrated circuits, logic circuits, microprocessors, GPUs, DSPs, or controllers from any desired family or manufacturer. The hardware processor 2012 may be a semiconductor-based (e.g., silicon-based) device. In this example, the processor 2012 implements an MPI compressed image decoder 460, a data interface 605, a texture stack decompressor 610, and an alpha stack decompressor 615.

[0130] The illustrated processor 2012 includes local memory 2013 (e.g., cache). The illustrated processor 2012 communicates with main memory, including volatile memory 2014 and non-volatile memory 2016, via link 2018. Link 2018 may be implemented as a bus, one or more point-to-point connections, etc., or a combination thereof. Volatile memory 2014 may be SDRAM, DRAM, etc. And / or any other type of random access memory device. Non-volatile memory 2016 can be implemented using flash memory and / or any other desired type of memory device. Access to main memory 2014, 2016 is controlled by the memory controller.

[0131] The processor platform 2000 illustrated also includes interface circuitry 2020. Interface circuitry 2020 can be implemented using any type of interface standard, such as an Ethernet interface, USB, etc. Interface, NFC interface, and / or PCI fast interface.

[0132] In the illustrated example, one or more input devices 2022 are connected to interface circuitry 2020. The input devices 2022 allow the user to input data and / or commands into processor 2012. These input devices may be implemented using, for example, audio sensors, microphones, cameras (still or video), keyboards, buttons, mice, touchscreens, touchpads, trackballs, trackbars (e.g., isopoints), speech recognition systems, and / or any other human-machine interface. Additionally, many systems, such as processor platform 2000, allow the user to control the computer system and provide data to the computer using physical gestures, such as, but not limited to, hand or body movements, facial expressions, and facial recognition.

[0133] One or more output devices 2024 are also connected to the interface circuitry 2020 illustrated in the figure. The output devices 2024 may be implemented, for example, by display devices (e.g., LED, OLED, LCD, CRT monitors, IP displays, touchscreens, etc.), haptic output devices, printers, and / or (one or more) speakers. The interface circuitry 2020 illustrated in the figure thus typically includes a graphics driver card, a graphics driver chip, and / or a graphics driver processor.

[0134] The interface circuit 2020 illustrated also includes communication devices such as transmitters, receivers, transceivers, modems, residential gateways, wireless access points, and / or network interfaces to facilitate data exchange with external machines (e.g., any type of computing device) via network 2026. Communication can be conducted via, for example, Ethernet connections, DSL connections, telephone line connections, coaxial cable systems, satellite systems, line-to-line wireless systems, cellular telephone systems, and so on.

[0135] The processor platform 2000 illustrated also includes one or more mass storage devices 2028 for storing software and / or data. Examples of such mass storage devices 2028 include floppy disk drives, hard disk drives, compact disk drives, Blu-ray disc drives, RAID systems, and DVD drives.

[0136] and Figures 10-14 The corresponding machine-executable instructions 2032 may be stored in mass storage device 2028, volatile memory 2014, non-volatile memory 2016, local memory 2013 and / or on removable non-transitory computer-readable storage media such as CD or DVD 2036.

[0137] exist Figure 21 The diagram illustrates the methods for distributing such... to third parties Figure 19 and / or Figure 20 A block diagram of an example software distribution platform 2105, such as example computer-readable instructions 1932 and / or 2032. The example software distribution platform 2105 can be implemented by any computer server, data facility, cloud service, etc., capable of storing software and transferring it to other computing devices. A third party can be a customer of an entity that owns and / or operates the software distribution platform. For example, the entity owning and / or operating the software distribution platform can be the software (e.g., software...). Figure 19 and / or Figure 20The developer, seller, and / or licensor of the software (example computer-readable instructions 1932 and / or 2032). A third party may be a consumer, user, retailer, OEM, etc., who purchases and / or licenses the software for use and / or resells and / or sublicenses. In the illustrated example, the software distribution platform 2105 includes one or more servers and one or more storage devices. The storage devices store computer-readable instructions 1932 and / or 2032, which may correspond to the instructions described above. Figures 10-14 and Figures 15-17 Examples of computer-readable instructions. One or more servers of the example software distribution platform 2105 communicate with network 2110, which may correspond to the Internet and / or any one or more of the example networks 1926 and / or 2026 described above. In some examples, as part of a commercial transaction, one or more servers respond to a request to transfer software to a requesting party. Payment for the delivery, sale, and / or licensing of the software may be processed by one or more servers of the software distribution platform and / or via a third-party payment entity. These servers enable purchasers and / or licensors to download computer-readable instructions 1932 and / or 2032 from the software distribution platform 2105. For example, it may be compatible with... Figures 10-14 The software corresponding to the example computer-readable instructions can be downloaded to the example processor platform 1900, which executes the computer-readable instructions 1932 to implement the MPI stack encoder 410. Additionally or alternatively, it can be used with... Figures 15-17 The software corresponding to the example computer-readable instructions can be downloaded to the example processor platform 2000, which executes the computer-readable instructions 2032 to implement the MPI stack decoder 420. In some examples, one or more servers of the software distribution platform 2105 periodically provide, transmit, and / or force software updates (e.g., Figure 19 and / or Figure 20 Examples of computer-readable instructions (1932 and / or 2032) are provided to ensure that improvements, patches, updates, etc., are distributed and applied to the software at the end-user device.

[0138] As will be clear from the foregoing, example methods, apparatuses, and articles of art for implementing multi-plane image compression have been disclosed. The disclosed methods, apparatuses, and articles of art improve the efficiency of using computing devices by reducing the amount of MPI data to be transmitted to a target platform for image rendering while maintaining the quality of the rendered image. Therefore, the example MPI compression techniques disclosed herein allow for lower bandwidth transmission to the target or rendering node compared to the transmission bandwidth required by other (e.g., conventional) techniques. Thus, the disclosed example MPI compression techniques can enable the execution of virtual reality and immersive media applications on resource-constrained platforms, but can also be used in any number of image processing systems and applications. The disclosed methods, apparatuses, and articles of art therefore target one or more improvements to the functionality of computers.

[0139] The foregoing disclosure provides example solutions for implementing multiplanar image compression. Further examples are disclosed herein, including subjects such as: apparatus for implementing multiplanar image compression, at least one non-transitory computer-readable medium comprising instructions that, when executed, cause at least one processor to implement multiplanar image compression, and associated methods. The disclosed examples may be implemented individually and / or in one or more combinations.

[0140] Example 1 includes an apparatus for compressing a multiplanar image stack, the apparatus comprising: an interface for accessing an input multiplanar image stack corresponding to a source camera viewpoint, the input multiplanar image stack including a plurality of texture images and corresponding plurality of alpha images, each of the alpha images including a pixel value representing the transparency of a corresponding pixel in each of the texture images; and a compressed image encoder for performing at least one of: (i) converting the plurality of texture images into a single composite texture image to generate the compressed multiplanar image stack, or (ii) converting the plurality of alpha images into a single composite alpha image to generate the compressed multiplanar image stack, the interface being used to output the compressed multiplanar image stack.

[0141] Example 2 includes the apparatus as described in Example 1, wherein the compressed image encoder is used to combine the plurality of texture images based on the plurality of alpha images to convert the plurality of texture images into the single composite texture image.

[0142] Example 3 includes the apparatus as described in Example 2, wherein, in order to combine the plurality of texture images based on the plurality of alpha images, the compressed image encoder is configured to weight the pixel values ​​of each texture image in the texture image according to the pixel values ​​of each alpha image in the alpha image corresponding to each texture image in the texture image to determine a plurality of alpha-weighted texture images.

[0143] Example 4 includes the apparatus as described in Example 3, wherein, in order to combine the plurality of texture images based on the plurality of alpha images, the compressed image encoder is used to average the alpha-weighted texture images to determine the single composite texture image, the single composite texture image including alpha-weighted pixel values ​​at each pixel location of the single composite texture image.

[0144] Example 5 includes an apparatus as described in Example 1, wherein the compressed image encoder is used to replace the plurality of texture images with source camera images associated with the source camera viewpoint corresponding to the input multiplanar image stack to convert the plurality of texture images into the single composite texture image.

[0145] Example 6 includes the apparatus as described in Example 1, wherein, in order to convert the plurality of alpha images into the single composite alpha image, the compressed image encoder is configured to: identify a first alpha plane index value corresponding to a first alpha image in the alpha images, wherein at a first pixel location, the first alpha image has the largest alpha value in the alpha images; identify a first alpha plane index value corresponding to a neighboring alpha image of the first alpha image in the alpha images, wherein at the first pixel location, the neighboring alpha image has the second largest alpha value in the alpha images; determine an offset value for the first pixel location, the offset value being based on the largest and second largest alpha values ​​at the first pixel location; and combine the first alpha plane index value and the offset value to determine a composite alpha value to be included in the single composite alpha image at the first pixel location.

[0146] Example 7 includes the apparatus as described in Example 1, wherein, in order to convert the plurality of alpha images into the single composite alpha image, the compressed image encoder is configured to: identify an alpha plane index value corresponding to a value of a source camera depth image at a first pixel location, the source camera depth image being associated with the source camera viewpoint corresponding to the input multi-plane image stack; determine an offset value for the first pixel location, the offset value being based on the alpha plane index value of the first pixel location and the source camera depth image at the first pixel location; and combine the alpha plane index value and the offset value to determine a composite alpha value to be included in the single composite alpha image at the first pixel location.

[0147] Example 8 includes an apparatus as described in Example 1, wherein the compressed image encoder is configured to (i) convert the plurality of texture images into the single composite texture image to generate the compressed multiplane image stack, and (ii) convert the plurality of alpha images into the single composite alpha image to generate the compressed multiplane image stack.

[0148] Example 9 includes at least one non-transitory computer-readable medium comprising computer-readable instructions that, when executed, cause at least one processor to perform at least the following operations: access an input multiplane image stack corresponding to a source camera viewpoint, the input multiplane image stack comprising a plurality of texture images and corresponding plurality of alpha images, each of the alpha images comprising a pixel value representing the transparency of a corresponding pixel in each of the texture images; perform at least one of the following: (i) convert the plurality of texture images into a single composite texture image to generate a compressed multiplane image stack, or (ii) convert the plurality of alpha images into a single composite alpha image to generate the compressed multiplane image stack; and output the compressed multiplane image stack.

[0149] Example 10 includes at least one non-transitory computer-readable medium as described in Example 9, wherein the instructions cause the at least one processor to combine the plurality of texture images based on the plurality of alpha images to convert the plurality of texture images into the single composite texture image.

[0150] Example 11 includes at least one non-transitory computer-readable medium as described in Example 10, wherein, in order to combine the plurality of texture images based on the plurality of alpha images, the instructions cause the at least one processor to weight the pixel values ​​of each texture image in the texture image according to the pixel values ​​of each alpha image in the alpha image corresponding to each texture image in the texture image to determine a plurality of alpha-weighted texture images.

[0151] Example 12 includes at least one non-transitory computer-readable medium as described in Example 11, wherein, in order to combine the plurality of texture images based on the plurality of alpha images, the instructions cause the at least one processor to average the alpha-weighted texture images to determine the single composite texture image, the single composite texture image including alpha-weighted pixel values ​​at each pixel location of the single composite texture image.

[0152] Example 13 includes at least one non-transitory computer-readable medium as described in Example 9, wherein the instructions cause the at least one processor to replace the plurality of texture images with source camera images associated with the source camera viewpoint corresponding to the input multiplanar image stack to transform the plurality of texture images into the single composite texture image.

[0153] Example 14 includes at least one non-transitory computer-readable medium as described in Example 9, wherein, in order to convert the plurality of alpha images into the single composite alpha image, the instructions cause the at least one processor to perform the following operations: identify a first alpha plane index value corresponding to a first alpha image in the alpha images, at a first pixel location, the first alpha image having the largest alpha value in the alpha images; identify a first alpha plane index value corresponding to a neighboring alpha image of the first alpha image in the alpha images, at the first pixel location, the neighboring alpha image having the second largest alpha value in the alpha images; determine a bias value for the first pixel location, the bias value being based on the largest and second largest alpha values ​​at the first pixel location; and combine the first alpha plane index value and the bias value to determine a composite alpha value to be included in the single composite alpha image at the first pixel location.

[0154] Example 15 includes at least one non-transitory computer-readable medium as described in Example 9, wherein, in order to convert the plurality of alpha images into the single composite alpha image, the instructions cause the at least one processor to perform the following operations: identify an alpha plane index value corresponding to a value of a source camera depth image at a first pixel location, the source camera depth image being associated with the source camera viewpoint corresponding to the input multi-plane image stack; determine an offset value for the first pixel location, the offset value being based on the alpha plane index value of the first pixel location and the source camera depth image at the first pixel location; and combine the alpha plane index value and the offset value to determine a composite alpha value to be included in the single composite alpha image at the first pixel location.

[0155] Example 16 includes at least one non-transitory computer-readable medium as described in Example 9, wherein the instructions cause the at least one processor to (i) convert the plurality of texture images into the single composite texture image to generate the compressed multiplane image stack, and (ii) convert the plurality of alpha images into the single composite alpha image to generate the compressed multiplane image stack.

[0156] Example 17 includes a method for compressing a multiplane image stack, the method comprising: accessing an input multiplane image stack corresponding to a source camera viewpoint, the input multiplane image stack including a plurality of texture images and corresponding plurality of alpha images, each of the alpha images including a pixel value representing the transparency of a corresponding pixel in each of the texture images; performing at least one of the following: (i) converting the plurality of texture images into a single composite texture image to generate the compressed multiplane image stack, or (ii) converting the plurality of alpha images into a single composite alpha image to generate the compressed multiplane image stack; and outputting the compressed multiplane image stack.

[0157] Example 18 includes the method as described in Example 17, wherein converting the plurality of texture images includes combining the plurality of texture images based on the plurality of alpha images to convert the plurality of texture images into the single composite texture image.

[0158] Example 19 includes the method as described in Example 18, wherein combining the plurality of texture images includes weighting the pixel values ​​of the texture images in the texture images according to the pixel values ​​of the alpha images in the alpha images corresponding to each texture image in the texture images to determine a plurality of alpha-weighted texture images.

[0159] Example 20 includes the method as described in Example 19, wherein combining the plurality of texture images includes averaging the alpha-weighted texture images to determine the single composite texture image, the single composite texture image including alpha-weighted pixel values ​​at each pixel location of the single composite texture image.

[0160] Example 21 includes the method as described in Example 17, wherein converting the plurality of texture images includes replacing the plurality of texture images with source camera images associated with the source camera viewpoint corresponding to the input multiplanar image stack to convert the plurality of texture images into the single composite texture image.

[0161] Example 22 includes the method as described in Example 17, wherein transforming the plurality of alpha images includes: identifying a first alpha plane index value corresponding to a first alpha image in the alpha images, at a first pixel location, the first alpha image having the largest alpha value in the alpha images; identifying a first alpha plane index value corresponding to a neighboring alpha image of the first alpha image in the alpha images, at the first pixel location, the neighboring alpha image having the second largest alpha value in the alpha images; determining a bias value for the first pixel location, the bias value being based on the largest and second largest alpha values ​​at the first pixel location; and combining the first alpha plane index value and the bias value to determine a composite alpha value to be included in the single composite alpha image at the first pixel location.

[0162] Example 23 includes the method as described in Example 17, wherein transforming the plurality of alpha images includes: identifying an alpha plane index value corresponding to a value of a source camera depth image at a first pixel location, the source camera depth image being associated with the source camera viewpoint corresponding to the input multi-plane image stack; determining a bias value for the first pixel location, the bias value being based on the alpha plane index value of the first pixel location and the source camera depth image at the first pixel location; and combining the alpha plane index value and the bias value to determine a composite alpha value to be included in the single composite alpha image at the first pixel location.

[0163] Example 24 includes the method as described in Example 17, wherein the method includes (i) converting the plurality of texture images into the single composite texture image to generate the compressed multiplane image stack, and (ii) converting the plurality of alpha images into the single composite alpha image to generate the compressed multiplane image stack.

[0164] Example 25 includes an apparatus for compressing a multiplanar image stack, the apparatus comprising: at least one memory; computer-readable instructions; and at least one processor for executing the computer-readable instructions to at least perform the following operations: accessing an input multiplanar image stack corresponding to a source camera viewpoint, the input multiplanar image stack comprising a plurality of texture images and corresponding plurality of alpha images, each of the alpha images comprising a pixel value representing the transparency of a corresponding pixel in each of the texture images; performing at least one of the following: (i) converting the plurality of texture images into a single composite texture image to generate the compressed multiplanar image stack, or (ii) converting the plurality of alpha images into a single composite alpha image to generate the compressed multiplanar image stack; and outputting the compressed multiplanar image stack.

[0165] Example 26 includes an apparatus as described in Example 25, wherein the at least one processor is configured to combine the plurality of texture images based on the plurality of alpha images to convert the plurality of texture images into the single composite texture image.

[0166] Example 27 includes the apparatus as described in Example 26, wherein, in order to combine the plurality of texture images based on the plurality of alpha images, the at least one processor is configured to weight the pixel values ​​of each texture image in the texture images according to the pixel values ​​of each alpha image in the alpha images corresponding to each texture image in the texture images to determine a plurality of alpha-weighted texture images.

[0167] Example 28 includes the apparatus as described in Example 27, wherein, in order to combine the plurality of texture images based on the plurality of alpha images, the at least one processor is configured to average the alpha-weighted texture images to determine the single composite texture image, the single composite texture image including alpha-weighted pixel values ​​at each pixel location of the single composite texture image.

[0168] Example 29 includes an apparatus as described in Example 25, wherein the at least one processor is configured to replace the plurality of texture images with source camera images associated with the source camera viewpoint corresponding to the input multiplanar image stack to convert the plurality of texture images into the single composite texture image.

[0169] Example 30 includes the apparatus as described in Example 25, wherein, in order to convert the plurality of alpha images into the single composite alpha image, the at least one processor is configured to: identify a first alpha plane index value corresponding to a first alpha image in the alpha images, wherein at a first pixel location, the first alpha image has the largest alpha value in the alpha images; identify a first alpha plane index value corresponding to a neighboring alpha image of the first alpha image in the alpha images, wherein at the first pixel location, the neighboring alpha image has the second largest alpha value in the alpha images; determine a bias value for the first pixel location, the bias value being based on the largest and second largest alpha values ​​at the first pixel location; and combine the first alpha plane index value and the bias value to determine a composite alpha value to be included in the single composite alpha image at the first pixel location.

[0170] Example 31 includes the apparatus as described in Example 25, wherein, in order to convert the plurality of alpha images into the single composite alpha image, the at least one processor is configured to: identify an alpha plane index value corresponding to a value of a source camera depth image at a first pixel location, the source camera depth image being associated with the source camera viewpoint corresponding to the input multi-plane image stack; determine an offset value for the first pixel location, the offset value being based on the alpha plane index value of the first pixel location and the source camera depth image at the first pixel location; and combine the alpha plane index value and the offset value to determine a composite alpha value to be included in the single composite alpha image at the first pixel location.

[0171] Example 32 includes an apparatus as described in Example 25, wherein the at least one processor is configured to (i) convert the plurality of texture images into the single composite texture image to generate the compressed multiplane image stack, and (ii) convert the plurality of alpha images into the single composite alpha image to generate the compressed multiplane image stack.

[0172] Example 33 includes an apparatus for compressing a multiplanar image stack, the apparatus comprising: an access means for accessing an input multiplanar image stack corresponding to a source camera viewpoint, the input multiplanar image stack including a plurality of texture images and corresponding plurality of alpha images, each of the alpha images including a pixel value representing the transparency of a corresponding pixel in each of the texture images; and an encoding means for encoding the input multiplanar image stack, the encoding means being configured to perform at least one of: (i) converting the plurality of texture images into a single composite texture image to generate the compressed multiplanar image stack, or (ii) converting the plurality of alpha images into a single composite alpha image to generate the compressed multiplanar image stack.

[0173] Example 34 includes an apparatus as described in Example 33, wherein the encoding means is used to combine the plurality of texture images based on the plurality of alpha images to convert the plurality of texture images into the single composite texture image.

[0174] Example 35 includes the apparatus as described in Example 34, wherein, in order to combine the plurality of texture images based on the plurality of alpha images, the encoding means is configured to weight the pixel values ​​of each texture image in the texture image according to the pixel values ​​of each alpha image in the alpha image corresponding to each texture image in the texture image to determine a plurality of alpha-weighted texture images.

[0175] Example 36 includes the apparatus as described in Example 35, wherein, in order to combine the plurality of texture images based on the plurality of alpha images, the encoding means is used to average the alpha-weighted texture images to determine the single composite texture image, the single composite texture image including alpha-weighted pixel values ​​at each pixel location of the single composite texture image.

[0176] Example 37 includes an apparatus as described in Example 33, wherein the encoding means is used to replace the plurality of texture images with source camera images associated with the source camera viewpoint corresponding to the input multiplanar image stack to convert the plurality of texture images into the single composite texture image.

[0177] Example 38 includes the apparatus as described in Example 33, wherein, in order to convert the plurality of alpha images into the single composite alpha image, the encoding means is configured to: identify a first alpha plane index value corresponding to a first alpha image in the alpha images, wherein at a first pixel location, the first alpha image has the largest alpha value in the alpha images; identify a first alpha plane index value corresponding to a neighboring alpha image of the first alpha image in the alpha images, wherein at the first pixel location, the neighboring alpha image has the second largest alpha value in the alpha images; determine an offset value for the first pixel location, the offset value being based on the largest and second largest alpha values ​​at the first pixel location; and combine the first alpha plane index value and the offset value to determine a composite alpha value to be included in the single composite alpha image at the first pixel location.

[0178] Example 39 includes the apparatus as described in Example 33, wherein, in order to convert the plurality of alpha images into the single composite alpha image, the encoding means is configured to: identify an alpha plane index value corresponding to a value of a source camera depth image at a first pixel location, the source camera depth image being associated with the source camera viewpoint corresponding to the input multi-plane image stack; determine an offset value for the first pixel location, the offset value being based on the alpha plane index value of the first pixel location and the source camera depth image at the first pixel location; and combine the alpha plane index value and the offset value to determine a composite alpha value to be included in the single composite alpha image at the first pixel location.

[0179] Example 40 includes an apparatus as described in Example 33, wherein the encoding means is configured to (i) convert the plurality of texture images into the single composite texture image to generate the compressed multiplane image stack, and (ii) convert the plurality of alpha images into the single composite alpha image to generate the compressed multiplane image stack.

[0180] Example 41 includes an apparatus for decompressing a compressed multiplanar image stack, the apparatus comprising: an interface for accessing the compressed multiplanar image stack, the compressed multiplanar image stack corresponding to a source camera viewpoint, the compressed multiplanar image stack including at least one of: (i) a single composite texture image representing a plurality of uncompressed texture images, or (ii) a single composite alpha image representing a plurality of uncompressed alpha images, each of the uncompressed alpha images including a pixel value representing the transparency of a corresponding pixel in each of the uncompressed texture images; and a compressed image decoder for performing at least one of: (i) decompressing the single composite texture image to obtain the plurality of uncompressed texture images, or (ii) decompressing the single composite alpha image to obtain the plurality of uncompressed alpha images, the interface being configured to output an uncompressed multiplanar image stack including the plurality of uncompressed texture images and the plurality of uncompressed alpha images.

[0181] Example 42 includes an apparatus as described in Example 41, wherein the compressed image decoder is configured to copy the single composite texture image to decompress the single composite texture image to obtain the plurality of uncompressed texture images.

[0182] Example 43 includes the apparatus as described in Example 41, wherein, in order to decompress the single composite alpha image, the compressed image decoder is configured to: set a first pixel position of a first uncompressed alpha image in the uncompressed alpha image to a first alpha value based on the value of the single composite alpha image at the first pixel position; set a first pixel position of a second uncompressed alpha image in the uncompressed alpha image to a second alpha value based on the value of the single composite alpha image at the first pixel position; and set the alpha value at the first pixel position of the remaining uncompressed alpha images in the uncompressed alpha image to zero.

[0183] Example 44 includes the apparatus as described in Example 43, wherein the compressed image decoder is configured to: determine a first alpha plane index value for the first pixel location based on the value of the single composite alpha image at the first pixel location; determine an offset value for the first pixel location based on the first alpha plane index value and the value of the single composite alpha image at the first pixel location; determine a second alpha plane index value for the first pixel location based on the offset value of the first pixel location; determine a second alpha value based on the offset value of the first pixel location; and determine the first alpha value based on the second alpha value.

[0184] Example 45 includes an apparatus as described in Example 41, wherein the compressed image decoder is configured to (i) decompress the single composite texture image to obtain the plurality of uncompressed texture images, and (ii) decompress the single composite alpha image to obtain the plurality of uncompressed alpha images.

[0185] Example 46 includes at least one non-transitory computer-readable medium comprising computer-readable instructions that, when executed, cause at least one processor to perform the following operations: at least access a compressed multiplane image stack corresponding to a source camera viewpoint, the compressed multiplane image stack comprising at least one of: (i) a single composite texture image representing a plurality of uncompressed texture images, or (ii) a single composite alpha image representing a plurality of uncompressed alpha images, each of the uncompressed alpha images comprising a pixel value representing the transparency of a corresponding pixel in each of the uncompressed texture images; perform at least one of: (i) decompressing the single composite texture image to obtain the plurality of uncompressed texture images, or (ii) decompressing the single composite alpha image to obtain the plurality of uncompressed alpha images; and output an uncompressed multiplane image stack comprising the plurality of uncompressed texture images and the plurality of uncompressed alpha images.

[0186] Example 47 includes at least one non-transitory computer-readable medium as described in Example 46, wherein the instructions cause the at least one processor to copy the single composite texture image to decompress the single composite texture image to obtain the plurality of uncompressed texture images.

[0187] Example 48 includes at least one non-transitory computer-readable medium as described in Example 46, wherein, in order to decompress the single composite alpha image, the instructions cause the at least one processor to perform the following operations: setting a first pixel position of a first uncompressed alpha image in the uncompressed alpha image to a first alpha value based on the value of the single composite alpha image at the first pixel position; setting a first pixel position of a second uncompressed alpha image in the uncompressed alpha image to a second alpha value based on the value of the single composite alpha image at the first pixel position; and setting the alpha value at the first pixel position of the remaining uncompressed alpha images in the uncompressed alpha image to zero.

[0188] Example 49 includes at least one non-transitory computer-readable medium as described in Example 48, wherein the instructions cause the at least one processor to perform the following operations: determine a first alpha plane index value for the first pixel location based on the value of the single composite alpha image at the first pixel location; determine an offset value for the first pixel location based on the first alpha plane index value and the value of the single composite alpha image at the first pixel location; determine a second alpha plane index value for the first pixel location based on the offset value of the first pixel location; determine a second alpha value based on the offset value of the first pixel location; and determine the first alpha value based on the second alpha value.

[0189] Example 50 includes at least one non-transitory computer-readable medium as described in Example 46, wherein the instructions cause the at least one processor to (i) decompress the single composite texture image to obtain the plurality of uncompressed texture images, and (ii) decompress the single composite alpha image to obtain the plurality of uncompressed alpha images.

[0190] Example 51 includes a method for decompressing a compressed multiplane image stack, the method comprising accessing the compressed multiplane image stack corresponding to a source camera viewpoint, the compressed multiplane image stack comprising at least one of: (i) a single composite texture image representing a plurality of uncompressed texture images, or (ii) a single composite alpha image representing a plurality of uncompressed alpha images, each of the uncompressed alpha images comprising a pixel value representing the transparency of a corresponding pixel in each of the uncompressed texture images; performing at least one of: (i) decompressing the single composite texture image to obtain the plurality of uncompressed texture images, or (ii) decompressing the single composite alpha image to obtain the plurality of uncompressed alpha images; and outputting an uncompressed multiplane image stack comprising the plurality of uncompressed texture images and the plurality of uncompressed alpha images.

[0191] Example 52 includes the method as described in Example 51, wherein decompressing the single composite texture image includes copying the single composite texture image.

[0192] Example 53 includes the method as described in Example 51, wherein decompressing the single composite alpha image includes: setting a first pixel position of a first uncompressed alpha image in the uncompressed alpha image to a first alpha value based on the value of the single composite alpha image at the first pixel position; setting a first pixel position of a second uncompressed alpha image in the uncompressed alpha image to a second alpha value based on the value of the single composite alpha image at the first pixel position; and setting the alpha value at the first pixel position of the remaining uncompressed alpha images in the uncompressed alpha image to zero.

[0193] Example 54 includes the method as described in Example 53, wherein decompressing the single composite alpha image includes: determining a first alpha plane index value for the first pixel location based on the value of the single composite alpha image at the first pixel location; determining an offset value for the first pixel location based on the first alpha plane index value and the value of the single composite alpha image at the first pixel location; determining a second alpha plane index value for the first pixel location based on the offset value of the first pixel location; determining a second alpha value based on the offset value of the first pixel location; and determining the first alpha value based on the second alpha value.

[0194] Example 55 includes the method as described in Example 51, wherein the method includes (i) decompressing the single composite texture image to obtain the plurality of uncompressed texture images, and (ii) decompressing the single composite alpha image to obtain the plurality of uncompressed alpha images.

[0195] Example 56 includes an apparatus for decompressing a compressed multiplane image stack, the apparatus comprising: at least one memory; computer-readable instructions; and at least one processor for executing the computer-readable instructions to at least perform the following operations: accessing a compressed multiplane image stack corresponding to a source camera viewpoint, the compressed multiplane image stack comprising at least one of: (i) a single composite texture image representing a plurality of uncompressed texture images, or (ii) a single composite alpha image representing a plurality of uncompressed alpha images, each of the uncompressed alpha images comprising a pixel value representing the transparency of a corresponding pixel in each of the uncompressed texture images; performing at least one of: (i) decompressing the single composite texture image to obtain the plurality of uncompressed texture images, or (ii) decompressing the single composite alpha image to obtain the plurality of uncompressed alpha images; and outputting an uncompressed multiplane image stack comprising the plurality of uncompressed texture images and the plurality of uncompressed alpha images.

[0196] Example 57 includes an apparatus as described in Example 56, wherein the at least one processor is configured to copy the single composite texture image to decompress the single composite texture image to obtain the plurality of uncompressed texture images.

[0197] Example 58 includes the apparatus as described in Example 56, wherein, in order to decompress the single composite alpha image, the at least one processor is configured to: set a first pixel position of a first uncompressed alpha image in the uncompressed alpha image to a first alpha value based on the value of the single composite alpha image at the first pixel position; set a first pixel position of a second uncompressed alpha image in the uncompressed alpha image to a second alpha value based on the value of the single composite alpha image at the first pixel position; and set the alpha value at the first pixel position of the remaining uncompressed alpha images in the uncompressed alpha image to zero.

[0198] Example 59 includes the apparatus as described in Example 58, wherein the at least one processor is configured to: determine a first alpha plane index value for the first pixel location based on the value of the single composite alpha image at the first pixel location; determine an offset value for the first pixel location based on the first alpha plane index value and the value of the single composite alpha image at the first pixel location; determine a second alpha plane index value for the first pixel location based on the offset value of the first pixel location; determine a second alpha value based on the offset value of the first pixel location; and determine the first alpha value based on the second alpha value.

[0199] Example 60 includes an apparatus as described in Example 56, wherein the at least one processor is configured to (i) decompress the single composite texture image to obtain the plurality of uncompressed texture images, and (ii) decompress the single composite alpha image to obtain the plurality of uncompressed alpha images.

[0200] Example 61 includes an apparatus for decompressing a compressed multiplanar image stack, the apparatus comprising: an access means for accessing the compressed multiplanar image stack, the compressed multiplanar image stack corresponding to a source camera viewpoint, the compressed multiplanar image stack including at least one of: (i) a single composite texture image representing a plurality of uncompressed texture images, or (ii) a single composite alpha image representing a plurality of uncompressed alpha images, each of the uncompressed alpha images including a pixel value representing the transparency of a corresponding pixel in each of the uncompressed texture images; and a decoding means for decoding the compressed multiplanar image stack, the decoding means being configured to perform at least one of: (i) decompressing the single composite texture image to obtain the plurality of uncompressed texture images, or (ii) decompressing the single composite alpha image to obtain the plurality of uncompressed alpha images.

[0201] Example 62 includes an apparatus as described in Example 61, wherein the decoding apparatus is configured to copy the single composite texture image to decompress the single composite texture image to obtain the plurality of uncompressed texture images.

[0202] Example 63 includes the apparatus as described in Example 61, wherein, in order to decompress the single composite alpha image, the decoding apparatus is configured to: set a first pixel position of a first uncompressed alpha image in the uncompressed alpha image to a first alpha value based on the value of the single composite alpha image at the first pixel position; set a first pixel position of a second uncompressed alpha image in the uncompressed alpha image to a second alpha value based on the value of the single composite alpha image at the first pixel position; and set the alpha value at the first pixel position of the remaining uncompressed alpha images in the uncompressed alpha image to zero.

[0203] Example 64 includes the apparatus as described in Example 63, wherein the decoding apparatus is configured to: determine a first alpha plane index value for the first pixel location based on the value of the single composite alpha image at the first pixel location; determine an offset value for the first pixel location based on the first alpha plane index value and the value of the single composite alpha image at the first pixel location; determine a second alpha plane index value for the first pixel location based on the offset value of the first pixel location; determine a second alpha value based on the offset value of the first pixel location; and determine the first alpha value based on the second alpha value.

[0204] Example 65 includes an apparatus as described in Example 61, wherein the decoding apparatus is configured to (i) decompress the single composite texture image to obtain the plurality of uncompressed texture images, and (ii) decompress the single composite alpha image to obtain the plurality of uncompressed alpha images.

[0205] While certain example methods, apparatuses, and articles of manufacture are disclosed herein, the scope of this patent is not limited thereto. Rather, this patent covers all methods, apparatuses, and articles of manufacture that fairly fall within the scope of the claims of this patent.

[0206] The appended claims are hereby incorporated by reference into this “Detailed Description” section, wherein each claim is an independent embodiment of this disclosure.

Claims

1. An encoder, comprising: instructions; and at least one programmable circuit, the at least one programmable circuit being programmed based on the instructions to: generate a composite texture image based on a plurality of texture images of a multi- plane image stack; generate a composite opacity image based on a plurality of opacity images of the multi- plane image stack; and encode the composite texture image and the composite opacity image into a bitstream. one or more of the at least one programmable circuit to: encode the composite texture image and the composite opacity image based on high efficiency video coding (HEVC).

2. The encoder of claim 1, wherein, one or more of the at least one programmable circuit to: generate the composite texture image based on opacity values included in one or more of the plurality of opacity images.

3. The encoder of claim 1, wherein, one or more of the at least one programmable circuit to: generate weights based on the opacity values; and 4. The encoder of claim 3, wherein, generate the composite texture image based on the weights. a first pixel value at a respective location in the composite texture image is associated with a corresponding second pixel at the respective location in the composite opacity image. one or more of the at least one programmable circuit to:

5. The encoder of claim 1, wherein, apply a filter to the composite opacity image prior to encoding.

6. The encoder of claim 1, wherein, the filter is a smoothing filter.

8. A computing method, comprising:

7. The encoder of claim 6, wherein, generating a composite texture image based on a plurality of texture images of a multi- plane image stack; generating a composite opacity image based on a plurality of opacity images of the multi- plane image stack; and encoding the composite texture image and the composite opacity image into a bitstream.

9. The method of claim 8, further comprising: encoding the composite texture image and the composite opacity image based on high efficiency video coding (HEVC).

10. The method of claim 8, further comprising: generating the composite texture image based on opacity values included in one or more of the plurality of opacity images.

11. The method of claim 10, further comprising: generating weights based on the opacity values; and generating the composite texture image based on the weights. a first pixel value at a respective location in the composite texture image is associated with a corresponding second pixel at the respective location in the composite opacity image.

13. The method of claim 8, further comprising: applying a filter to the composite opacity image prior to encoding. the filter is a smoothing filter.

12. The method of claim 8, wherein, 15. One or more computer-readable media storing instructions that, in response to execution by one or more processors, cause the one or more processors to perform the method of any one of claims 8 to 14.

16. A computing apparatus comprising means for performing the method of any one of claims 8 to 14.

17. A computer program product comprising instructions that, in response to execution by one or more processors, cause the one or more processors to perform the method of any one of claims 8 to 14.

14. The method of claim 13, wherein, ​ ​ ​ ​

Images