Encoding / decoding a video sequence associated with alpha channel information

Abstract

This application relates to methods and apparatus for encoding / decoding a video sequence associated with alpha channel information. An alpha channel indicator is signaled as part of the alpha channel information. The alpha channel indicator indicates which type of post-decoding processing the alpha channel information associated with the video sequence is intended for.

Description

[0001] Cross-references to related applications

[0002] This application claims priority and benefit to European Patent Application No. 22305949.4, filed with the European Patent Office on 30 June 2022, the entire contents of which are incorporated herein by reference. Technical Field

[0003] This application generally relates to encoding / decoding video sequences and signaling associated alpha channel information. In particular, but not exclusively, this application relates to an encoding / decoding method; means including tools for performing one of the methods; a bitstream formatted to include encoded data obtained according to the encoding method and an alpha channel indicator; a computer program product including instructions that, when executed by one or more processors, cause the one or more processors to perform the encoding / decoding method; and a non-transitory storage medium carrying instructions for performing the encoding / decoding method. Background Technology

[0004] This section is intended to introduce the reader to various aspects of the art that may relate to aspects of at least one exemplary embodiment of the present application described below and / or claimed. This discussion is intended to help provide the reader with background information to facilitate a better understanding of the various aspects of the present application. Therefore, it should be understood that these statements should be read in this context rather than as an admission of prior art.

[0005] A pixel corresponds to the smallest display unit on a screen, and it can be composed of one or more light sources (one for a monochrome screen and three or more for a color screen).

[0006] A video sequence is a time series of video frames.

[0007] A video frame (also known as a picture or image) includes at least one component (also known as a channel) defined by a specific image / video format that specifies all information related to the sample values ​​and all information that can be used by a display unit and / or any other device to display and / or decode video image data associated with the video frame in order to generate pixel values.

[0008] A video frame includes at least one component, typically represented in the shape of a 2D array of samples.

[0009] Monochrome video frames consist of a single component, while color video frames (also known as textured video frames) may consist of three components.

[0010] For example, when the image / video format is a well-known (Y,Cb,Cr) format, a color video frame may include a luminance (or brightness) component and two chrominance components, or when the image / video format is a well-known (R,G,B) format, a color video frame may include three color components (one for red, one for green, and one for blue).

[0011] Each component of a video frame may include multiple samples relative to the number of pixels on the screen on which the video image is intended to be displayed. For example, the number of samples included in a component may be the same as, or a multiple (or fraction) of, the number of pixels on the screen on which the video frame is intended to be displayed.

[0012] The number of samples contained in a component may be a multiple (or fraction) of the number of samples contained in another component in the same video frame.

[0013] For example, in a video format that includes a luminance component and two chrominance components (such as the (Y,Cb,Cr) format), the chrominance component may contain half the number of samples in width and / or height relative to the luminance component, depending on the color format under consideration.

[0014] A sample is the smallest unit of visual information that makes up a video frame. Sample values ​​can be, for example, luminance or chrominance values, or color values ​​of the red, green, or blue components in (R,G,B) format.

[0015] For monochrome video images, the pixel value of the screen can be represented by a single sample. For color video frames, the pixel value of the screen can be represented by multiple co-located samples. A co-located sample associated with a pixel refers to the sample corresponding to the position of the pixel on the screen.

[0016] Video frames are typically considered as a set of pixel values, with each pixel represented by at least one sample.

[0017] In computer graphics, alpha compositing or alpha blending is the process of combining an image with a background to create a partially or completely transparent appearance.

[0018] It is often useful to render image elements (pixels) in individual paths or layers and then combine the resulting 2D images into a single final image (called a composite image). Compositing is widely used in film when computer-rendered image elements are combined with live footage.

[0019] Alpha blending is also used in 2D computer graphics to place rasterized foreground elements on top of the background.

[0020] Alpha channels are used in combination. If alpha channels are used in an image, two common representations are available: direct (non-associated) alpha and pre-multiplied (associated) alpha.

[0021] For Direct Alpha, the RGB components of an image represent the color of an object or pixel, regardless of its opacity.

[0022] For premultiplication alpha, the RGB components represent the emission of an object or pixel, while the alpha represents occlusion.

[0023] The alpha channel can also be used for adapting the transparency level of see-through (transparent) displays (screens).

[0024] Historically, the first see-through displays used LCD display technology from the mid-2010s. However, LCD transparent screens only filtered incident light from behind the TV to illuminate the pixels, which effectively meant these screens couldn't work in a dark room. This is one of the reasons why OLED-based see-through displays quickly became a more promising approach, because OLED screens have self-emissive pixels. That is, each pixel of an OLED screen contains its own light source. In short, OLED-based see-through displays are made from ordinary OLED screens with numerous holes punched in them by manufacturers so that light can pass through from behind. It's important to note that because OLED screens emit light in only one direction, a viewer behind the screen will not be able to see what is displayed.

[0025] For both LCD and OLED-based transparent screens, the ability of a viewer to perceive the scene behind the TV, or a portion of it, is determined by the pixel values. There are two extreme cases. The first is when the user cannot see anything behind the screen at all, i.e., completely opaque. The second is when the user can fully see what's behind the TV and there are no other images displayed on the TV, i.e., completely transparent. Between these two cases, there exists a continuous range of transparency levels, where the user sees the image displayed on the screen superimposed on top of the scene behind the TV. In all these cases, for the best transparent display technologies such as current OLED displays, completely opaque, completely transparent, and medium transparent can be used for the entire screen or at a pixel-level granularity.

[0026] Depending on whether it's LCD or OLED technology, each effect described above (i.e., completely opaque, completely transparent, and medium transparent) is achieved in a different way. For LCDs, a pixel is black when as much light as possible from the light source is blocked. Therefore, dark pixels don't allow light from behind the TV to pass through and will appear opaque to the viewer. Conversely, for OLEDs, a pixel is white when all light passes through all sub-color pixels to form a white beam. Therefore, light from behind can also pass through the pixel through the aperture, allowing the viewer to see objects behind it, thus creating a transparent effect.

[0027] Both LCD and OLED screens can display a grayscale gradient from left (white) to right (black). On a transparent OLED, black pixels (with the lights off) allow light from behind to pass through the screen. This effectively means that in this simplified version of a transparent OLED screen, black cannot be displayed. Conversely, if the scene behind the TV is not white, it is impossible to display white pixels on a transparent LCD screen.

[0028] There are OLED TV models that have an additional layer placed directly behind the screen. This additional layer is responsible for dimming the light coming from behind. The purpose of this dimming layer is to improve the transparency of OLED-based screens, making dark pixels opaque as well by activating the layer. Another use of this layer is switching between traditional opaque TV and transparent TV modes. In practice, there may be situations where content is not designed for transparent screens, leading to a poor user experience; in such cases, being able to switch to traditional opaque TV mode would be beneficial for the user. As of today, it appears possible to activate this dimming layer at different dimming intensity levels, but only for the entire screen at a time. That is, the dimming layer does not seem to be able to target specific areas / pixels of the screen. However, this feature can be expected in future commercial versions.

[0029] Perceived transparency depends on:

[0030] - The intensity level of the sample, which results in brighter pixels, and therefore less transparent for OLEDs (and less transparent for LCDs).

[0031] opposite);

[0032] - The amount of light from behind the TV, since the objects behind will be visible, therefore the more light from behind the TV...

[0033] The brighter the image, the more transparent it appears.

[0034] - The ability to block light from behind the TV via a dimming back panel.

[0035] These three methods (and potentially more in the future) can be used to adjust the perceived pixel transparency level to match the content creator's intent.

[0036] One way to adjust image samples is by performing an alpha blending operation, treating the displayed image as an overlay on a black (for OLED screens) background. This way, portions of the image that are expected to be more transparent will have a low alpha coefficient, which increases the level of black in the composite image due to a factor equal to (1-α). However, this doesn't account for the potential increase in brightness (or luminance) that might be needed in the opposite scenario (where the area should be opaque), even though possible natural light might come from behind the TV. If an area of ​​the image has a high coefficient value (say, 1), alpha blending will simply keep the samples the same (value multiplied by alpha); however, due to, for example, strong ambient lighting, it might be necessary to increase the values ​​of those luminance samples by a factor larger than simply alpha (value 1). Furthermore, if there is a backlight that dims the light behind the TV, the receiver might not even need to adjust some sample values ​​to make the area opaque. Activating the dimming panel is sufficient, if possible, to position it towards this area in the image.

[0037] Therefore, the alpha channel for transmitting transparent screens is relevant, but simply applying alpha blending is insufficient to recreate the perceived transparency effect.

[0038] Alpha channels and alpha mixing have already been implemented in video coding standards based on auxiliary images associated with the main image and SEI (Supplemental Enhancement Information) messages.

[0039] Basically, during alpha mixing, due to the information in the SEI message, the samples of the auxiliary image are interpreted as an indication of the degree of opacity (or equivalently, the degree of transparency) associated with the corresponding brightness samples of the main image.

[0040] For example, the current draft of the VSEI (General Supplemental Enhancement Information Messages for Encoded Video Bitstreams) standard (JVET-Y2006, Additional SEI Messages for VSEI (Draft 6), 25th Meeting of ITU-T SG 16WP 3 and ISO / IEC JTC 1 / SC 29, JVET, 12-21 January 2022) defines the Alpha Channel Information (ACI) SEI message (Section 8.23) and the Scalability Dimension Information (SDI) SEI message (Section 8.19). The SDI SEI message provides scalability dimension information for each layer, such as 1) the view ID of each layer when there may be multiple views; and 2) the auxiliary ID of each layer when there may be auxiliary information (such as depth or alpha) carried by one or more layers. The Alpha Channel Information (ACI) SEI message provides information about the alpha channel sample values ​​and the post-decoding processing of the alpha plane encoded in the auxiliary images and one or more associated decoded master images.

[0041] The ACI SEI signaling in the VSEI specification does not cover the intended transparency level for the purpose of using transparent screens (e.g., chosen by the film director / producer at the production company), but primarily assumes conventional alpha blending operation.

[0042] The ACI SEI signaling in the VSEI specification does not allow post-decoding processing related to transparent screens, and since alpha mixing is different, intents should be distinguished (even if some required parameter semantics will be the same).

[0043] Furthermore, given that the syntax and semantics of auxiliary images in current video coding standards will be hijacked for transparent screen purposes, signaling similar to VSEI does not prevent the interpretation and application of auxiliary image processing to the upstream equipment (source) of the TV, rather than processing specifically for transparent screens, which may vary particularly depending on the underlying technology used (OLED vs. LCD).

[0044] If both the alpha channel and transparency intent should be transmitted simultaneously, VSEI signaling does not provide any mechanism to determine how this should be done (processing order, identification of the appropriate SEI for appropriate processing, etc.).

[0045] As understood from existing technology, the concept of image compositing assumes two images, such as a foreground image F and a background image B, where the foreground image is superimposed on top of the background image. To calculate the resulting pixel from the synthesis of corresponding pixels from the two images, a mathematical formula has been established: O = F × α + B × (1 - α). Therefore, a coefficient α for each pixel position needs to be known. Based on this coefficient name, the alpha plane (also called the alpha channel) is conceptualized, and the related alpha mixing for image compositing is conceptualized. Thus, there is a coupling between the design of alpha mixing and the alpha channel. This coupling can also be observed in video transmission specifications such as video coding and video and image delivery formats, where the alpha channel is always used for the purpose of alpha mixing.

[0046] However, there are other use cases, such as displaying images and videos on transparent screens, where the alpha channel can be transmitted along with the visual content, allowing the receiving device to adapt the pixel transparency intensity of the displayed content based on the signaling information presented earlier.

[0047] There is a need to improve known alpha channel signaling in existing technologies to decouple the assumed alpha mixing post-decoding process from other types of post-decoding processes. Furthermore, another need is to address the ambiguity of signaling alpha channel sequences when their purpose is not specified (as seen in several video coding standards), enabling interoperability. Finally, another need is to improve compression efficiency by signaling only relevant parameters associated with the post-decoding process. For example, parameters in the alpha channel signaling associated only with alpha mixing are syntactically excluded from other post-decoding processes, allowing for the preservation of bit information for transmission and avoiding the need to specify spurious values ​​when alpha mixing is part of the post-decoding process.

[0048] In view of the foregoing, at least one exemplary embodiment of this application has been designed. Summary of the Invention

[0049] The following section presents a simplified outline of at least one exemplary embodiment to provide a basic understanding of some aspects of this application. This outline is not an exhaustive summary of the exemplary embodiments. It is not intended to identify key or essential elements of the exemplary embodiments. The following outline presents only some aspects of at least one exemplary embodiment in a simplified form as a prelude to a more detailed description provided elsewhere in the document.

[0050] According to a first aspect of this application, a method is provided for encoding a video sequence associated with alpha channel information into a bitstream, wherein the method includes:

[0051] - Write an alpha channel indicator into the bitstream. This alpha channel indicator indicates which type of post-processing of the video sequence the alpha channel information associated with the video sequence is intended to be used for.

[0052] - If the alpha channel indicator indicates that the alpha channel information is intended for pixel transparency adjustment on a transparent screen;

[0053] - Encode the video sequence as data into a bitstream;

[0054] - Determine the transparency information mask associated with the video sequence; and

[0055] - The alpha channel information representing the transparency information mask is encoded as data into the bitstream.

[0056] According to a second aspect of this application, a method is provided for decoding a video sequence associated with alpha channel information from a bitstream, wherein the method includes:

[0057] - Read the alpha channel indicator from the bitstream, which indicates what type of post-processing the alpha channel information is intended for the video sequence;

[0058] - If the alpha channel indicator indicates that the alpha channel information is intended for adjusting the pixel transparency of a transparent screen;

[0059] - Decode encoded data from the bitstream in the decoded video sequence;

[0060] - Decode encoded data from the bitstream in the decoded alpha channel video sequence that transmits alpha channel information;

[0061] - Determine the transparency information mask based on the transparency information associated with the decoded alpha channel video sequence; and

[0062] - Adjust the pixel transparency of the transparent screen based on a transparency information mask, wherein the pixels are derived from sample values ​​of the decoded video sequence.

[0063] In one exemplary embodiment, the decoding method further includes determining whether a dimming adjustment of the dimming backplane of the transparent screen is requested, and if a dimming adjustment is requested, adjusting the pixel transparency by adjusting the dimming backplane.

[0064] In one exemplary embodiment, the decoding method further acquires ambient lighting condition data and adjusts pixel transparency based on the transparency information mask and the ambient lighting conditions.

[0065] In one exemplary embodiment, the alpha channel indicator is a specific syntax element (alpha_channel_idc, alpha_channel_id) included in the alpha channel information SEI message.

[0066] In one exemplary embodiment, a specific syntax element (alpha_channel_idc)

[0067] - Equals 0 to indicate that the alpha channel information SEI message is intended for alpha mixing of decoded video images with alpha channel information;

[0068] - equals 1 to indicate Alpha channel information. SEI messages are intended for adjusting pixel transparency.

[0069] In one exemplary embodiment, when equal to a specific value, the specific syntax element (alpha_channel_idc) further indicates that parameters of the alpha channel information SEI message not involved in the pixel transparency adjustment are not written into or read from the bitstream.

[0070] In one exemplary embodiment, the specific syntax element (alpha_channel_id) is a number belonging to a range of values ​​that indicate an alternative post-decoding process or a cascade of alternative post-decoding processes intended to be applied to the video sequence being decoded.

[0071] In one exemplary embodiment, a specific syntax element (alpha_channel_id) further indicates how to cascade post-decoding processing intended to be applied to the decoded video image.

[0072] In one exemplary embodiment, when equal to a specific value, the specific syntax element (alpha_channel_id) further indicates that parameters of the alpha channel information SEI message not involved in the transparency adjustment of the pixels of the decoded video image are not written into or read from the bitstream.

[0073] In one exemplary embodiment, the alpha channel indicator is a specific value of the existing syntax element (alpha_channel_use_idc) of the alpha channel information SEI message, which indicates the transparency adjustment of pixels in the alpha channel information intended for use in decoding video sequences.

[0074] In one exemplary embodiment, the alpha channel indicator is a specific type of auxiliary image that is signaled in the SEI message.

[0075] According to a third aspect of this application, a bitstream is provided, which is formatted to include encoded data obtained from the method according to the first aspect and an alpha channel indicator.

[0076] According to a fourth aspect of this application, an apparatus includes tools for performing one of the methods according to the first and / or second aspects of this application.

[0077] According to a fifth aspect of this application, a computer program product includes instructions that, when the program is executed by one or more processors, cause the one or more processors to perform the method according to the first and / or second aspects of this application.

[0078] According to a sixth aspect of this application, a non-transitory storage medium (or storage medium) is provided, carrying instructions for performing program code according to any one of the methods of the first and / or second aspects of this application.

[0079] The specific nature of at least one embodiment in the exemplary embodiments, as well as other objects, advantages, features, and uses of the at least one embodiment in the exemplary embodiments, will become even more apparent from the following description of the examples taken in conjunction with the accompanying drawings. Attached Figure Description

[0080] Reference will now be made to the accompanying drawings, which illustrate exemplary embodiments of this application, and wherein:

[0081] Figure 1 A schematic block diagram illustrating the steps of a method 100 for encoding a video sequence associated with alpha channel information and intended for display on a transparent screen into a bitstream, according to an exemplary embodiment;

[0082] Figure 2 A schematic block diagram illustrating the steps of a method 200 for decoding a video sequence associated with alpha channel information from a bitstream and intended for display on a transparent screen, according to an exemplary embodiment; and

[0083] Figure 3 A schematic block diagram illustrating an example of a system in which various aspects and exemplary embodiments are implemented is shown.

[0084] Similar or identical elements are indicated by the same reference numerals. Detailed Implementation

[0085] At least one of the exemplary embodiments will be described more fully below with reference to the accompanying drawings, which depict examples of at least one of the exemplary embodiments. However, the exemplary embodiments may be implemented in many alternative forms and should not be construed as limited to the examples set forth herein. Thus, it should be understood that the exemplary embodiments are not intended to be limited to the specific forms disclosed. Rather, this application is intended to cover all modifications, equivalents, and alternatives falling within the spirit and scope of this application.

[0086] Generally, the alpha channel indicator is signaled as part of the alpha channel information associated with the video sequence. The alpha channel indicator indicates what type of post-decoding processing the alpha channel information is intended to be used for.

[0087] Then, the alpha mixing is applied to the decoded video sequence only if the alpha channel indicator indicates that the alpha channel information is intended for alpha mixing.

[0088] Different post-decoding processes can be applied to the decoded video sequence to recreate the intended transparency level for use with transparent screens.

[0089] The signaling of the alpha channel indicator indicates that the post-decoding processing and alpha mixing related to the transparent screen are different, and therefore the intent should be distinguished (even if some of the required parameter semantics will be the same).

[0090] The signaling of the alpha channel indicator avoids processing of auxiliary images for interpretation and application to the upstream equipment (source) of the TV, rather than processing dedicated to the transparent screen, which may vary especially depending on the underlying technology used (OLED vs. LCD).

[0091] The signaling of the alpha channel indicator improves upon known alpha channel signaling in the prior art because it allows decoupling of the assumed alpha hybrid decoding post-processing from other types of decoding post-processing.

[0092] Furthermore, signaling the alpha channel indicator resolves the ambiguity of signaling alpha channel sequences when no specific purpose is specified (as seen in several video coding standards), enabling interoperability.

[0093] Finally, the signaling of the alpha channel indicator improves compression efficiency by only signaling relevant parameters associated with the post-decoding process. For example, parameters in the alpha channel signaling that are only associated with alpha mixing are syntactically excluded from other post-decoding processes, which allows for the preservation of bit information for transmission and avoids the need to specify pseudo-values ​​when alpha mixing is a post-decoding process.

[0094] The following exemplary embodiments are described to illustrate how to modify the VSEI syntax and semantics of the Alpha Channel SEI message in order to implement this disclosure.

[0095] This disclosure is not limited to the current version of the VSEI syntax, but extends to any editable variants of the current VSEI syntax.

[0096] In a first exemplary embodiment, the alpha channel indicator is an additional syntax element included in the current alpha channel information SEI message (ACI SEI message) of the VSEI.

[0097] Therefore, adding an additional syntax element to the ACI SEI message separates roles: the additional syntax element is designed to canonically influence the syntax of the ACI SEI message, while the current syntax element alpha_channel_use_idc is used as a sub-registration scope.

[0098] In a first variant of the first embodiment, the additional syntax element is the syntax element alpha_channel_idc included in the current ACI SEI message of VSEI, which is:

[0099] - equals 0 to indicate that the ACI SEI message is intended for Alpha Hybrid purposes;

[0100] - equals 1 to indicate that the ACI SEI message is intended for the transparency of pixels on a transparent screen;

[0101] - equals 2 to indicate that the ACI SEI message is intended for an unspecified purpose.

[0102] Table 1 provides an example of the syntax of the modified Alpha Channel Information SEI message as defined in VSEI for the first variant according to the first exemplary embodiment.

[0103]

[0104] Table 1

[0105] In a variant of the first variant, when equal to a specific value (=0 or =1), the syntax element alpha_channel_idc further indicates that parameters of the ACI SEI message not involved in the pixel transparency adjustment are not written to or read from the bitstream.

[0106] Table 2 provides an example of the syntax for the modified Alpha Channel Information (SEI) message according to the first variant. In this example, when the syntax element alpha_channel_idc is not equal to 0, i.e., when the post-decoding processing is not alpha mixing, the syntax elements alpha_transparent_value, alpha_opaque_value, alpha_channel_incr_flag, alpha_channel_clip_flag, and alpha_channel_clip_type_flag are discarded because these syntax elements are only related to alpha mixing.

[0107]

[0108] Table 2

[0109] In a second variant of the first exemplary embodiment, the additional syntax element is the syntax element alpha_channel_id included in the current ACI SEI message of VSEI, which is a number belonging to a range of values ​​indicating alternative post-decoding processes or cascades of alternative post-decoding processes intended to be applied to the video sequence.

[0110] For example, when more than one ACI SEI message exists with the same value for the syntax element alpha_channel_id, the content of these ACI SEI messages should be identical. When ACI SEI messages exist with more than one value for the syntax element alpha_channel_id, this can indicate that the different values ​​of alpha_channel_id indicate alternative decoding processes provided for different purposes, or that the concatenation of decoding processes can be applied sequentially. The value of the syntax element alpha_channel_id can be between 0 and 2. 32 The range of -2 includes 0 and 2. 32 -2.

[0111] In the variants, values ​​from 0 to 255 (inclusive) and values ​​from 512 to 2... 31 -1 (including 512 and 2) 31 The value of the syntax element alpha_channel_id (-1) can be used as determined by the external application. The decoder encounters syntax elements whose alpha_channel_id values ​​are in the range of 256 to 511 (inclusive).

[0112] Table 3 provides an example of the syntax of the Alpha Channel Information SEI message as defined in VSEI and modified according to the second variant of the first exemplary embodiment.

[0113]

[0114] Table 3

[0115] In a variant of the second variant of the first exemplary embodiment, the additional syntax element alpha_channel_id further indicates how to cascade post-decoding processing intended to be applied to the decoded video image.

[0116] For example, the value of the syntax element alpha_channel_id, ranging from 0 to 255 (inclusive), indicates the alpha channel information used for alpha mixing purposes. In variants, different alpha mixing operations can be designed to be performed in ascending order of the alpha_channel_id values, regardless of the q-layer order.

[0117] The values ​​of the syntax element alpha_channel_id, ranging from 256 to 511 (inclusive), indicate the alpha channel information used for transparency display purposes. Furthermore, the transparency adjustment operations for different pixels are intended to be performed in ascending order of alpha_channel_id values, regardless of layer order.

[0118] In the variant, different post-decoding processing, alpha blending, and pixel transparency adjustment operations are designed to be performed in ascending order of the alpha_channel_id values ​​of the syntax element, regardless of the layer order and the nature of the post-decoding processing.

[0119] In another variant of the second variant, when equal to a specific value, the syntax element alpha_channel_id further indicates that parameters of the ACI SEI message not involved in the pixel transparency adjustment are not written to or read from the bitstream.

[0120] Table 4 provides an example of the syntax for the modified Alpha Channel Information (SEI) message according to the second variant. In this example, when the syntax element alpha_channel_id is greater than or equal to a specific value (256 in this case), i.e., when the post-decoding processing is not alpha mixing, the syntax elements alpha_transparent_value, alpha_opaque_value, alpha_channel_incr_flag, alpha_channel_clip_flag, and alpha_channel_clip_type_flag are discarded because these syntax elements are only related to alpha mixing.

[0121]

[0122] Table 4

[0123] In the second exemplary embodiment, the alpha channel indicator is a specific value of the existing syntax element alpha_channel_use_idc of the VSEI ACI SEI message.

[0124] The specific value of the syntax element alpha_channel_use_idc indicates a specific post-decoding process for transparent display purposes; that is, the specific value indicates that the alpha channel information is intended for adjusting the transparency of pixels on a transparent screen.

[0125] For example, the syntax element alpha_channel_use_idc

[0126] - Equals 0 to indicate that, for the purpose of alpha mixing, after being output from the decoding process, during the display process, the decoded sample of the associated main image should be multiplied by the interpreted sample value of the auxiliary encoded image;

[0127] - Equals 1 to indicate that, for the purpose of alpha mixing, after being output from the decoding process, during the display process, the decoded sample of the associated main image should not be multiplied by the interpreted sample value of the auxiliary encoded image;

[0128] - equals 2 to indicate the use of an unspecified auxiliary image; and

[0129] -According to this disclosure, 3 indicates that, for the purpose of transparent display, after being output from the decoding process, during the display process, the transparency of the pixels of the transparent screen is adjusted using the interpretation sample values ​​of the image of the auxiliary encoded image.

[0130] In a third exemplary embodiment, the alpha channel indicator is a specific type of auxiliary image that is signaled in the VSEI SCI SEI message.

[0131] While it's still possible to signal an alpha channel indicator, as part of the ACI SEI message, indicating what type of post-decoding post-processing the alpha channel is intended for, it's beneficial to readily identify whether the alpha channel information is intended for alpha blending or for transparent display. Easy identification of the purpose means a unique location expressing that purpose, i.e., a unique parameter. One way to achieve this is to signal the alpha channel indicator as a specific type of auxiliary image within the VSEI's SDI SEI message.

[0132] In VSEI (Section 8.19.2), a table is defined for mapping the type of auxiliary image to the values ​​of the so-called array sdi_aux_id[i] of the i-th layer. A value sdi_aux_id[i] equal to 0 indicates that the i-th layer in the current CVS does not contain an auxiliary image. A value sdi_aux_id[i] equal to 1 indicates that the auxiliary image in the i-th layer corresponds to alpha channel information. A value sdi_aux_id[i] equal to 2 indicates that the auxiliary image in the i-th layer corresponds to depth information.

[0133] According to the third exemplary embodiment, the value sdi_aux_id[i] equal to 3 indicates that the auxiliary image in layer i is intended for transparent display purposes. In this case, layer i also contains associated ACI SEI messages, but these are generated for the purpose of displaying decoded video on a transparent display.

[0134] Table 5 provides an example of a VSEI table as modified by the third exemplary embodiment.

[0135]

[0136] Table 5

[0137] Figure 1 A schematic block diagram illustrates the steps of a method 100 for encoding a video sequence associated with alpha channel information and intended for display on a transparent screen into a bitstream, according to an exemplary embodiment.

[0138] In step 110, an alpha channel indicator is written into the bitstream. The alpha channel indicator indicates which type of post-processing of the video sequence the alpha channel information associated with the video sequence is intended to be used for.

[0139] When the goal is to display a video sequence on the transparent screen, one type of post-processing of the video sequence can be adjusting the transparency of the pixels on the transparent screen.

[0140] Another type of post-processing for video sequences can be alpha mixing.

[0141] If the alpha channel indicator indicates that the alpha channel information is intended for the transparency adjustment of pixels on a transparent screen, then steps 120 through 140 follow step 110.

[0142] In step 120, the video sequence is encoded as data into a bitstream.

[0143] In step 130, a transparency information mask associated with the video sequence is determined.

[0144] In step 140, the alpha channel information representing the transparency information mask is encoded as coded data into the bitstream.

[0145] In an exemplary embodiment of method 100, the alpha channel information associated with an image of the video sequence is an alpha channel image.

[0146] In one exemplary embodiment, an Alpha Channel SEI (Supplemental Enhancement Information) message is also written into the bitstream. The content of the SEI message describes post-decoding processing intended for use with the decoded video image. Specifically, the Alpha Channel SEI message includes information representing an Alpha Channel indicator.

[0147] Any conventional video encoder can be used to encode video sequences and associated alpha channel images, as well as possible SEI messages.

[0148] For example, VVC can be used.

[0149] Figure 2 A schematic block diagram illustrates the steps of a method 200 for decoding a video sequence associated with alpha channel information from a bitstream and intended for display on a transparent screen, according to an exemplary embodiment.

[0150] In step 210, the SEI parser detects and extracts the alpha channel SEI message from the bitstream. The alpha channel SEI message transmits the alpha channel indicator.

[0151] In step 220, the Alpha Channel Indicator is read from the Alpha Channel SEI message.

[0152] If the alpha channel indicator indicates that the alpha channel information is intended for adjusting the transparency of pixels on a transparent screen, then steps 230 through 250 follow step 220.

[0153] In step 230, encoded data is decoded from the bitstream in the decoded video sequence.

[0154] In step 240, data encoded from the bitstream is decoded in the decoded alpha channel video sequence that transmits alpha channel information.

[0155] In one exemplary embodiment, the decoded video sequence may correspond to a primary (main) video sequence, and the decoded alpha channel video sequence may correspond to an auxiliary video sequence associated with the primary video sequence.

[0156] In step 250, a transparency information mask is determined based on the transparency information associated with the decoded alpha channel video sequence.

[0157] In step 260, the transparency of the pixels of the transparent screen is adjusted based on the transparency information mask.

[0158] For example, a transparency information mask contains an array of adjustment coefficients associated with samples of the decoded video sequence to adjust the perceived transparency of pixels on a transparent screen. In a simple case, pixel transparency adjustment might resemble an alpha mixing operation, where the background video sequence would be a black video sequence for an OLED screen or a white video sequence for an LCD screen, to adjust pixel transparency. By doing so, low coefficients (say, 0) in the transparency information mask would cause the associated samples in the decoded video sequence to be canceled, leaving either samples from the black video sequence (i.e., black samples (values ​​0 or close to 0)) or samples from the white video sequence (i.e., white samples (values ​​255 or close to 255 for 8-bit depth)), which in turn makes the pixel at that location completely transparent on the transparent screen.

[0159] Any conventional video decoder can be used to decode video sequences and associated alpha channel images, as well as possible SEI messages.

[0160] For example, VVC can be used.

[0161] In one exemplary embodiment, method 200 further includes determining (step 270) whether a dimming adjustment of the dimming backplane of the transparent screen is requested, and if a dimming adjustment is requested, adjusting (step 260) the transparency of the pixels of the transparent screen by adjusting the dimming backplane.

[0162] In one exemplary embodiment, method 200 further includes acquiring (step 280) ambient lighting condition data and adjusting (step 260) the transparency of pixels of the transparent screen based on the transparency information mask and the ambient lighting conditions.

[0163] In one exemplary embodiment of method 100 and method 200, the alpha channel indicator is a specific syntax element alpha_channel_idc or alpha_channel_id as described above.

[0164] Figure 3 A schematic block diagram illustrating an example of a system 300 in which various aspects and exemplary embodiments are implemented is shown.

[0165] System 300 can be embedded as one or more devices, including the various components described below. In various exemplary embodiments, system 300 can be configured to implement one or more aspects described in this application.

[0166] Examples of equipment that may constitute all or part of system 300 include personal computers, laptop computers, smartphones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, connected vehicles and their associated processing systems, head-mounted display devices (HMDs, glasses), projectors, "caves" (systems including multiple displays), servers, video encoders, video decoders, post-processors that process the output from the video decoder, pre-processors that provide input to the video encoder, web servers, video servers (e.g., broadcast servers, video-on-demand servers, or network servers), still or video cameras, encoding or decoding chips, or any other communication devices. The elements of system 300 may be implemented individually or in combination in a single integrated circuit (IC), multiple ICs, and / or discrete components. For example, in at least one exemplary embodiment, the processing and encoder / decoder elements of system 300 may be distributed across multiple ICs and / or discrete components. In various exemplary embodiments, system 300 may be communicatively coupled to other similar systems or other electronic devices via, for example, a communication bus or through dedicated input and / or output ports.

[0167] System 300 may include at least one processor 310 configured to execute instructions loaded therein for implementing various aspects, such as those described in this application. Processor 310 may include embedded memory, input / output interfaces, and various other circuitry known in the art. System 300 may include at least one memory 320 (e.g., a volatile memory device and / or a non-volatile memory device). System 300 may include a storage device 340, which may include non-volatile memory and / or volatile memory, including but not limited to electrically erasable programmable read-only memory (EEPROM), read-only memory (ROM), programmable read-only memory (PROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, disk drives, and / or optical disk drives. As a non-limiting example, storage device 340 may include internal storage devices, attached storage devices, and / or network-accessible storage devices.

[0168] System 300 may include an encoder / decoder module 330 configured, for example, to process data to provide encoded / decoded video image data, and the encoder / decoder module 330 may include its own processor and memory. The encoder / decoder module 330 may represent one or more modules that can be included in a device to perform encoding and / or decoding functions. As is known, a device may include one or both of the encoding and decoding modules. Furthermore, the encoder / decoder module 330 may be implemented as a separate element of system 300, or may be integrated into processor 310 as a combination of hardware and software known to those skilled in the art.

[0169] Program code to be loaded onto processor 310 or encoder / decoder 330 to execute the various aspects described in this application may be stored in storage device 340 and subsequently loaded onto memory 320 for execution by processor 310. According to various exemplary embodiments, during the execution of the processes described in this application, one or more of processor 310, memory 320, storage device 340, and encoder / decoder module 330 may store one or more of various items. Such stored items may include, but are not limited to, video image data, information data for encoding / decoding video image data, bitstreams, matrices, variables, and intermediate or final results of equations, formulas, operations, and arithmetic logic processing.

[0170] In several exemplary embodiments, the memory within the processor 310 and / or encoder / decoder module 330 may be used to store instructions and provide working memory for processes that can be performed during encoding or decoding.

[0171] However, in other exemplary embodiments, external memory (e.g., the processing device may be processor 310 or encoder / decoder module 330) is used for one or more of these functions. External memory may be memory 320 and / or storage device 340, such as volatile memory and / or non-volatile flash memory. In several exemplary embodiments, external non-volatile flash memory is used to store the television's operating system. In at least one exemplary embodiment, fast external volatile memory such as RAM may be used as working memory for video encoding and decoding operations, for example, for MPEG-2 Part 2 (also known as ITU-T Recommendation H.262 and ISO / IEC 13818-2, also known as MPEG-2 video), AVC, HEVC, EVC, VVC, AVI, etc.

[0172] As indicated in box 390, input to the components of system 300 can be provided through various input devices. Such input devices include, but are not limited to, (i) an RF section capable of receiving RF signals transmitted over the air, for example by a broadcast device, (ii) a composite input terminal, (iii) a USB input terminal, (iv) an HDMI input terminal, and (v) a bus such as CAN (Controller Area Network), CAN FD (Controller Area Network Flexible Data Rate), FlexRay (ISO 17458), or Ethernet (ISO / IEC 802-3) bus when this disclosure is implemented in the automotive field.

[0173] In various exemplary embodiments, the input device of block 390 has associated corresponding input processing elements, as known in the art. For example, the RF section may be associated with elements necessary for: (i) selecting a desired frequency (also known as selecting a signal, or limiting the signal band to a band), (ii) down-converting the selected signal, (iii) further limiting the band to a narrower band to select (e.g.,) a signal band that may be referred to as a channel in some exemplary embodiments, (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select a desired data packet stream. The RF section of various exemplary embodiments may include one or more elements performing these functions, such as a frequency selector, signal selector, band limiter, channel selector, filter, downconverter, demodulator, error corrector, and demultiplexer. The RF section may include tuners performing various functions among these functions, including, for example, down-converting a received signal to a lower frequency (e.g., intermediate frequency or near-baseband frequency) or baseband.

[0174] In one set-top box embodiment, the RF section and its associated input processing elements can receive RF signals transmitted over a wired (e.g., cable) medium. The RF section can then perform frequency selection by filtering, down-converting, and re-filtering to the desired frequency band.

[0175] Various exemplary embodiments may rearrange the order of the above (and other) elements, remove some of these elements, and / or add other elements that perform similar or different functions.

[0176] Adding components may include inserting components between existing components, such as, for example, inserting amplifiers and analog-to-digital converters. In various exemplary embodiments, the RF portion may include an antenna.

[0177] Furthermore, the USB and / or HDMI terminals may include corresponding interface processors for connecting system 300 to other electronic devices via USB and / or HDMI connections. It should be understood that various aspects of input processing (e.g., Reed-Solomon error correction) may be implemented, for example, within a separate input processing IC or within processor 310, when necessary. Similarly, various aspects of USB or HDMI interface processing may be implemented within a separate interface IC or within processor 310, when necessary. The demodulated, error-corrected, and demultiplexed streams may be provided to various processing elements, including, for example, processor 310 and encoder / decoder 330, which operate in conjunction with memory and storage elements to process the data streams for presentation on an output device when necessary.

[0178] Various components of system 300 can be provided within an integrated housing. Within the integrated housing, suitable connection arrangements 390, such as internal buses (including I2C buses), wiring, and printed circuit boards known in the art, can be used to interconnect various components and transfer data between them.

[0179] System 300 may include a communication interface 350 that enables communication with other devices via a communication channel 351. The communication interface 350 may include, but is not limited to, a transceiver configured to send and receive data on the communication channel 351. The communication interface 350 may include, but is not limited to, a modem or network interface card (NIC), and the communication channel 351 may be implemented, for example, within a wired and / or wireless medium.

[0180] In various exemplary embodiments, a Wi-Fi network such as IEEE 802.11 can be used to stream data to system 300. The Wi-Fi signals in these exemplary embodiments can be received via a communication channel 351 and a communication interface 350 suitable for Wi-Fi communication. The communication channel 351 in these exemplary embodiments can typically be connected to an access point or router that provides access to external networks, including the Internet, to allow streaming applications and other over-the-top cloud communications.

[0181] Other exemplary embodiments may use a set-top box to provide streaming data to system 300, the set-top box delivering the data via an HDMI connection in input block 390.

[0182] Other exemplary embodiments may use the RF connection of input block 390 to provide streaming data to system 300.

[0183] Streamed data can be used as signaling information by system 300. Signaling information may include information such as the number of pixels in a bitstream and / or a video image, any encoding / decoding settings, alignment status, alignment reference data, overlap status, resampled data, interpolation data, and / or calibration data.

[0184] It should be recognized that signaling can be implemented in a variety of ways. For example, in various exemplary embodiments, one or more syntax elements, flags, etc., can be used to signal information to the corresponding decoder.

[0185] System 300 can provide output signals to various output devices, including a display 361, a speaker 371, and other peripheral devices 381. In various examples of exemplary embodiments, other peripheral devices 381 may include one or more of a standalone DVR, a disc player, a stereo system, a lighting system, and other devices that provide functionality based on the output of system 300.

[0186] In various exemplary embodiments, control signals may be communicated between system 300 and display 361, speaker 371 or other peripheral devices 381 using signaling such as AV.Link (audio / video link), CEC (consumer electronics control), or other communication protocols that enable device-to-device control with or without user intervention.

[0187] Output devices can be communicatively coupled to system 300 via dedicated connections through the corresponding interfaces 360, 370 and 380.

[0188] Alternatively, the output device can be connected to the system 300 via the communication interface 350 using the communication channel 351. The display 361 and speaker 371 can be integrated into a single unit with other components of the system 300 in electronic devices such as, for example, televisions.

[0189] In various exemplary embodiments, the display interface 360 ​​may include a display driver, such as, for example, a timing controller (TCon) chip.

[0190] For example, if the RF portion of input 390 is part of a separate set-top box, then display 361 and speaker 371 may optionally be separate from one or more other components. In various exemplary embodiments where display 361 and speaker 371 can be external components, output signals may be provided via dedicated output connections, including, for example, an HDMI port, a USB port, or a COMP output.

[0191] exist Figures 1 to 3This document describes various methods, each comprising one or more steps or actions to implement the described method. Unless the correct operation of the method requires a specific order of steps or actions, the order and / or use of specific steps and / or actions may be modified or combined.

[0192] Examples of block diagrams and / or operation flowcharts are described. Each block represents a portion of circuitry, a module, or code, which includes one or more executable instructions for implementing one or more specified logical functions. It should also be noted that in other implementations, the functions (one or more) marked in the blocks may occur out of order. For example, depending on the functions involved, two blocks shown sequentially may actually execute substantially concurrently, or sometimes these blocks may be executed in reverse order.

[0193] The embodiments and aspects described herein may be implemented in, for example, methods or processes, apparatus, computer programs, data streams, bit streams, or signals. Even if discussed only in the context of a single form of implementation (e.g., discussed only as a method), implementations of the discussed features may be implemented in other forms (e.g., apparatus or computer programs).

[0194] The method can be implemented in, for example, a processor, which generally refers to a processing device, including, for example, a computer, microprocessor, integrated circuit, or programmable logic device. Processors also include communication devices.

[0195] Furthermore, the method can be implemented by instructions executed by a processor, and such instructions (and / or data values ​​generated by the implementation) can be stored in a computer-readable storage medium, such as, for example, storage device 340. Figure 3 The computer-readable storage medium may take the form of a computer-readable program product implemented in one or more computer-readable media and having computer-readable program code executable thereon. Considering the inherent ability to store information therein and the inherent ability to retrieve information provided therefrom, the computer-readable storage medium as used herein can be considered a non-transitory storage medium. The computer-readable storage medium may be, for example, but not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any suitable combination of the foregoing. It should be understood that while more specific examples of computer-readable storage media to which this exemplary embodiment may be applied are provided below, they are merely illustrative and not an exhaustive list, as will be readily apparent to those skilled in the art: portable computer floppy disks; hard disks; read-only memory (ROM); erasable programmable read-only memory (EPROM or flash memory); portable optical disc read-only memory (CD-ROM); optical storage devices; magnetic storage devices; or any suitable combination of the foregoing.

[0196] Instructions can form applications that are tangibly implemented on processor-readable media.

[0197] For example, instructions can be found in hardware, firmware, software, or a combination thereof. Instructions can be found, for example, in an operating system, a standalone application, or a combination of both. Therefore, a processor can be characterized as, for example, a device configured to execute a process and a device including a processor-readable medium (such as a storage device) having instructions for executing the process. Additionally, in addition to or instead of instructions, the processor-readable medium can store data values ​​generated by the implementation.

[0198] The device can be implemented, for example, in appropriate hardware, software, and firmware. Examples of such devices include personal computers, laptops, smartphones, tablets, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected home appliances, head-mounted display devices (HMDs, see-through glasses), projectors, "caves" (systems comprising multiple displays), servers, video encoders, video decoders, post-processors that process the output from the video decoder, pre-processors that provide input to the video encoder, web servers, set-top boxes, and any other devices used for processing video images, or other communication devices. It should be clear that the equipment can be mobile and even mounted in mobile vehicles.

[0199] The computer software may be implemented by the processor 310 or by hardware, or by a combination of hardware and software. As a non-limiting example, exemplary embodiments may also be implemented by one or more integrated circuits. The memory 320 may be of any type suitable for the technical environment and may be implemented using any suitable data storage technology, such as optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory, as non-limiting examples. The processor 310 may be of any type suitable for the technical environment and may encompass one or more of microprocessors, general-purpose computers, special-purpose computers, and processors based on multi-core architectures, as non-limiting examples.

[0200] Based on this application, as will be apparent to those skilled in the art, implementations can generate various signals formatted to carry, for example, information that can be stored or transmitted. The information may include, for example, instructions for performing a method or data generated by one of the described implementations. For example, the signal may be formatted to carry a bitstream of the described exemplary embodiments. Such a signal may be formatted as, for example, electromagnetic waves (e.g., using the radio frequency portion of the spectrum) or baseband signals. Formatting may include, for example, encoding the data stream and modulating a carrier wave with the encoded data stream. The information carried by the signal may be, for example, analog or digital information. As is known, signals can be transmitted via various wired or wireless links. The signal may be stored on a processor-readable medium.

[0201] The terminology used herein is for the purpose of describing particular exemplary embodiments only and is not intended to be limiting. As used herein, the singular forms “a,” “an,” and “the” may also be intended to include the plural forms unless the context clearly indicates otherwise. It will be further understood that, when used in this specification, the terms “include / comprise” and / or “including / comprising” may specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. Moreover, when an element is referred to as “responding,” “connected,” or “associated” to another element, it may directly respond to or connect to or associate with the other element, or there may be intermediate elements. Conversely, when an element is referred to as “directly responding” or “directly connected” to or “directly associated” with another element, there are no intermediate elements.

[0202] It should be recognized that, for example, in the cases of “A / B,” “A and / or B,” and “at least one of A and B,” the use of any of the symbols / terms “ / ,” “and / or,” and “at least one” can be intended to cover the selection of only the first listed option (A), or only the second listed option (B), or the selection of both options (A and B). As a further example, in the cases of “A, B, and / or C” and “at least one of A, B, and C,” such wording is intended to cover the selection of only the first listed option (A), or only the second listed option (B), or only the third listed option (C), or only the first and second listed options (A and B), or only the first and third listed options (A and C), or only the second and third listed options (B and C), or the selection of all three options (A, B, and C). As will be apparent to those skilled in the art and related fields, this can be extended to as many items as are listed.

[0203] Various numerical values ​​may be used in this application. Specific values ​​may be used for illustrative purposes and the aspects described are not limited to these specific values.

[0204] It will be understood that while the terms first, second, etc., may be used herein to describe various elements, these elements are not limited by these terms. These terms are used only to distinguish one element from another. For example, without departing from the teachings of this application, a first element may be referred to as a second element, and similarly, a second element may be referred to as a first element. There is no implied order between the first element and the second element.

[0205] The references to “an exemplary embodiment” or “an exemplary embodiment” or “an implementation” or “implementation” and other variations thereof are frequently used to convey that a particular feature, structure, characteristic, etc. (described in connection with the embodiment / implementation) is included in at least one embodiment / implementation. Therefore, the phrases “in an exemplary embodiment” or “in an exemplary embodiment” or “in one implementation” or “in one implementation” appearing throughout this application, as well as any other variations, do not necessarily refer to the same exemplary embodiment.

[0206] Similarly, the references to "according to an exemplary embodiment / example / implementation" or "in an exemplary embodiment / example / implementation" and their variations are frequently used to convey that a particular feature, structure, or characteristic (described in conjunction with an exemplary embodiment / example / implementation) may be included in at least one exemplary embodiment / example / implementation. Therefore, the expressions "according to an exemplary embodiment / example / implementation" or "in an exemplary embodiment / example / implementation" appearing throughout this application do not necessarily refer to the same exemplary embodiment / example / implementation, nor are individual or alternative exemplary embodiments / examples / implementations necessarily mutually exclusive with other exemplary embodiments / examples / implementations.

[0207] The reference numerals appearing in the claims are for illustrative purposes only and do not limit the scope of the claims. Although not explicitly described, these exemplary embodiments / examples and variations may be employed in any combination or sub-combination.

[0208] When a diagram is presented as a flowchart, it should be understood that it also provides a block diagram of the corresponding apparatus. Similarly, when a diagram is presented as a block diagram, it should be understood that it also provides a flowchart of the corresponding method / process.

[0209] While some diagrams include arrows along the communication path to indicate the main direction of communication, it should be understood that communication can occur in the opposite direction to the arrows depicted.

[0210] Various implementations involve decoding. As used herein, "decoding" can encompass all or part of a process performed, for example, on a received video sequence (which may include a received bitstream encoded with one or more video sequences) to produce a final output suitable for display or further processing in a reconstructed video domain. In various exemplary embodiments, such processes include one or more processes typically performed by a decoder. In various exemplary embodiments, such processes may also, or optionally, include processes performed by a decoder of the various embodiments described herein.

[0211] As a further example, in one exemplary embodiment, "decoding" may refer only to dequantization; in another exemplary embodiment, "decoding" may refer to entropy decoding; in yet another exemplary embodiment, "decoding" may refer only to differential decoding; and in yet another exemplary embodiment, "decoding" may refer to a combination of dequantization, entropy decoding, and differential decoding. It will be clear, and believed to be well understood by those skilled in the art, whether the phrase "decoding process" is intended to specifically refer to a subset of operations or generally to a broader decoding process, depending on the context of the specific description.

[0212] Various implementations involve encoding. In a manner similar to the above discussion of "decoding," the term "encoding" as used herein can encompass all or part of a process, for example, performing on an input video sequence to produce a bitstream of output. In various exemplary embodiments, such processes include one or more processes typically performed by an encoder. In various exemplary embodiments, such processes also include, or optionally include, processes performed by an encoder of the various embodiments described herein.

[0213] As a further example, in one exemplary embodiment, "encoding" may refer only to quantization; in another exemplary embodiment, "encoding" may refer only to entropy encoding; in yet another exemplary embodiment, "encoding" may refer only to differential encoding; and in still another exemplary embodiment, "encoding" may refer to a combination of quantization, differential encoding, and entropy encoding. It will be clear, and believed to be well understood, by those skilled in the art, whether the phrase "encoding process" is intended to specifically refer to a subset of operations or generally to a broader encoding process, depending on the context of the particular description.

[0214] Furthermore, this application may refer to "obtaining" various types of information. Obtaining information may include one or more of the following: estimated information, calculated information, predicted information, or information retrieved from memory, processed information, moved information, copied information, erased information, calculated information, determined information, predicted information, or estimated information.

[0215] Furthermore, this application may refer to "receiving" various types of information. Receiving information may include, for example, access information or information received from a communication network, one or more of these.

[0216] Moreover, as used herein, the word "signal" specifically refers to instructing the corresponding decoder to do something. For example, in some exemplary embodiments, the encoder signals specific information, such as encoding parameters or encoded video image data. In this way, in exemplary embodiments, the same parameter can be used on both the encoder and decoder sides. Thus, for example, the encoder can transmit (explicit signaling) a specific parameter to the decoder so that the decoder can use the same specific parameter. Conversely, if the decoder already has the specific parameter as well as other parameters, then signaling can be used without transmission (implicit signaling) to simply allow the decoder to know and select the specific parameter. Bit savings are achieved in various exemplary embodiments by avoiding the transmission of any actual functionality. It should be recognized that signaling can be accomplished in a variety of ways. For example, in various exemplary embodiments, one or more syntax elements, flags, etc., are used to signal information to the corresponding decoder. Although the verb form of the word "signal" has been referred to above, the word "signal" can also be used as a noun herein.

[0217] Several implementations have been described. However, it should be understood that various modifications can be made. For example, elements of different implementations can be combined, supplemented, modified, or removed to produce other implementations. Furthermore, those skilled in the art will understand that other structures and processes can replace the disclosed structures and processes, and the resulting implementations will perform at least substantially the same functions in at least substantially the same manner to achieve at least substantially the same results as the disclosed implementations. Therefore, these and other implementations are contemplated in this application.

Claims

1. A method (100) for encoding a video sequence associated with alpha channel information into a bitstream, wherein the method comprises: - Write an alpha channel indicator into the bitstream (110), the alpha channel indicator indicating which type of post-processing of the video sequence the alpha channel information associated with the video sequence is intended to be used for; - If the alpha channel indicator indicates that the alpha channel information is intended for pixel transparency adjustment of a transparent screen; - Encode the video sequence as coded data (120) into the bitstream; - Determine (130) the transparency information mask associated with the video sequence; as well as - The alpha channel information representing the transparency information mask is encoded as data (140) into the bitstream.

2. The method according to claim 1, wherein the alpha channel indicator is a specific syntax element included in the alpha channel information SEI message, and the specific syntax element includes any one of the following: alpha_channel_idc, alpha_channel_id.

3. The method of claim 1, wherein the alpha channel indicator is a specific value of the existing syntax element alpha_channel_use_idc of the alpha channel information SEI message, the specific value indicating that the alpha channel information is intended for pixel transparency adjustment.

4. The method of claim 1, wherein the alpha channel indicator is a specific type of auxiliary image that is signaled in the SEI message.

5. A method (200) for decoding a video sequence associated with alpha channel information from a bitstream, wherein the method comprises: - Read (220) an alpha channel indicator from the bitstream, the alpha channel indicator indicating which type of post-processing of the video sequence the alpha channel information is intended to be used for; - If the alpha channel indicator indicates that the alpha channel information is intended for adjusting the pixel transparency of a transparent screen; - Decode the data encoded from the bitstream in the decoded video sequence (230); - Data encoded from the bitstream in the decoded alpha channel video sequence that transmits the alpha channel information; - Determine (250) a transparency information mask based on the transparency information associated with the decoded alpha channel video sequence; as well as - Adjust the pixel transparency of the transparent screen (260) based on the transparency information mask, the pixels being derived from sample values ​​of the decoded video sequence.

6. The method of claim 5, wherein the method further comprises determining (270) whether a dimming adjustment of a dimming backplane of a transparent screen is requested, and if a dimming adjustment is requested, adjusting (260) the pixel transparency by adjusting the dimming backplane.

7. The method according to claim 5 or 6, wherein the method further comprises acquiring (280) ambient lighting condition data and adjusting (260) the pixel transparency based on the transparency information mask and the ambient lighting conditions.

8. The method according to any one of claims 5 to 7, wherein the alpha channel indicator is a specific syntax element included in the alpha channel information SEI message, the specific syntax element including any one of the following: alpha_channel_idc, alpha_channel_id.

9. The method of claim 8, wherein the specific syntax element alpha_channel_idc - Equals 0 to indicate that the Alpha Channel Information SEI message is intended for alpha mixing of the decoded video image with the Alpha Channel Information; - Equals 1 to indicate that the alpha channel information SEI message is intended for adjusting the pixel transparency of the (150) transparent screen.

10. The method of claim 9, wherein when equal to a specific value, the specific syntax element alpha_channel_idc further indicates that parameters of the alpha channel information SEI message not involved in the transparency adjustment of the pixel are not written into or read from the bitstream.

11. The method of claim 8, wherein the particular syntax element alpha_channel_id is a number belonging to a range of values ​​indicating an alternative post-decoding process or a cascade of alternative post-decoding processes intended to be applied to the decoded video sequence.

12. The method of claim 11, wherein the particular syntax element alpha_channel_id further indicates how to cascade post-decoding processing intended to be applied to the decoded video image.

13. The method of claim 11 or 12, wherein when equal to a specific value, the specific syntax element alpha_channel_id further indicates that parameters of the alpha channel information SEI message not involved in the transparency adjustment of the pixel are not written into or read from the bitstream.

14. The method according to any one of claims 5 to 7, wherein the alpha channel indicator is a specific value of the existing syntax element alpha_channel_use_idc of the alpha channel information SEI message, the specific value indicating that the alpha channel information is intended for pixel transparency adjustment.

15. The method according to any one of claims 5 to 7, wherein the alpha channel indicator is a specific type of auxiliary image that is signaled in the SEI message.

16. An apparatus for encoding a video sequence associated with alpha channel information into a bitstream, comprising a module for performing the method of any one of claims 1 to 4.

17. An apparatus for decoding a video sequence associated with alpha channel information from a bitstream, comprising a module for performing the method of any one of claims 5 to 15.

18. A computer program product comprising instructions that, when executed by one or more processors, cause the one or more processors to perform the method of any one of claims 1 to 15.

19. A non-transitory storage medium carrying instructions for performing program code for any one of claims 1 to 15.

20. An electronic device, comprising: processor; and Memory for storing instructions that can be executed by the processor; The processor is configured to implement the method of any one of claims 1 to 15.

21. A method for storing a bit stream, characterized in that, Generate a bitstream by performing the method of any one of claims 1 to 4; and store the bitstream.

22. A method for transmitting a bit stream, characterized in that, Generate a bit stream by performing the method of any one of claims 1 to 4; and transmit the bit stream.

23. A computer-readable storage medium storing a computer program / instructions and a bit stream thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the method of any one of claims 1 to 4 to generate the bit stream.

Images