Encoding / decoding video sequences associated with alpha channel information

Abstract

The invention relates to a method and an apparatus for encoding / decoding a video sequence associated with alpha channel information. The alpha channel indicator is signaled as part of the alpha channel information. The alpha channel indicator indicates which type of post-decoding processing the alpha channel information associated with the video sequence is intended for.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to and the benefit of European Patent Application No. 22305949.4 filed with the European Patent Office on June 30, 2022, the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present application generally relates to encoding / decoding video sequences and signaling associated alpha channel information. In particular, but not exclusively, the present application relates to an encoding / decoding method; an apparatus comprising means for performing one of the methods; a bitstream formatted to include encoded data obtained according to the encoding method and an alpha channel indicator; a computer program product comprising instructions that, when the program is executed by one or more processors, cause the one or more processors to perform the encoding / decoding method; and a non-transitory storage medium carrying instructions of a program code for performing the encoding / decoding method. Background Art

[0004] This section is intended to introduce the reader to various aspects of the art that may be related to various aspects of at least one exemplary embodiment of the present application described and / or claimed below. This discussion is believed to be helpful in providing the reader with background information to facilitate a better understanding of various aspects of the present application. Therefore, it should be understood that these statements should be read in this light, and not as admissions of prior art.

[0005] A pixel corresponds to the smallest display unit on the screen, which can be composed of one or more light sources (1 for a monochrome screen and 3 or more for a color screen).

[0006] A video sequence is a temporal series of video frames.

[0007] A video frame (also called a picture or an image) comprises at least one component (also called a channel) determined by a specific image / video format, which specifies all information related to sample values ​​and all information that can be used by a display unit and / or any other device to display and / or decode video image data related to the video frame in order to generate pixel values.

[0008] A video frame comprises at least one component which is typically represented in the shape of a 2D array of samples.

[0009] A monochrome video frame includes a single component, whereas a color video frame (also called a texture video frame) may include three components.

[0010] For example, a color video frame may include a luminance (or luminance) component and two chrominance components when the image / video format is the well-known (Y,Cb,Cr) format, or may include three color components (one for red, one for green, and one for blue) when the image / video format is the well-known (R,G,B) format.

[0011] Each component of a video frame may include a number of samples relative to the number of pixels of the screen on which the video image is intended to be displayed. For example, the number of samples contained in a component may be the same as the number of pixels of the screen on which the video frame is intended to be displayed, or a multiple (or fraction) thereof.

[0012] The number of samples contained in a component may be a multiple (or fraction) of the number of samples contained in another component of the same video frame.

[0013] For example, in case the video format includes a luma component and two chroma components (such as a (Y, Cb, Cr) format), the chroma components may contain half the number of samples in width and / or height relative to the luma components, depending on the color format considered.

[0014] A sample is the smallest visual information unit of a component that makes up a video frame. A sample value can be, for example, a luminance or chrominance value or a color value of a red, green or blue component in (R, G, B) format.

[0015] For a monochrome video image, the pixel value of the screen can be represented by one sample, and for a color video frame, the pixel value of the screen can be represented by multiple co-located samples. The co-located sample associated with a pixel refers to a sample corresponding to the position of the pixel in the screen.

[0016] A video frame is usually considered as a set of pixel values, with each pixel represented by at least one sample.

[0017] In computer graphics, alpha compositing or alpha blending is the process of combining an image with a background to create the appearance of partial or full transparency.

[0018] It is often useful to render image elements (pixels) in separate passes or layers, and then combine the resulting 2D pictures into a single final picture, called a composited picture. Compositing is used extensively in film when combining computer-rendered picture elements with live action footage.

[0019] Alpha blending is also used in 2D computer graphics to place rasterized foreground elements on top of the background.

[0020] Alpha blending uses an alpha channel. If an alpha channel is used in an image, two common representations are available: direct (non-contextual) alpha and pre-multiplied (contextual) alpha.

[0021] With direct alpha, the RGB components of the image represent the color of the object or pixel, regardless of its opacity.

[0022] For premultiplied alpha, the RGB components represent the emission of an object or pixel, while the alpha represents the occlusion.

[0023] The alpha channel can also be used to adapt the transparency level of a see-through (transparent) display (screen).

[0024] Historically, the first version of see-through displays used LCD display technology in the mid-2010s. However, LCD transparent screens only filter the incident light from behind the TV to illuminate the pixels, which effectively means that these screens cannot work in a dark room. This is one of the reasons why OLED-based see-through displays have quickly become a more promising approach, because OLED screens have self-luminous pixels. That is, each pixel of an OLED screen contains its light source. In short, an OLED-based see-through display is made from an ordinary OLED screen, in which the manufacturer punches a large number of holes so that light can pass through the screen from behind. It should be noted that since OLED screens only emit light in one direction, viewers behind the screen will not be able to see what is displayed on the screen.

[0025] For both LCD and OLED based transparent screens, the fact that the viewer can perceive the scene or part of the scene behind the TV is determined by the value of the pixel. There are two extreme cases. The first case is when the user can not see what is behind the screen at all, that is, completely opaque. The second case is when the user can see what is behind the TV completely and no other pictures are displayed on the TV, completely transparent. Between these two cases, there is a range of continuous transparency levels, in which the user sees the picture displayed on the screen superimposed on the top of the scene behind the TV. In all these cases, for the best perspective display technology such as current OLED displays, completely opaque, completely transparent and medium transparency can be used for the entire screen or positioned to the granularity of pixels.

[0026] Each effect described above (i.e., fully opaque, fully transparent, and moderately transparent) is achieved in a different way, depending on the LCD or OLED technology. For LCD, a pixel is black when as much light as possible from the light source is blocked. Therefore, a dark pixel will not let light from behind the TV pass through and will appear opaque to the viewer. In contrast, for OLED, a pixel is white when all light passes through all sub-color pixels to form a white beam. Therefore, light from behind can also pass through the pixel through the hole and thus allow the viewer to see the object behind, thereby creating a transparent effect.

[0027] Both LCD and OLED screens can display a grayscale gradient from left (white) to right (black). On a transparent OLED, black pixels (light off) let light from behind pass through the screen. This actually means that in this simple version of a transparent OLED screen, black cannot be displayed on the screen. Conversely, if the scene on the back of the TV is not white, it is impossible to display white pixels on a transparent LCD screen.

[0028] There are OLED TV models in which an additional layer is placed just behind the screen. This additional layer is responsible for dimming the light coming from behind. The purpose of this dimming layer is to improve OLED-based transparent screens so that by activating this layer dark pixels can also be made opaque. Another use of this layer is to switch between traditional opaque TV and transparent TV modes. There may actually be cases where the content is not made for transparent screens and results in a bad user experience, in which case being able to switch to traditional opaque TV mode would be advantageous for the user. As of today, it looks like it will be possible to activate this dimming layer with different dimming intensity levels, but only for the entire screen at a time. That is, the dimming layer does not yet seem to be able to be targeted to certain areas / pixels of the screen. However, this feature can be expected in future commercial versions.

[0029] Perceived transparency depends on:

[0030] -Intensity level of the sample, which results in brighter pixels and therefore more opaque for OLEDs (and less so for LCDs)

[0031] opposite);

[0032] -The amount of light coming from behind the TV, since objects behind it will be visible, the more light comes from behind the TV.

[0033] The brighter the image, the more transparent the perceived image is;

[0034] - Ability to block light from behind the TV via a dimming backplane.

[0035] Using these three methods (and likely more in the future), the user-perceived level of pixel transparency can be adjusted to match the content creator's intent.

[0036] One way to adjust the picture samples is by performing an alpha blending operation, treating the displayed picture as superimposed on a black (for OLED screens) background. This way, parts of the picture that are desired to be more transparent will have a low alpha coefficient, which increases the black level in the composite picture by a factor equal to (1-α). However, this does not take into account the increase in brightness that may be needed in the opposite case (where the area should be opaque), despite possible natural light coming from behind the TV. If an area of ​​the picture has a high coefficient value (say 1), alpha blending will simply keep the samples the same (value multiplied by alpha), however, due to e.g. strong ambient lighting, it may be necessary to increase the values ​​of those luminance samples by a factor larger than just alpha (value 1). Furthermore, if there is a backplane to dim the light behind the TV, the receiver may not even need to adjust some sample values ​​to make the area opaque. It is sufficient to activate the dimming plane, if possible positioning it to this area in the picture.

[0037] Therefore, transmitting the alpha channel of a transparent screen is relevant, but simply applying alpha blending is not sufficient to recreate the perceived transparency effect.

[0038] Alpha channels and alpha blending have been implemented in video coding standards based on auxiliary pictures associated with a main picture and SEI (Supplemental Enhancement Information) messages.

[0039] Basically, in the alpha blending process, samples of the auxiliary image are interpreted as being indicative of the degree of opacity (or equivalently, the degree of transparency) associated with corresponding luma samples of the primary image, thanks to the information of the SEI message.

[0040] For example, the current draft of the VSEI (Generic Supplementary Enhancement Information Message for Coded Video Bitstream) standard (JVET-Y2006, Additional SEI Messages for VSEI (Draft 6), ITU-T SG 16 WP 3 and ISO / IEC JTC 1 / SC 29, 25th Meeting JVET, January 12-21, 2022) defines an alpha channel information SEI message (Section 8.23) and a scalability dimension information (SDI) SEI message (Section 8.19). The SDI SEI message provides scalability dimension information for each layer, such as 1) a view ID for each layer when multiple views are possible; and 2) an auxiliary ID for each layer when there may be auxiliary information (such as depth or alpha) carried by one or more layers. The alpha channel information (ACI) SEI message provides information about alpha channel sample values ​​and post-decoding processing applied to decoded alpha planes encoded in auxiliary images and one or more associated decoded primary images.

[0041] The ACI SEI signaling in the VSEI specification does not cover the transparency level intent for transparent screen usage purposes (eg, selected on the production side by a movie director / producer), but rather mainly assumes conventional alpha blending operation.

[0042] The ACI SEI signaling in the VSEI specification does not allow decoding post-processing related to transparent screens, and since alpha blending is different, the intent should be distinguished (even though some required parameter semantics will be the same).

[0043] Furthermore, in cases where the syntax and semantics of auxiliary images from current video coding standards would be hijacked for transparent screen purposes, VSEI-like signaling does not avoid interpreting and applying processing of auxiliary images to upstream equipment (the source) of the TV, other than processing specific to transparent screens, which may differ, inter alia, depending on the underlying technology used (OLED vs. LCD).

[0044] If both the alpha channel and the transparency intent should be transmitted simultaneously, VSEI signaling does not provide any mechanism to determine how this should be done (processing order, identification of the appropriate SEI for the proper processing, ...).

[0045] As can be understood from the prior art, the concept of picture synthesis assumes two pictures, such as a foreground F and a background B picture, where the foreground picture is superimposed on top of the background picture. In order to calculate the resulting pixel based on the synthesis of corresponding pixels from the two pictures, a mathematical formula has been established, namely O=F×α+B×(1-α). It is therefore necessary to know the coefficient α for each pixel position. According to this coefficient name, the alpha plane (also called alpha channel) is conceptualized and the related picture synthesis alpha blending is conceptualized. Therefore, there is a coupling through the design of alpha blending and alpha channel. This coupling can also be observed in video transmission specifications such as video coding and video and picture transmission formats, where the alpha channel is always used for the purpose of alpha blending.

[0046] However, there are other usage scenarios, such as displaying pictures and videos on a transparent screen, where the alpha channel can be transmitted together with the visual content so that the receiving device adapts the pixel transparency strength of the displayed content based on the signaled information as previously presented.

[0047] There is a need to improve the alpha channel signaling known in the prior art to decouple the assumed alpha blending post-decoding process from other types of post-decoding processes. In addition, another need is to resolve the ambiguity of signaling the alpha channel sequence when the usage is not specified (as seen in several video coding standards) so that interoperability can be achieved. Finally, another need is to improve compression efficiency by signaling only the relevant parameters associated with the post-decoding process, for example, the parameters in the alpha channel signaling related only to alpha blending are syntactically excluded from other post-decoding processes, so that bit information can be saved for transmission, and the need to specify dummy values ​​when alpha blending is a post-decoding process is avoided.

[0048] At least one exemplary embodiment of the present application has been designed in view of the foregoing. Summary of the invention

[0049] The following section presents a simplified summary of at least one exemplary embodiment in order to provide a basic understanding of some aspects of the present application. This summary is not an exhaustive overview of the exemplary embodiments. It is not intended to identify the key or important elements of the exemplary embodiments. The following summary only presents some aspects of at least one exemplary embodiment in a simplified form as a prelude to a more detailed description provided elsewhere in the document.

[0050] According to a first aspect of the present application, there is provided a method for encoding a video sequence associated with alpha channel information into a bitstream, wherein the method comprises:

[0051] - writing an alpha channel indicator into the bitstream, the alpha channel indicator indicating for which type of post-processing of the video sequence the alpha channel information associated with the video sequence is intended;

[0052] - if the alpha channel indicator indicates that the alpha channel information is intended for pixel transparency adjustment for a transparent screen;

[0053] - encoding a video sequence as coded data into a bitstream;

[0054] - determining a transparency information mask associated with the video sequence; and

[0055] - encoding alpha channel information representing said transparency information mask into a bitstream as encoded data.

[0056] According to a second aspect of the present application, there is provided a method for decoding a video sequence associated with alpha channel information from a bitstream, wherein the method comprises:

[0057] - reading an alpha channel indicator from the bitstream, the alpha channel indicator indicating for which type of post-processing of the video sequence the alpha channel information is intended;

[0058] - if the alpha channel indicator indicates that the alpha channel information is intended to be used to adjust pixel transparency for a transparent screen;

[0059] - decoding the encoded data from the bitstream in the decoded video sequence;

[0060] - decoding data encoded in a decoded alpha channel video sequence conveying alpha channel information from a bitstream;

[0061] - determining a transparency information mask based on transparency information associated with a decoded alpha channel video sequence; and

[0062] -Adjusting the transparency of pixels of the transparent screen based on the transparency information mask, the pixels being derived from sample values ​​of the decoded video sequence.

[0063] In an exemplary embodiment, the decoding method further includes determining whether a dimming adjustment of a dimming backplane of the transparent screen is requested, and if the dimming adjustment is requested, adjusting the pixel transparency by adjusting the dimming backplane.

[0064] In an exemplary embodiment, the decoding method further obtains ambient lighting condition data and adjusts pixel transparency based on the transparency information mask and the ambient lighting condition.

[0065] In an exemplary embodiment, the alpha channel indicator is a specific syntax element (alpha_channel_idc, alpha_channel_id) included in the alpha channel information SEI message.

[0066] In an exemplary embodiment, a specific syntax element (alpha_channel_idc)

[0067] - equal to 0 to indicate that the alpha channel information SEI message is intended for alpha blending of decoded video pictures with alpha channel information;

[0068] - equal to 1 to indicate that the alpha channel information SEI message is intended for adjusting pixel transparency.

[0069] In an exemplary embodiment, when equal to a specific value, the specific syntax element (alpha_channel_idc) further indicates that parameters of the alpha channel information SEI message not involved in transparency adjustment of the pixel are not written to or read from the bitstream.

[0070] In one exemplary embodiment, the particular syntax element (alpha_channel_id) is a number belonging to a range of values ​​indicating an alternative post-decoding process or a cascade of alternative post-decoding processes intended to be applied to the decoded video sequence.

[0071] In one exemplary embodiment, a specific syntax element (alpha_channel_id) further indicates how to concatenate the decoding post-processing intended to be applied to the decoded video picture.

[0072] In an exemplary embodiment, when equal to a specific value, the specific syntax element (alpha_channel_id) further indicates that parameters of the alpha channel information SEI message not involved in transparency adjustment of pixels of the decoded video image are not written to or read from the bitstream.

[0073] In one exemplary embodiment, the alpha channel indicator is a specific value of an existing syntax element (alpha_channel_use_idc) of the alpha channel information SEI message, which indicates that the alpha channel information is intended to be used for transparency adjustment of pixels of the decoded video sequence.

[0074] In one exemplary embodiment, the alpha channel indicator is a specific type of auxiliary picture that is signaled in the SEI message.

[0075] According to a third aspect of the present application, there is provided a bit stream formatted to include the encoded data obtained from the method according to the first aspect and an alpha channel indicator.

[0076] According to a fourth aspect of the present application, an apparatus includes means for executing one of the methods according to the first and / or second aspect of the present application.

[0077] According to a fifth aspect of the present application, a computer program product comprises instructions, and when the program is executed by one or more processors, the instructions cause the one or more processors to perform the method according to the first and / or second aspect of the present application.

[0078] According to a sixth aspect of the present application, a non-transitory storage medium (or storage medium) is provided, which carries instructions of program code for executing any one of the methods according to the first and / or second aspects of the present application.

[0079] Specific properties of at least one embodiment of the exemplary embodiments and other objects, advantages, features and uses of at least one embodiment of the exemplary embodiments will become even more apparent from the following description of the examples taken in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] Reference will now be made, by way of example, to the accompanying drawings which show exemplary embodiments of the present application, and in which:

[0081] Figure 1 A schematic block diagram showing the steps of a method 100 of encoding a video sequence associated with alpha channel information and intended to be displayed on a transparent screen into a bitstream according to an exemplary embodiment;

[0082] Figure 2 a schematic block diagram showing the steps of a method 200 of decoding from a bitstream a video sequence associated with alpha channel information and intended to be displayed on a transparent screen according to an exemplary embodiment; and

[0083] Figure 3 A schematic block diagram illustrating an example of a system in which various aspects and exemplary embodiments are implemented is shown.

[0084] Similar or identical elements are denoted by the same reference numerals. DETAILED DESCRIPTION

[0085] At least one of the exemplary embodiments will be described more fully below with reference to the accompanying drawings, wherein an example of at least one of the exemplary embodiments is depicted. However, the exemplary embodiments may be implemented in many alternative forms and should not be construed as being limited to the examples set forth herein. Thus, it should be understood that the exemplary embodiments are not intended to be limited to the specific forms disclosed. On the contrary, the present application is intended to cover all modifications, equivalents, and alternatives falling within the spirit and scope of the present application.

[0086] Generally speaking, the alpha channel indicator is signaled as part of the alpha channel information associated with the video sequence.The alpha channel indicator indicates for which type of post-decoding processing the alpha channel information is intended.

[0087] Then, alpha blending is applied to the decoded video sequence only if the alpha channel indicator indicates that the alpha channel information is intended for alpha blending.

[0088] Different post-decoding processes may be applied to the decoded video sequence to recreate the transparency level intended for transparent screen usage purposes.

[0089] Signaling the alpha channel indicator allows the decoding post-processing and alpha blending related to transparent screens to be different, so the intent should be differentiated (even though some required parameter semantics will be the same).

[0090] Signaling the alpha channel indicator avoids interpreting and applying to the upstream equipment (source) of the TV processing of auxiliary images other than processing specific to the transparent screen, which may differ depending on, among other things, the underlying technology used (OLED vs. LCD).

[0091] Signaling said alpha channel indicator improves the alpha channel signaling known in the prior art, since it allows to decouple the assumed alpha blending decoding post-processing from other types of decoding post-processing.

[0092] Furthermore, signaling the alpha channel indicator resolves the ambiguity of signaling an alpha channel sequence when the usage is not specified (as seen in several video coding standards), so that interoperability can be achieved.

[0093] Finally, signaling the alpha channel indicator improves compression efficiency by signaling only relevant parameters associated with post-decoding processing, for example, parameters in the alpha channel signaling that are only related to alpha blending are syntactically excluded from other post-decoding processing, so that bit information can be saved for transmission, and the need to specify dummy values ​​when alpha blending is a post-decoding process is avoided.

[0094] The following exemplary embodiments are described to show how to modify the VSEI syntax and semantics of the alpha channel SEI message in order to implement the present disclosure.

[0095] The present disclosure is not limited to the current version of the VSEI syntax, but extends to any edited variant of the current VSEI syntax.

[0096] In the first exemplary embodiment, the alpha channel indicator is an additional syntax element included in a current alpha channel information SEI message (ACI SEI message) of the VSEI.

[0097] Therefore, adding an additional syntax element in the ACI SEI message separates the roles: the additional syntax element is intended to normatively affect the syntax of the ACI SEI message, while the current syntax element alpha_channel_use_idc is used as a sub-registration usage scope.

[0098] In a first variant of the first embodiment, the additional syntax element is a syntax element alpha_channel_idc included in the current ACI SEI message of the VSEI, the syntax element alpha_channel_idc:

[0099] - equal to 0 to indicate that the ACI SEI message is intended for alpha blending purposes;

[0100] - equal to 1 to indicate that the ACI SEI message is intended for transparency of pixels of a transparent screen;

[0101] - equal to 2 to indicate that the ACI SEI message is intended for an unspecified purpose.

[0102] Table 1 provides an example of the syntax of a modified alpha channel information SEI message as defined in VSEI according to the first variant of the first exemplary embodiment.

[0103]

[0104] Table 1

[0105] In a variation of the first variation, when equal to a specific value (=0 or =1), the syntax element alpha_channel_idc further indicates that parameters of the ACI SEI message not involved in transparency adjustment of the pixel are not written to or read from the bitstream.

[0106] Table 2 provides an example of the syntax of the modified alpha channel information SEI message according to the variant of the first variant. In this example, when the syntax element alpha_channel_idc is not equal to 0, that is, when the decoding post-processing is not alpha blending, the syntax elements alpha_transparent_value, alpha_opaque_value, alpha_channel_incr_flag, alpha_channel_clip_flag and alpha_channel_clip_type_flag are discarded because these syntax elements are only related to alpha blending.

[0107]

[0108] Table 2

[0109] In a second variant of the first exemplary embodiment, the additional syntax element is a syntax element alpha_channel_id comprised in the current ACI SEI message of the VSEI, the syntax element alpha_channel_id being a number belonging to a range of values ​​indicating an alternative post-decoding process or a cascade of alternative post-decoding processes intended to be applied to the video sequence.

[0110] For example, when more than one ACI SEI message exists with the same value of the syntax element alpha_channel_id, the contents of these ACI SEI messages should be the same. When an ACI SEI message exists with more than one value of the syntax element alpha_channel_id, this may indicate that the post-decoding processes indicated by the different values ​​of the syntax element alpha_channel_id are alternatives provided for different purposes, or that a cascade of post-decoding processes may be applied in a sequential order. The value of the syntax element alpha_channel_id may be between 0 and 2. 32 -2, including 0 and 2 32 -2.

[0111] In the variant, from 0 to 255 (inclusive) and from 512 to 2 31 -1 (including 512 and 2 31 -1) may be used as determined by an external application. The decoder encounters a value of the syntax element alpha_channel_id in the range of 256 to 511, inclusive.

[0112] Table 3 provides an example of the syntax of the alpha channel information SEI message as defined in the VSEI and modified according to the second variant of the first exemplary embodiment.

[0113]

[0114] Table 3

[0115] In a variation of the second variant of the first exemplary embodiment, an additional syntax element alpha_channel_id further indicates how to concatenate decoding post-processing intended to be applied to the decoded video images.

[0116] For example, a value of the syntax element alpha_channel_id from 0 to 255 inclusive indicates that the alpha channel information is used for alpha blending purposes. In a variant, different alpha blending operations may be intended to be performed in increasing order of syntax element alpha_channel_id values ​​regardless of the q-layer order.

[0117] The values ​​of the syntax element alpha_channel_id from 256 to 511, inclusive, indicate that the alpha channel information is used for transparent display purposes. Furthermore, transparency adjustment operations for different pixels are intended to be performed in increasing order of alpha_channel_id values, regardless of the layer order.

[0118] In a variant, different post-decoding processing, alpha blending and transparency adjustment operations of pixels are intended to be performed in increasing order of syntax element alpha_channel_id values, regardless of the layer order and the nature of the post-decoding processing.

[0119] In another variation of the second variation, when equal to a specific value, the syntax element alpha_channel_id further indicates that parameters of the ACI SEI message not involved in transparency adjustment of the pixel are not written to or read from the bitstream.

[0120] Table 4 provides an example of the syntax of the alpha channel information SEI message modified according to the variant of the second variant. In this example, when the syntax element alpha_channel_id is greater than or equal to a specific value (here 256), that is, when the post-decoding processing is not alpha blending, the syntax elements alpha_transparent_value, alpha_opaque_value, alpha_channel_incr_flag, alpha_channel_clip_flag and alpha_channel_clip_type_flag are discarded because these syntax elements are only related to alpha blending.

[0121]

[0122] Table 4

[0123] In the second exemplary embodiment, the alpha channel indicator is a specific value of the existing syntax element alpha_channel_use_idc of the ACI SEI message of the VSEI.

[0124] A specific value of the syntax element alpha_channel_use_idc indicates a specific post-decoding process for transparent display purposes, ie, the specific value indicates that the alpha channel information is intended to be used for transparency adjustment of pixels of a transparent screen.

[0125] For example, the syntax element alpha_channel_use_idc

[0126] - equal to 0 to indicate that for alpha blending purposes, upon output from the decoding process, during display, the decoded samples of the associated primary picture should be multiplied by the interpreted sample values ​​of the auxiliary coded picture;

[0127] - equal to 1 to indicate that for alpha blending purposes, after output from the decoding process, the decoded samples of the associated primary picture shall not be multiplied by the interpreted sample values ​​of the auxiliary coded picture during display;

[0128] - equal to 2 to indicate that the use of auxiliary images is not specified; and

[0129] - according to the present disclosure, equal to 3 to indicate that for the purpose of transparent display, after being output from the decoding process, in the display process, the interpreted sample values ​​of the auxiliary coded image are used to adjust the transparency of the pixels of the transparent screen.

[0130] In a third exemplary embodiment, the alpha channel indicator is a specific type of auxiliary picture signaled in the SCI SEI message of the VSEI.

[0131] Although it is still possible to signal an alpha channel indicator as part of an ACI SEI message indicating which type of post-decoding processing the alpha channel is intended for, it is beneficial to very easily identify the purpose of the alpha channel information for alpha blending or for transparent display. Easily identifying the purpose means a unique location, i.e., a unique parameter, that expresses the purpose. One way to achieve this is to signal the alpha channel indicator as a specific type of auxiliary picture in the SDI SEI message of the VSEI.

[0132] In the VSEI (Section 8.19.2), a table is defined for mapping the types of auxiliary pictures to the values ​​of the so-called array sdi_aux_id[i] of the i-th layer. The value sdi_aux_id[i] is equal to 0 to indicate that the i-th layer in the current CVS does not contain auxiliary pictures. The value sdi_aux_id[i] is equal to 1 to indicate that the auxiliary picture in the i-th layer corresponds to alpha channel information. The value sdi_aux_id[i] is equal to 2 to indicate that the auxiliary picture in the i-th layer corresponds to depth information.

[0133] According to a third exemplary embodiment, the value sdi_aux_id[i] equal to 3 indicates that the auxiliary pictures in layer i are intended for transparent display purposes. In this case, layer i also contains associated ACI SEI messages, but generated for the purpose of displaying the decoded video on a transparent display.

[0134] Table 5 provides an example of a table of VSEI as modified by the third exemplary embodiment.

[0135]

[0136] Table 5

[0137] Figure 1 A schematic block diagram showing the steps of a method 100 of encoding into a bitstream a video sequence associated with alpha channel information and intended to be displayed on a transparent screen according to an exemplary embodiment.

[0138] An alpha channel indicator is written into the bitstream at step 110. The alpha channel indicator indicates for which type of post-processing of the video sequence the alpha channel information associated with the video sequence is intended.

[0139] When the video sequence is intended to be displayed on said transparent screen, one type of post-processing of the video sequence may be a transparency adjustment of the pixels of the transparent screen.

[0140] Another type of post-processing of video sequences can be alpha blending.

[0141] If the alpha channel indicator indicates that the alpha channel information is intended for transparency adjustment of pixels of a transparent screen, step 110 is followed by steps 120 to 140 .

[0142] In step 120, the video sequence is encoded as encoded data into a bitstream.

[0143] In step 130, a transparency information mask associated with the video sequence is determined.

[0144] In step 140, alpha channel information representing the transparency information mask is encoded into a bitstream as encoded data.

[0145] In an exemplary embodiment of the method 100 , the alpha channel information associated with an image of the video sequence is an alpha channel image.

[0146] In an exemplary embodiment, an alpha channel SEI (Supplemental Enhancement Information) message is also written into the bitstream. The content of the SEI message describes the post-decoding processing intended to be applied to the decoded video image. In particular, the alpha channel SEI message includes information representing an alpha channel indicator.

[0147] Any conventional video encoder may be used to encode the video sequence and the associated alpha channel picture and possibly the SEI message.

[0148] For example, VVC may be used.

[0149] Figure 2 A schematic block diagram showing the steps of a method 200 of decoding from a bitstream a video sequence associated with alpha channel information and intended to be displayed on a transparent screen according to an exemplary embodiment.

[0150] In step 210, the SEI parser detects and extracts an alpha channel SEI message from the bitstream. The alpha channel SEI message conveys an alpha channel indicator.

[0151] At step 220, the alpha channel indicator is read from the alpha channel SEI message.

[0152] If the alpha channel indicator indicates that the alpha channel information is intended to be used for adjusting the transparency of pixels of the transparent screen, step 220 is followed by steps 230 to 250 .

[0153] In step 230, the encoded data is decoded from the bitstream in the decoded video sequence.

[0154] In step 240, data encoded in a decoded alpha channel video sequence conveying alpha channel information is decoded from a bitstream.

[0155] In an exemplary embodiment, the decoded video sequence may correspond to a primary (master) video sequence, and the decoded alpha channel video sequence may correspond to an auxiliary video sequence associated with the primary video sequence.

[0156] In step 250, a transparency information mask is determined based on transparency information associated with the decoded alpha channel video sequence.

[0157] In step 260 , the transparency of the pixels of the transparent screen is adjusted based on the transparency information mask.

[0158] For example, the transparency information mask contains an array of adjustment coefficients associated with samples of the decoded video sequence in order to adjust the transparency of the pixel perceived on the transparent screen. In a simple case, the transparency adjustment of the pixel may be similar to an alpha blending operation, where the background video sequence will be a black video sequence for an OLED screen, or a white video sequence for an LCD screen, to adjust the transparency of the pixel. By doing so, a low coefficient (say 0) in the transparency information mask will result in the cancellation of the associated sample in the decoded video sequence, and leave samples from the black video sequence (i.e., black samples (values ​​of 0 or close to 0)), or samples from the white video sequence (i.e., white samples (values ​​of 255 or close to 255 for 8-bit depth), which in turn makes the pixel at that position fully transparent on the transparent screen.

[0159] Any conventional video decoder can be used to decode the video sequence and the associated alpha channel picture and possible SEI messages.

[0160] For example, VVC may be used.

[0161] In an exemplary embodiment, the method 200 further includes determining (step 270) whether dimming adjustment of a dimming backplane of the transparent screen is requested, and if dimming adjustment is requested, adjusting (step 260) the transparency of pixels of the transparent screen by adjusting the dimming backplane.

[0162] In an exemplary embodiment, the method 200 further includes acquiring (step 280) ambient lighting condition data, and adjusting (step 260) the transparency of pixels of the transparent screen based on the transparency information mask and the ambient lighting condition.

[0163] In an exemplary embodiment of the method 100 and the method 200 , the alpha channel indicator is a specific syntax element alpha_channel_idc or alpha_channel_id as described above.

[0164] Figure 3 A schematic block diagram illustrating an example of a system 300 in which various aspects and exemplary embodiments are implemented is shown.

[0165] System 300 may be embedded as one or more devices, including various components described below. In various exemplary embodiments, system 300 may be configured to implement one or more aspects described in this application.

[0166] Examples of equipment that may constitute all or part of system 300 include personal computers, laptop computers, smart phones, tablet computers, digital multimedia set-top boxes, digital television receivers, personal video recording systems, connected household appliances, connected vehicles and their associated processing systems, head-mounted display devices (HMD, perspective glasses), projectors (projectors), "caves" (systems including multiple displays), servers, video encoders, video decoders, post-processors for processing outputs from video decoders, pre-processors for providing inputs to video encoders, web servers, video servers (e.g., broadcast servers, video-on-demand servers, or network servers), static or video cameras, encoding or decoding chips, or any other communication devices. The elements of system 300 may be implemented individually or in combination in a single integrated circuit (IC), multiple ICs, and / or discrete components. For example, in at least one exemplary embodiment, the processing and encoder / decoder elements of system 300 may be distributed across multiple ICs and / or discrete components. In various exemplary embodiments, system 300 may be coupled to other similar systems or other electronic devices via, for example, a communication bus or by dedicated input and / or output ports.

[0167] The system 300 may include at least one processor 310 configured to execute instructions loaded therein for implementing, for example, various aspects described in the present application. The processor 310 may include embedded memory, input / output interfaces, and various other circuits known in the art. The system 300 may include at least one memory 320 (e.g., a volatile memory device and / or a non-volatile memory device). The system 300 may include a storage device 340, which may include a non-volatile memory and / or a volatile memory, including but not limited to an electrically erasable programmable read-only memory (EEPROM), a read-only memory (ROM), a programmable read-only memory (PROM), a random access memory (RAM), a dynamic random access memory (DRAM), a static random access memory (SRAM), a flash memory, a disk drive, and / or an optical drive. As a non-limiting example, the storage device 340 may include an internal storage device, an attached storage device, and / or a network accessible storage device.

[0168] The system 300 may include an encoder / decoder module 330, which is configured to, for example, process data to provide encoded / decoded video image data, and the encoder / decoder module 330 may include its own processor and memory. The encoder / decoder module 330 may represent a module (one or more) that may be included in a device to perform encoding and / or decoding functions. As known, a device may include one or both of the encoding and decoding modules. In addition, the encoder / decoder module 330 may be implemented as a separate element of the system 300, or may be incorporated into the processor 310 as a combination of hardware and software known to those skilled in the art.

[0169] Program code to be loaded onto the processor 310 or the encoder / decoder 330 to perform various aspects described in the present application may be stored in the storage device 340 and subsequently loaded onto the memory 320 to be executed by the processor 310. According to various exemplary embodiments, during the execution of the processes described in the present application, one or more of the processor 310, the memory 320, the storage device 340, and the encoder / decoder module 330 may store one or more of various items. Such stored items may include, but are not limited to, video image data, information data for encoding / decoding video image data, bit streams, matrices, variables, and intermediate or final results of equations, formulas, operations, and operation logic processing.

[0170] In several exemplary embodiments, memory internal to processor 310 and / or encoder / decoder module 330 may be used to store instructions and provide working memory for processes that may be performed during encoding or decoding.

[0171] However, in other exemplary embodiments, memory external to the processing device (e.g., the processing device may be the processor 310 or the encoder / decoder module 330) is used for one or more of these functions. The external memory may be a memory 320 and / or a storage device 340, such as a dynamic volatile memory and / or a non-volatile flash memory. In several exemplary embodiments, the external non-volatile flash memory is used to store the operating system of the television. In at least one exemplary embodiment, a fast external dynamic volatile memory such as RAM may be used as working memory for video encoding and decoding operations, such as for MPEG-2 Part 2 (also known as ITU-T Recommendation H.262 and ISO / IEC 13818-2, also known as MPEG-2 Video), AVC, HEVC, EVC, VVC, AV1, etc.

[0172] As indicated in block 390, input to the elements of system 300 may be provided through various input devices. Such input devices include, but are not limited to, (i) an RF section that may receive an RF signal transmitted over the air, for example, by a broadcast device, (ii) a composite input terminal, (iii) a USB input terminal, (iv) an HDMI input terminal, (v) when the present disclosure is implemented in the automotive field, a bus such as a CAN (Controller Area Network), CAN FD (Controller Area Network Flexible Data Rate), FlexRay (ISO 17458), or Ethernet (ISO / IEC 802-3) bus.

[0173] In various exemplary embodiments, the input device of block 390 has associated corresponding input processing elements, as known in the art. For example, the RF portion may be associated with elements necessary for: (i) selecting a desired frequency (also referred to as selecting a signal, or band-limiting a signal to a frequency band), (ii) down-converting the selected signal, (iii) band-limiting the frequency band again to a narrower frequency band to select a signal band that may be referred to as a channel in certain exemplary embodiments, (iv) demodulating the down-converted and band-limited signal, (v) performing error correction, and (vi) demultiplexing to select a desired packet stream. The RF portion of various exemplary embodiments may include one or more elements that perform these functions, such as a frequency selector, a signal selector, a band limiter, a channel selector, a filter, a down-converter, a demodulator, an error corrector, and a demultiplexer. The RF portion may include a tuner that performs various of these functions, including, for example, down-converting a received signal to a lower frequency (e.g., an intermediate frequency or a near-baseband frequency) or a baseband.

[0174] In one set-top box embodiment, the RF section and its associated input processing elements can receive RF signals transmitted over a wired (e.g., cable) medium. The RF section can then perform frequency selection by filtering, down-converting, and filtering again to a desired frequency band.

[0175] Various exemplary embodiments rearrange the order of the above-described (and other) elements, remove some of these elements, and / or add other elements that perform similar or different functions.

[0176] Adding components may include inserting components between existing components, such as, for example, inserting an amplifier and an analog-to-digital converter.In various exemplary embodiments, the RF portion may include an antenna.

[0177] In addition, the USB and / or HDMI terminals may include corresponding interface processors for connecting the system 300 to other electronic devices via USB and / or HDMI connections. It should be understood that various aspects of input processing (e.g., Reed-Solomon error correction) may be implemented, for example, in a separate input processing IC or in the processor 310 when necessary. Similarly, various aspects of USB or HDMI interface processing may be implemented in a separate interface IC or in the processor 310 when necessary. The demodulated, error-corrected, and demultiplexed streams may be provided to various processing elements, including, for example, the processor 310 and the encoder / decoder 330, which operate in conjunction with memory and storage elements to process the data streams for presentation on output devices when necessary.

[0178] The various elements of system 300 may be provided within an integrated housing. Within the integrated housing, the various elements may be interconnected and data transferred between them using a suitable connection arrangement 390, such as an internal bus (including an I2C bus), wiring, and printed circuit boards known in the art.

[0179] System 300 may include a communication interface 350 that enables communication with other devices via a communication channel 351. Communication interface 350 may include, but is not limited to, a transceiver configured to send and receive data over communication channel 351. Communication interface 350 may include, but is not limited to, a modem or a network card, and communication channel 351 may be implemented, for example, within a wired and / or wireless medium.

[0180] In various exemplary embodiments, a Wi-Fi network such as IEEE 802.11 may be used to stream data to the system 300. The Wi-Fi signals of these exemplary embodiments may be received via a communication channel 351 suitable for Wi-Fi communications and a communication interface 350. The communication channel 351 of these exemplary embodiments may typically connect to an access point or router that provides access to external networks including the Internet to allow streaming applications and other over-the-top communications.

[0181] Other exemplary embodiments may provide streamed data to system 300 using a set top box that delivers the data through an HDMI connection to input block 390 .

[0182] Still other exemplary embodiments may provide streaming data to the system 300 using an RF connection to input block 390 .

[0183] The streamed data may be used as a means of signaling information used by the system 300. The signaling information may include a bitstream and / or information such as the number of pixels of a video image, any encoding / decoding setting parameters, alignment status, alignment reference data, overlap status, resampling data, interpolation data, and / or calibration data.

[0184] It should be appreciated that signaling may be implemented in a variety of ways. For example, in various exemplary embodiments, one or more syntax elements, flags, etc. may be used to signal information to a corresponding decoder.

[0185] The system 300 may provide output signals to various output devices, including a display 361, speakers 371, and other peripheral devices 381. In various examples of the exemplary embodiments, the other peripheral devices 381 may include one or more of a stand-alone DVR, a disk player, a stereo system, a lighting system, and other devices that provide functionality based on the output of the system 300.

[0186] In various exemplary embodiments, control signals may be communicated between the system 300 and the display 361, speakers 371, or other peripherals 381 using signaling such as AV.Link (Audio / Video Link), CEC (Consumer Electronics Control), or other communication protocols that enable device-to-device control with or without user intervention.

[0187] Output devices may be communicatively coupled to system 300 via dedicated connections through respective interfaces 360 , 370 , and 380 .

[0188] Optionally, output devices may be connected to system 300 using communication channel 351 via communication interface 350. Display 361 and speaker 371 may be integrated into a single unit with other components of system 300 in an electronic device such as, for example, a television.

[0189] In various exemplary embodiments, the display interface 360 ​​may include a display driver, such as, for example, a timing controller (TCon) chip.

[0190] For example, if the RF portion of input 390 is part of a separate set-top box, then display 361 and speaker 371 may optionally be separate from one or more of the other components. In various exemplary embodiments where display 361 and speaker 371 may be external components, output signals may be provided via dedicated output connections including, for example, an HDMI port, a USB port, or a COMP output.

[0191] exist Figures 1 to 3In the present invention, various methods are described herein, and each method includes one or more steps or actions to implement the described method. Unless a specific order of steps or actions is required for the correct operation of the method, the order and / or use of specific steps and / or actions can be modified or combined.

[0192] Some examples are described about block diagrams and / or operational flow charts. Each square block represents a portion of a circuit element, module or code, which includes one or more executable instructions for implementing (one or more) specified logical functions. It should also be noted that, in other embodiments, the (one or more) functions marked in the square block may not occur in the order indicated. For example, depending on the functions involved, two square blocks shown in succession can actually be executed substantially concurrently, or sometimes these square blocks can be executed in reverse order.

[0193] The embodiments and aspects described herein may be implemented in, for example, a method or process, an apparatus, a computer program, a data stream, a bit stream, or a signal. Even if only discussed in the context of a single form of embodiment (e.g., discussed only as a method), the embodiments of the features discussed may also be implemented in other forms (e.g., an apparatus or a computer program).

[0194] The method may be implemented in, for example, a processor, which generally refers to a processing device, including, for example, a computer, a microprocessor, an integrated circuit, or a programmable logic device. The processor also includes a communication device.

[0195] Furthermore, the methods may be implemented by instructions executed by a processor, and such instructions (and / or data values ​​produced by the embodiments) may be stored in a computer-readable storage medium, such as, for example, storage device 340 ( Figure 3 ). The computer-readable storage medium may take the form of a computer-readable program product implemented in one or more computer-readable media and having a computer-readable program code implemented thereon that can be executed by a computer. Computer-readable storage media as used herein may be considered to be non-transitory storage media, given the inherent ability to store information therein and the inherent ability to provide retrieval of information therefrom. The computer-readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared or semiconductor system, device or apparatus, or any suitable combination of the foregoing. It should be appreciated that the following, although more specific examples of computer-readable storage media to which the present exemplary embodiment may be applied are provided, are merely illustrative and non-exhaustive lists, as those of ordinary skill in the art will readily appreciate: portable computer floppy disk; hard disk; read-only memory (ROM); erasable programmable read-only memory (EPROM or flash memory); portable compact disk read-only memory (CD-ROM); optical storage device; magnetic storage device; or any suitable combination of the foregoing.

[0196] The instructions may form an application program tangibly embodied on a processor-readable medium.

[0197] For example, instructions may be in hardware, firmware, software, or a combination. For example, instructions may be found in an operating system, a separate application, or a combination of both. Thus, a processor may be characterized as, for example, a device configured to perform a process and a device including a processor-readable medium (such as a storage device) having instructions for performing the process. Additionally, in addition to or in lieu of instructions, a processor-readable medium may store data values ​​generated by an embodiment.

[0198] Device can be realized in suitable hardware, software and firmware for example.The example of such device comprises personal computer, laptop computer, smart phone, tablet computer, digital multimedia set-top box, digital television receiver, personal video recording system, connected household appliances, head-mounted display device (HMD, perspective glasses), projector (projector), "cave" (system including multiple displays), server, video encoder, video decoder, post-processor for processing the output from video decoder, pre-processor for providing input to video encoder, web server, set-top box, and any other equipment for processing video image, or other communication equipment.It should be clear that equipment can be mobile and even installed in a mobile vehicle.

[0199] The computer software may be implemented by the processor 310 or by hardware, or by a combination of hardware and software. As a non-limiting example, the exemplary embodiments may also be implemented by one or more integrated circuits. The memory 320 may be of any type suitable for the technical environment and may be implemented using any appropriate data storage technology (such as optical memory devices, magnetic memory devices, semiconductor-based memory devices, fixed memory, and removable memory, as non-limiting examples). The processor 310 may be of any type suitable for the technical environment and may include one or more of a microprocessor, a general-purpose computer, a special-purpose computer, and a processor based on a multi-core architecture, as non-limiting examples.

[0200] Based on the present application, as will be apparent to one of ordinary skill in the art, embodiments may generate various signals formatted to carry information that may be stored or transmitted, for example. The information may include, for example, instructions for executing a method or data generated by one of the described embodiments. For example, a signal may be formatted to carry a bit stream of the described exemplary embodiments. Such a signal may be formatted as, for example, an electromagnetic wave (e.g., using a radio frequency portion of a spectrum) or a baseband signal. Formatting may include, for example, encoding a data stream and modulating a carrier with the encoded data stream. The information carried by the signal may be, for example, analog or digital information. As is known, the signal may be transmitted over a variety of different wired or wireless links. The signal may be stored on a processor-readable medium.

[0201] The terms used herein are only used to describe the purpose of specific exemplary embodiments and are not intended to be limited. As used herein, the singular "a / kind (a)", "an / kind (an)" and "the / said (the)" may also be intended to include plural forms, unless the context clearly indicates otherwise. It will be further understood that, when used in this specification, the terms "include / comprise" and / or "including / comprising" may specify the existence of stated, for example, features, integers, steps, operations, elements and / or components, but do not exclude the existence or addition of one or more other features, integers, steps, operations, elements, components and / or their groups. Moreover, when an element is referred to as "response" or "connection" or "associated" to another element, it may directly respond or be connected to another element or be associated with another element, or there may be an intermediate element. On the contrary, when an element is referred to as "direct response" or "direct connection" to another element or "directly associated" with another element, there is no intermediate element.

[0202] It should be appreciated that use of any of the symbols / terms " / ", "and / or", and "at least one of" may be intended to encompass selection of only the first listed option (A), or only the second listed option (B), or both options (A and B), such as in the case of "A / B", "A and / or B", and "at least one of A and B". As a further example, in the case of "A, B, and / or C" and "at least one of A, B, and C", such wording is intended to encompass selection of only the first listed option (A), or only the second listed option (B), or only the third listed option (C), or only the first and second listed options (A and B), or only the first and third listed options (A and C), or only the second and third listed options (B and C), or all three options (A and B and C). This may be extended to as many items as listed, as will be apparent to one of ordinary skill in this and related arts.

[0203] Various numerical values ​​may be used in the present application. Specific values ​​may be used for example purposes and the described aspects are not limited to these specific values.

[0204] It will be understood that although the terms first, second, etc. can be used to describe various elements in this article, these elements are not limited by these terms. These terms are only used to distinguish one element from another element. For example, without departing from the teaching of this application, the first element can be referred to as the second element, and similarly, the second element can be referred to as the first element. There is no suggestion of sorting between the first element and the second element.

[0205] References to "an exemplary embodiment" or "exemplary embodiments" or "one implementation" or "implementation" and other variations thereof are frequently used to convey that a particular feature, structure, characteristic, etc. (described in conjunction with an embodiment / implementation) is included in at least one embodiment / implementation. Thus, the appearances of the phrases "in an exemplary embodiment" or "in an exemplary embodiment" or "in one implementation" or "in an implementation" and any other variations appearing in various places in this application do not necessarily all refer to the same exemplary embodiment.

[0206] Similarly, references herein to "according to an exemplary embodiment / example / implementation" or "in an exemplary embodiment / example / implementation" and other variations thereof are frequently used to convey that a particular feature, structure, or characteristic (described in conjunction with an exemplary embodiment / example / implementation) may be included in at least one exemplary embodiment / example / implementation. Therefore, the expressions "according to an exemplary embodiment / example / implementation" or "in an exemplary embodiment / example / implementation" appearing in various places in this application do not necessarily all refer to the same exemplary embodiment / example / implementation, nor are separate or alternative exemplary embodiments / examples / implementations necessarily mutually exclusive of other exemplary embodiments / examples / implementations.

[0207] Reference numerals appearing in the claims are for illustration purposes only and have no limiting effect on the scope of the claims.The present exemplary embodiments / examples and variants may be employed in any combination or sub-combination although not explicitly described.

[0208] When a figure is presented as a flow chart, it should be understood that it also provides a block diagram of the corresponding apparatus. Similarly, when a figure is presented as a block diagram, it should be understood that it also provides a flow chart of the corresponding method / process.

[0209] While some diagrams include arrows on communication paths to illustrate a primary direction of communication, it should be understood that communication can occur in the opposite direction to the depicted arrows.

[0210] Various embodiments relate to decoding. As used in the present application, "decoding" may encompass, for example, all or part of a process performed on a received video sequence (which may include a received bitstream that encodes one or more video sequences) to produce a final output suitable for display or further processing in a reconstructed video domain. In various exemplary embodiments, such a process includes one or more of the processes typically performed by a decoder. In various exemplary embodiments, such a process also or alternatively includes a process performed by a decoder of the various embodiments described in the present application.

[0211] As a further example, in one exemplary embodiment, "decoding" may refer only to dequantization, in one exemplary embodiment, "decoding" may refer to entropy decoding, in another exemplary embodiment, "decoding" may refer only to differential decoding, and in another exemplary embodiment, "decoding" may refer to a combination of dequantization, entropy decoding, and differential decoding. Whether the phrase "decoding process" may be intended to specifically refer to a subset of operations, or to generally refer to a broader decoding process will be clear, and is believed to be well understood by those skilled in the art, based on the context of the particular description.

[0212] Various embodiments relate to encoding. In a manner similar to the above discussion of "decoding", "encoding" as used in this application may encompass, for example, all or part of a process performed on an input video sequence to produce an output bitstream. In various exemplary embodiments, such processes include one or more of the processes typically performed by an encoder. In various exemplary embodiments, such processes also include or alternatively include processes performed by an encoder of the various embodiments described in this application.

[0213] As a further example, in one exemplary embodiment, "encoding" may refer only to quantization, in one exemplary embodiment, "encoding" may refer only to entropy coding, in another exemplary embodiment, "encoding" may refer only to differential coding, and in another exemplary embodiment, "encoding" may refer to a combination of quantization, differential coding, and entropy coding. Based on the context of the particular description, whether the phrase "encoding process" may be intended to specifically refer to a subset of operations, or to generally refer to a broader encoding process will be clear and is believed to be well understood by those skilled in the art.

[0214] Additionally, the present application may refer to "obtaining" various information. Obtaining information may include, for example, one or more of estimating information, calculating information, predicting information, or retrieving information from a memory, processing information, moving information, copying information, erasing information, calculating information, determining information, predicting information, or estimating information.

[0215] Additionally, this application may refer to "receiving" various information. Receiving information may include, for example, one or more of accessing the information or receiving the information from a communication network.

[0216] Moreover, as used herein, the word "signal" especially refers to indicating something to a corresponding decoder, etc. For example, in some exemplary embodiments, the encoder signals specific information, such as encoding parameters or encoded video image data. In this way, in exemplary embodiments, the same parameter can be used on the encoder side and the decoder side. Therefore, for example, the encoder can transmit (explicit signaling) specific parameters to the decoder so that the decoder can use the same specific parameters. On the contrary, if the decoder already has specific parameters and other parameters, signaling can be used without transmission (implicit signaling) to simply allow the decoder to know and select specific parameters. By avoiding the transmission of any actual function, bit saving is achieved in various exemplary embodiments. It should be recognized that signaling can be completed in a variety of ways. For example, in various exemplary embodiments, one or more syntax elements, flags, etc. are used to signal information to the corresponding decoder. Although the verb form of the word "signal" is mentioned above, the word "signal" can also be used as a noun in this article.

[0217] A number of embodiments have been described. However, it should be understood that various modifications may be made. For example, elements of different embodiments may be combined, supplemented, modified, or removed to produce other embodiments. In addition, it will be understood by those of ordinary skill that other structures and processes may replace the disclosed structures and processes, and that the resulting embodiments will perform at least substantially the same (one or more) functions in at least substantially the same (one or more) manners to achieve at least substantially the same (one or more) results as the disclosed embodiments. Thus, the present application contemplates these and other embodiments.

Claims

1. A method (100) of encoding a video sequence associated with alpha channel information into a bitstream, wherein the method comprises: - writing (110) an alpha channel indicator into the bitstream, the alpha channel indicator indicating for which type of post-processing of the video sequence the alpha channel information associated with the video sequence is intended; - if the alpha channel indicator indicates that the alpha channel information is intended to be used for pixel transparency adjustment of a transparent screen; - encoding (120) the video sequence as encoded data into the bitstream; - determining (130) a transparency information mask associated with said video sequence; as well as - encoding (140) alpha channel information representing said transparency information mask into said bitstream as encoded data.

2. A method (200) for decoding a video sequence associated with alpha channel information from a bitstream, wherein the method comprises: - reading (220) an alpha channel indicator from a bitstream, the alpha channel indicator indicating for which type of post-processing of a video sequence the alpha channel information is intended; - if the alpha channel indicator indicates that the alpha channel information is intended to be used for adjusting pixel transparency of a transparent screen; - decoding (230) coded data from said bitstream in a decoded video sequence; - decoding (240) data encoded from the bitstream in a decoded alpha channel video sequence conveying the alpha channel information; - determining (250) a transparency information mask based on transparency information associated with said decoded alpha channel video sequence; and - adjusting (260) the transparency of the pixels of the transparent screen based on the transparency information mask, the pixels being derived from sample values ​​of the decoded video sequence.

3. The method of claim 2, wherein the method further comprises determining (270) whether a dimming adjustment of a dimming backplane of a transparent screen is requested, and if a dimming adjustment is requested, adjusting (260) the pixel transparency by adjusting the dimming backplane.

4. The method according to claim 2 or 3, wherein the method further comprises acquiring (280) ambient lighting condition data, and adjusting (260) the pixel transparency based on the transparency information mask and the ambient lighting condition.

5. The method according to one of claims 1 to 4, wherein the alpha channel indicator is a specific syntax element (alpha_channel_idc, alpha_channel_id) included in an alpha channel information SEI message.

6. The method according to claim 5, wherein the specific syntax element (alpha_channel_idc) - equal to 0 to indicate that the alpha channel information SEI message is intended for alpha blending the decoded video picture with alpha channel information; - equal to 1 to indicate that the alpha channel information SEI message is intended for adjusting (150) pixel transparency of a transparent screen.

7. The method according to claim 6, wherein when equal to a specific value, the specific syntax element (alpha_channel_idc) further indicates that parameters of the alpha channel information SEI message not involved in transparency adjustment of the pixel are not written to or read from the bitstream.

8. The method of claim 5, wherein the specific syntax element (alpha_channel_id) is a number belonging to a range of values ​​indicating an alternative post-decoding process or a cascade of alternative post-decoding processes intended to be applied to the decoded video sequence.

9. The method according to claim 8, wherein the specific syntax element (alpha_channel_id) further indicates how to cascade decoding post-processing intended to be applied to the decoded video picture.

10. The method according to claim 8 or 9, wherein when equal to a specific value, the specific syntax element (alpha_channel_id) further indicates that parameters of the alpha channel information SEI message not involved in transparency adjustment of the pixel are not written into the bitstream or not read from the bitstream.

11. The method according to one of claims 1 to 4, wherein the alpha channel indicator is a specific value of an existing syntax element (alpha_channel_use_idc) of the alpha channel information SEI message, the specific value indicating that the alpha channel information is intended to be used for transparency adjustment of pixels.

12. The method of one of claims 1 to 4, wherein the alpha channel indicator is a specific type of auxiliary picture signaled in a SEI message.

13. A bitstream formatted to include the encoded data obtained from the method of claim 1 and an alpha channel indicator.

14. An apparatus comprising means for carrying out one of the methods according to any one of claims 1 to 12.

15. A computer program product comprising instructions which, when the program is executed by one or more processors, cause the one or more processors to perform the method of any one of claims 1 to 12.

16. A non-transitory storage medium carrying program code instructions for executing the method of any one of claims 1 to 12.

17. An electronic device comprising: processor; and a memory for storing instructions executable by the processor; The processor is configured to implement the method of any one of claims 1 to 12.

Images