Video decoding method and device, computer equipment and storage medium

Abstract

The invention relates to a video decoding method and device, computer equipment, a storage medium and a computer program product. The method comprises the following steps: performing bit stream filtering processing on video coding data with transparency based on a hierarchical identifier of the video coding data to obtain a base layer code stream and an alpha layer code stream; respectively decoding the base layer code stream and the alpha layer code stream through a decoder of the base layer code stream to obtain a base layer image stream and an alpha layer image stream; and performing image synthesis processing on the base layer image stream and the alpha layer image stream to obtain a video image stream with transparency. By adopting the method, the decoder of the base layer code stream can be directly adopted to decode the video coding data with transparency, so that the transparent video can be effectively decoded on the basis of not introducing the decoder for the video with transparency.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a video decoding method, apparatus, computer equipment, storage medium and computer program product. Background Art

[0002] With the development of computer technology and video technology, transparent video technology has emerged, that is, videos with transparency. For example, the H.265 encoding standard supports videos with transparency. Videos with transparency have a wide range of applications, such as making animation effects, video background replacement, etc. Videos with transparency can be realized using PNG image sequences.

[0003] However, the encoding and decoding of transparent videos requires new encoders and decoders to support it. Currently, most decoders do not support video encoding formats with transparency, cannot decode, or lose transparency information during decoding. Summary of the invention

[0004] Based on this, it is necessary to provide a video method, apparatus, computer device, computer-readable storage medium and computer program product that can effectively implement video decoding with transparency in response to the above technical problems.

[0005] In a first aspect, the present application provides a video decoding method. The method comprises:

[0006] Performing bitstream filtering processing on the video coding data with transparency based on the layer identifier of the video coding data to obtain a base layer code stream and an alpha layer code stream, wherein the layer identifier of the alpha layer code stream is rewritten as the layer identifier of the base layer code stream;

[0007] Decoding the base layer code stream and the alpha layer code stream respectively by a base layer code stream decoder to obtain a base layer image stream and an alpha layer image stream;

[0008] The base layer image stream and the alpha layer image stream are subjected to image synthesis processing to obtain a video image stream with transparency.

[0009] In a second aspect, the present application further provides a video decoding device. The device comprises:

[0010] A bitstream filtering module, configured to perform bitstream filtering processing on the video coding data with transparency based on a layer identifier of the video coding data to obtain a base layer bitstream and an alpha layer bitstream, wherein the layer identifier of the alpha layer bitstream is rewritten as the layer identifier of the base layer bitstream;

[0011] A video decoding module, used for respectively decoding the base layer code stream and the alpha layer code stream through a base layer code stream decoder to obtain a base layer image stream and an alpha layer image stream;

[0012] The image synthesis module is used to perform image synthesis processing on the base layer image stream and the alpha layer image stream to obtain a video image stream with transparency.

[0013] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0014] Performing bitstream filtering processing on the video coding data with transparency based on the layer identifier of the video coding data to obtain a base layer code stream and an alpha layer code stream, wherein the layer identifier of the alpha layer code stream is rewritten as the layer identifier of the base layer code stream;

[0015] Decoding the base layer code stream and the alpha layer code stream respectively by a base layer code stream decoder to obtain a base layer image stream and an alpha layer image stream;

[0016] The base layer image stream and the alpha layer image stream are subjected to image synthesis processing to obtain a video image stream with transparency. In a fourth aspect, the present application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0017] Performing bitstream filtering processing on the video coding data with transparency based on the layer identifier of the video coding data to obtain a base layer code stream and an alpha layer code stream, wherein the layer identifier of the alpha layer code stream is rewritten as the layer identifier of the base layer code stream;

[0018] Decoding the base layer code stream and the alpha layer code stream respectively by a base layer code stream decoder to obtain a base layer image stream and an alpha layer image stream;

[0019] The base layer image stream and the alpha layer image stream are subjected to image synthesis processing to obtain a video image stream with transparency. In a fifth aspect, the present application also provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0020] Performing bitstream filtering processing on the video coding data with transparency based on the layer identifier of the video coding data to obtain a base layer code stream and an alpha layer code stream, wherein the layer identifier of the alpha layer code stream is rewritten as the layer identifier of the base layer code stream;

[0021] Decoding the base layer code stream and the alpha layer code stream respectively by a base layer code stream decoder to obtain a base layer image stream and an alpha layer image stream;

[0022] The base layer image stream and the alpha layer image stream are subjected to image synthesis processing to obtain a video image stream with transparency.

[0023] The above-mentioned video decoding method, apparatus, computer equipment, storage medium and computer program product perform bitstream filtering on the video coding data with transparency based on the layer identifier of the video coding data to obtain the base layer code stream and the alpha layer code stream, and the layer identifier of the alpha layer code stream is rewritten as the layer identifier of the base layer code stream; that is, the video coding data is split into the base layer code stream and the alpha layer code stream that can be directly decoded by the decoder of the base layer code stream through bitstream filtering, and the layer identifier of the alpha layer code stream is rewritten so that the decoder of the base layer code stream can also directly decode the alpha layer code stream. Therefore, the base layer code stream and the alpha layer code stream can be decoded by the decoder of the base layer code stream respectively to obtain the corresponding base layer image stream and alpha layer image stream; finally, the base layer image stream and the alpha layer image stream are subjected to image synthesis processing to obtain a decoded video image stream with transparency. In the present application, the bitstream of the video coding data with transparency is filtered based on the layered identifier of the video coding data to obtain the base layer code stream and the alpha layer code stream, and the layered identifier of the alpha layer code stream is rewritten to decode the video and synthesize the video image to obtain a video with transparency. The decoder of the base layer code stream can be directly used to complete the decoding of the video coding data with transparency, thereby effectively realizing the decoding of the transparent video without introducing a decoder for the video with transparency. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 A diagram showing an application environment of a video decoding method in an embodiment;

[0025] Figure 2 is a schematic flow chart of a video decoding method in one embodiment;

[0026] Figure 3 A schematic diagram of the structure of a network abstraction layer unit header in one embodiment;

[0027] Figure 4 is an H.265 video coding data structure with transparency in one embodiment;

[0028] Figure 5 A schematic diagram of a filtering and splicing process of a bitstream filtering process in one embodiment;

[0029] Figure 6 1 is a schematic diagram of a process flow of a bitstream filtering process in an embodiment;

[0030] Figure 7 A schematic diagram of a background synthesis process of a video image stream with transparency in one embodiment;

[0031] Figure 8 is a schematic flow chart of a video decoding method in one embodiment;

[0032] Fig. 9 is a structural block diagram of a video decoding device in one embodiment;

[0033] Fig.10 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0034] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0035] The video decoding method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or it can be placed on the cloud or other servers. When the user needs to decode the existing video with transparency, if there is no decoder for the video with transparency, but only an ordinary video decoder, the video decoding method of the present application can be used to complete the decoding of the transparent video. First, the user submits the video encoding data to the server 104 through the terminal 102, and the server 104 performs bitstream filtering on the video encoding data with transparency based on the layered identifier of the video encoding data to obtain the base layer code stream and the alpha layer code stream, and the layered identifier of the alpha layer code stream is rewritten as the layered identifier of the base layer code stream; the base layer code stream and the alpha layer code stream are decoded by the decoder of the base layer code stream respectively to obtain the base layer image stream and the alpha layer image stream; the base layer image stream and the alpha layer image stream are subjected to image synthesis processing to obtain a video image stream with transparency. The terminal 102 may be, but is not limited to, various desktop computers, laptop computers, smart phones, tablet computers, IoT devices, and portable wearable devices. The IoT devices may be smart speakers, smart TVs, smart air conditioners, smart car-mounted devices, etc. The portable wearable devices may be smart watches, smart bracelets, head-mounted devices, etc. The server 104 may be implemented as an independent server or a server cluster consisting of multiple servers.

[0036] In one embodiment, Figure 2 As shown, a video decoding method is provided, which is applied to Figure 1 The server 104 in the example is used for explanation. Of course, the video decoding method of the present application can also be applied to a terminal. When it is applied to the server 104, it specifically includes the following steps:

[0037] Step 201, based on the layer identifier of the video coding data, bit stream filtering is performed on the video coding data with transparency to obtain a base layer code stream and an alpha layer code stream, and the layer identifier of the alpha layer code stream is rewritten as the layer identifier of the base layer code stream.

[0038] Among them, video coding data refers to the data obtained through video coding. Because the amount of video data collected directly is large, it is difficult to realize actual storage and transmission of these video data without compression coding. Therefore, the video coding standard can generally be used to compress the collected video in a specific video coding format. The layered identifier (nuh_layer_id) refers to the identifier of layered coding in video coding data with transparency. In a specific embodiment, the video coding data is specifically video coding data in H.265-HEVC (High Efficiency Video Coding) format, and its bit stream is composed of network abstraction layer (NAL) units. The first two bytes of the network abstraction layer unit are the unit header, followed by the payload data. The unit header structure can be specifically referred to. Figure 3As shown, it includes F: the first bit, which is always 0; nal_unit_type: network abstraction layer unit type; nuh_layer_id: layer identifier; nuh_temp_id_plus1: temporal layer identifier. These four parts, if the layer identifier = 0 represents the base layer, and if it is greater than 0, it represents the enhancement layer. For transparent videos in H.265 format, if its layer identifier is greater than 0 (usually selected to be equal to 1), it is the transparent (alpha) layer of the video, which represents the transparency information of different pixel positions. Video encoding data with transparency refers to video encoding data with background transparency information attached to the video encoding data. Video with transparency can be used in the production of animation special effects, video background replacement and other fields. Bitstream filtering refers to filtering the bit stream of video encoding data, thereby separating the streams of different layer identifiers to obtain the base layer stream and the alpha layer stream. Bitstream refers to the 0 / 1 bit data stream after the video is compressed and encoded. Bitstream filtering processing for video coding data can be achieved through specific bitstream filters. For example, for video coding data in H.265-HEVC format, the base layer codestream refers to the codestream data composed of bitstream data with a layer identifier equal to 0, while the alpha layer codestream refers to the codestream data composed of bitstream data with a layer identifier greater than 0.

[0039] Specifically, the scheme of the present application is mainly applicable to the decoding process of video coding data with transparency through the existing video decoder without transparency. Therefore, when the user of the terminal 102 submits the video coding data with transparency, the server 104 can perform bitstream filtering on the video coding data with transparency based on the layer identifier of the video coding data. The filtering process is specifically implemented based on the bitstream filter carried by the server 104. The bitstream filter can identify the layer identifier of each network abstraction layer unit in the video coding data, and then divide the video coding data based on the layer identifier, that is, the bitstream filtering process, and obtain the base layer code stream and alpha layer code stream after the diversion. After the alpha layer code stream is obtained by the bitstream filtering process, in order to effectively decode the alpha layer code stream, the layer identifier of the alpha layer code stream can also be rewritten, that is, the layer identifier of the alpha layer code stream is rewritten as the layer identifier of the base layer code stream. For example, for video coding data in H.265-HEVC format, because the existing H.265 decoder device that does not support transparent video decoding cannot correctly process the code stream with a layer identifier greater than 0. After the layer identifier is rewritten, the output alpha layer code stream is completely consistent with the ordinary non-transparent base layer code stream, and old devices can process and decode it normally.

[0040] Step 203: The base layer code stream and the alpha layer code stream are decoded by a base layer code stream decoder to obtain a base layer image stream and an alpha layer image stream.

[0041] The decoder of the base layer code stream refers to a decoder device used to decode the existing video encoding data without transparency (i.e., the base layer code stream), wherein the decoder is a program (video player) or device used to restore the decoding operation of the encoded digital video. The decoding process is the reverse process of the encoding process, which refers to the process of decompressing the compressed video encoding data into a video image. The video image generated by decoding the base layer code stream is the base layer image stream, and the video image generated by decoding the alpha layer code stream is the alpha layer image stream.

[0042] Specifically, in the scheme of the present application, after the video coding data is subjected to bitstream filtering processing to obtain the base layer codestream and the alpha layer codestream, the base layer codestream and the alpha layer codestream can be decoded and processed respectively by the existing base layer codestream decoder. Since the decoder for the base layer codestream can be used to decode the base layer codestream, the base layer image stream corresponding to the base layer codestream can be directly obtained. As for the alpha layer codestream, since the layer identifier of the alpha layer codestream is rewritten as the layer identifier of the base layer codestream during the bitstream filtering process, the decoder of the base layer codestream can also directly decode the rewritten alpha layer codestream to obtain the alpha image stream corresponding to the alpha codestream. For example, for video coding data in H.265 format, it includes a format that supports transparency coding. However, the existing H.265 video decoding device may not be able to decode H.265 video coding data for transparency, that is, it cannot correctly process the alpha layer codestream whose layer identifier is greater than zero. But after the rewrite, since the output alpha layer code stream is exactly the same as the ordinary non-transparent H.265 base layer code stream, these old devices can also decode the alpha layer code stream normally to obtain the alpha layer image stream.

[0043] Step 205: Perform image synthesis processing on the base layer image stream and the alpha layer image stream to obtain a video image stream with transparency.

[0044] The image synthesis process refers to adding transparency information to the base layer image stream based on the alpha layer image stream, thereby obtaining a video image stream with transparency.

[0045] Specifically, after obtaining the base layer image stream and the alpha layer image stream, wherein the base layer image stream contains the color data of the original video coding data, and the alpha layer image stream contains the transparency data of the original video coding data, in order to obtain a video image stream with transparency, the base layer image stream and the alpha layer image stream can be subjected to image synthesis fusion processing, so that the transparency system information of the alpha layer can be added on the basis of the original base layer image stream. When performing image fusion in a specific embodiment, the transparent channel related data of the video coding data can be obtained through the alpha layer image stream, and then the transparent channel data contained in the alpha layer image stream is fused with the color related channel data originally contained in the base layer image stream to obtain image information containing transparency, and then obtain a video image stream with transparency.

[0046] The above-mentioned video decoding method performs bitstream filtering on the video coding data with transparency based on the layer identifier of the video coding data to obtain the base layer code stream and the alpha layer code stream, and the layer identifier of the alpha layer code stream is rewritten as the layer identifier of the base layer code stream; that is, the video coding data is split into the base layer code stream and the alpha layer code stream that can be directly decoded by the decoder of the base layer code stream through bitstream filtering, and the layer identifier of the alpha layer code stream is rewritten so that the decoder of the base layer code stream can also directly decode the alpha layer code stream. Therefore, the base layer code stream and the alpha layer code stream can be decoded by the decoder of the base layer code stream respectively to obtain the corresponding base layer image stream and alpha layer image stream; finally, the base layer image stream and the alpha layer image stream are subjected to image synthesis processing to obtain a decoded video image stream with transparency. In the present application, the bitstream of the video coding data with transparency is filtered based on the layered identifier of the video coding data to obtain the base layer code stream and the alpha layer code stream, and the layered identifier of the alpha layer code stream is rewritten to decode the video and synthesize the video image to obtain a video with transparency. The decoder of the base layer code stream can be directly used to complete the decoding of the video coding data with transparency, thereby effectively realizing the decoding of the transparent video without introducing a decoder for the video with transparency.

[0047] In one embodiment, step 201 includes: identifying a network abstraction layer unit of video coding data with transparency; determining a unit type and a layer identifier of the network abstraction layer unit; and performing bitstream filtering on the video coding data based on the unit type and the layer identifier of the network abstraction layer unit to obtain a base layer code stream and an alpha layer code stream.

[0048] Among them, the network abstraction layer specifically includes two categories, one is the video coding layer (VCL), which carries the actual coded data; the other is the non-video coding layer (non-VCL), which contains coding parameter information such as video width and height, as well as user-defined data. For example, for H.265 format video coding data, the unit of the video coding layer is the VCL unit, while the typical non-VCL unit includes the video parameter set (VPS), sequence parameter set (SPS), and picture parameter set (PPS). The video parameter set is mainly used to transmit video classification information, which is conducive to the extension of compatible standards in scalable video coding or multi-view video. The sequence parameter set contains the shared coding parameters of all coded images in a coded video sequence (CVS). The picture parameter set contains the common parameters used for an image, that is, all slices (SS) in an image refer to the same PPS. These three network abstraction layer units contain coding configuration information. If they are lost, the decoder cannot decode. The layer identifier is specifically used to indicate whether the current network abstraction layer unit belongs to the base layer or the transparent layer. In one embodiment, the H.265 video coding data structure with transparency can refer to Figure 4 As shown, it starts with the VPS unit, followed by three network abstraction layer units: SPS, PPS, and VCL. The layer identifier is in brackets after the unit. For H.265 video encoding data with transparency, a layer identifier of 0 indicates that the network abstraction layer unit belongs to the base layer, and a layer identifier of 1 indicates that the network abstraction layer unit belongs to the alpha layer.

[0049] Specifically, when filtering the bitstream of the video coding data, it is necessary to obtain two bitstream data, the base layer bitstream and the alpha layer bitstream, through filtering and diversion. Both the base layer bitstream and the alpha layer bitstream require specific network abstraction layer units as the beginning of decoding. For example, for the video coding data in H.265 format, its bitstream needs to start with the video parameter set as the beginning of the bitstream, followed by other network abstraction layer units. Therefore, when filtering and diverting the video coding data with transparency, it is necessary to first identify the unit type and layer identifier of the network abstraction layer unit of the video coding data with transparency. The beginning part of the bitstream can be determined by the unit type of the network abstraction layer unit, and the layer identifier of the network abstraction layer unit can be used to determine whether the network abstraction layer unit should be diverted to the base layer bitstream or the alpha layer bitstream. After all the network abstraction layer units in the video coding data are filtered and diverted to the base layer bitstream and the alpha layer bitstream, it can be determined that the filtering process is completed and the final complete base layer bitstream and alpha layer bitstream are obtained. The specific filtering process of the bitstream can be implemented by a pre-built bitstream filter, which can identify the unit type and layer identifier of the network abstraction layer unit, and then implement the shunting filtering process of the network abstraction layer unit. In this embodiment, the bitstream filtering process is performed based on the unit type and layer identifier of the network abstraction layer unit, which can effectively ensure the accuracy of the bitstream filtering process, thereby ensuring the processing effect of the video filtering and video decoding process.

[0050] In one embodiment, the network abstraction layer unit includes a first type of network abstraction layer unit whose unit type is a video parameter set, and a second type of network abstraction layer unit whose unit type is not a video parameter set. Based on the unit type and layer identifier of the network abstraction layer unit, bitstream filtering processing is performed on video coding data to obtain a base layer code stream and an alpha layer code stream, including: when the network abstraction layer unit is a first type of network abstraction layer unit, copying the network abstraction layer unit of the video parameter set to obtain a code stream header; when the network abstraction layer unit is a second type of network abstraction layer unit, classifying the network abstraction layer unit that is not a video parameter set based on the layer identifier of the network abstraction layer unit to obtain a classification result; based on the classification result, combining the code stream header and the second type of network abstraction layer unit to obtain the base layer code stream and the alpha layer code stream.

[0051] Specifically, the unit types of the network abstraction layer unit include video coding layer units, and video parameter set units, sequence parameter units and image parameter set units under non-video coding layer units. When the unit type of the network abstraction layer unit is a video parameter set, it means that the unit belongs to the code stream header. For the filtering processing of video coding data with transparency, the complete video coding data can be split into a base layer code stream and an alpha layer code stream. Therefore, two corresponding code stream headers are required to serve as the beginning of the base layer code stream and the alpha layer code stream respectively, and the network abstraction layer unit of the video parameter set needs to be copied to obtain the code stream header. At the same time, the network abstraction layer unit is classified to determine the first type of network abstraction layer unit whose unit type is a video parameter set, and the second type of network abstraction layer unit whose unit type is not a video parameter set. When the unit type of the network abstraction layer unit is not a video parameter set, that is, the network abstraction layer unit is a video coding layer unit, and there is a sequence parameter unit or an image parameter set unit in a non-video coding layer unit, it means that the network abstraction layer unit outputs the subsequent data of the code stream header. In this way, the network abstraction layer units in the video coding data can be divided into code stream headers and non-code stream headers, and then the network abstraction layer units that are not video parameter sets are classified based on the layer identifier of the network abstraction layer unit to determine whether the current network abstraction layer unit belongs to the base layer code stream and the alpha layer code stream. If the classification result indicates that the network abstraction layer unit belongs to the base layer code stream, the second type of network abstraction layer unit belonging to the base layer code stream is combined with the first code stream header, and when the classification result indicates that the network abstraction layer unit belongs to the alpha layer code stream, the second type of network abstraction layer unit belonging to the alpha layer code stream is combined with the second code stream header. Combining with the code stream header here means splicing one by one to the network abstraction layer units following the header in sequence. At the same time, in order to decode the alpha layer code stream by the existing decoder for decoding the base layer code stream, the layer identifier of the second type of network abstraction layer unit belonging to the alpha layer code stream can be rewritten as the layer identifier of the base layer code stream; and then the network abstraction layer unit after the layer identifier is rewritten is combined with the code stream header of the alpha layer code stream. By rewriting, the layer identifiers of the network abstraction layer units of the alpha layer code stream are synchronized with the base layer code stream, so the alpha layer code stream after the layer identifier is rewritten can be decoded by the decoder of the base layer code stream. Figure 5As shown, for H.265 format video coding data with transparency, the network abstraction layer units in the coding data are VPS, SPS(0), PPS(0), SPS(1), PPS(1), VCL(0), VCL(1), VCL(0), VCL(1)... in sequence. By splitting the stream, the base layer code stream can be obtained as VPS, SPS(0), PPS(0), VCL(0), VCL(0). By splitting the stream and rewriting the layer identifier, the alpha layer code stream can be obtained as VPS, SPS(0), PPS(0), VCL(0), VCL(0). For the bitstream filtering process, please refer to Figure 6 As shown, first extract and check the unit type of the network abstraction layer unit from the bit stream of the input video coding data. If the unit type of the network abstraction layer unit is VPS, make an additional copy (total two copies). If the network abstraction layer unit is not VPS, check the layer identifier. If the layer identifier is equal to 0, determine that the network abstraction layer unit belongs to the base layer, combine it with the VPS (VPS in front, other units in the back), and output it as the base layer code stream; if the layer identifier is greater than 0, determine that the network abstraction layer unit belongs to the alpha transparency layer, and rewrite the layer identifier of the network abstraction layer unit to 0. Combine the rewritten alpha layer network abstraction layer unit with the copied VPS and output it as the alpha layer code stream; if the code stream data is not finished, jump to the initial judgment step to continue processing until all code streams of the video coding data are processed. In this embodiment, the classification and rewriting of the network abstraction layer unit are implemented through the unit type of the network abstraction layer unit, which can effectively rewrite the alpha layer code stream that cannot be directly decoded into a decodable basic layer code stream structure, thereby realizing the decoding processing of the alpha layer code stream, and can effectively realize the decoding processing of video encoding data with transparency, thereby ensuring the efficiency and accuracy of video decoding processing.

[0052] In one embodiment, step 205 includes: extracting a brightness channel image stream of the alpha layer image stream; and synthesizing the brightness channel image stream of the alpha layer image stream with the base layer image stream to obtain a video image stream with transparency.

[0053] Among them, the brightness channel image stream refers to the Y channel data in the YUV color encoding. YUV is a color encoding method. It is often used in various video processing components. When encoding photos or videos, YUV allows the bandwidth of chrominance to be reduced, taking into account human perception. "Y" represents brightness (Luminance or Luma), which is the grayscale value, and "U" and "V" represent chrominance (Chrominance or Chroma), which is used to describe the color and saturation of the image and is used to specify the color of the pixel.

[0054] Specifically, the base layer code stream is mainly used to describe the color information of the original video coding data, so the base layer image stream is an image of only the color layer, lacking the transparency-related information of the video. As for the alpha layer image stream, it mainly contains the transparency-related information of the original video. Therefore, if you want to obtain a video image stream with transparency, you need to perform image synthesis processing on the base layer image stream and the alpha layer image stream. The specific method of image processing can be to add transparency information on the basis of the base layer image stream. For the alpha layer image stream under YUV encoding, the transparency information of the video can generally be stored in the brightness channel, that is, the Y channel. Therefore, when synthesizing the image, the brightness channel image stream of the alpha layer image stream can be directly extracted, and the U channel and the V channel are discarded. Then, the brightness channel image stream of the alpha layer image stream is synthesized with the base layer image stream to obtain a video image stream with transparency. As for the specific process of synthesizing the brightness channel image stream of the alpha layer image stream with the base layer image stream, the brightness channel image stream of the alpha layer image stream is synthesized as the alpha channel image stream with the base layer image stream to obtain a video image stream with transparency. That is, the brightness channel image stream of the alpha layer image stream is used as a new channel, namely, the alpha channel image stream, and then the alpha channel image stream is synthesized with the base layer image stream to obtain a video image stream with transparency. The obtained video image stream contains a YUVA channel, wherein the A channel is the alpha channel, and information related to transparency is stored. In this embodiment, by synthesizing the brightness channel image stream of the alpha layer image stream as the alpha channel image stream with the base layer image stream to obtain a video image stream with transparency, it can be effectively ensured that the video obtained after the base layer image stream and the alpha layer image stream are synthesized can retain the relevant information of transparency, thereby ensuring the availability of the video with transparency.

[0055] In one embodiment, the method further includes: performing node labeling processing on the video image stream with transparency based on the appearance period of the target object in the video image stream with transparency to obtain labeled nodes; and extracting the target segment from the video image stream based on the labeled nodes to obtain video special effects material containing the target object.

[0056] The target object may be the material content used to produce video special effects, and the target object may specifically be a person or object that appears in the image.

[0057] Specifically, the video content with transparency decoded in the present application can be used as the material content of video special effects, and when extracting the material content, it can be cut and extracted based on the target object appearing in the image. Therefore, the video image stream with transparency can be firstly marked with nodes based on the appearance period of the target object in the video image stream with transparency, and the marked nodes corresponding to the appearance time of the target object in the video image stream are marked. Then, since the video special effect only needs the part where the target object appears, the target segment is directly cut out from the video image stream based on the marked node to obtain the video special effect material containing the target object. When the video special effect material needs to be used, the content can be added to the transparent part of the video with transparency to perform special effect processing and obtain the special effect video. In a specific embodiment, for a geographically related video image stream containing forest grassland, it is necessary to extract the part where the trees are shaking as the special effect material. At this time, the time period when the trees appear can be determined in the video image stream, and then this part is marked and extracted as the target segment to obtain the video special effect material containing the target object. In this embodiment, through node marking and segment capture, the required segments can be effectively captured from the video to generate materials for the special effects video, thereby ensuring the generation efficiency and generation effect of the special effects video.

[0058] In one embodiment, the method further includes: receiving a color layer image extraction request; generating a color information image stream based on the base layer image stream, and feeding back the color information image stream.

[0059] Specifically, the color layer image extraction request is used to request the server to extract the color layer image in the video coding data, that is, the base layer image stream obtained by decoding the base layer code stream. Therefore, after obtaining the color layer image extraction request, when the bit stream filtering processing is performed on the video coding data with transparency based on the layer identifier of the video coding data, only the base layer code stream is retained and the alpha layer code stream is discarded, and then the base layer code stream is directly decoded by the decoder of the base layer code stream to obtain the base layer code stream, and finally a color information image stream is generated based on the base layer image stream, and the color information image stream is fed back. In addition, after the base layer code stream and the alpha layer code stream are respectively decoded by the decoder of the base layer code stream to obtain the base layer image stream and the alpha layer image stream, the alpha layer image stream is discarded, and only the base layer image stream is retained, and then a color information image stream is generated based on the base layer image stream, and the color information image stream is fed back. In addition, in another embodiment, when the color information of the base layer code stream is not needed, that is, when the transparent layer image extraction request is received; a transparency information image stream can be generated based on the alpha layer image stream, and the transparency information image stream can be fed back. In this embodiment, when the alpha layer transparency information is needed, the alpha layer code stream output by the bitstream filtering can be discarded; therefore, the designed video decoding process only needs to be combined with one decoder instance, which can easily realize the decoding of the base layer color image, avoid decoding the alpha layer image, reduce hardware resource overhead, and save power consumption. Conversely, the base layer code stream output by the bitstream filtering can be discarded, and only the alpha layer code stream can be decoded and output, which is used in special business scenarios that only require information related to video transparency.

[0060] In one embodiment, the method further includes: obtaining a background video corresponding to the video encoding data; and performing a superposition process on the video image stream with transparency and the background video to generate a target video.

[0061] Specifically, in the production process of transparent videos, technologies such as artificial intelligence are generally used to remove the background from the original video material, retain the target foreground (such as a portrait), and save it as a transparent video. That is, the video encoding data with transparency is mainly used to make the background part of the video transparent while retaining the foreground video. Transparent videos can be freely combined with different background videos and superimposed on the background video to form a new video. Therefore, in the process of making videos, after decoding and synthesizing a video image stream with transparency, the video image stream with transparency can be merged with the background video, and the target video with background is obtained by superimposing the video image stream with transparency and the background video. In one embodiment, the overall process is as follows: Figure 7As shown, when the video material is shot and the original video is obtained, the video encoding process generates a transparent video in H.265 format by removing the background, and then in the decoding process, different types of backgrounds can be superimposed on the transparent video in H.265 format to obtain videos with different backgrounds, and the video with the replaced background can be directly rendered and output, or compressed and encoded again to be saved as a video file. In this embodiment, by superimposing the corresponding background video on the basis of the video image stream with transparency to generate the target video after replacing the background, a background replacement video that meets the requirements can be effectively generated on the basis of the transparent video, and the processing efficiency of the video generation process can be guaranteed.

[0062] The present application also provides an application scenario, and the application scenario applies the above-mentioned video decoding method. Specifically, the application of the video decoding method in the application scenario is as follows:

[0063] When a user wants to play a transparent video in H.265 format based on an existing H.265 video decoder that is not adapted to transparency, the existing video decoding device that does not support transparent video decoding cannot correctly process the code stream with a layer identifier greater than 0. Therefore, the video decoding method of the present application can be used with the help of an existing video decoding device that does not support transparent video decoding to achieve decoding processing of transparent videos in H.265 format.

[0064] First, the user provides the video encoding data of the H.265 format transparent video to the server equipped with the video decoding method of the present application, and the server performs a decoding operation based on the video encoding data provided by the user to obtain playable video data. The overall process can be referred to Figure 8As shown, the server will input the input H.265 format transparent video into the bitstream filter for filtering. During the bitstream filtering process, the bitstream filter can identify the unit type and layer identifier of the network abstraction layer unit of the video coding data with transparency, and perform bitstream filtering on the video coding data based on the unit type and layer identifier of the network abstraction layer unit to obtain the base layer code stream and the alpha layer code stream. The network abstraction layer unit includes a first type of network abstraction layer unit whose unit type is a video parameter set, and a second type of network abstraction layer unit whose unit type is not a video parameter set. Specifically, the filtering process is: when the network abstraction layer unit is a first type of network abstraction layer unit, the network abstraction layer unit of the video parameter set is copied to obtain a code stream header; when the network abstraction layer unit is a second type of network abstraction layer unit, the network abstraction layer unit that is not a video parameter set is classified based on the layer identifier of the network abstraction layer unit to obtain a classification result; based on the classification result, the code stream header and the second type of network abstraction layer unit are combined to obtain the base layer code stream and the alpha layer code stream. At the same time, when the classification result indicates that the network abstraction layer unit belongs to the base layer code stream, the second type of network abstraction layer unit belonging to the base layer code stream is combined with the first code stream header; when the classification result indicates that the network abstraction layer unit belongs to the alpha layer code stream, the layer identifier of the second type of network abstraction layer unit belonging to the alpha layer code stream is rewritten as the layer identifier of the base layer code stream; the second type of network abstraction layer unit after the layer identifier is rewritten is combined with the code stream header of the alpha layer code stream. After the filtering is completed, the base layer code stream and the alpha layer code stream are decoded by the decoder 1 and decoder 2 of the base layer code stream respectively to obtain the base layer image stream and the alpha layer image stream; and the base layer image stream and the alpha layer image stream are subjected to image synthesis processing to obtain a video image stream with transparency.

[0065] As for the specific synthesis process, the brightness channel image stream of the alpha layer image stream can be extracted; the brightness channel image stream of the alpha layer image stream is used as the alpha channel image stream and synthesized with the base layer image stream to obtain a video image stream with transparency. When the video image stream with transparency is parsed, the user can extract video materials from it, that is, based on the appearance period of the target object in the video image stream with transparency, the node annotation processing is performed on the video image stream with transparency to obtain the annotation node; based on the annotation node, the target segment is cut out from the video image stream to obtain the video special effects material containing the target object. In addition, the target video with background can be synthesized based on the existing background video, that is, the background video corresponding to the video encoding data is obtained; and the video image stream with transparency and the background video are superimposed to generate the target video.

[0066] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0067] Based on the same inventive concept, the embodiment of the present application also provides a video decoding device for implementing the above-mentioned video decoding method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above-mentioned method, so the specific limitations in one or more video decoding device embodiments provided below can refer to the limitations on the video decoding method above, and will not be repeated here.

[0068] In one embodiment, Fig. 9 As shown, a video decoding device is provided, comprising:

[0069] A bitstream filtering module 902 is used to perform bitstream filtering processing on the video coding data with transparency based on the layer identifier of the video coding data to obtain a base layer bitstream and an alpha layer bitstream, and the layer identifier of the alpha layer bitstream is rewritten as the layer identifier of the base layer bitstream;

[0070] The video decoding module 904 is used to decode the base layer code stream and the alpha layer code stream respectively through the base layer code stream decoder to obtain the base layer image stream and the alpha layer image stream;

[0071] The image synthesis module 906 is used to perform image synthesis processing on the base layer image stream and the alpha layer image stream to obtain a video image stream with transparency.

[0072] In one embodiment, the bitstream filtering module 902 is specifically used to: identify the network abstraction layer unit of the video coding data with transparency; determine the unit type and layer identifier of the network abstraction layer unit; and perform bitstream filtering on the video coding data based on the unit type and layer identifier of the network abstraction layer unit to obtain the base layer bitstream and the alpha layer bitstream.

[0073] In one embodiment, the network abstraction layer unit includes a first type of network abstraction layer unit whose unit type is a video parameter set, and a second type of network abstraction layer unit whose unit type is not a video parameter set. The code stream filtering module 902 is specifically used to: when the network abstraction layer unit is a first type of network abstraction layer unit, copy the network abstraction layer unit of the video parameter set to obtain a code stream header; when the network abstraction layer unit is a second type of network abstraction layer unit, classify the network abstraction layer unit that is not a video parameter set based on the hierarchical identifier of the network abstraction layer unit to obtain a classification result; based on the classification result, combine the code stream header and the second type of network abstraction layer unit to obtain a base layer code stream and an alpha layer code stream.

[0074] In one embodiment, the code stream header includes a first code stream header corresponding to the base layer code stream and a second code stream header corresponding to the alpha layer code stream, and the code stream filtering module 902 is specifically used to: when the classification result indicates that the network abstraction layer unit belongs to the base layer code stream, combine the second type of network abstraction layer unit belonging to the base layer code stream with the first code stream header; when the classification result indicates that the network abstraction layer unit belongs to the alpha layer code stream, combine the second type of network abstraction layer unit belonging to the alpha layer code stream with the second code stream header.

[0075] In one embodiment, the code stream filtering module 902 is specifically used to: rewrite the layer identifier of the second type of network abstraction layer unit belonging to the alpha layer code stream into the layer identifier of the base layer code stream; and combine the second type of network abstraction layer unit after the layer identifier is rewritten with the code stream header of the alpha layer code stream.

[0076] In one embodiment, the image synthesis module 906 is specifically used to: perform image synthesis processing on the base layer image stream and the alpha layer image stream to obtain a video image stream with transparency, including: extracting the brightness channel image stream of the alpha layer image stream; synthesizing the brightness channel image stream of the alpha layer image stream with the base layer image stream to obtain a video image stream with transparency.

[0077] In one embodiment, the image synthesis module 906 is specifically used to: use the brightness channel image stream of the alpha layer image stream as the alpha channel image stream to perform synthesis processing with the base layer image stream to obtain a video image stream with transparency.

[0078] In one embodiment, it also includes a material extraction module, which is used to: perform node annotation processing on the video image stream with transparency based on the appearance period of the target object in the video image stream with transparency to obtain annotated nodes; based on the annotated nodes, cut out the target segment from the video image stream to obtain the video special effects material containing the target object.

[0079] In one embodiment, it also includes a color information extraction module, which is used to: receive a color layer image extraction request; generate a color information image stream based on the base layer image stream, and feed back the color information image stream.

[0080] In one embodiment, it also includes a transparency information extraction module, which is used to: receive a transparency layer image extraction request; generate a transparency information image stream based on the alpha layer image stream, and feed back the transparency information image stream.

[0081] In one embodiment, it also includes a background synthesis module, which is used to: obtain the background video corresponding to the video encoding data; and superimpose the video image stream with transparency and the background video to generate a target video.

[0082] Each module in the above video decoding device can be implemented in whole or in part by software, hardware or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0083] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Fig.10 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store video decoding data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a video decoding method is implemented.

[0084] Those skilled in the art will understand that Fig.10 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0085] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0086] In one embodiment, a computer-readable storage medium is provided, storing a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0087] In one embodiment, a computer program product or computer program is provided, the computer program product or computer program includes computer instructions, the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps in the above-mentioned method embodiments.

[0088] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.

[0089] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0090] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0091] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A video decoding method, characterized in that: The method comprises: Performing bitstream filtering processing on the video coding data with transparency based on the layer identifier of the video coding data to obtain a base layer code stream and an alpha layer code stream, wherein the layer identifier of the alpha layer code stream is rewritten as the layer identifier of the base layer code stream; Decoding the base layer code stream and the alpha layer code stream respectively by a base layer code stream decoder to obtain a base layer image stream and an alpha layer image stream; The base layer image stream and the alpha layer image stream are subjected to image synthesis processing to obtain a video image stream with transparency.

2. The method according to claim 1, characterized in that The bitstream filtering process of the video coding data with transparency based on the layered identifier of the video coding data to obtain the base layer code stream and the alpha layer code stream includes: identifying a network abstraction layer unit of the video coding data with transparency; determining a unit type and a layer identifier of the network abstraction layer unit; The video coding data is subjected to bitstream filtering processing based on the unit type and the layer identifier of the network abstraction layer unit to obtain a base layer bitstream and an alpha layer bitstream.

3. The method according to claim 2, characterized in that The network abstraction layer unit includes a first type of network abstraction layer unit whose unit type is a video parameter set, and a second type of network abstraction layer unit whose unit type is not a video parameter set, and the bitstream filtering processing is performed on the video coding data based on the unit type and the layer identifier of the network abstraction layer unit to obtain the base layer code stream and the alpha layer code stream, including: When the network abstraction layer unit is a first type network abstraction layer unit, copying the network abstraction layer unit of the video parameter set to obtain a code stream header; When the network abstraction layer unit is a second type network abstraction layer unit, classifying the network abstraction layer unit that is not a video parameter set based on the hierarchical identifier of the network abstraction layer unit to obtain a classification result; Based on the classification result, the bitstream header and the second type of network abstraction layer unit are processed in combination to obtain a base layer bitstream and an alpha layer bitstream.

4. The method according to claim 1, characterized in that: The code stream header includes a first code stream header corresponding to the base layer code stream and a second code stream header corresponding to the alpha layer code stream, and the code stream header and the second type of network abstraction layer unit are combined and processed based on the classification result to obtain the base layer code stream and the alpha layer code stream, including: When the classification result indicates that the network abstraction layer unit belongs to the base layer code stream, combining the second type of network abstraction layer unit belonging to the base layer code stream with the first code stream header; When the classification result indicates that the network abstraction layer unit belongs to an alpha layer code stream, a second type of network abstraction layer unit belonging to the alpha layer code stream is combined with the second code stream header.

5. The method according to claim 4, characterized in that The combining of the second type of network abstraction layer unit belonging to the alpha layer code stream with the second code stream header comprises: Rewrite the layer identifier of the second type of network abstraction layer unit belonging to the alpha layer code stream into the layer identifier of the base layer code stream; The second type of network abstraction layer unit after rewriting the layer identifier is combined with the code stream header of the alpha layer code stream.

6. The method according to claim 1, characterized in that The performing image synthesis processing on the base layer image stream and the alpha layer image stream to obtain a video image stream with transparency comprises: Extracting a brightness channel image stream of the alpha layer image stream; The brightness channel image stream of the alpha layer image stream is synthesized with the base layer image stream to obtain a video image stream with transparency.

7. The method according to claim 6, characterized in that The synthesizing the brightness channel image stream of the alpha layer image stream and the base layer image stream to obtain a video image stream with transparency comprises: The brightness channel image stream of the alpha layer image stream is used as the alpha channel image stream to be synthesized with the base layer image stream to obtain a video image stream with transparency.

8. The method according to any one of claims 1 to 7, characterized in that The method further comprises: Based on the appearance period of the target object in the video image stream with transparency, performing node labeling processing on the video image stream with transparency to obtain labeled nodes; A target segment is cut out from the video image stream based on the marked node to obtain a video special effect material containing the target object.

9. The method according to any one of claims 1 to 7, characterized in that: The method further comprises: receiving a color layer image extraction request; A color information image stream is generated based on the base layer image stream, and the color information image stream is fed back.

10. The method according to any one of claims 1 to 7, characterized in that The method further comprises: Receiving a transparent layer image extraction request; A transparency information image stream is generated based on the alpha layer image stream, and the transparency information image stream is fed back.

11. The method according to any one of claims 1 to 7, characterized in that: The method further comprises: Obtaining a background video corresponding to the video encoding data; The video image stream with transparency and the background video are superimposed to generate a target video.

12. A video decoding device, characterized in that: The device comprises: A bitstream filtering module, configured to perform bitstream filtering processing on the video coding data with transparency based on a layer identifier of the video coding data to obtain a base layer bitstream and an alpha layer bitstream, wherein the layer identifier of the alpha layer bitstream is rewritten as the layer identifier of the base layer bitstream; A video decoding module, used for respectively decoding the base layer code stream and the alpha layer code stream through a base layer code stream decoder to obtain a base layer image stream and an alpha layer image stream; The image synthesis module is used to perform image synthesis processing on the base layer image stream and the alpha layer image stream to obtain a video image stream with transparency.

13. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 11 are implemented.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.

15. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.

Images