Video decoding method, video processing device, medium and product

Abstract

Embodiments of the present application provide a video decoding method, a video processing device, a computer-readable storage medium, and a computer program product, the video decoding method including: obtaining reference frame information of a video frame to be decoded; obtaining a reference image list of the video frame to be decoded based on the reference frame information and complementary frame information; decoding the video frame to be decoded based on the reference image list, and obtaining a first reconstructed image and a decoded image of the video frame to be decoded.

Description

[Technical Field]

[0001] [CROSS-REFERENCE TO RELATED APPLICATIONS] This application claims the benefit of priority to a Chinese patent application bearing application number 202211263037.4 and filed on October 14, 2022, the entire contents of which are incorporated herein by reference.

[0002] [Technical field] TECHNICAL FIELD Embodiments of the present application relate to the technical field of video processing, and in particular to a video decoding method, a video processing device, a computer-readable storage medium and a computer program product. [Background technology]

[0003] With the development of neural networks (NNs), their nonlinear fitting capabilities become increasingly stronger. Therefore, video processing techniques based on neural networks are widely applied in video processing, such as video encoding and video decoding.

[0004] In the related art, neural network-based loop filtering is one of the effective means for improving video encoding / decoding performance, and the selection and modification of the neural network filtering mode and the conventional filtering mode are both block-based. The image obtained by the above method may have good overall performance, but may have poor local performance, resulting in local distortion. Reference frame distortion is likely to cause error propagation. Furthermore, in the inter-prediction process, the reference image list of some video frames contains duplicated reference frames, which lacks diversity and results in undesirable inter-prediction results. Therefore, how to further improve video quality in the video processing process is an issue that needs to be urgently discussed and resolved. Summary of the Invention [Problem to be solved by the invention]

[0005] Embodiments of the present application provide a video decoding method, a video processing device, a computer-readable storage medium and a computer program product for improving the image quality of a video. [Means for solving the problem]

[0006] In aspect 1, the present embodiment comprises: Obtaining reference frame information of a video frame to be decoded; obtaining a reference image list for the video frame to be decoded based on the reference frame information and the complementary frame information; decoding a video frame to be decoded based on the reference image list to obtain a first reconstructed image and a decoded image of the video frame to be decoded; A video decoding method is provided.

[0007] In aspect 2, the present embodiment comprises: 1. A video decoding method applied to a video frame including a temporal layer identifier, comprising: performing the video decoding method of any one of aspects 1 on a video frame waiting to be decoded if the temporal layer identifier of the video frame is less than a predetermined threshold. A video decoding method is provided.

[0008] In aspect 3, the present embodiment comprises: a memory; a processor; and a computer program stored in the memory and executable by the processor, the computer program, when executed by the processor, implementing the video decoding method of any one of aspect 1 or aspect 2; Provides video processing equipment.

[0009] In aspect 4, the present embodiment comprises: a memory device storing computer-executable instructions for performing the video decoding method of any one of aspect 1 or aspect 2; A computer-readable storage medium is provided.

[0010] In aspect 5, the present embodiment comprises: a computer program or computer instructions stored in a computer-readable storage medium, wherein a processor of a computing device reads the computer program or the computer instructions from the computer-readable storage medium, and the processor executes the computer program or the computer instructions such that the computing device performs the video decoding method of any one of Aspect 1 or Aspect 2; A computer program product is provided. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a flowchart of video encoding according to the related art. [Figure 2] 1 is a flowchart of video decoding according to the related art; [Figure 3] 1 is a flowchart of loop filtering according to the related art. [Figure 4] FIG. 1 is a schematic diagram illustrating the configuration of random access video encoding according to the related art; [Figure 5] FIG. 1 is a schematic diagram illustrating the settings of low-delay B video encoding according to the related art. [Figure 6] 1 is a flowchart of a NN-based coded video encoding method according to the related art; [Figure 7] 1 is a flowchart of decoding encoded video based on a NN according to the related art; [Figure 8] 1 is a schematic diagram of a system architecture of an application scenario of a video decoding method according to an embodiment of the present application; [Figure 9] 2 is a flowchart of a video decoding method according to an embodiment of the present application; [Figure 10] 2 is a flowchart of a video decoding method according to an embodiment of the present application; [Figure 11] 10A and 10B are schematic diagrams illustrating the insertion of reference frames at different positions according to the present example. [Figure 12]10 is a flowchart illustrating a process for replacing a reference frame according to the present example. [Figure 13] 10 is a flowchart illustrating a process for storing luminance components in a decoded image buffer according to this example. [Figure 14] 10 is a flowchart of video decoding according to the present example. [Figure 15] 10 is a flowchart of video decoding according to the present example. [Figure 16] 1 is a structural schematic diagram of a video processing device according to an embodiment of the present application; DETAILED DESCRIPTION OF THE INVENTION

[0012] In order to clarify the purpose, technical solution and advantages of the present application, the present application will be described in more detail below with reference to the drawings and examples. It should be understood that the specific examples described herein are only for the purpose of interpreting the present application, and are not intended to limit the present application.

[0013] Although the schematic diagram of the device is divided into functional modules and the flowchart shows a logical order, in some cases the steps shown or described may be performed in a manner different from the division of the modules in the device or the order in the flowchart. Terms such as "first," "second," etc. in the specification, claims, and drawings are used to distinguish between similar objects and are not necessarily used to describe a particular order or priority.

[0014] In describing the embodiments of the present application, unless otherwise expressly limited, terms such as installation, mounting, connection, etc. should be interpreted broadly, and those skilled in the art can reasonably determine the specific meaning of the above terms in the embodiments of the present application in combination with the specific content of the technical solutions.

[0015] In the embodiments of the present application, terms such as "further," "exemplary," or "preferably" are used to describe an example, illustration, or explanation, and should not be construed as preferred or advantageous over other embodiments or designs. The use of terms such as "further," "exemplary," or "preferably" is intended to present the relevant concept in a concrete manner.

[0016] Fig. 1 is a flowchart of video coding according to the related art. As shown in Fig. 1, the coding framework of the next-generation video coding standard H.266 / VVC developed by the Joint Video Project of ITU-T and ISO / IEC includes functional modules such as intra prediction, inter prediction, transform, quantization, loop filtering, and entropy coding. The video coding flow is as follows: operation At least includes.

[0017] 1st action In the step 100, an input video frame image is divided into blocks to form coding tree units (CTUs).

[0018] 2nd action In the above, the divided CTUs are sent to an intra / inter prediction module for predictive coding, where the intra prediction module is mainly used to remove spatial correlation of the image and predict the current pixel block using the reconstructed block information after coding to remove spatial redundancy information, while the inter prediction module is mainly used to remove temporal correlation of the image and use the coded image as the reference image of the current frame to obtain motion information of each block and remove temporal redundancy.

[0019] 3rd actionIn the method, the obtained prediction value is subtracted from the original block to obtain a residual value, and then the residual value is transformed and quantized to remove frequency domain correlation and perform lossy data compression. Transform coding converts the image from a spatial domain signal into the frequency domain, concentrating energy in the low-frequency region. The quantization module can reduce the dynamic range of the image coding.

[0020] 4th action Finally, all the coding parameters and residual values ​​are entropy coded to form a binary stream for storage or transmission, and the output data of the entropy coding module is a code stream after compressing the original video.

[0021] 5th action In step S100, the predicted value is added to the residual value after inverse quantization and inverse transformation to obtain a block reconstruction value, and finally a reconstruction image is formed.

[0022] 6th action In this process, the reconstructed image is filtered using a loop filter and stored in an image buffer for future reference. Loop filtering techniques in H.266 / VVC include luma mapping with chroma scaling (LMCS), deblocking filter (DBF), sample adaptive offset (SAO), and adaptive loop filter (ALF). LMCS improves compression efficiency by reassigning codewords to information within the dynamic range, DBF reduces blocking effects, SAO improves ringing effects, and ALF reduces decoding errors.

[0023] FIG. 2 is a flowchart of video decoding according to the related art. As shown in FIG. 2, the flow of video decoding is as follows: operation At least includes.

[0024] 1st action In this case, the code stream is analyzed. The residual values ​​and predictions Obtain the measurement.

[0025] 2nd action In the step 1, the code stream is analyzed and the residual values ​​obtained are inversely transformed and inversely quantized.

[0026] 3rd action In step S100, the predicted value is added to the residual value after inverse quantization and inverse transformation to obtain a block reconstruction value, and finally a reconstruction image is formed.

[0027] 4th action In step 1, the reconstructed image is filtered by a loop filter and stored in an image buffer to be used as a reference image thereafter.

[0028] 3 is a flowchart of loop filtering according to the related art. As shown in the figure, the reconstructed image first passes through an LMCS module, which gradually and linearly changes the dynamic range of the amplitude distribution of the input video signal before encoding to improve coding efficiency and can be reversely restored on the decoding side. Then, the reconstructed image passes through a DBF module, which adds a longer filter and a luminance adaptive filtering mode specially designed for high dynamic video. Then, the reconstructed image passes through an SAO module, which can reduce the loss of high frequency components and compensate for ringing areas from the pixel domain without reducing coding efficiency. Finally, the reconstructed image passes through an ALF module, which employs diamond filters based on luminance and chrominance, and selects one filter from the transmitted multiple groups for each block.

[0029] With the development of neural networks (NNs), their nonlinear fitting capabilities have become increasingly stronger. Because neural networks can learn the mapping relationship between the original domain and the reconstructed domain, NN-based video coding is the future development direction for video coding in the field of video coding / decoding. Here, loop filtering using NNs is one effective way to improve coding performance. Currently, NNVC, based on the next-generation video coding standard H.266 / VVC, supports two network structures. Switching between the two types of networks and their on / off settings can be controlled by syntax elements in the sequence parameter set (SPS).

[0030] An input video frame is a basic unit that constitutes a video, and one video sequence may include multiple video frames. The input video frame may be a video frame captured in real time, for example, a video frame captured in real time by a camera of a terminal, or a video frame corresponding to a stored video. The input video frame may be an I-frame, a P-frame, or a B-frame, where an I-frame is an intra-predicted frame, a P-frame is a forward-predicted frame, and a B-frame is a bidirectionally predicted frame.

[0031] A reference frame is a video frame that needs to be referenced when encoding a video frame to be decoded. A reference frame is a video frame obtained by reconstructing encoded data corresponding to a video frame that can be used as a reference frame. The reference frame corresponding to a video frame to be decoded may be a forward reference frame or a bidirectional reference frame depending on the inter-prediction type, and the number of current reference frames corresponding to a video frame to be encoded may be one or more. For example, if a video frame to be encoded is a P frame, there may be one corresponding reference frame. If a video frame to be encoded is a B frame, there may be two corresponding reference frames. The reference frame corresponding to a video frame to be encoded may be obtained based on a reference relationship, and the reference relationship may differ depending on the encoding / decoding standard for each video.

[0032] Reference frame management is an important technology in video coding. It mainly manages the decoded picture buffer (DPB), selects the optimal reference method from it, and creates a reference picture queue. The DPB is a buffer for storing decoded images in video coding. To eliminate temporal redundancy, the current coding frame uses the decoded image in the DPB as a reference frame, and inter prediction is used to transmit only inter prediction residual values, further improving coding efficiency.

[0033] Furthermore, the list storing the forward reference frame of the current frame is called the forward reference image list L0, which is also referred to as the first reference image list in the embodiment of the present application, and the list storing the backward reference frame of the current frame is called the backward prediction reference image list L1, which is also referred to as the second reference image list in the embodiment of the present application. If the video frame to be decoded is unidirectionally predicted, the candidate list only has the L0 list, and if the video frame to be decoded is bidirectionally predicted, the candidate lists have both the L0 and L1 lists.

[0034] In the video encoding process, the reference frames used for each frame are usually determined based on the encoding configuration (Config). reference_pictures_L0 / L1 indicate the distance between the reference frames in L0 and L1 and the current frame, respectively, and ref_pics_L0 / L1 indicate the maximum number of available reference frames in L0 / L1 for the current frame. ref_pics_active_L0, ref_pics_active_L1 indicate the number of available reference frames in L0 / L1. The encoding side transmits the POC number of the reference frame used for each frame to the codestream.

[0035] On the video decoding side, the codestream is parsed to obtain the reference frames of the current frame.

[0036] Commonly used video coding settings include a Random Access (RA) setting and a low-delay (loW-delay B) setting.

[0037] 4 is a schematic diagram illustrating the configuration of random access video encoding according to the related art. As shown in FIG. 4, each rectangle in the diagram represents a frame and carries reference frame information and an encoding / decoding number. The reference frame information may be a picture order count (POC), which indicates the playback order of the video frames after decoding, and the encoding / decoding number indicates the order of the video frames in the encoding / decoding process. For example, a P frame with a POC of 32 is a forward predicted frame, and its corresponding reference frame is an I frame with a POC of 0. A B frame with a POC of 16 is a bidirectionally predicted frame, and its corresponding reference frames are two: an I frame with a POC of 0 and a P frame with a POC of 32. The same can be applied to the following.

[0038] FIG. 5 is a schematic diagram illustrating the configuration of low-delay B video encoding according to related technology. As shown in FIG. 5, the first encoded image is an I-image, and the remaining encoded images are B-images or P-images that only have previous reference images in display order, and the display order of the images is the same as the decoding order. Arrows indicate the reference relationships between images, and the arrows point to the reference images. Each frame image only references the reconstructed frame that precedes the current encoded image in playback order, and the video sequence is encoded / decoded according to playback order, eliminating the need to wait for the encoding / decoding of images that follow the current image but precede it in playback order. This results in relatively small delays, making it known as a low-delay structure and suitable for situations with high latency requirements, such as live performances and video calls.

[0039] FIG. 6 is a flowchart of a video encoding process based on a neural network according to the related art. As shown in FIG. 6, the video encoding process includes the following steps: operation At least includes.

[0040] 1st action The encoding side constructs the reference frame list L0, L1 based on the POC difference of the reference frame indicated by cfg. Here, the reference frame can be expressed as a "short-term reference frame," a "non-reference frame," or a "long-term reference frame."

[0041] 2nd action In the above, prediction, transformation, quantization, inverse quantization, and inverse transformation are performed on cu as a unit to obtain a reconstructed block.

[0042] 3rd action In the above, after the whole frame prediction is completed, the LMCS step is performed.

[0043] 4th action In this section, conventional filtering is performed on the reconstructed image after LMCS.

[0044] 5th action In this step, NN filtering is performed on the reconstructed image after LMCS.

[0045] 6th action In, adaptation and modification are performed on the NN and conventional filtering based on the original image, and syntax elements related to the modifications are determined.

[0046] 7th action In, the ALF filtering operation is performed on the NN filtered modified frame of the above process.

[0047] 8th action In this step, the current frame is stored in the DPB as a "short-term reference frame."

[0048] FIG. 7 is a flowchart of the decoding of coded video based on a neural network according to the related art. As shown in FIG. 7, the video decoding flow is as follows: operation At least includes.

[0049] 1st action In the above, a reference frame list is constructed based on the codestream information. Reference lists L0 and L1 are constructed based on the POC differences of the reference frames analyzed in the codestream. Here, the reference frames can be referred to as "short-term reference frames," "non-reference frames," or "long-term reference frames."

[0050] 2nd action In the method, cu is used as a unit and prediction, transformation, quantization, inverse quantization, and inverse transformation are performed based on information analyzed in the code stream to obtain a reconstructed block.

[0051] 3rd action In the above, after the whole frame prediction is completed, the LMCS step is performed.

[0052] 4th action In step 1, conventional filtering is performed on the corresponding blocks in the reconstructed image after LMCS according to the results analyzed in the codestream.

[0053] 5th actionIn step 1, NN filtering is performed on the corresponding blocks in the reconstructed image after LMCS according to the results analyzed in the codestream.

[0054] 6th action In this method, adaptation is performed on the NN and conventional filtering according to the results analyzed in the codestream.

[0055] 7th action In, the ALF filtering operation is performed on the NN filtered modified frame of the above process.

[0056] 8th action In this step, the current frame is stored in the DPB as a "short-term reference frame."

[0057] In related art, the model parameters of the NN filtering module may be preset or transmitted via the code stream. Existing NN filtering networks are generally offline networks, and the model is trained offline using a large amount of data. When encoding a video image, the image obtained by the NN filtering network may be better than that obtained by a traditional filtering method, or the image obtained by the NN filtering network at some pixel points may be worse than that obtained by a traditional filtering method. Based on this, in NN-based video encoding, the image obtained by the NN filtering and the reconstruction value of traditional filtering are generally combined to perform a modification operation to obtain a balanced filtering effect based on the original image, and information related to the modification is written into the code stream and transmitted to the decoding side.

[0058] In existing neural network (NN) modification processes, the selection and modification of NN filtering modes and conventional filtering modes are performed on a block-by-block basis. To reduce the amount of information written into the codestream, large blocks such as 64x64, 128x128, and 256x256 are generally used. This may result in good overall performance for a block, but poor local performance. In inter-prediction structures, the modified reconstructed frame is used as a reference frame for subsequent frames. Currently, two problems remain: One is that local distortions may occur in images after neural network filtering, which is likely to cause error propagation; and The other is that in the inter-prediction process, the reference image lists of some video frames contain duplicated reference frames, which lack diversity and result in undesirable inter-prediction results. Currently, there are no effective solutions to these problems.

[0059] Based on this, embodiments of the present application provide a video decoding method, a video processing device, a computer-readable storage medium, and a computer program product, in which the video decoding method is based on a hybrid coding framework and provides reference frames processed by multiple loop filtering methods for use in subsequent frames, thereby increasing the diversity of reference frames, improving detail quality issues, and improving the overall image quality of the video.

[0060] Hereinafter, the embodiments of the present invention will be further described with reference to the drawings.

[0061] FIG. 8 is a schematic diagram of a system architecture for an application scenario of a video decoding method according to an embodiment of the present application. As shown in FIG. 8, the application environment includes a terminal 110 and a server 120. The terminal 110 or the server 120 can perform video encoding using an encoder and video decoding using a decoder. The terminal 110 or the server 120 can also perform video encoding by executing a video encoding program using a processor, or perform video decoding by executing a video decoding program using a processor. After receiving encoded data transmitted from the terminal 110 via an input interface, the server 120 may directly transmit the encoded data to the processor for decoding, or may store the encoded data in a database for later decoding. After encoding the original video frames using a processor to obtain encoded data, the server 120 may directly transmit the encoded data to the terminal 110 via an output interface, or may store the encoded data in a database for later transmission.

[0062] The video decoding method may be completed by the terminal 110 or the server 120. The terminal 110 may encode input video frames using a video encoding method and then transmit the encoded data to the server 120, or may receive the encoded data from the server 120, decode it, and then generate decoded video frames. The server 120 may encode the video frames, in which case the video encoding method is completed by the server 120. If the server 120 needs to decode the encoded data, the video decoding method is completed by the server 120. Of course, after receiving the encoded data transmitted from the terminal 110, the server 120 may transmit the encoded data to a corresponding receiving terminal, which may then decode it. It should be understood that the encoding side and the decoding side may be the same side or different sides, and that the above-mentioned computer device, such as a terminal or a server, may be the encoding side or the decoding side.

[0063] The terminal 110 and the server 120 are connected via a network. The terminal 110 in the embodiment of the present application is a device related to image and video playback, such as a mobile phone, a tablet computer, a computer, a laptop, a wearable device, an in-vehicle device, a liquid crystal display, a cathode ray tube display, a holographic display, or a projector, and other terminal devices, but the embodiment of the present application is not limited thereto. The server 120 can be implemented as an independent server or a server cluster consisting of multiple servers.

[0064] It should be noted that the video decoding method according to the embodiment of the present application is based on the NN video coding framework, so that the flow architecture in FIG. 6 or FIG. 7 can be referred to, and the description thereof will be omitted here.

[0065] 9 is a flowchart of a video processing method such as a video decoding method according to one embodiment of the present application. As shown in FIG. 9, the video decoding method is applied to a video processing device. In the embodiment of FIG. 9, the video decoding method includes steps S1000, S2000, and S3000. of It may include, but is not limited to:

[0066] In step S1000, reference frame information of a video frame waiting to be decoded is obtained.

[0067] In one embodiment, reference frame information corresponding to a video frame to be decoded is obtained by analyzing a video encoding code stream, and the reference frame information includes a picture order value POC of one reference frame. For example, if the video frame to be decoded has only one reference frame, the reference frame information includes one POC, which indicates the frame position of the reference frame. Taking the example shown in FIG. 4 as an example, assuming that the frame corresponding to POC=32 is the current frame and the frame corresponding to POC=0 is the reference frame corresponding to the current frame, the current frame has only one reference frame, i.e., the frame corresponding to POC=0. Correspondingly, the reference frame information includes POC=0.

[0068] In another embodiment, reference frame information corresponding to a video frame to be decoded is obtained by analyzing a video encoding code stream, and the reference frame information includes picture order values ​​POC of a plurality of reference frames. For example, if the reference frame of the video frame to be decoded has two frames, the reference frame information includes two POCs, each indicating the frame position of the reference frame. Taking the example shown in FIG. 4 as an example, assuming that the frame corresponding to POC=16 is the current frame, and the frame corresponding to POC=0 and the frame corresponding to POC=32 are both reference frames corresponding to the current frame, there are two reference frames corresponding to the current frame, namely, the frame corresponding to POC=0 and the frame corresponding to POC=32. Correspondingly, the reference frame information includes POC=0 and POC=32.

[0069] The reference frame is used to reconstruct the video frame to be decoded, and the reference frame information may be information that can represent other image playback orders or image playback positions other than image sequence values, and this embodiment is not limited thereto.

[0070] It can be understood that the reference frame may be a frame forward of the current frame, a frame backward of the current frame, or a frame forward and a frame backward of the current frame at the same time. The number of reference frames may be one or more.

[0071] In step S2000, a reference picture list of a video frame to be decoded is obtained based on the reference frame information and the complementary frame information.

[0072] In one embodiment, the frame position of the reference frame corresponding to the video frame to be decoded is obtained based on the reference frame information. Specifically, the frame position of the reference frame corresponding to the video frame to be decoded is obtained based on the picture order value POC of the reference frame. Furthermore, the image corresponding to the reference frame is obtained based on the interpolation frame information isInsertFlag. That is, the frame position corresponding to the reference frame is obtained based on the POC of the reference frame, and the image corresponding to the reference frame is obtained based on the isInsertFlag of the reference frame. Finally, the corresponding image is searched / extracted from the image buffer based on the above content. In this manner, all images of the reference frames corresponding to the video frame to be decoded can be obtained and a reference image list can be constructed.

[0073] In one possible embodiment, the reference frame information corresponding to the video frame to be decoded is obtained by parsing the video encoding codestream.

[0074] For example, assume that the frame corresponding to POC=16 is the current frame, and the frames corresponding to POC=0 and POC=32 are both reference frames corresponding to the current frame. By analyzing the video encoding code stream, obtaining reference frame information corresponding to the current frame specifically includes POC=0 and POC=32. Furthermore, based on isInsertFlag, images included in the frame position of POC=0, for example, isInsertFlag=0 and isInsertFlag=1, can be found. Different image sublists can be constructed based on different POCs, such as a first reference image list L0={POC=0, isInsertFlag=0, isInsertFlag=1} and a second reference image list L1={POC=32, isInsertFlag=0, isInsertFlag=1}. The reference image lists include a first reference image list and a second reference image list.

[0075] The first reference picture list L0 and the second reference picture list L1 may use different division methods. For example, images whose POC values ​​corresponding to reference frames are smaller than the POC corresponding to the video frames to be decoded may be set in the first reference picture list L0, and images whose POC values ​​corresponding to reference frames are larger than the POC corresponding to the video frames to be decoded may be set in the second reference picture list L1. Alternatively, images whose POC values ​​corresponding to reference frames are larger than the POC corresponding to the video frames to be decoded may be set in the first reference picture list L0, and images whose POC values ​​corresponding to reference frames are smaller than the POC corresponding to the video frames to be decoded may be set in the second reference picture list L1. Furthermore, division may be performed using other specified methods, and is not limited thereto.

[0076] For example, assume that the frame corresponding to POC=16 is the current frame, and the frames corresponding to POC=0 and POC=32 are both reference frames corresponding to the current frame. All reference frame images are obtained based on the reference frame information POC and the complementary frame information isInsertFlag.

[0077] All reference frame images are divided into different decoded images according to isInsertFlag. Exemplarily, an image with isInsertFlag=0 constitutes a first decoded image. The first decoded image is divided into different reference lists according to different POCs. For example, POC=0 is divided into the first reference image list, i.e., the first reference image list L0={POC=0, isInsertFlag=0}, and POC=32 is divided into the second reference image list, i.e., the second reference image list L1={POC=32, isInsertFlag=0}. That is, the image order value of the first decoded image is equal to one of the image order values ​​of the reference frames, and the complementary frame information of the first decoded image is equal to the first complementary frame information (i.e., isInsertFlag=0).

[0078] Furthermore, the image with isInsertFlag=1 constitutes the second decoded image. The second decoded image is divided into different reference lists according to different POCs, for example, POC=0 is divided into the first reference image list and POC=32 is divided into the second reference image list. The second decoded image may be located at a different position in the reference image list, for example, at the beginning of the first reference image list or the second reference image list, at the end of the first reference image list or the second reference image list, or at the penultimate position of the first reference image list or the second reference image list. The second decoded image may be located at a specified position based on the instruction information in the video encoding code stream. Furthermore, the location of the first decoded image may be replaced.

[0079] In step S3000, a video frame to be decoded is decoded based on the reference image list to obtain a first reconstructed image and a decoded image of the video frame to be decoded.

[0080] In one embodiment, a suitable reference image is selected based on the reference image list to perform video processing such as decoding, thereby obtaining a reconstructed or restored image of the video frame to be decoded. At least two reconstructed images are included, illustratively a first reconstructed image and a decoded image, respectively. Both reconstructed images such as the first reconstructed image and the decoded image are stored in an image buffer and can be used as reference images for the next frame (i.e., the current frame at the next time) to restore the image of the next frame (i.e., the current frame at the next time). The decoded image may be output as a decoded image.

[0081] In another embodiment, complementary frame information can be set for the reconstructed images to facilitate searching and distinguishing between different reconstructed images, e.g., first complementary frame information for the first reconstructed image such as isInsertFlag=1, and second complementary frame information for the decoded image such as isInsertFlag=0.

[0082] In one embodiment, the first reconstructed image is different from the decoded image. Illustratively, the first reconstructed image has not undergone loop filtering, while the decoded image has undergone loop filtering. The loop filtering may include deblocking loop filtering (DBF), sample adaptive offset (SAO), luma mapping with chroma scaling (LMCS), neural network filter (NNF), and adaptive loop filter (ALF).

[0083] In another embodiment, the first reconstructed image and the decoded image are subjected to different loop filtering processes. For example, the first reconstructed image is subjected to luminance mapping LMCS processing based on chroma scaling, and the decoded image is subjected to LMCS, deblocking loop filtering DBF, sample adaptive compensation loop filtering SAO, neural network-based loop filtering NNF, or adaptive loop filtering. The first reconstructed image and the decoded image may be subjected to different loop filtering processes, and the present invention is not limited thereto.

[0084] The technical solution of this embodiment is not only applicable to the video frame waiting to be decoded, but also to the video / image frame waiting to be processed, or the target video / image frame, or the current video / image frame.

[0085] The selection and modification of the neural network loop filtering mode and the conventional filtering mode is performed on a block-by-block basis, and the local performance of the image after the neural network filtering process is poor. If the decoded image after the neural network loop filtering process continues to be used as a reference frame for the subsequent frame, local distortion will occur. The image quality of the conventional filtering technique is stable. In this embodiment, the conventional filtering process is performed on the image. Ta By storing the subsequent first reconstructed image in the decoded image buffer and using it as a reference frame, the diversity of reference frames can be increased, and the predictive reconstructed image quality of the frame can be improved.

[0086] Since the representation of local details in a reconstructed image obtained by conventional filtering after ALF filtering is worse than that in a reconstructed image obtained by conventional filtering without ALF filtering, this embodiment stores the reconstructed image without ALF filtering in the decoded image buffer as the first reconstructed image and uses it as a reference frame for the subsequent frame, thereby increasing the predicted reconstructed image quality of the subsequent frame.

[0087] 10 is a flowchart of a video decoding method according to an embodiment of the present application. As shown in FIG. 10, the video decoding method is applied to a video processing device. In the embodiment of FIG. 10, the video decoding method includes steps S1000, S2000, S3000, and S4000. of It may include, but is not limited to:

[0088] Steps S1000, S2000, and S3000 in the above-described embodiment are applied to this embodiment, and the description thereof will be omitted here.

[0089] In step S4000, a second reconstructed image is generated based on the luminance component of the first reconstructed image and the chrominance component of the decoded image. 2 A reconstructed image is acquired.

[0090] In one embodiment,2 The reconstructed image is stored in an image buffer. 2 To facilitate the search and identification of the reconstructed images, 2 Set second complementary frame information such as isInsertFlag=0 for the reconstructed image.

[0091] An embodiment of the present application provides a video processing method, such as a video decoding method, that is applied to a video frame having a temporal layer identifier, and when the temporal layer identifier of the video frame to be decoded is smaller than a predetermined threshold, the video processing method described in FIG. 9 or 10 is performed on the video frame to be decoded. For example, as shown in FIG. 4, 32 frames are divided into six temporal layers from top to bottom, with the upper frames in the figure being in the lower temporal layer and the lower frames in the figure being in the higher temporal layer. Because the higher temporal layer is less likely to be used as a reference frame for a frame than the lower temporal layer, the higher temporal layer does not achieve as great an improvement in storing reconstructed frames after DBF as compared to the lower temporal layer. Therefore, considering the rational use of resources, when dividing the current temporal layer TL, it is possible to store reconstructed frames after DBF for low temporal layer frames whose temporal layer TL is smaller than the threshold T, thereby saving unnecessary storage overhead.

[0092] Hereinafter, the application process of the video decoding method according to the embodiment of the present application will be described in detail with six specific examples.

[0093] [Example 1] The order of the reference frames affects the reference frame ID numbers during transmission and has a certain effect on the compression results, so this example will explain the insertion position of one of them.

[0094] 11 is a schematic diagram of inserting reference frames at different positions according to this example. operation At least includes.

[0095] 1st action In, a reference image list is constructed.

[0096] The decoding side analyzes the picture order value POC of the reference frame of the obtained video frame to be decoded in the codestream, and constructs a first reference picture list L0 and a second reference picture list L1 from the decoded picture buffer based on the picture order value POC, where the first reference picture list and the second reference picture list constitute a reference picture list. The reference picture list includes a first decoded picture and a second decoded picture, where the first decoded picture corresponds to the picture with the isInsertFlag identifier value = 0, and the second decoded picture corresponds to the picture with the isInsertFlag identifier value = 1. Here, the reference frame can be referred to as a "short-term reference frame," a "non-reference frame," or a "long-term reference frame."

[0097] In this example, the reference frames can be referred to as "short-term reference frames," "non-reference frames," or "long-term reference frames."

[0098] 2nd action At this time, if an image with isInsertFlag=1 exists in the decoded image buffer and the corresponding POC number appears in L0 or L1, the image can be inserted into multiple positions such as the first, last, or penultimate position of the corresponding reference image list, or the position of the inserted image can be flexibly determined based on a specific cost.

[0099] 3rd action In the coding unit (CU), prediction, transformation, quantization, inverse quantization, and inverse transformation are performed based on information analyzed in the codestream to obtain a reconstructed block.

[0100] 4th action In the above, after the whole frame prediction is completed, the LMCS step is performed.

[0101] 5th actionIn accordance with the result of analyzing the codestream, conventional filtering is performed on the corresponding block in the reconstructed image after LMCS, and the reconstructed image after filtering is stored in the decoded image buffer as a first reconstructed image, the image order value of the video frame to be decoded is recorded, and the complementary frame information is set as the second complementary frame information.

[0102] 6th action In step 100, NN filtering is performed on the corresponding blocks in the reconstructed image after LMCS according to the results analyzed in the codestream, to prepare the reconstructed frame for subsequent conventional filtering and NN filtering.

[0103] 7th action In this method, adaptation is performed for NN filtering and conventional filtering according to the results analyzed in the code stream.

[0104] 8th action In , re Perform ALF filtering operations on the constituent frames.

[0105] 9th action In the step 10, the decoded image is designated as a "short-term reference frame" and the complementary frame information is set to the first complementary frame information. The decoded image is stored in a decoded image buffer.

[0106] [Example 2] Inserting a reference frame increases the range of motion search and increases the temporal complexity, so in this example, the operation of inserting a reference frame is changed to replacing a reference frame, which reduces the temporal complexity to a certain extent.

[0107] FIG. 12 is a flowchart showing the replacement of reference frames according to this example. As shown in the figure, operation At least includes.

[0108] 1st action In, a reference image list is constructed.

[0109] The decoding side analyzes the picture order value POC of the reference frame of the obtained video frame to be decoded in the codestream, and constructs a first reference picture list L0 and a second reference picture list L1 from the decoded picture buffer based on the picture order value POC, where the first reference picture list and the second reference picture list constitute a reference picture list. The reference picture list includes a first decoded picture and a second decoded picture, where the first decoded picture corresponds to the picture with the isInsertFlag identifier value = 0, and the second decoded picture corresponds to the picture with the isInsertFlag identifier value = 1. Here, the reference frame can be referred to as a "short-term reference frame," a "non-reference frame," or a "long-term reference frame."

[0110] 2nd action At this time, if an image with isInsertFlag=1 exists in the decoded image buffer and the corresponding POC number appears in L0 and L1, the original reference image with isInsertFlag=0 is replaced with this image.

[0111] 3rd action In the method, cu is used as a unit and prediction, transformation, quantization, inverse quantization, and inverse transformation are performed based on information analyzed in the code stream to obtain a reconstructed block.

[0112] 4th action In the above, after the whole frame prediction is completed, the LMCS step is performed.

[0113] 5th action In accordance with the result analyzed in the codestream, perform conventional filtering on the corresponding block in the reconstructed image after LMCS, and store the result after conventional filtering in a decoded image buffer, record the POC number corresponding to the video frame to be decoded, and set isInsertFlag to 1.

[0114] 6th action In step 1, NN filtering is performed on the corresponding blocks in the reconstructed image after LMCS according to the results analyzed in the codestream.

[0115] 7th action In this method, adaptation is performed on the NN and conventional filtering according to the results analyzed in the codestream.

[0116] 8th action In, the ALF filtering operation is performed on the reconstructed frame of the above process.

[0117] 9th action In step 100, the decoded image is stored in the decoded image buffer as a "short-term reference frame" and isInsertFlag is set to 0.

[0118] [Example 3] Because the probability that the high temporal layer is used as a reference frame is much smaller than that of the low temporal layer, the improvement of storing the reconstructed frame after DBF for the high temporal layer is not as great as that for the low temporal layer. To save storage overhead, this example only uses the low temporal layer.

[0119] 1st action In, a reference image list is constructed.

[0120] The decoding side analyzes the picture order values ​​POC of the reference frames of the obtained video frame to be decoded in the codestream, and constructs a first reference picture list L0 and a second reference picture list L1 from the decoded picture buffer based on the picture order values ​​POC, where the first reference picture list and the second reference picture list constitute a reference picture list. The reference picture list includes a first decoded picture and a second decoded picture, where the first decoded picture corresponds to the picture with the value of the isInsertFlag identifier = 0, and the second decoded picture corresponds to the picture with the value of the isInsertFlag identifier = 1.

[0121] 2nd actionAt this time, if the current temporal layer TL is smaller than the threshold T (T∈[0,5], which can be set by hand as needed), determine whether there is an image with isInsertFlag=1 in the decoded image buffer, and if the corresponding POC number appears in L0, L1, insert the image after the corresponding reference image list.

[0122] 3rd action In the method, cu is used as a unit and prediction, transformation, quantization, inverse quantization, and inverse transformation are performed based on information analyzed in the code stream to obtain a reconstructed block.

[0123] 4th action In the above, after the whole frame prediction is completed, the LMCS step is performed.

[0124] 5th action In accordance with the result analyzed in the codestream, perform conventional filtering on the corresponding block in the reconstructed image after LMCS, and store the result after conventional filtering in a decoded image buffer, record the POC number corresponding to the video frame to be decoded, and set isInsertFlag to 1.

[0125] 6th action In step 1, NN filtering is performed on the corresponding blocks in the reconstructed image after LMCS according to the results analyzed in the codestream.

[0126] 7th action In this method, adaptation is performed on the NN and conventional filtering according to the results analyzed in the codestream.

[0127] 8th action In, the ALF filtering operation is performed on the reconstructed frame of the above process.

[0128] 9th action In step 100, the decoded image is stored in the decoded image buffer as a "short-term reference frame" and isInsertFlag is set to 0.

[0129] [Example 4] Since the performance of conventional filtering of chroma is much worse than the result after NN, it is not suitable to use the chroma after conventional filtering as a reference frame. Therefore, this example stores only the luma component after conventional filtering in the decoded image buffer, and keeps the chroma component as the result after NN and conventional adaptation.

[0130] FIG. 13 is a flowchart showing the process of storing luminance components in the decoded image buffer according to this example. operation At least includes.

[0131] 1st action In, a reference image list is constructed.

[0132] The decoding side analyzes the picture order value POC of the reference frame of the obtained video frame to be decoded in the codestream, and constructs a first reference picture list L0 and a second reference picture list L1 from the decoded picture buffer based on the picture order value POC, where the first reference picture list and the second reference picture list constitute a reference picture list. The reference picture list includes a first decoded picture and a second decoded picture, where the first decoded picture corresponds to the picture with the isInsertFlag identifier value = 0, and the second decoded picture corresponds to the picture with the isInsertFlag identifier value = 1. Here, the reference frame can be referred to as a "short-term reference frame," a "non-reference frame," or a "long-term reference frame."

[0133] 2nd action At this time, if an image with isInsertFlag=1 exists in the decoded image buffer and the corresponding POC number appears in L0 and L1, the image is inserted after the corresponding reference image list.

[0134] 3rd action In the method, cu is used as a unit and prediction, transformation, quantization, inverse quantization, and inverse transformation are performed based on information analyzed in the code stream to obtain a reconstructed block.

[0135] 4th actionIn the above, after the whole frame prediction is completed, the LMCS step is performed.

[0136] 5th action In the step S100, according to the result analyzed in the codestream, perform conventional filtering on the corresponding block in the reconstructed image after LMCS, and store the result of the luminance component after conventional filtering in the decoded image buffer, record the POC number corresponding to the video frame to be decoded, and set isInsertFlag to 1.

[0137] 6th action In step 1, NN filtering is performed on the corresponding blocks in the reconstructed image after LMCS according to the results analyzed in the codestream.

[0138] 7th action In this method, adaptation is performed on the NN and conventional filtering according to the results analyzed in the codestream.

[0139] 8th action In the above process, the ALF filtering operation is performed on the reconstructed frame. The result of the chrominance components after ALF is 5th action The POC number corresponding to the video frame waiting to be decoded is recorded, and isInsertFlag is set to 1.

[0140] 9th action In step 100, the decoded image is stored in the decoded image buffer as a "short-term reference frame" and isInsertFlag is set to 0.

[0141] [Example 5] FIG. 14 is a flowchart of the video decoding according to this example. As shown in the figure, operation At least includes.

[0142] 1st action In, a reference image list is constructed.

[0143] The decoding side analyzes the picture order value POC of the reference frame of the obtained video frame to be decoded in the codestream, and constructs a first reference picture list L0 and a second reference picture list L1 from the decoded picture buffer based on the picture order value POC, where the first reference picture list and the second reference picture list constitute a reference picture list. The reference picture list includes a first decoded picture and a second decoded picture, where the first decoded picture corresponds to the picture with the isInsertFlag identifier value = 0, and the second decoded picture corresponds to the picture with the isInsertFlag identifier value = 1. Here, the reference frame can be referred to as a "short-term reference frame," a "non-reference frame," or a "long-term reference frame."

[0144] 2nd action At this time, if an image with isInsertFlag=1 exists in the decoded image buffer and the corresponding POC number appears in L0 and L1, the image is inserted after the corresponding reference image list.

[0145] 3rd action In the method, cu is used as a unit and prediction, transformation, quantization, inverse quantization, and inverse transformation are performed based on information analyzed in the code stream to obtain a reconstructed block.

[0146] 4th action In the above, after the whole frame prediction is completed, the LMCS step is performed.

[0147] 5th action In step 100, DBF filtering is performed on the reconstructed image after LMCS, and the result after conventional filtering is stored in the decoded image buffer, the POC number corresponding to the video frame waiting to be decoded is recorded, and isInsertFlag is set to 1.

[0148] 6th action In step 2, SAO filtering is performed on the reconstructed image after the decoded image buffer.

[0149] 7th action In this case, the ALF filtering operation is performed.

[0150] 8th action In step 100, the decoded image is stored in the decoded image buffer as a "short-term reference frame" and isInsertFlag is set to 0.

[0151] [Example 6] When multiple neural network filters are adopted in the video encoding process, images after different NN filtering modifications can be used as reference frames for subsequent frames, thereby increasing the diversity of reference frames.

[0152] 15 is a flowchart of the video decoding according to this example. As shown in the figure, the video decoding process includes at least the following operations:

[0153] 1st action In, a reference image list is constructed.

[0154] The decoding side analyzes the picture order value POC of the reference frame of the obtained video frame to be decoded in the codestream, and constructs a first reference picture list L0 and a second reference picture list L1 from the decoded picture buffer based on the picture order value POC, where the first reference picture list and the second reference picture list constitute a reference picture list. The reference picture list includes a first decoded picture and a second decoded picture, where the first decoded picture corresponds to the picture with the isInsertFlag identifier value = 0, and the second decoded picture corresponds to the picture with the isInsertFlag identifier value = 1. Here, the reference frame can be referred to as a "short-term reference frame," a "non-reference frame," or a "long-term reference frame."

[0155] 2nd action At this time, if an image with isInsertFlag=1 exists in the decoded image buffer and the corresponding POC number appears in L0 and L1, the image is inserted after the corresponding reference image list.

[0156] 3rd action In the method, cu is used as a unit and prediction, transformation, quantization, inverse quantization, and inverse transformation are performed based on information analyzed in the code stream to obtain a reconstructed block.

[0157] 4th action In the above, after the whole frame prediction is completed, the LMCS step is performed.

[0158] 5th action In accordance with the result of analyzing the codestream, perform neural network filtering on the corresponding block in the reconstructed image after LMCS, using a first neural network filter NN1, and store the result after filtering by NN1 in a decoded image buffer, record the POC number corresponding to the video frame to be decoded, and set isInsertFlag to 1.

[0159] 6th action In step S100, according to the result of analyzing the code stream, neural network filtering is performed on the corresponding block in the reconstructed image after LMCS, and a second neural network filter NN2 is adopted.

[0160] 7th action In the method, the multiple neural network filtering results are adapted according to the results analyzed in the codestream.

[0161] 8th action In, the ALF filtering operation is performed on the reconstructed frame of the above process.

[0162] 9th action In step 100, the decoded image is stored in the decoded image buffer as a "short-term reference frame" and isInsertFlag is set to 0.

[0163] Fig. 16 is a structural schematic diagram of a video processing device according to an embodiment of the present application. As shown in Fig. 16, the video processing device 2000 includes a memory 2100 and a processor 2200. The number of memories 2100 and processors 2200 may be one or more. Fig. 16 shows one memory 2100 and one processor 2200 as an example, and the memory 2100 and processor 2200 may be connected via a bus or other methods.

[0164] The memory 2100 can be used as a computer-readable storage medium to store software programs, computer-executable programs and modules, such as program instructions / modules corresponding to the methods according to any embodiment of the present application. The processor 2200 executes the software programs, instructions and modules stored in the memory 2100 to implement the methods.

[0165] The memory 2100 may primarily include a program storage area and a data storage area, where the program storage area can store an operating system and / or application programs required for at least one function. The memory 2100 may also include high-speed random access memory and may further include non-volatile memory such as at least one magnetic disk storage device, flash memory, or other non-volatile solid-state storage device. In some embodiments, the memory 2100 may include memory located remotely from the processor 2200, and these remote memories may be connected to the device via a network. Examples of such networks may include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0166] An embodiment of the present application further provides a computer-readable storage medium having stored thereon computer-executable instructions for performing the video processing method according to any embodiment of the present application.

[0167] An embodiment of the present application further provides a computer program product, comprising a computer program or computer instructions stored in a computer-readable storage medium, wherein a processor of a computing device reads the computer program or computer instructions from the computer-readable storage medium, and the processor executes the computer program or computer instructions such that the computing device performs a video processing method according to any embodiment of the present application.

[0168] The system architecture and application scenarios described in the embodiments of the present application are intended to more clearly explain the technical solutions of the embodiments of the present application, and are not intended to limit the technical solutions of the embodiments of the present application. Those skilled in the art will understand that with the evolution of system architecture and the emergence of new application scenarios, the technical solutions of the embodiments of the present application can be similarly applied to similar technical problems.

[0169] Those skilled in the art will understand that all or part of the steps in the above-disclosed methods, systems, and functional modules / units in the devices can be implemented as software, firmware, hardware, or a suitable combination thereof.

[0170] In hardware embodiments, the division among functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components. For example, one physical component may have multiple functions, or one function or step may be performed cooperatively by multiple physical components. Some or all of the components may be implemented as software executed by a processor, such as a central processor, digital signal processor, or microprocessor, or as hardware or integrated circuits, such as application-specific integrated circuits. Such software may be distributed on computer-readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). As known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (e.g., computer-readable instructions, data structures, program modules, or other data). Computer storage media may include, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk memory, magnetic cartridges, magnetic tape, magnetic disk memory or other magnetic storage devices, or any other medium used to store the desired information and accessible by a computer. Also, as known to those skilled in the art, communication media typically include computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier or other transport mechanism, and may include any information delivery media.

[0171] As used herein, terms such as "component," "module," and "system" are used to refer to computer-related entities, hardware, firmware, a combination of hardware and software, software, or software in execution. For example, a component may be, but is not limited to, a process running on a processor, a processor, an object, an executable file, a thread of execution, a program, or a computer. By way of illustration, both an application running on a computing device and the computing device may be components. One or more components may reside in a process or thread of execution, and components may be located on one computer or distributed among two or more computers. These components may also be implemented by various computer-readable media having various data structures stored thereon. Components may communicate with local or remote processes, for example, via signals comprising one or more data packets (e.g., data originating from two components automatically interacting with another component across a local system, a distributed system, or a network, e.g., the Internet, which interacts with other systems via signals).

[0172] Although the above describes some embodiments of the present application with reference to the drawings, the scope of the present application is not limited thereto. Any modifications, equivalent substitutions, improvements, etc. made by those skilled in the art without departing from the scope and substance of the present application should be included within the scope of protection of the present application.

Claims

1. Obtaining reference frame information of a video frame to be decoded; obtaining a reference image list for the video frame to be decoded based on the reference frame information and the complementary frame information; decoding a video frame to be decoded based on the reference image list to obtain a first reconstructed image and a decoded image of the video frame to be decoded; Video decoding methods.

2. storing a first reconstructed image of the video frame to be decoded and the decoded image in a decoded image buffer; and setting complementary frame information for the first reconstructed image of the video frame to be decoded and the decoded image. The method of claim 1.

3. the first reconstructed image is different from the decoded image, The first reconstructed image has not been subjected to loop filtering; Alternatively, the first reconstructed image and the decoded image have been subjected to different loop filtering processes.

3. The method according to claim 1 or 2.

4. The loop filtering process includes at least Deblocking loop filtering (DBF); Sample adaptive compensation loop filtering SAO; Luminance mapping LMCS based on chroma scaling; Neural network based loop filtering (NNF), The method of claim 3.

5. Obtaining reference frame information for a video frame to be decoded includes: analyzing the video encoding codestream to obtain reference frame information corresponding to the video frame to be decoded; the reference frame information includes a picture order value of at least one of the reference frames; 10. The method according to claim 1, 2 or 4.

6. obtaining a reference image list for the video frame to be decoded based on the reference frame information and the complementary frame information, obtaining a first reference image list and a second reference image list based on the image order values ​​of the reference frames; The method of claim 5.

7. the first reference image list includes a first decoded image; the second reference image list includes the first decoded image; the image order value of the first decoded image is equal to one of the image order values ​​of the reference frame, and the complementary frame information of the first decoded image is equal to the first complementary frame information; The method of claim 6.

8. the first reference image list further includes a second decoded image; and / or the second reference image list further includes a second decoded image; the image order value of the second decoded image is equal to the image order value of the first decoded image, and the complementary frame information of the second decoded image is equal to the second complementary frame information; The method of claim 7.

9. The second decoded image is being located at the beginning of the first reference image list or the second reference image list; being located at the end of the first reference image list or the second reference image list; being located at the second to last position in the first reference image list or the second reference image list; replacing the first decoded image with the second decoded image; placing the video encoded codestream at a designated location based on the instruction information; At least one of The method of claim 8.

10. Storing the reconstructed images in an image buffer and setting corresponding reference subframe information for the reconstructed images includes: storing the first reconstructed image and the decoded image in a decoded image buffer, setting first complementary frame information for the first reconstructed image, and setting second complementary frame information for the decoded image; The method of claim 3.

11. obtaining a third reconstructed image based on a luminance component of the first reconstructed image and a chroma component of the decoded image; storing the third reconstructed image in an image buffer and setting second complementary frame information for the third reconstructed image; The method of claim 2 or 10.

12. 1. A video decoding method applied to a video frame including a temporal layer identifier, comprising: performing the video decoding method of any one of claims 1 to 11 on a video frame waiting to be decoded if the temporal layer identifier of the video frame is less than a predetermined threshold. Video decoding methods.

13. a memory, a processor, and a computer program stored in the memory and executable by the processor; When the processor executes the computer program, it implements the video decoding method of any one of claims 1 to 12. Video processing equipment.

14. 13. A method for decoding a video signal comprising: storing computer-executable instructions for performing the video decoding method of any one of claims 1 to 12; A computer-readable storage medium.

15. A computer program or computer instructions stored on a computer-readable storage medium, a processor of the computer device reading the computer program or the computer instructions from the computer-readable storage medium; The processor executes the computer program or the computer instructions such that the computer device performs the video decoding method of any one of claims 1 to 12. Computer program products.

Images