Method and device for determining weight based on template cost and storage medium

Abstract

The invention provides a method and device for determining weight based on template cost and a storage medium, and the method for determining weight based on template cost is applied to a coding end, and comprises the steps: dividing each image frame in a to-be-coded video stream into a plurality of blocks; performing intra / inter prediction mode decision on the current block in the plurality of blocks; if it is determined that the current block uses a joint intra inter prediction (CIIP) mode, determining an intra prediction weight and an inter prediction weight based on a template cost; determining a joint prediction value of the current block according to the intra-frame prediction value, the intra-frame prediction weight, the inter-frame prediction value and the inter-frame prediction weight of the current block, and taking the joint prediction value of the current block as a prediction value of the current block; and transmitting related syntax elements of the current block for determining the intra-frame prediction weight and the inter-frame prediction weight based on the template cost to a decoding end.

Description

Technical Field

[0001] The present application relates to the field of video coding and decoding, and in particular to a method, device and storage medium for determining weights based on template costs. Background Art

[0002] Today's world is a multimedia society surrounded by images and videos. For example, advertising, medical care, security, entertainment, conference television and other fields are everywhere, all of which involve media content such as images and videos. With the improvement of video quality: gradually turning from high-definition video to ultra-high-definition video such as 4K and 8K, the increase of various video types such as screen content and panoramic video, as well as the widespread application of network streaming and real-time communication, the amount of data transmitted or stored in the video has been greatly increased, and extremely high requirements have been placed on the video transmission bandwidth. Video compression is a technology that uses various encoding tools to remove redundant information in space and time as much as possible from the original video, effectively reducing the amount of video data while ensuring video quality. It is also called video coding, which is one of the most popular research directions in the field of multimedia technology research.

[0003] In 2020, the Joint Video Expert Teams (JVET) of the Moving Pictures Experts Group (MPEG) of the International Organization for Standardization (ISO) / International Electrotechnical Commission (IEC) and the Video Code Expert Group (VCEG) of the International Telecommunication Union-Telecommunication standardization sector (ITU-T) released the Versatile Video Coding (H.266 / VVC) standard, which can better support a variety of new video types such as 8K ultra-high definition, screen content, high dynamic and 360° panoramic video, as well as applications such as streaming media and real-time communication with adaptive bandwidth and resolution. Since then, the Joint Expert Group has continued to explore the next generation of video coding standards (Enhanced Compression Model, ECM). The ECM standard uses the coding framework of H.266 / VVC: a block-based hybrid coding framework. The hybrid coding framework mainly includes modules such as intra-frame prediction, inter-frame prediction, transformation, quantization, entropy coding and loop filtering. Each module covers a variety of new coding technologies, further improving the coding gain, but also requires a lot of memory.

[0004] Combined inter and intra prediction (CIIP) mode is a prediction technology in H.266 / VVC. The basic principle is that the current coding unit (Coding Unit, block) uses intra prediction and inter prediction to obtain two prediction blocks, and the weighted average is used as the final prediction value. In order to avoid high computational complexity, H.266 / VVC limits CIIP intra prediction to only use the planar mode, and inter prediction to only use the conventional merge mode.

[0005] At the JVET-W meeting in July 2021, it was proposed to replace the original Planar mode with the intra-frame prediction mode obtained by the template-based intra mode derivation (TIMD) method for the intra-frame prediction mode of the CIIP mode, in order to refine the intra-frame prediction mode of the CIIP mode without increasing signaling transmission. When the CIIP mode is combined with TIMD, that is, the CIIP-TIMD mode, improvements are also proposed for the weight of the CIIP mode. If the intra-frame prediction mode derived by TIMD is Planar or (Direct Block, DC) mode, the weight coefficient remains unchanged.

[0006] At the JVET-X meeting in October 2021, the CIIP-Template Matching (CIIP-TM) mode was proposed based on the CIIP-TIMD mode, that is, the inter-frame prediction part is based on the motion vector (MV) candidate after TM refinement, and the inter-frame prediction mode at this time is Template Matching-Merge (TM-Merge). In the CIIP mode using template matching, the weights of inter-frame prediction and intra-frame prediction are still determined based on the existing template, and the weight coefficients are not accurate enough.

[0007] Application Contents

[0008] In view of this, embodiments of the present application hope to provide a method, device and storage medium for determining weights based on template costs.

[0009] In a first aspect, an embodiment of the present application provides a method for determining a weight based on a template cost, which is applied to an encoding end and includes:

[0010] Dividing each image frame in the video stream to be encoded into a plurality of blocks;

[0011] Performing intra / inter prediction mode decision on a current block among the multiple blocks;

[0012] If it is determined that the current block uses the CIIP mode, the intra-frame prediction weight and the inter-frame prediction weight are determined based on the template cost;

[0013] Determine a joint prediction value of the current block according to the intra prediction value, the intra prediction weight, the inter prediction value and the inter prediction weight of the current block, and use the joint prediction value of the current block as the prediction value of the current block;

[0014] The relevant syntax elements for determining the intra-frame prediction weight and the inter-frame prediction weight of the current block based on the template cost are transmitted to the decoding end.

[0015] In a second aspect, an embodiment of the present application provides a method for determining a weight based on a template cost, which is applied to a decoding end, including:

[0016] Receive the encoded video stream and parse the video stream;

[0017] Obtain blocks in the parsed video bitstream, and read syntax elements related to intra-frame / inter-frame prediction in the parsed video bitstream;

[0018] Determine the intra / inter prediction mode of the current block according to the parsed current block and the intra / inter prediction related syntax elements;

[0019] If it is determined that the current block uses the CIIP mode, the intra-frame prediction weight and the inter-frame prediction weight are determined based on the template cost;

[0020] Determine a joint prediction value of the current block according to the intra prediction value, the intra prediction weight, the inter prediction value and the inter prediction weight of the current block, and use the joint prediction value of the current block as a reconstruction value of the current block;

[0021] A decoded video stream is obtained according to the reconstructed value of each of the parsed multiple blocks.

[0022] In a third aspect, an embodiment of the present application provides a device for determining a weight based on a template cost, which is arranged at an encoding end and includes:

[0023] a segmentation module configured to divide each image frame in the video stream to be encoded into a plurality of blocks;

[0024] A decision module, configured to make an intra / inter prediction mode decision for a current block among the multiple blocks;

[0025] A weight determination module, configured to determine an intra-frame prediction weight and an inter-frame prediction weight based on a template cost if it is determined that the current block uses the CIIP mode;

[0026] A joint prediction module is configured to determine a joint prediction value of the current block according to an intra prediction value, an intra prediction weight, an inter prediction value, and an inter prediction weight of the current block, and use the joint prediction value of the current block as a prediction value of the current block;

[0027] The transmission module is configured to transmit relevant syntax elements for determining intra-frame prediction weights and inter-frame prediction weights of the current block based on the template cost to a decoding end.

[0028] In a fourth aspect, an embodiment of the present application provides a device for determining a weight based on a template cost, which is arranged at a decoding end and includes:

[0029] A receiving module, configured to receive the encoded video stream and parse the video stream;

[0030] An acquisition module, configured to acquire blocks in the parsed video bitstream and read syntax elements related to intra / inter prediction in the parsed video bitstream;

[0031] A mode determination module, configured to determine the intra / inter prediction mode of the current block according to the parsed current block and the intra / inter prediction related syntax elements;

[0032] A weight determination module, configured to determine an intra-frame prediction weight and an inter-frame prediction weight based on a template cost if it is determined that the current block uses the CIIP mode;

[0033] a joint prediction module, configured to determine a joint prediction value of the current block according to an intra prediction value, an intra prediction weight, an inter prediction value, and an inter prediction weight of the current block, and use the joint prediction value of the current block as a reconstruction value of the current block;

[0034] The reconstruction module is configured to obtain a decoded video stream according to a reconstruction value of each of the parsed multiple blocks.

[0035] In a fifth aspect, an embodiment of the present application provides a device for determining a weight based on a template cost, including:

[0036] a memory configured to store a program;

[0037] The processor is configured to execute a program, and when the program is executed, the method for determining weights based on template costs as in any implementation of the first aspect is performed.

[0038] In a sixth aspect, an embodiment of the present application provides a device for determining a weight based on a template cost, including:

[0039] a memory configured to store a program;

[0040] The processor is configured to execute a program, and when the program is executed, the method for determining weights based on template costs as in any implementation of the second aspect is performed.

[0041] In a seventh aspect, an embodiment of the present application provides a non-volatile storage medium, the storage medium including a stored program, characterized in that when the program is running, the method for determining weights based on template costs of any implementation of the first aspect or any implementation of the second aspect is executed. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 Schematic diagram of the hybrid coding framework of H.266 / VVC

[0043] Figure 2 This is a schematic diagram of the Planar mode principle;

[0044] Figure 3 This is a schematic diagram of the derivation of CIIP weights;

[0045] Figure 4a and Figure 4b Schematic diagram of the division method of CIIP-TIMD angle mode;

[0046] Figure 5 Schematic diagram of template matching near the initial MV;

[0047] Figure 6 A flowchart of a method for determining weights based on template costs provided in an embodiment of the present application;

[0048] Figure 7a and Figure 7b Schematic diagram of template weighting for CIIP-TM angle mode;

[0049] Figure 8 A flowchart of another method for determining weights based on template costs provided in an embodiment of the present application;

[0050] Fig. 9 A schematic diagram of the structure of a device for determining weights based on template costs provided in an embodiment of the present application;

[0051] Fig.10 A schematic diagram of the structure of another device for determining weights based on template costs provided in an embodiment of the present application;

[0052] Fig.11 A schematic diagram of the structure of a device for determining weights based on template costs provided in an embodiment of the present application;

[0053] Fig.12 A schematic diagram of the structure of a device for determining weights based on template costs provided in an embodiment of the present application. DETAILED DESCRIPTION

[0054] In order to make the application objectives, technical solutions and beneficial effects of the present application clearer, the embodiments of the present application are described below in conjunction with the accompanying drawings. It should be noted that, unless there is a conflict, the embodiments in the present application and the features in the embodiments can be combined with each other arbitrarily.

[0055] Figure 1 Schematic diagram of the hybrid coding framework of H.266 / VVC, Figure 1 As shown in FIG. 1 , the hybrid coding framework mainly includes modules such as intra-frame prediction, inter-frame prediction, transformation, quantization, entropy coding and loop filtering. Each module covers a variety of new coding technologies, which further improves the coding gain, but also requires a large amount of memory.

[0056] Among the modules of the H.266 / VVC coding framework, the intra-frame prediction module mainly removes the spatial correlation of the video image, and uses the pixel information of the adjacent reconstructed blocks that have been encoded in the frame to predict the pixel value of the current block, with a low compression rate; the inter-frame prediction module mainly removes the temporal correlation of the video image, and uses the inter-frame encoded image as the reference image of the current frame to obtain the motion information of each block of the current frame. It can use unidirectional and bidirectional reference images for prediction, and generally speaking, the compression rate is higher than that of the intra-frame prediction coding. After the video image block passes through the intra-frame / inter-frame prediction module, the predicted value of the current block is obtained. The residual data can be obtained by subtracting the original image block from the predicted block, and the residual data is transformed, that is, from the time domain described in pixel form to the frequency domain, so that the image energy is concentrated in the low-frequency area, which can effectively remove the frequency domain correlation of the image, that is, the correlation of the transform coefficients is small; usually, the transformed coefficients have a large dynamic range, and the quantization module is needed to reduce the dynamic range to achieve better compression; the entropy coding module encodes the intra-frame prediction data, motion information, quantization coefficients, and coding control data into a binary stream, that is, the compressed code stream of the original video, and then realizes storage or transmission.

[0057] In addition, in the hybrid coding framework, in order to build a decoded image cache for inter-frame prediction as a reference image, an image reconstruction process is also implemented, which mainly includes inverse quantization, inverse transformation and loop filtering modules. The specific process is to inverse quantize and inverse transform the quantized transform coefficients to obtain residual data, and then add it to the prediction information to reconstruct the image data; the loop filtering module is to improve the subjective quality and compression efficiency of the image and reduce decoding errors. The reason is that video coding is based on block processing, and usually the reconstructed image will have block effects, ringing effects and other phenomena. This module mainly includes the following filtering technologies: Luma Mapping Chroma Scaling (LMCS), De-blocking Filter (DBF), Sample Adaptive Offset (SAO) and Adaptive Loop Filter (ALF). Luma Mapping and Chroma Scaling is to redistribute codewords for information within the dynamic range, which can improve compression efficiency; De-blocking Filter is to smooth the block boundary, which can effectively reduce and remove the block effect; Sample Adaptive Offset is to use different compensation values ​​for different categories of reconstructed pixels, which can improve the ringing effect caused by the loss of high-frequency information and effectively improve the subjective and objective quality of the reconstructed image; Adaptive Loop Filter is to perform Wiener filtering on the reconstructed image pixels, which can effectively reduce decoding errors.

[0058] Figure 1 The overall framework process shown is as follows:

[0059] (1) The input video frame (image) is divided into blocks: the frame is first divided into multiple coding tree units (CTUs). Each CTU can be divided into four coding units (CUs) of the same size according to a quadtree, or recursively divided into CUs of different sizes according to a multiple type tree (MTT), that is, a binary tree or a ternary tree structure. CTU contains all color channels, that is, a CTU consists of a luminance block (Coding Tree Block, CTB) and two chrominance blocks CTB; the block is the basic unit of video coding, the maximum size of the luminance block is 128*128, the minimum size is 4*4, the maximum size of the chrominance block is 64*64, and the minimum size is 2*2; the size of the final divided block is related to the video image content.

[0060] (2) The intra-frame / inter-frame prediction mode decision determines whether the coding block is to be intra-frame prediction coded or inter-frame prediction coded. The main purpose of intra-frame prediction coding is to remove the spatial correlation of the image; the main purpose of inter-frame prediction coding is to remove the temporal correlation of the image.

[0061] (3) Subtract the predicted block obtained by intra-frame / inter-frame coding from the original coded block to obtain the residual data of the current block. The residual data is transformed and quantized to remove the frequency domain correlation, achieve lossy compression of the image residual data, and further improve the compression efficiency.

[0062] (4) The quantized two-dimensional transform coefficients are Z-scanned and run-length encoded to obtain one-dimensional transform coefficients. Finally, the transform coefficients, coding control information, prediction data, motion vector data and other information are used as input to the entropy coding module and encoded into a binary stream for storage or transmission.

[0063] (5) The quantized transform coefficients are dequantized and inversely transformed to obtain the residual data of the reconstructed block, which is then added to the predicted value obtained by intra-frame / inter-frame coding to obtain the reconstructed value of the coding block, thereby obtaining the reconstructed image.

[0064] (6) The reconstructed image needs to be filtered through the loop filter module to obtain the final reconstructed image and build a decoded image cache as a reference image for inter-frame prediction.

[0065] In fact, the image reconstruction process of the hybrid coding framework is the image decoding process, but the decoding end adds a bit stream parsing step.

[0066] Planar mode is a traditional non-angle prediction mode suitable for areas where pixel values ​​change slowly. Its mode index is 0. If the current block uses Planar mode, the predicted pixel at a certain position in the block is the weighted average of the four reference pixels in the horizontal and vertical directions corresponding to the position according to the distance from the pixel at that position. Figure 2 This is a schematic diagram of the Planar mode principle, as shown in Figure 2 As shown, the arrow line in the horizontal direction represents the weighted average of the two reference pixels in the horizontal direction (the 0th column and the W+1th column) according to the distance, and the arrow line in the vertical direction represents the weighted average of the two reference pixels in the vertical direction (the 0th row and the H+1th row) according to the distance, and the average of the horizontal and vertical directions is used as the predicted pixel value of the current position.

[0067] The conventional Merge mode is an inter-frame prediction mode that directly uses the motion vector information (reference frame index, motion vector) of the motion vector prediction (motion vector prediction, MVP) as the motion vector information of the current block. First, a Merge candidate list (Merge candidate list) needs to be established, and then one of the candidates MVP in the Merge MVP list is selected as the motion vector (motion vector, MV) of the current block. When encoding, only the index value in the list where the selected MVP is located needs to be encoded, and there is no need to transmit the motion vector difference (motion vector difference, MVD) and the reference frame index. The pixel prediction value of the current block can be obtained based on the selected MV information, and the prediction residual can be further encoded from the original pixel value and the predicted pixel value of the current block to complete the encoding of the current block.

[0068] When the current block adopts CIIP mode, the intra-frame prediction value P is obtained by Planar mode. intra , the inter-frame prediction value P is obtained by the conventional Merge mode inter , and weight it to get the predicted value of the current block:

[0069] P CIIP =(w inter *P inter +w intra *P intra +2)>>2

[0070] Among them, w inter is the inter-frame prediction weight, w intra is the intra-frame prediction weight, whose value is derived from the coding mode of the adjacent blocks on the top (Top) and left (Left) of the current block. Figure 3 This is a schematic diagram of CIIP weight derivation, as shown in Figure 3As shown, the weight of the current block is derived from the coding modes of the upper and left adjacent blocks. The specific weight values ​​are shown in Table 1:

[0071] Table 1 CIIP weight settings

[0072] Upper neighbor block coding mode Left adjacent block coding mode <![CDATA[Current block (w intra , w inter )]]> Intra-frame coding Intra-frame coding (3,1) Intra-frame coding Interframe Coding (2,2) Interframe Coding Intra-frame coding (2,2) Interframe Coding Interframe Coding (1,3)

[0073] After the encoder encodes the block in CIIP mode, it needs to send the syntax elements related to CIIP mode to the decoder. There are three syntax elements related to CIIP mode, as shown below:

[0074] (1) sps_ciip_enabled_flag: SPS level, identifies whether the current sequence is allowed to use the CIIP mode. 1 indicates allowed, and 0 indicates not allowed.

[0075] (2) ciip_flag: block level, identifies whether the current block uses CIIP mode, 1 means enabled, 0 means disabled.

[0076] (3) merge_idx: block level, identifies the index of the current block MVP in the Merge MVP list.

[0077] Whether the current block adopts CIIP mode can be explicitly specified by ciip_flag as 1, or by implicitly inferring ciip_flag as 1. If the following four conditions are met, ciip_flag is inferred to be 1:

[0078] (1) SPS level syntax element sps_ciip_enabled_flag=1.

[0079] (2) The block is not in Skip mode.

[0080] (3) The block belongs to the Merge mode, but not the regular Merge mode, SbTMVP mode, or affine Merge mode.

[0081] (4) The width and height of the block are both less than 128, and the block contains at least 64 brightness pixels (width*height>=64).

[0082] At the JVET-W meeting in July 2021, for the intra prediction mode of the CIIP mode, it was proposed to replace the original Planar mode with the intra prediction mode obtained based on the TIMD method, in order to refine the intra prediction mode of the CIIP mode without increasing signaling transmission. TIMD is a technology that derives intra prediction modes based on templates (L-shape templates). The principle is to first derive the candidate list of TIMD from the most probable mode (MPM) list and expand the number of intra prediction modes from 67 to 131; then for each candidate prediction mode in the candidate list, use its L-shape template reference pixels to generate the template prediction pixels of the current block in the corresponding mode; then calculate the minimum transform domain absolute error and cost (sum of absolute transformed difference cost, SATD cost) based on the reconstructed (or reference) pixels and predicted pixels of the current block template, select the mode corresponding to the minimum SATD cost as the intra prediction mode of the current block, and map the prediction mode to one of the original 67 conventional prediction modes through a table.

[0083] When the CIIP mode is combined with TIMD, i.e., the CIIP-TIMD mode, the weight of the CIIP mode is also improved. If the intra prediction mode derived by TIMD is Planar or DC mode, the weight coefficient remains unchanged, as shown in Table 1; otherwise, i.e., the intra prediction mode derived by TIMD is angular mode, the current block needs to be equally divided into 4 sub-blocks, and different weight coefficients are used for each sub-block: for modes close to the horizontal direction, i.e., near-horizontal modes (intra mode index is less than 34), the current block is equally divided into 4 sub-blocks vertically, as shown in Figure 4(a); for modes close to the vertical direction, i.e., near-vertical modes (intra mode index is greater than or equal to 34), the current block is equally divided into 4 sub-blocks horizontally, as shown in Figure 4(b), and the weight coefficient of each sub-block is shown in Table 2. In addition, CIIP-TIMD requires that the current block contains at most 1024 pixels, i.e., the width*height of the CU is less than or equal to 1024. Figure 4a and Figure 4b Schematic diagram of the division method of CIIP-TIMD angle mode.

[0084] Table 1 Weight settings of CIIP-TIMD angle mode

[0085] Sub-block index <![CDATA[Current block (w intra , w inter )]]> 0 (6,2) 1 (5,3) 2 (3,5) 3 (2,6)

[0086] At the JVET-X conference in October 2021, the CIIP-TM mode was proposed based on the CIIP-TIMD mode, that is, the inter-frame prediction part is based on the MV candidate after TM refinement, and the inter-frame prediction mode at this time is TM-Merge. The TM mode is a decoding-side MV derivation method that refines the motion information of the current block, and the MV is refined by searching for the best matching template (L-shape or Top or Left template) between the current frame and the reference frame. Figure 5 It is a schematic diagram of template matching near the initial MV, such as Figure 5 As shown, the best matching template is searched near the initial MV to obtain a better refined MV, and the search area is the [-8,8] pixel range of the current block. In TM-Merge mode, the merge candidates indicated by the merge index also use a similar search method. When searching based on TM, 1 / 8 pixel MVD accuracy can be achieved or MVD accuracy of more than half a pixel can be skipped, depending on whether an alternative interpolation filter is used based on the merge motion information. In addition, template matching can be used as an independent process or an additional MV refinement process based on block-based and sub-block-based bilateral matching methods.

[0087] In CIIP-TM mode, it is necessary to construct a CIIP-TM Merge candidate list. The construction method is the same as that of the regular Merge list. The difference is that the CIIP-TM Merge candidate list has at most 2 candidates; the candidates in the TM-Merge list are MV candidates after TM refinement, and reordering is achieved through adaptive reordering of mergecandidates (ARMC) (the same as the regular CIIP mode).

[0088] If the current block is encoded in CIIP mode, the SPS-level syntax element sps_ciip_enabled_flag is marked as 1, and the block-level syntax element ciip_flag is marked as 1. Whether the current block adopts the CIIP-TM mode depends on the other two block-level syntax elements: pdpc_flag and tm_merge_flag. If pdpc_flag is 1, the intra-frame prediction mode of the CIIP-TM mode defaults to the Planar mode, and the PDPC process is applied, otherwise its intra-frame prediction mode is the mode derived from TIMD, and the PDPC process is not applied; if tm_merge_flag is 1, it indicates that the inter-frame prediction mode of the CIIP-TM mode is the TM-Merge mode, otherwise its inter-frame prediction mode is the conventional Merge mode.

[0089] At the encoding end, the encoder determines the value of pdpc_flag through rate distortion optimized (RDO) decision, and determines the value of tm_merge_flag through the competition between the conventional Merge mode and the TM-Merge mode, or through the competition between the CIIP-merge and CIIP-TM-merge processes. Then, the above four syntax elements and the merge index or TM-merge index are transmitted to the decoding end. After receiving the relevant syntax elements, the decoder performs the corresponding correct decoding to obtain the reconstructed value of the current block. The sps_ciip_enabled_flag determines whether the current sequence allows CIIP mode encoding. If allowed, the ciip_flag determines whether the current block adopts the CIIP mode. If adopted, When using the CIIP mode, it is further determined according to pdpc_flag whether the intra-frame prediction mode of the CIIP-TM mode is the mode derived from TIMD, and according to tm_merge_flag whether the inter-frame prediction mode of the CIIP-TM mode is the TM-Merge mode; and the intra-frame prediction reconstruction value of the current block is obtained according to the intra-frame prediction mode determined by decoding, and the same merge list is constructed at the decoding end in the same way as the merge list is constructed at the encoding end in the conventional Merge mode, and the MV candidate of the current block is determined by the merge index or the TM-merge index, and the inter-frame prediction reconstruction value of the current block is obtained, and finally the intra-frame and inter-frame prediction reconstruction values ​​are weighted averaged to obtain the reconstruction value of the current block, and correct decoding is completed.

[0090] Figure 6 A flowchart of a method for determining weights based on template costs provided in an embodiment of the present application is shown in FIG. Figure 6 As shown, the method for determining weights based on template costs provided in this embodiment includes:

[0091] Step S610: divide each image frame in the video stream to be encoded into a plurality of blocks.

[0092] The method for determining weights based on template cost provided in this embodiment is used to optimize weights in the process of encoding a video image stream to be transmitted. The method for determining weights based on template cost provided in this embodiment is applied to the encoding end, that is, one end of the video stream to be transmitted. Figure 1 As shown in the hybrid coding framework, the video image stream needs to go through several processes before transmission to compress the video image. The method provided in this embodiment is based on Figure 1 The hybrid coding framework shown only improves the way of determining prediction weights during intra-frame prediction and inter-frame prediction.

[0093] Through the above description and Figures 1 to 5It can be seen from the records that in order to improve the accuracy of intra-frame prediction and inter-frame prediction, a joint intra-frame and inter-frame prediction mode, namely, the CIIP mode, can be used at present. However, in the current CIIP mode, the intra-frame prediction weights and the inter-frame prediction weights are both obtained by table lookup, that is, the intra-frame prediction weights and the inter-frame prediction weights are both values ​​in a fixed template, but this is obviously not flexible enough. The intra-frame prediction weights and the inter-frame prediction weights in the fixed template do not necessarily match the actual intra-frame prediction weights and the inter-frame prediction weights. Therefore, when the current CIIP mode is used for joint intra-frame and inter-frame prediction, there may be errors in the prediction accuracy.

[0094] Therefore, in this embodiment, a method for determining weights based on template cost is proposed. The method does not determine the values ​​of intra-frame prediction weights and inter-frame prediction weights based solely on the weight template, but rather determines the intra-frame prediction weights and inter-frame prediction weights based on the judgment and calculation of multiple factors and the comprehensive determination based on the template cost, thereby achieving accurate prediction of blocks in the video image and improving video encoding performance.

[0095] First, each image frame in the video stream to be encoded needs to be divided into multiple block elements. That is, the original video to be encoded is input into the hybrid coding framework. The video image to be encoded consists of a frame sequence composed of multiple image frames. For each image frame, it is first divided into multiple CTUs. For example, each frame of the image can be divided into 128*128 CTUs. Then each CTU is divided into blocks according to the quadtree or multi-type tree recursive structure, or it can be further divided, that is, the block size is the same as the CTU. However, there are two points worth noting: first, quadtree division cannot be performed after division using the multi-type tree recursive structure; second, the CIIP-TM encoding tool has requirements for block size, that is, the block contains at least 64 pixels and at most 1024 pixels, that is, the product of width and height is greater than or equal to 64 and less than or equal to 1024, otherwise CIIP-TM cannot be enabled.

[0096] Step S620: performing intra / inter prediction mode decision on a current block among the multiple blocks.

[0097] For each of the multiple blocks after division, it is necessary to make an intra / inter prediction mode decision, that is, to determine the prediction mode used for prediction of each block. For a block, the aforementioned intra prediction mode can be used, the inter prediction mode can be used, or the joint intra / inter prediction mode, that is, the CIIP mode, can be used. For the current block among the multiple blocks, the encoding end makes an intra / inter prediction mode decision through RDO decision and the competition between various prediction modes. The specific decision method has been described in detail in the aforementioned content.

[0098] Performing an intra / inter prediction mode decision on a current block among multiple blocks may be performing an intra / inter prediction mode decision on a current block among multiple blocks to determine whether the current block uses inter prediction coding or intra prediction coding; if the current block uses inter prediction coding, then determining whether the current block uses the CIIP mode.

[0099] Step S630: If it is determined that the current block uses the CIIP mode, then the intra prediction weight and the inter prediction weight are determined based on the template cost.

[0100] After making an intra / inter prediction mode decision for the current block, if it is determined that the current block uses the CIIP mode, that is, joint intra and inter prediction, then the intra prediction weight and the inter prediction weight are determined based on the template cost. According to the above records, two joint prediction modes, the CIIP-TIMD mode and the CIIP-TM mode, are currently proposed based on the CIIP mode, but the above two joint prediction modes still use preset weights as the weights of intra prediction and inter prediction, and then perform joint prediction by weighted average to obtain the prediction value of the joint prediction. In this embodiment, the intra prediction weight and the inter prediction weight are determined based on the template cost, and the CIPP mode based on the template cost is proposed, which can be expressed as TM-CIIP. The difference from CIIP-TM is that the method provided in this embodiment does not perform joint prediction of intra and inter prediction according to preset weights, but determines the intra and inter prediction weights based on the template cost.

[0101] Specifically, in the embodiment, if it is determined that the current block uses the CIIP mode, including determining that the current block uses the CIIP mode or the CIIP-TM mode, then the intra-frame prediction mode of the CIIP / CIIP-TM mode is the Planar mode or the mode derived from TIMD, and the inter-frame prediction mode is the conventional merge or TM-Merge mode. The weight coefficient of the weighted average of the CIIP / CIIP-TM mode is determined by the template cost corresponding to the intra-frame prediction mode and the inter-frame prediction mode, that is, the sum of absolute difference cost (SAD cost), so that the template information can be fully utilized to achieve better prediction and improve the encoding performance.

[0102] Further, determine whether the current block uses the CIIP mode, that is, determine whether the SPS-level syntax element sps_ciip_enabled_flag is 1, and whether the block-level syntax element ciip_flag is 1; and further determine whether the CIIP-TM mode is enabled for the current block. This can be determined by the value of the block-level syntax element pdpc_flag to determine whether the intra-frame prediction mode of the CIIP-TM tool is the default Planar mode, or the prediction mode derived from TIMD.

[0103] Step S640, determining a joint prediction value of the current block according to the intra prediction value, intra prediction weight, inter prediction value and inter prediction weight of the current block, and taking the joint prediction value of the current block as the prediction value of the current block.

[0104] When performing joint intra-frame and inter-frame prediction on the current block based on the template cost, the intra-frame prediction weight and the inter-frame prediction weight can be determined, as well as the intra-frame prediction value and the inter-frame prediction value of the current block. In this way, the joint prediction value of the current block can be determined based on the intra-frame prediction value, the intra-frame prediction weight, the inter-frame prediction value and the inter-frame prediction weight of the current block, and the joint prediction value of the current block is used as the prediction value of the current block. In fact, the weighted average is performed based on the intra-frame prediction value, the intra-frame prediction weight, the inter-frame prediction value and the inter-frame prediction weight of the current block to obtain the joint prediction value of the current block as the final prediction value.

[0105] Step S650: transmitting the syntax elements related to determining the intra-frame prediction weight and the inter-frame prediction weight of the current block based on the template cost to the decoding end.

[0106] After the coding segment performs joint intra-frame and inter-frame prediction on the current block based on the template cost, it is necessary to transmit the relevant syntax elements for determining the intra-frame prediction weights and inter-frame prediction weights of the current block based on the template cost to the decoding end, so that the decoding end can know the coding method adopted by the current block and complete the reconstruction of the current block according to the relevant syntax elements for determining the intra-frame prediction weights and inter-frame prediction weights of the current block based on the template cost.

[0107] Among them, the relevant syntax elements for determining the intra-frame prediction weight and inter-frame prediction weight of the current block based on the template cost include: the SPS-level flag allowing the CIIP mode to be enabled and the enabling flag sps_ciip_enabled_flag and sps_tmcost_dir_enabled_flag allowing the weight to be determined based on the template cost; the CU-level CIIP mode enabling flag ciip_flag and the enabling flag tmcost_dir_flag allowing the weight to be determined based on the template cost.

[0108] In addition, the encoding end may also transmit the syntax elements related to the current block and the intra prediction and inter prediction to the decoding end, that is, transmit other syntax elements except the syntax elements related to the current block determining the intra prediction weight and the inter prediction weight based on the template cost to the decoding end. The syntax elements related to the current block and the intra prediction and inter prediction include: the identifier pdpc_flag for determining the intra prediction mode at the block level, the identifier tm_merge_flag for determining the inter prediction mode at the CU level, and the corresponding candidate list index merge_idx.

[0109] The method for determining weights based on template cost provided in the present embodiment divides each image frame in the video stream to be encoded into multiple blocks; makes an intra / inter prediction mode decision for the current block among the multiple blocks; if it is determined that the current block uses the CIIP mode, determines the intra prediction weight and the inter prediction weight based on the template cost; determines the joint prediction value of the current block according to the intra prediction value, intra prediction weight, inter prediction value and inter prediction weight of the current block, and uses the joint prediction value of the current block as the prediction value of the current block; transmits the relevant syntax elements for determining the intra prediction weight and the inter prediction weight of the current block based on the template cost to the decoding end. The method provided in the present embodiment makes full use of the template information of the current block, improves the prediction accuracy of the CIIP-TM tool, does not increase too many extra transmission bits, and effectively improves the encoding performance.

[0110] exist Figure 6 In the illustrated embodiment, the intra-frame prediction weight and the inter-frame prediction weight are determined based on the template cost. The intra-frame prediction weight and the inter-frame prediction weight can be determined directly based on the template cost; or the intra-frame prediction weight and the inter-frame prediction weight can be determined indirectly based on the template cost. The following will respectively focus on the two different methods of direct determination and indirect determination. Figure 6 The method for determining weights based on template costs provided in the embodiment of the present application is further described in detail.

[0111] Among them, for the scheme of directly determining the intra-frame prediction weight and inter-frame prediction weight based on the template cost, first, when the CIIP / CIIP-TM mode is enabled for the current coding block (block), that is, the intra-frame prediction mode of the CIIP-TM mode is the Planar mode or the mode derived from TIMD, and the inter-frame prediction mode is the conventional Merge or TM-Merge mode, the weighting coefficient is determined by the template cost of the intra-frame and inter-frame prediction; then the intra-frame and inter-frame prediction values ​​are weighted averaged as the prediction of the current block (block). In addition, the weighting coefficient determined based on the template cost can be used not only for the luminance component, but also for the chrominance component. That is, in step S620, it is determined that the current block uses the CIIP mode. First, it is necessary to add an SPS-level syntax element sps_tmcost_dir_enabled_flag (an enable flag that allows template cost to determine weights) to indicate whether the sequence where the current block is located allows the weighting coefficient to be directly determined using the template cost; it is also necessary to add a CU-level syntax element tmcost_dir_flag (an enable flag that allows template cost to determine weights) to indicate whether the current block uses the template cost to directly determine the weighting coefficient.

[0112] At this time, the specific method for determining the intra-frame prediction weight and the inter-frame prediction weight based on the template cost includes:

[0113] (1.1) When the CIIP / CIIP-TM mode is enabled in the current coding block, the inter-frame prediction mode adopted is determined by tm_merge_flag (block-level identification of the inter-frame prediction mode), which may be the conventional Merge mode or the TM-Merge mode, and the corresponding candidate list is constructed. Then, each candidate in the list is traversed to calculate the rate-distortion cost and determine the inter-frame template cost corresponding to each MV candidate. inter .

[0114] (1.2) The intra prediction mode of the CIIP / CIIP-TM tool is determined by pdpc_flag. If the mode derived from TIMD is used as the intra prediction mode, the intra template cost cost is determined at this time. intra It is worth noting that the mode derived from TIMD may be Planar or DC mode or the block width is less than or equal to 4 or the block height is less than or equal to 4, or it may be an angle mode. There are two optional implementations for these two situations, and their intra-frame template costs are determined respectively, as shown in optional implementations (1-a) and (1-b); according to the two template costs cost intra and cost inter Determine the weighting coefficients as follows

[0115] w inter =cost intra / (cost intra +cost inter )

[0116] w intra =1-w inter

[0117] (1.3) Using the weights obtained in step (1.2), perform weighted averaging on the intra-frame prediction value of the CIIP / CIIP-TM mode and the prediction values ​​of each MV candidate between frames, and select the optimal weighted result as the prediction value of the current block:

[0118] P CIIP =(w inter *P inter +w intra *P intra +1)>>1

[0119] (1.4) The syntax elements related to the method of directly determining the weights of the inter-frame and intra-frame modes based on the template cost are transmitted to the decoding end.

[0120] The syntax elements related to this embodiment include: the flag sps_ciip_enabled_flag that allows the CIIP mode to be enabled at the SPS level of the current block, the flag sps_tmcost_dir_enabled_flag that allows the template cost to determine the weight, the CIIP mode enabling flag ciip_flag at the CU level, the flag pdpc_flag that determines the intra-frame prediction mode at the CU level, the flag tm_merge_flag that determines the inter-frame prediction mode at the CU level, the enabling flag tmcost_dir_flag that uses the template cost to directly determine the weight at the CU level, and the corresponding candidate list index. These syntax elements need to be transmitted to the decoding end in the bitstream. After receiving the relevant flags, the decoding end can determine whether to enable the CIIP / CIIP-TM mode and whether to use the template cost to directly determine the weight coefficients of the inter-frame and intra-frame modes, and complete the correct decoding process according to the same operation as the encoding end.

[0121] (1-a) Directly determine the non-angle mode weight based on the template cost. If the intra-frame prediction mode of the CIIP / CIIP-TM mode is the Planar or DC mode derived from TIMD or the block width / height is less than or equal to 4, and the inter-frame prediction mode is the conventional Merge or TM-Merge mode, the weight coefficient is determined using the template cost of the intra-frame and inter-frame prediction modes, and no longer refers to Table 1 to set the fixed weight; otherwise, that is, the intra-frame prediction mode is the angle mode, the current block is still divided according to the existing ECM algorithm, and the fixed weight coefficients of the four sub-blocks are set, refer to Table 2; then the intra-frame and inter-frame prediction values ​​are weighted averaged as the prediction of the current block.

[0122] (1-b) Directly determine the angle mode weight based on the template cost. If the intra prediction mode of the CIIP / CIIP-TM mode is the angle mode derived from TIMD, and the inter prediction mode is the conventional Merge or TM-Merge mode, the weight coefficient is determined using the template cost of the intra and inter predictions. It is worth noting that: at this time, there is no need to divide the current block into four equal parts, but the template cost of the entire current block in the intra angle mode is used as costIntra to calculate the weight coefficient; otherwise, that is, the intra prediction mode is the Planar or DC mode derived from TIMD, the fixed weight coefficient is still set according to the existing ECM algorithm, refer to Table 1; then the intra and inter prediction values ​​are weighted averaged as the prediction of the current block.

[0123] For the scheme of indirectly determining intra prediction weight and inter prediction weight based on template cost, when CIIP / CIIP-TM mode is enabled for the current coding block, that is, the intra prediction mode is Planar mode or mode derived from TIMD, and the inter prediction mode is conventional Merge or TM-Merge mode, the intra and inter prediction templates are weighted respectively using the existing fixed weight coefficients, thereby obtaining the template costs under different weights, and the fixed weight coefficient corresponding to the template with the minimum template cost is used as the weight coefficient of the current block; then the intra and inter prediction values ​​are weighted averaged as the prediction of the current block. The weight coefficient indirectly determined based on template cost can be used not only for luminance components, but also for chrominance components.

[0124] Compared with the above-mentioned scheme of directly determining the weights, the difference is that the weighting coefficient at this time is not directly calculated using the template cost according to the formula of Example 1, but one of the fixed values ​​is selected from the fixed weights set by the existing algorithm (Table 1 or Table 2). The selection basis is firstly calculating the weighted values ​​of the intra-frame and inter-frame prediction templates under different fixed weights, and then determining one of the fixed weights as the optimal weighting coefficient through the minimum template cost.

[0125] Similar to the above-mentioned scheme of directly determining weights, this embodiment also needs to add syntax elements related to template costs: an SPS-level enable flag sps_tmcost_indir_enabled_flag, which indicates whether the sequence in which the current block is located allows the use of template costs to indirectly determine weighting coefficients; and a block-level syntax element tmcost_indir_flag needs to be added to indicate whether the current block uses template costs to indirectly determine weighting coefficients.

[0126] At this time, the specific method for determining the intra-frame prediction weight and the inter-frame prediction weight based on the template cost includes:

[0127] (2.1) When the CIIP / CIIP-TM mode is enabled in the current coding block, the inter-frame prediction mode is determined by merge_flag or tm_merge_flag, and a corresponding candidate list is constructed. Then, each candidate in the list is traversed to calculate the rate-distortion cost and determine the inter-frame prediction template corresponding to each candidate.

[0128] (2.2) The intra-frame prediction mode of CIIP / CIIP-TM is determined by pdpc_flag. If the mode derived from TIMD is used as the intra-frame prediction mode, the weighted values ​​of the intra-frame and inter-frame prediction templates are calculated respectively as the final prediction templates according to the fixed weights (Table 1 or Table 2) selected for the prediction mode at this time, and the fixed weight corresponding to the prediction template with the minimum template cost between the final prediction template and the current block reconstruction template is used as the weighting coefficient.

[0129] If the intra prediction template is Tintra , the inter-frame prediction template is T inter , the fixed weight values ​​for intra-frame and inter-frame prediction are w intra and w inter , then the weighted final prediction template T pred As shown below:

[0130] T pred =(w inter *T inter +w intra *T intra +2)>>2

[0131] It is worth noting that the mode derived from TIMD may be Planar or DC mode or block width / height less than or equal to 4, or it may be an angle mode. These two cases correspond to different fixed weight values, as shown in Table 1 and Table 2. Therefore, this embodiment has two optional implementations, as shown in optional implementations (2-a) and (2-b).

[0132] (2.3) Use the weights obtained in step (2.2) to perform weighted averaging on the intra-frame prediction value and each MV candidate prediction value between frames, and select the optimal weighted result as the prediction value of the current block:

[0133] P CIIP =(w inter *P inter +w intra *P intra +2)>>2

[0134] (2.4) The syntax elements related to the method for indirectly determining the weights of the inter-frame and intra-frame modes based on the template cost are transmitted to the decoding end.

[0135] The syntax elements related to this embodiment include the flag sps_ciip_enabled_flag of the current block at the SPS level that allows the CIIP mode to be enabled, the flag sps_tmcost_indir_enabled_flag that allows the template cost to determine the weight, the CIIP mode enabling flag ciip_flag at the block level, the flag pdpc_flag that determines the intra-frame prediction mode at the block level, the flag tm_merge_flag that determines the inter-frame prediction mode at the block level, the enabling flag tmcost_indir_flag that uses the template cost to directly determine the weight at the block level, and the corresponding candidate list index. These syntax elements need to be transmitted to the decoding end in the bitstream. After receiving the relevant flags, the decoding end can determine whether to enable the CIIP / CIIP-TM mode and whether to use the template cost to indirectly determine the weight coefficients of the inter-frame and intra-frame modes, and complete the correct decoding process according to the same operation as the encoding end.

[0136] (2-a) Indirect determination of non-angle mode weights based on template cost. If the intra prediction mode of the CIIP / CIIP-TM mode is the Planar or DC mode derived from TIMD or the block width / height is less than or equal to 4, and the inter prediction mode is the conventional Merge or TM-Merge mode, the intra and inter prediction templates are weighted according to the original fixed weights (the three groups of weighting coefficients shown in Table 1: (wintra, winter) = {(3, 1), (2, 2), (1, 3)}) to obtain three different final prediction templates, and then the SATD cost between these three prediction templates and the current block reconstruction template is calculated, that is, the template cost, and a set of fixed weights corresponding to the template with the minimum template cost is used as the weighting coefficient; otherwise, that is, the intra prediction mode is the angle mode, the current block is still divided according to the existing ECM algorithm, and the fixed weighting coefficients of the four sub-blocks are set, refer to Table 2; then the intra and inter prediction values ​​are weighted averaged as the prediction of the current block.

[0137] (2-b) Indirectly determine the angle mode weight based on the template cost. If the intra prediction mode of the CIIP / CIIP-TM mode is the angle mode derived from TIMD, and the inter prediction mode is the conventional Merge or TM-Merge mode, the current block is vertically or horizontally divided into 4 sub-blocks according to the angle mode, and the intra and inter prediction templates are weighted according to the original fixed weights (the 4 groups of weighting coefficients shown in Table 2: (wintra,winter) = {(6,2), (5,3), (3,5), (2,6)}) to obtain 4 different final prediction templates, and then calculate the SATD cost between these four prediction templates and the current block reconstruction template, that is, the template cost, and use a set of fixed weights corresponding to the template with the minimum template cost as the weighting coefficient; otherwise, that is, the intra prediction mode is Planar or DC mode, the fixed weighting coefficients are still set for the intra and inter prediction of the current block according to the existing ECM algorithm, refer to Table 1; then the intra and inter prediction values ​​are weighted averaged as the prediction of the current block.

[0138] It is worth noting that if the angle mode is horizontal, Figure 7a As shown in the figure, the intra-frame and inter-frame prediction templates are Top templates, and the template weights are calculated in the form of four equal parts to obtain four different prediction templates; similarly, for the angle mode, the vertical direction is biased, that is, according to Figure 7b As shown in the figure, the intra-frame and inter-frame prediction templates are Left templates, and the template weights are calculated in a four-equal division form to obtain four different prediction templates. Figure 7a and Figure 7b Schematic diagram of template weighting for CIIP-TM angle mode.

[0139] Figure 8 A flowchart of another method for determining weights based on template cost provided in an embodiment of the present application, such as Figure 8 As shown, the method for determining weights based on template costs provided in this embodiment includes:

[0140] Step S810, receiving the encoded video code stream and parsing the video code stream.

[0141] The method for determining weights based on template cost provided in this embodiment is used to optimize weights in the decoding process of a received coded video image stream. The method for determining weights based on template cost provided in this embodiment is applied to the decoding end, that is, the end receiving the video stream to be transmitted. After receiving the coded video image stream, several processes are required to reconstruct the video image. The method provided in this embodiment is based on Figure 1 The hybrid coding framework shown only improves the way of determining prediction weights during intra-frame prediction and inter-frame prediction.

[0142] First, the decoder receives the encoded video bitstream, that is, the bitstream containing video information, processes the Network Abstraction Layer (NAL), reads the Network Abstraction Layer Unit (NALU), and checks whether the NALU is normal. If it is normal, it means that a new picture is input into the bitstream and needs to be decoded. Otherwise, it means that no new picture is input. Then the decoder needs to parse the video bitstream, that is, to obtain the information contained in the video bitstream.

[0143] Step S820: Obtain blocks in the parsed video bitstream, and read syntax elements related to intra / inter prediction in the parsed video bitstream.

[0144] The decoder at the decoding end performs loop filtering and a series of decomposition operations on the decoded frame of the picture, and performs different types of decoding according to different NAL types; when the decoder decodes a frame of the picture, it first traverses the slice and then traverses each CTU. Each CTU needs to be divided to obtain the coding block s, and checks whether the SPS-level syntax element sps_ciip_enabled_flag of the current block is marked as turned on. That is, read the syntax elements related to intra-frame / inter-frame prediction in the parsed video bitstream.

[0145] Step S830: determining the intra / inter prediction mode of the current block according to the parsed current block and the intra / inter prediction related syntax elements.

[0146] After reading the syntax elements related to intra / inter prediction in the parsed video bitstream, the intra / inter prediction mode of the current block is determined according to the parsed current block and the syntax elements related to intra / inter prediction, that is, whether each block in a frame of the picture is encoded using the CIIP mode.

[0147] Specifically, first determine whether the current block uses inter-frame prediction coding or intra-frame prediction coding according to the parsed current block and intra-frame / inter-frame prediction related syntax elements; if the current block uses inter-frame prediction coding, determine whether the current block uses CIIP mode.

[0148] Specifically, before decoding each block, it is necessary to check whether the current block is in Skip mode, palette mode, and whether it is in intra-frame or inter-frame prediction mode; if it is determined that the current block uses inter-frame prediction coding, it is further determined whether the conventional Merge mode is used. If the conventional Merge mode is not used, it is further determined whether the CIIP mode is used, that is, whether the ciip_flag is 1. It is further checked whether the CIIP-TM mode is enabled for the current block. If pdpc_flag = 0, the intra-frame prediction mode used is the mode derived from TIMD, otherwise it is the default Planar mode; if tm_merge_flag = 1, the inter-frame prediction mode used is the TM-Merge mode, otherwise it is the default conventional Merge mode.

[0149] Step S840: If it is determined that the current block uses the CIIP mode, then the intra-frame prediction weight and the inter-frame prediction weight are determined based on the template cost.

[0150] If it is determined that the current block uses the CIIP mode, the intra prediction weight and the inter prediction weight are determined based on the template cost, that is, at the decoding end, the same method as the encoding end is used to perform joint intra and inter prediction. The specific method is the same as step S630.

[0151] Step S850, determining a joint prediction value of the current block according to the intra prediction value, intra prediction weight, inter prediction value and inter prediction weight of the current block, and using the joint prediction value of the current block as a reconstructed value of the current block.

[0152] This step is the same as step S640, and after determining the intra prediction weight and the inter prediction weight, and determining the intra prediction value and the inter prediction value of the current block, the joint prediction value of the current block is determined according to the intra prediction value, the intra prediction weight, the inter prediction value and the inter prediction weight of the current block, and the joint prediction value of the current block is used as the prediction value of the current block. At the decoding end, the final prediction value is further used as the reconstruction value of the current block.

[0153] Step S860, obtaining a decoded video stream according to the reconstructed value of each of the multiple blocks after parsing.

[0154] After decoding and reconstructing each block of each frame of the video stream, a reconstructed image of each decoded frame can be obtained, and then a reconstructed video stream can be obtained, thus completing video decoding.

[0155] The method for determining weights based on template cost provided in the present embodiment receives an encoded video bitstream and parses the video bitstream; obtains a block in the parsed video bitstream and reads syntax elements related to intra / inter prediction in the parsed video bitstream; determines the intra / inter prediction mode of the current block according to the parsed current block and syntax elements related to intra / inter prediction; if it is determined that the current block uses the CIIP mode, determines the intra prediction weight and the inter prediction weight based on the template cost; determines the joint prediction value of the current block according to the intra prediction value, intra prediction weight, inter prediction value and inter prediction weight of the current block, and uses the joint prediction value of the current block as the reconstruction value of the current block; obtains the decoded video stream according to the reconstruction value of each of the multiple parsed blocks. The method provided in the present embodiment makes full use of the template information of the current block, improves the prediction accuracy of the CIIP-TM tool, does not increase too many extra transmission bits, and effectively improves the encoding performance.

[0156] exist Figure 8 In the illustrated embodiment, the intra-frame prediction weight and the inter-frame prediction weight are determined based on the template cost. The intra-frame prediction weight and the inter-frame prediction weight can be determined directly based on the template cost; or the intra-frame prediction weight and the inter-frame prediction weight can be determined indirectly based on the template cost. The following will respectively focus on the two different methods of direct determination and indirect determination. Figure 8 The method for determining weights based on template costs provided in the illustrated embodiment is further described in detail.

[0157] Among them, for the scheme of directly determining the intra-frame prediction weight and the inter-frame prediction weight based on the template cost, the decoder first checks the high-level syntax element sps_ciip_enabled_flag in the code stream to indicate whether the sequence where the current block is located allows the CIIP mode to be enabled; under the premise that the SPS level allows the flag to be enabled, it further checks whether the block-level flag ciip_flag is turned on. If it is turned on, it checks the block-level flag pdpc_flag, which indicates whether the intra-frame prediction mode of the CIIP mode of the current block is a mode derived from TIMD, and the block-level flag tm_merge_flag, which indicates whether the inter-frame prediction mode of the CIIP mode of the current block is the TM-Merge mode, that is, determines whether the current block adopts the CIIP-TM mode, and checks the merge index or TM-merge index to determine the MV reconstruction value of the current block.

[0158] In addition, the decoder also needs to check the high-level syntax element sps_tmcost_dir_enabled_flag to indicate whether the sequence in which the current block is located allows the weighting coefficient to be directly determined based on the template cost; when the SPS level allows the enabling flag to be turned on, the block-level flag tmcost_dir_flag is further checked to indicate whether the current block uses the template cost to directly determine the weight.

[0159] The syntax structure is as follows:

[0160] … sps_ciip_enabled_flag sps_tmcost_dir_enabled_flag … ciip_flag if(ciip_flag){ pdpc_flag tm_merge_flag tmcost_dir_flag } …

[0161] The semantics of the above syntax are as follows:

[0162] sps_ciip_enabled_flag: equal to 1, indicating that the sequence where the current block is located allows the CIIP mode to be enabled; equal to 0, indicating that the sequence where the current block is located prohibits the CIIP mode from being enabled;

[0163] sps_tmcost_dir_enabled_flag: equal to 1 indicates that the sequence where the current block is located allows the weight coefficient to be directly determined based on the template cost; equal to 0 indicates that the sequence where the current block is located prohibits the weight coefficient to be directly determined based on the template cost;

[0164] ciip_flag: 1 indicates that the current block adopts CIIP mode, and 0 indicates that the current block does not adopt CIIP mode;

[0165] pdpc_flag: 1 indicates that the current block uses the PDPC process, and 0 indicates that the current block does not use the PDPC process;

[0166] tm_merge_flag: 1 indicates that the inter-frame prediction mode of the current block CIIP mode adopts the TM-Merge mode; 0 indicates that the inter-frame prediction mode of the current block CIIP mode does not adopt the TM-Merge mode;

[0167] tmcost_dir_flag: 1 indicates that the current block uses the template cost to directly determine the weighting coefficient; 0 indicates that the current block does not use the template cost to directly determine the weighting coefficient.

[0168] The decoding end determines the intra-frame prediction mode and the inter-frame prediction mode according to the received relevant syntax elements, and whether the current block adopts the weighting coefficient directly determined based on the template cost. Then, as with the encoding end, it constructs a merge or TM-Merge candidate list, determines the reconstruction value of the current block MV by the candidate index, and determines the intra-frame prediction value by the intra-frame prediction mode derived from TIMD or the default Planar mode. It also indicates whether the current block calculates the weighting coefficient of the TM-CIIP mode through the template cost according to the flag for enabling template cost determination weights, completes the weighted average of intra-frame and inter-frame predictions, and obtains the reconstructed block.

[0169] At this time, the specific method for determining the intra-frame prediction weight and the inter-frame prediction weight based on the template cost includes:

[0170] (3.1) Determine the MV reconstruction value and intra prediction mode of the current block, and directly determine the weight coefficient of the TM-CIIP mode of the current block based on the template cost.

[0171] When the SPS level flags sps_ciip_enabled_flag is 1, sps_tmcost_dir_enabled_flag is 1, and the block level flags ciip_flag is 1, tmcost_dir_flag is 1, the current block adopts the TM-CIIP mode in which the weighting coefficient is directly determined based on the template cost.

[0172] Refer to the process of building a list at the encoder end to build a conventional merge or TM-merge candidate list, and determine the MV candidate of the current block according to the corresponding candidate index; refer to the derivation process of TIMD or Planar mode to determine the intra-frame prediction mode of the current block

[0173] According to the intra prediction mode derived from TIMD, select the optional implementation (1-a) or (1-b) accordingly, refer to Figure 6 Steps (1.1) and (1.2) in the illustrated embodiment obtain the weighting coefficients of the TM-CIIP mode.

[0174] (3.2) Perform weighted averaging on the intra-frame and inter-frame predictions using the weighting coefficients determined in step (3.1) to determine the reconstructed value of the current decoding block.

[0175] For the scheme of indirectly determining the intra-frame prediction weight and inter-frame prediction weight based on template cost, the decoder first checks the high-level syntax element sps_ciip_enabled_flag in the bitstream to indicate whether the sequence where the current block is located allows the CIIP mode to be enabled; under the premise that the SPS level allows the flag to be enabled, it further checks whether the block-level flag ciip_flag is turned on. If it is turned on, it checks the block-level flag pdpc_flag, which indicates whether the intra-frame prediction mode of the CIIP mode of the current block is a mode derived from TIMD, and the block-level flag tm_merge_flag, which indicates whether the inter-frame prediction mode of the CIIP mode of the current block is the TM-Merge mode, and checks the corresponding candidate index to determine the MV reconstruction value of the current block.

[0176] In addition, the decoder also needs to check the SPS-level high-level syntax element sps_tmcost_indir_enabled_flag to indicate whether the sequence in which the current block is located allows the use of weighting coefficients indirectly determined based on template costs; when the SPS-level enabling flag is turned on, the block-level flag tmcost_indir_flag is further checked to indicate whether the current block uses indirect weight determination based on template costs.

[0177] The syntax structure is as follows:

[0178] … sps_ciip_enabled_flag sps_tmcost_indir_enabled_flag … ciip_flag if(ciip_flag){ pdpc_flag tm_merge_flag tmcost_indir_flag } …

[0179] The semantics of the above syntax are as follows:

[0180] sps_ciip_enabled_flag: equal to 1, indicating that the sequence where the current block is located allows the CIIP mode to be enabled; equal to 0, indicating that the sequence where the current block is located prohibits the CIIP mode from being enabled;

[0181] sps_tmcost_indir_enabled_flag: equal to 1, indicating that the sequence where the current block is located allows the indirect determination of weighting coefficients based on template cost, and equal to 0, indicating that the sequence where the current block is located prohibits the indirect determination of weighting coefficients based on template cost;

[0182] ciip_flag: 1 indicates that the current block adopts CIIP mode, and 0 indicates that the current block does not adopt CIIP mode;

[0183] pdpc_flag: 1 indicates that the current block uses the PDPC process, and 0 indicates that the current block does not use the PDPC process;

[0184] tm_merge_flag: 1 indicates that the inter-frame prediction mode of the current block CIIP mode adopts the TM-Merge mode; 0 indicates that the inter-frame prediction mode of the current block CIIP mode does not adopt the TM-Merge mode;

[0185] tmcost_indir_flag: 1 indicates that the current block uses the template cost to indirectly determine the weighting coefficient; 0 indicates that the current block does not use the template cost to indirectly determine the weighting coefficient.

[0186] The decoding end determines the intra-frame prediction mode and the inter-frame prediction mode according to the received relevant syntax elements, and whether the current block adopts the weighting coefficient indirectly determined based on the template cost. Then, as with the encoding end, it constructs a merge or TM-Merge candidate list, determines the reconstruction value of the current block MV by the candidate index, and determines the intra-frame prediction value by the intra-frame prediction mode or Planar mode derived by TIMD, and indicates whether the current block adopts the weighting coefficient indirectly determined based on the template cost according to the flag of enabling the indirect determination of the weight by the template cost, that is, the prediction templates are calculated separately according to the originally set fixed weights, and then the template costs between different prediction templates and the reconstruction template of the current block are determined, and the fixed weight corresponding to the template with the minimum template cost is used as the weighting coefficient of the current block, and the weighted average of the intra-frame and inter-frame predictions is completed to obtain the reconstructed block.

[0187] At this time, the specific method for determining the intra-frame prediction weight and the inter-frame prediction weight based on the template cost includes:

[0188] (4.1) Determine the MV reconstruction value and intra prediction mode of the current block, and indirectly determine the weight coefficient of the TM-CIIP mode of the current block based on the template cost;.

[0189] When the SPS level flag sps_ciip_enabled_flag is 1, sps_tmcost_indirect_enabled_flag is 1, and the block level flag ciip_flag is 1, tmcost_indirect_flag is 1, the current block adopts the TM-CIIP mode in which the weighting coefficient is indirectly determined based on the template cost.

[0190] Refer to the process of building a list at the encoding end to build a regular merge or TM-merge candidate list, and determine the MV candidate of the current block according to the corresponding candidate index; refer to the derivation process of TIMD or the Planar mode to determine the intra-frame prediction mode of the current block.

[0191] According to the intra prediction mode derived from TIMD, select the optional implementation (2-a) or (2-b) accordingly, refer to Figure 6Step 2.2 in the illustrated embodiment obtains the weighting coefficients of the TM-CIIP mode.

[0192] (4.2) Perform weighted averaging on the intra-frame and inter-frame predictions using the weighting coefficients determined in step (4.1) to determine the reconstructed value of the current decoding block.

[0193] The present application is an embodiment that proposes a TM-CIIP inter-frame prediction method based on template cost to determine weights for the traditional CIIP-TM inter-frame coding tool of ECM. The method for determining weights based on template cost includes: directly calculating new weighting coefficients by intra-frame and inter-frame prediction template costs; and calculating intra-frame and inter-frame prediction templates under different weights according to the existing fixed weights, and taking the fixed weight corresponding to the prediction template with the minimum template cost as the optimal weighting coefficient.

[0194] The method of directly determining the weight based on the template cost is to use the intra-frame prediction template cost intra and inter-frame prediction template cost inter Calculate the weighting coefficients to replace the fixed weights set in the existing algorithm; the method of indirectly determining the weights based on the template cost is to keep the fixed weights set by the existing algorithm, but the final weighting coefficients select a set of optimal weighting coefficients from the fixed weights, which is based on using the currently set multiple sets of fixed weights to further solve multiple prediction templates within and between frames, and use a set of fixed weights corresponding to the template with the minimum template cost as the optimal weighting coefficients.

[0195] In addition, since the intra-frame prediction mode in the CIIP-TM mode is derived from TIMD, and the intra-frame prediction mode derived from TIMD can be divided into two categories: one is the Planar or DC mode or the block width / height is less than or equal to 4 (that is, the non-angle mode), and the other is the angle mode, these two types of intra-frame prediction modes are processed differently in the current algorithm. Therefore, in the two proposed TM-CIIP methods for determining weights based on template costs, namely, Example 1 and Example 2, there are two optional implementation methods respectively.

[0196] The solution provided by the embodiment of the present application fully utilizes the template information of the current block, improves the prediction accuracy of the CIIP-TM tool, does not increase too many extra transmission bits, and effectively improves the coding performance.

[0197] Fig. 9 A schematic diagram of a structure of a device for determining weights based on template cost provided in an embodiment of the present application, such as Fig. 9 As shown, the device for determining weights based on template costs provided in this embodiment includes:

[0198] A segmentation module 91 is configured to divide each image frame in a video stream to be encoded into a plurality of blocks; a decision module 92 is configured to make an intra / inter prediction mode decision for a current block among the plurality of blocks; a weight determination module 93 is configured to determine an intra prediction weight and an inter prediction weight based on a template cost if it is determined that the current block uses the CIIP mode; a joint prediction module 94 is configured to determine a joint prediction value of the current block according to the intra prediction value, the intra prediction weight, the inter prediction value and the inter prediction weight of the current block, and use the joint prediction value of the current block as the prediction value of the current block; and a transmission module 95 is configured to transmit relevant syntax elements for determining the intra prediction weight and the inter prediction weight of the current block based on the template cost to a decoding end.

[0199] The side link channel selection device provided in this embodiment is arranged at the encoding end and is used to perform Figure 6 The implementation principle and technical effect of the method for determining weights based on template cost in the illustrated embodiment are similar and will not be described in detail here.

[0200] Fig.10 A schematic diagram of the structure of another device for determining weights based on template cost provided in an embodiment of the present application, such as Fig.10 As shown, the device for determining weights based on template costs provided in this embodiment includes:

[0201] The receiving module 101 is configured to receive the encoded video code stream and parse the video code stream; the obtaining module 102 is configured to obtain the blocks in the parsed video code stream and read the syntax elements related to intra-frame / inter-frame prediction in the parsed video code stream; the mode determination module 103 is configured to determine the intra-frame / inter-frame prediction mode of the current block according to the parsed current block and the syntax elements related to intra-frame / inter-frame prediction; the weight determination module 104 is configured to determine the intra-frame prediction weight and the inter-frame prediction weight based on the template cost if it is determined that the current block uses the CIIP mode; the joint prediction module 105 is configured to determine the joint prediction value of the current block according to the intra-frame prediction value, the intra-frame prediction weight, the inter-frame prediction value and the inter-frame prediction weight of the current block, and use the joint prediction value of the current block as the reconstruction value of the current block; the reconstruction module 106 is configured to obtain the decoded video stream according to the reconstruction value of each of the multiple parsed blocks.

[0202] The side link channel selection device provided in this embodiment is arranged at the decoding end and is used to perform Figure 8 The method for determining weights based on template costs in the illustrated embodiment has similar implementation principles and technical effects, which will not be described in detail here.

[0203] Fig.11 A schematic diagram of a device for determining weights based on template costs provided in an embodiment of the present application is shown in FIG. Fig.11As shown, the device for determining weights based on template costs includes a processor 111, a memory 112, a receiver 113, and a transmitter 114; the number of processors 111 in the device for determining weights based on template costs may be one or more. Fig.11 In the figure, a processor 111 is taken as an example; the processor 111, the memory 112, the receiver 113 and the transmitter 114 in the edge link channel selection device can be connected via a bus or other means. Fig.11 The example of connecting through bus is taken in the following.

[0204] The memory 112 is a computer-readable storage medium that can be used to store software programs, computer executable programs, and modules. Figure 6 The program instructions / modules corresponding to the method for determining weights based on template costs in the embodiment (segmentation module 91, decision module 92, weight determination module 93, joint prediction module 94, transmission module 95). The processor 111 applies various functions and data processing of the device for determining weights based on template costs by running the software programs, instructions and modules stored in the memory 112, that is, realizing the above-mentioned method for determining weights based on template costs.

[0205] The memory 112 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required for at least one function; the data storage area may store data created according to the use of a device whose weight is determined based on a template cost, etc. In addition, the memory 112 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage devices.

[0206] The receiver 113 is any device / module having data receiving capability or a combination of multiple devices / modules having data receiving capability, and the transmitter 114 is any device / module having data sending capability or a combination of multiple devices / modules having data sending capability.

[0207] Fig.12 A schematic diagram of a device for determining weights based on template costs provided in an embodiment of the present application is shown in FIG. Fig.12 As shown, the device for determining weights based on template costs includes a processor 121, a memory 122, a receiver 123, and a transmitter 124; the number of processors 121 in the device for determining weights based on template costs may be one or more. Fig.12 In the figure, a processor 121 is taken as an example; the processor 121, the memory 122, the receiver 123 and the transmitter 124 in the edge link channel selection device can be connected via a bus or other means. Fig.12 The example of connecting through bus is taken in the following.

[0208] The memory 122 is a computer-readable storage medium that can be used to store software programs, computer executable programs, and modules, such as the present application. Figure 6 The program instructions / modules corresponding to the method for determining weights based on template costs in the embodiment (receiving module 101, acquisition module 102, mode determination module 103, weight determination module 104, joint prediction module 105, reconstruction module 106). The processor 121 applies various functions and data processing of the device for determining weights based on template costs by running the software programs, instructions and modules stored in the memory 122, that is, realizing the above-mentioned method for determining weights based on template costs.

[0209] The memory 122 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required for at least one function; the data storage area may store data created according to the use of a device whose weight is determined based on a template cost, etc. In addition, the memory 122 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage devices.

[0210] The receiver 123 is any device / module having data receiving capability or a combination of multiple devices / modules having data receiving capability, and the transmitter 124 is any device / module having data sending capability or a combination of multiple devices / modules having data sending capability.

[0211] An embodiment of the present application also provides a storage medium containing computer executable instructions, which are used to perform a method for determining weights based on template costs when executed by a computer processor. The method includes: dividing each image frame in a video stream to be encoded into multiple coding unit blocks; making an intra-frame / inter-frame prediction mode decision for a current block among the multiple blocks; if it is determined that the current block uses a joint intra-frame and inter-frame prediction CIIP mode, determining an intra-frame prediction weight and an inter-frame prediction weight based on the template cost; determining a joint prediction value of the current block according to the intra-frame prediction value, intra-frame prediction weight, inter-frame prediction value and inter-frame prediction weight of the current block, and using the joint prediction value of the current block as the prediction value of the current block; and transmitting relevant syntax elements for determining the intra-frame prediction weight and the inter-frame prediction weight of the current block based on the template cost to a decoding end.

[0212] The embodiment of the present application also provides a storage medium containing computer executable instructions, which are used to perform a method for determining weights based on template costs when executed by a computer processor. The method includes: receiving an encoded video stream and parsing the video stream; obtaining a coding unit block in the parsed video stream, and reading syntax elements related to intra-frame / inter-frame prediction in the parsed video stream; determining the intra-frame / inter-frame prediction mode of the current block according to the parsed current block and the syntax elements related to intra-frame / inter-frame prediction; if it is determined that the current block uses the joint intra-frame inter-frame prediction CIIP mode, determining the intra-frame prediction weight and the inter-frame prediction weight based on the template cost; determining the joint prediction value of the current block according to the intra-frame prediction value, intra-frame prediction weight, inter-frame prediction value and inter-frame prediction weight of the current block, and using the joint prediction value of the current block as the reconstruction value of the current block; and obtaining a decoded video stream according to the reconstruction value of each of the multiple parsed blocks.

[0213] Although the implementation methods disclosed in this application are as above, the contents are only implementation methods adopted for facilitating understanding of the technical solutions of this application, and are not used to limit this application. Any technician in the technical field to which this application belongs can make any modifications and changes in the form and details of implementation without departing from the core technical solutions disclosed in this application, but the scope of protection defined in this application shall still be subject to the scope defined in the attached claims.

Claims

1. A method for determining weights based on template cost, characterized in that: Applied to the encoding end, including: Dividing each image frame in the video stream to be encoded into a plurality of coding unit blocks; Performing intra / inter prediction mode decision on a current block among the multiple blocks; If it is determined that the current block uses the joint intra-frame and inter-frame prediction CIIP mode, the intra-frame prediction weight and the inter-frame prediction weight are determined based on the template cost; Determine a joint prediction value of the current block according to the intra prediction value, the intra prediction weight, the inter prediction value and the inter prediction weight of the current block, and use the joint prediction value of the current block as the prediction value of the current block; The syntax elements related to determining the intra-frame prediction weight and the inter-frame prediction weight of the current block based on the template cost are transmitted to the decoding end.

2. The method according to claim 1, characterized in that The performing intra / inter prediction mode decision on a current block among the multiple blocks comprises: Performing intra-frame / inter-frame prediction mode decision on a current block among the multiple blocks, and determining whether the current block adopts inter-frame prediction coding or intra-frame prediction coding; If the current block adopts inter-frame prediction coding, it is determined whether the current block uses the CIIP mode.

3. The method according to claim 1, characterized in that The determining of the intra-frame prediction weight and the inter-frame prediction weight based on the template cost includes: Directly determine intra-frame prediction weights and inter-frame prediction weights based on template costs; Alternatively, the intra prediction weight and the inter prediction weight are indirectly determined based on the template cost.

4. The method according to claim 3, characterized in that The directly determining the intra-frame prediction weight and the inter-frame prediction weight based on the template cost includes: The intra prediction weight coefficient and the inter prediction weight coefficient are determined by using the template cost of the intra prediction and the template cost of the inter prediction.

5. The method according to claim 3, characterized in that: The indirect determination of the intra-frame prediction weight and the inter-frame prediction weight based on the template cost includes: The intra-frame prediction template and the inter-frame prediction template are weighted respectively by using a fixed weight coefficient to obtain a joint prediction template; Calculate the template cost of the joint prediction template and the reconstructed template of the current block under different weights; The fixed weight coefficient corresponding to the template with the minimum template cost is used as the intra-frame prediction weight coefficient and the inter-frame prediction weight coefficient of the current coding block.

6. The method according to any one of claims 1 to 5, characterized in that: The determining of the intra-frame prediction weight and the inter-frame prediction weight based on the template cost includes: If the intra prediction mode of the current block using the CIIP mode is a non-angle mode, the intra prediction weight and the inter prediction weight are determined based on the template cost, otherwise a fixed weight coefficient is used as the intra prediction weight and the inter prediction weight; Or if the intra prediction mode of the CIIP mode used in the current block is the angle mode, the intra prediction weight and the inter prediction weight are determined based on the template cost, otherwise fixed weighting coefficients are used as the intra prediction weight and the inter prediction weight.

7. The method according to any one of claims 1 to 5, characterized in that: The determining of the joint prediction value of the current block according to the intra prediction value, the intra prediction weight, the inter prediction value and the inter prediction weight of the current block includes: The intra-frame prediction weight and the inter-frame prediction weight of the current block are used to perform weighted averaging on the intra-frame prediction value and the inter-frame prediction value of the current block to obtain a joint prediction value of the current block.

8. The method according to any one of claims 1 to 5, characterized in that: The relevant syntax elements for determining the intra-frame prediction weight and the inter-frame prediction weight of the current block based on the template cost include: an SPS-level flag allowing the CIIP mode to be enabled and an enabling flag allowing the weight to be determined based on the template cost, a CU-level CIIP mode enabling flag and an enabling flag for determining the weight based on the template cost.

9. The method according to any one of claims 1 to 5, characterized in that: Also includes: The syntax elements related to the current block, intra-frame prediction and inter-frame prediction are transmitted to a decoding end.

10. The method according to claim 9, characterized in that The syntax elements related to the intra-frame prediction and inter-frame prediction of the current block include: an identifier of the intra-frame prediction mode determined at the block level, an identifier of the inter-frame prediction mode determined at the block level, and a corresponding candidate list index.

11. The method according to any one of claims 1 to 5, characterized in that: The step of dividing each image frame in the video stream to be encoded into a plurality of blocks comprises: Each image frame in the video stream to be encoded is divided into a plurality of coding tree units CTU, and each CTU is further divided into one or more blocks.

12. A method for determining weights based on template cost, characterized in that: Applied to the decoding end, including: Receiving an encoded video code stream and parsing the video code stream; Obtain the coding unit block in the parsed video bitstream, and read the syntax elements related to intra-frame / inter-frame prediction in the parsed video bitstream; Determine the intra / inter prediction mode of the current block according to the parsed current block and the intra / inter prediction related syntax elements; If it is determined that the current block uses the joint intra-frame and inter-frame prediction CIIP mode, the intra-frame prediction weight and the inter-frame prediction weight are determined based on the template cost; Determine a joint prediction value of the current block according to the intra prediction value, the intra prediction weight, the inter prediction value and the inter prediction weight of the current block, and use the joint prediction value of the current block as a reconstruction value of the current block; A decoded video stream is obtained according to the reconstructed value of each of the parsed multiple blocks.

13. The method according to claim 12, characterized in that The step of determining the intra / inter prediction mode of the current block according to the parsed current block and the intra / inter prediction related syntax elements includes: Determine whether the current block adopts inter-frame prediction coding or intra-frame prediction coding according to the parsed current block and intra-frame / inter-frame prediction related syntax elements; If the current block adopts inter-frame prediction coding, it is determined whether the current block uses the CIIP mode.

14. The method according to claim 12, characterized in that The determining of the intra-frame prediction weight and the inter-frame prediction weight based on the template cost includes: Directly determine intra-frame prediction weights and inter-frame prediction weights based on template costs; Alternatively, the intra prediction weight and the inter prediction weight are indirectly determined based on the template cost.

15. The method according to claim 14, characterized in that The directly determining the intra-frame prediction weight and the inter-frame prediction weight based on the template cost includes: The intra prediction weight coefficient and the inter prediction weight coefficient are determined by using the template cost of the intra prediction and the template cost of the inter prediction.

16. The method according to claim 14, characterized in that The indirect determination of the intra-frame prediction weight and the inter-frame prediction weight based on the template cost includes: The intra-frame prediction template and the inter-frame prediction template are weighted respectively by using a fixed weight coefficient to obtain a joint prediction template; Calculate the template cost of the joint prediction template and the reconstructed template of the current block under different weights; The fixed weight coefficient corresponding to the template with the minimum template cost is used as the intra-frame prediction weight coefficient and the inter-frame prediction weight coefficient of the current coding block.

17. The method according to any one of claims 12 to 16, characterized in that: The determining of the intra-frame prediction weight and the inter-frame prediction weight based on the template cost includes: If the intra prediction mode of the current block using the CIIP mode is a non-angle mode, the intra prediction weight and the inter prediction weight are determined based on the template cost, otherwise a fixed weight coefficient is used as the intra prediction weight and the inter prediction weight; Or if the intra prediction mode of the CIIP mode used in the current block is the angle mode, the intra prediction weight and the inter prediction weight are determined based on the template cost, otherwise fixed weighting coefficients are used as the intra prediction weight and the inter prediction weight.

18. The method according to any one of claims 12 to 16, characterized in that: The step of determining a joint prediction value of the current block according to an intra prediction value, an intra prediction weight, an inter prediction value, and an inter prediction weight of the current block, and using the joint prediction value of the current block as a reconstruction value of the current block includes: The intra-frame prediction weight and the inter-frame prediction weight of the current block are used to perform weighted averaging on the intra-frame prediction value and the inter-frame prediction value of the current block to obtain a joint prediction value of the current block, and the joint prediction value of the current block is used as a reconstruction value of the current block.

19. The method according to any one of claims 12 to 16, characterized in that: The syntax elements related to intra / inter prediction include syntax elements related to determining intra prediction weights and inter prediction weights based on template costs; The relevant syntax elements for determining intra-frame prediction weights and inter-frame prediction weights based on template cost include: an SPS-level flag allowing the CIIP mode to be enabled and an enabling flag allowing weights to be determined based on template cost, a CU-level CIIP mode enabling flag and an enabling flag for determining weights based on template cost.

20. The method according to claim 19, characterized in that The intra / inter prediction related syntax elements also include: Syntax elements related to intra prediction and inter prediction.

21. The method according to claim 20, characterized in that The syntax elements related to intra-frame prediction and inter-frame prediction include: an identifier for determining an intra-frame prediction mode at a block level, an identifier for determining an inter-frame prediction mode at a block level, and a corresponding candidate list index.

22. A device for determining weights based on template cost, arranged at an encoding end, characterized in that: include: a segmentation module, configured to divide each image frame in the video stream to be encoded into a plurality of encoding unit blocks; A decision module, configured to make an intra / inter prediction mode decision for a current block among the multiple blocks; A weight determination module, configured to determine an intra prediction weight and an inter prediction weight based on a template cost if it is determined that the current block uses a joint intra-frame and inter-frame prediction CIIP mode; A joint prediction module is configured to determine a joint prediction value of the current block according to an intra prediction value, an intra prediction weight, an inter prediction value, and an inter prediction weight of the current block, and use the joint prediction value of the current block as a prediction value of the current block; The transmission module is configured to transmit the relevant syntax elements of the current block for determining the intra-frame prediction weight and the inter-frame prediction weight based on the template cost to the decoding end.

23. A device for determining weights based on template cost, arranged at a decoding end, characterized in that: include: A receiving module, configured to receive the encoded video stream and parse the video stream; An acquisition module, configured to acquire a coding unit block in a parsed video bitstream and read syntax elements related to intra-frame / inter-frame prediction in the parsed video bitstream; A mode determination module, configured to determine the intra / inter prediction mode of the current block according to the parsed current block and the intra / inter prediction related syntax elements; A weight determination module, configured to determine an intra prediction weight and an inter prediction weight based on a template cost if it is determined that the current block uses a joint intra-frame and inter-frame prediction CIIP mode; a joint prediction module, configured to determine a joint prediction value of the current block according to an intra prediction value, an intra prediction weight, an inter prediction value, and an inter prediction weight of the current block, and use the joint prediction value of the current block as a reconstruction value of the current block; The reconstruction module is configured to obtain a decoded video stream according to a reconstruction value of each of the parsed multiple blocks.

24. A device for determining weights based on template costs, characterized in that: include: a memory configured to store a program; The processor is configured to execute a program, and when the program is executed, the method for determining a weight based on a template cost according to any one of claims 1 to 11 is performed.

25. A device for determining weights based on template costs, characterized in that: include: a memory configured to store a program; The processor is configured to execute a program, and when the program is executed, the method for determining a weight based on a template cost as claimed in any one of claims 12 to 21 is performed.

26. A non-volatile storage medium, the storage medium comprising a stored program, characterized in that: When the program is run, the method for determining weights based on template costs according to any one of claims 1 to 11 or 12 to 21 is executed.

Images