Image coding prediction method, electronic equipment and storage medium

Abstract

The embodiment of the invention provides an image coding prediction method, electronic equipment and a storage medium, and the method comprises the steps: obtaining a coding unit of a to-be-coded video, and determining a hybrid prediction mode according to an enabled coding mode of the coding unit; determining an inter-frame prediction signal and an intra-frame prediction signal of the coding unit according to the hybrid prediction mode, and determining a hybrid prediction signal of the inter-frame prediction signal and the intra-frame prediction signal; and transmitting the syntax element of the hybrid prediction mode to a decoding end. According to the embodiment of the invention, joint compression coding of intra-frame prediction and inter-frame prediction is realized, the image coding quality can be improved, and the picture quality of a video image is enhanced.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image encoding method, an electronic device and a storage medium. Background Art

[0002] At present, images and videos have become an indispensable part of the current multimedia society, and are widely used in advertising, medical treatment, security, entertainment, conference television and other fields. With the development of video technology, video quality has been continuously improved, gradually shifting from high-definition video to 4K ultra-clear video, 8K ultra-high-definition video, screen content, panoramic video, etc., which puts higher requirements on network streaming and real-time communication. At the same time, the amount of video transmission and data storage has greatly increased, which also puts higher requirements on video transmission bandwidth. Based on this, video compression technology came into being. Video compression is a technology that uses various encoding tools to remove redundant information in space and time as much as possible from the original video, effectively reducing the amount of video data while ensuring video quality. It is also called video coding, which is one of the most popular research directions in the field of multimedia technology research. The Joint Video Expert Teams (JVET) of ISO / IEC's MPEG and ITU-T's VCEG released the Versatile Video Coding standard (H.266 / VVC), which can better support a variety of new video types such as 8K ultra-high definition, screen content, high dynamic and 360° panoramic video, as well as applications such as streaming media and real-time communication with adaptive bandwidth and resolution. However, the inter-frame prediction technology provided by the existing traditional video coding will greatly affect the quality of video coding. How to improve the image coding prediction technology to improve the quality of video coding has become a problem that needs to be solved urgently. Summary of the invention

[0003] The embodiments of the present application aim to provide an image coding prediction method, an electronic device and a storage medium, the purpose of which is to realize a joint coding within a frame and between frames to improve the image coding quality, which can improve the picture quality of the video image.

[0004] The present application provides an image coding prediction method, which is applied to an encoding end. The method includes:

[0005] Obtaining a coding unit of a video to be encoded, and determining a hybrid prediction mode according to an enabled coding mode of the coding unit;

[0006] Determine an inter-frame prediction signal and an intra-frame prediction signal of the coding unit according to the hybrid prediction mode, and determine a hybrid prediction signal of the inter-frame prediction signal and the intra-frame prediction signal;

[0007] The syntax elements of the hybrid prediction mode are transmitted to a decoding end.

[0008] The embodiment of the present application also provides an image coding prediction method, applied to a decoding end, the method comprising:

[0009] Parsing a syntax element indicating a hybrid prediction mode according to a bitstream transmitted by an encoding end;

[0010] Determining a hybrid prediction mode of a decoding unit according to the syntax element;

[0011] An inter-frame prediction signal and an intra-frame prediction signal of the decoding unit are determined according to the hybrid prediction mode, and a hybrid reconstruction signal is determined according to the inter-frame prediction signal and the intra-frame prediction signal.

[0012] The present application also provides an electronic device, which includes:

[0013] one or more processors;

[0014] A memory for storing one or more programs;

[0015] When the one or more programs are executed by the one or more processors, the one or more processors implement the image coding prediction method as described in any one of the embodiments of the present application.

[0016] An embodiment of the present application also provides a computer-readable storage medium, wherein the computer-readable storage medium stores one or more programs, and the one or more programs are executed by one or more processors to implement the image coding prediction method as described in any one of the embodiments of the present application.

[0017] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0019] Figure 1 is an example diagram of an image coding framework provided by an embodiment of the present application;

[0020] Figure 2 is an example diagram of a Planar mode image prediction provided by an embodiment of the present application;

[0021] Figure 3 is an example diagram of CIIP weight derivation provided in an embodiment of the present application;

[0022] Figure 4 This is an example diagram of a division method of a CIIP-TIMD angle mode provided in an embodiment of the present application;

[0023] Figure 5 This is another example of a division method of the CIIP-TIMD angle mode provided in an embodiment of the present application.

[0024] Figure 6 This is an example diagram of template matching near an initial MV provided in an embodiment of the present application;

[0025] Figure 7 is a flowchart of an image coding prediction method provided by an embodiment of the present application;

[0026] Figure 8 is a flowchart of another image coding prediction method provided by an embodiment of the present application;

[0027] Fig. 9 is a flowchart of another image coding prediction method provided by an embodiment of the present application;

[0028] Fig.10 is a flowchart of another image coding prediction method provided by an embodiment of the present application;

[0029] Fig.11 is an example diagram of image predictive coding provided by an embodiment of the present application;

[0030] Fig.12 This is an example diagram of an implementation of image predictive coding provided by an embodiment of the present application;

[0031] Fig.13 is a flowchart of an image coding prediction method provided by an embodiment of the present application;

[0032] Fig.14 is a flowchart of another image coding prediction method provided by an embodiment of the present application;

[0033] Fig.15 is a flowchart of another image coding prediction method provided by an embodiment of the present application;

[0034] Fig.16 is a flowchart of another image coding prediction method provided by an embodiment of the present application;

[0035] Fig.17 This is an example diagram of an implementation of image predictive coding provided by an embodiment of the present application;

[0036] Fig.18 It is a structural schematic diagram of an image coding prediction device provided in an embodiment of the present application;

[0037] Fig.19 is a structural schematic diagram of another image coding prediction device provided in an embodiment of the present application;

[0038] Fig. 20 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0039] It should be understood that the specific implementations described herein are only used to explain the present application and are not used to limit the present application.

[0040] In the subsequent description, suffixes such as "module", "component" or "unit" used to represent elements are only used to facilitate the description of the present application and have no specific meaning themselves. Therefore, "module", "component" or "unit" may be used interchangeably.

[0041] Currently, the ECM standard uses the H.266 / VCC coding framework, see Figure 1 , the coding framework is a block-based hybrid coding framework, which mainly includes modules such as intra-frame prediction, inter-frame prediction, transformation, quantization, entropy coding and loop filtering. Each module covers a variety of new coding technologies, which further improves the coding gain, but also requires a lot of memory. Among the various modules of the H.266 / VVC coding framework, the intra-frame prediction module mainly removes the spatial correlation of the video image, and uses the pixel information of the adjacent reconstructed blocks that have been encoded in the frame to predict the pixel value of the current block, with a low compression rate; the inter-frame prediction module mainly removes the temporal correlation of the video image, and uses the inter-frame encoded image as the reference image of the current frame to obtain the motion information of each block of the current frame. It can use unidirectional and bidirectional reference images for prediction, and generally speaking, the compression rate is higher than that of intra-frame prediction coding. After the video image block passes through the intra-frame / inter-frame prediction module, the predicted value of the current block is obtained. The residual data can be obtained by subtracting the original image block from the predicted block, and the residual data is transformed, that is, from the time domain described in pixel form to the frequency domain, so that the image energy is concentrated in the low-frequency area, which can effectively remove the frequency domain correlation of the image, that is, the correlation of the transformation coefficients is small; usually the transformed coefficients have a large dynamic range, and the quantization module is needed to reduce the dynamic range to achieve better compression; the entropy coding module encodes the intra-frame prediction data, motion information, quantization coefficients, coding control and other data into a binary stream, that is, the compressed code stream of the original video, so as to achieve storage or transmission.

[0042] In addition, in the hybrid coding framework, in order to build a decoded image cache for inter-frame prediction as a reference image, an image reconstruction process is also implemented, which mainly includes inverse quantization, inverse transformation and loop filtering modules. The specific process is to inverse quantize and inverse transform the quantized transform coefficients to obtain residual data, and then add it to the prediction information to reconstruct the image data; the loop filtering module is to improve the subjective quality and compression efficiency of the image and reduce decoding errors. The reason is that video coding is based on block processing, and usually the reconstructed image will have block effects, ringing effects and other phenomena. This module mainly includes the following filtering technologies: Luma Mapping Chroma Scaling (LMCS), De-blocking Filter (DBF), Sample Adaptive Offset (SAO) and Adaptive Loop Filter (ALF). Luma Mapping and Chroma Scaling is to redistribute codewords for information within the dynamic range, which can improve compression efficiency; De-blocking Filter is to smooth the block boundary, which can effectively reduce and remove the block effect; Sample Adaptive Offset is to use different compensation values ​​for different categories of reconstructed pixels, which can improve the ringing effect caused by the loss of high-frequency information and effectively improve the subjective and objective quality of the reconstructed image; Adaptive Loop Filter is to perform Wiener filtering on the reconstructed image pixels, which can effectively reduce decoding errors.

[0043] The overall framework process of the encoding end includes the following:

[0044] (1) The input video frame (image) is divided into blocks: the frame is first divided into multiple coding tree units (CTUs). Each CTU can be divided into four coding units (CUs) of the same size according to a quadtree, or recursively divided into CUs of different sizes according to a multiple type tree (MTT), that is, a binary tree or a ternary tree structure. CTU contains all color channels, that is, a CTU consists of a luma coding tree block (CTB) and two chroma CTBs; CU is the basic unit of video coding, the maximum size of the luma CU is 128*128, the minimum size is 4*4, the maximum size of the chroma CU is 64*64, and the minimum size is 2*2; the size of the coding unit CU finally divided is related to the video image content.

[0045] (2) The divided blocks are sent to the intra-frame / inter-frame prediction module for predictive coding. The main purpose of the intra-frame prediction module is to remove the spatial correlation of the image. The reconstructed block after encoding is used to predict the current block pixel to remove spatial redundant information. The main purpose of the inter-frame prediction coding is to remove the temporal correlation of the image. The encoded image is used as the reference image of the current frame to obtain the motion information of each block in the current frame.

[0046] (3) Subtract the predicted value obtained by the intra-frame / inter-frame coding module from the original block pixel value to obtain residual data. The residual data is transformed and quantized to remove frequency domain correlation. The transformation module transforms the image from a spatial domain signal to a frequency domain signal to concentrate the energy in the low-frequency area. The quantization module reduces the dynamic range of image coding, thereby achieving lossy compression of image residual data and further improving compression efficiency.

[0047] (4) Finally, all the coding parameters and residual data are used as the input of the entropy coding module to obtain a binary stream, that is, a code stream after the original image data is compressed, for storage or transmission.

[0048] (5) The quantized transform coefficients are dequantized and inversely transformed to obtain the residual data of the reconstructed block, which is then added to the predicted value obtained by intra-frame / inter-frame coding to obtain the reconstructed value of the coding block, thereby obtaining the reconstructed image.

[0049] (6) The reconstructed image needs to be filtered through the loop filter module to obtain the final reconstructed image and build a decoded image cache as a reference image for inter-frame prediction.

[0050] The image reconstruction process of the hybrid coding framework is the image decoding process, except that the decoding end adds a bitstream parsing step. The overall framework process of the decoding end is as follows:

[0051] (1) Analyze the bitstream to determine the prediction mode and obtain the prediction value;

[0052] (2) Dequantizing and detransforming the residual value after bitstream analysis to obtain the residual data before transformation and quantization;

[0053] (3) Adding the reconstructed residual data of the current block to the prediction value obtained by intra-frame / inter-frame coding to obtain a block reconstruction value, thereby obtaining a reconstructed image;

[0054] (4) The reconstructed image needs to be filtered through the loop filter module and a decoded image cache is constructed as a reference image for inter-frame prediction.

[0055] Combined Inter and Intra Prediction (CIIP) mode is a prediction technology in H.266 / VVC. The basic principle is that the current CU uses intra prediction and inter prediction to obtain two prediction blocks, and uses the weighted average as the final prediction value. In order to avoid excessive computational complexity, H.266 / VVC limits CIIP intra prediction to only Planar mode, inter prediction to only conventional Merge mode, CU width and height are both less than 128, and CU contains at least 64 pixels.

[0056] Planar mode is a traditional non-angle prediction mode, suitable for areas where pixel values ​​change slowly. Its mode index is 0. If the current CU adopts Planar mode, the predicted pixel at a certain position in the CU is the weighted average of the four reference pixels in the horizontal and vertical directions corresponding to the position according to the distance from the pixel at that position, such as Figure 2 As shown, the blue line represents the mean of the two reference pixels in the horizontal direction (column 0 and column W+1) weighted by the distance, and the red line represents the mean of the two reference pixels in the vertical direction (row 0 and row H+1) weighted by the distance, and the mean of the horizontal and vertical directions is used as the predicted pixel value of the current position.

[0057] The conventional Merge mode is an inter-frame prediction mode that directly uses the motion vector information (reference frame index, motion vector) of the motion vector prediction (MVP) as the motion vector information of the current CU. First, a Merge MVP candidate list needs to be established, and then one of the candidates in the Merge MVP list is selected as the motion vector (MV) of the current CU. When encoding, only the index value in the list where the selected MVP is located needs to be encoded, and there is no need to transmit the motion vector difference (MVD) and the reference frame index. The pixel prediction value of the current CU can be obtained based on the selected MV information, and the prediction residual can be further encoded from the original pixel value and the predicted pixel value of the current CU to complete the encoding of the current CU.

[0058] When the current CU adopts the CIIP mode, the intra-frame prediction value P_intra is obtained by the Planar mode, and the inter-frame prediction value P_inter is obtained by the conventional Merge mode, and the prediction value of the current CU is obtained by weighting them:

[0059] P CIIP =(w inter *P inter +w intra *Pintra +2)>>2

[0060] Among them, w_inter is the inter-frame prediction weight, w_intra is the intra-frame prediction weight, and its value is derived from the coding mode of the top and left adjacent CUs of the current CU, such as Figure 3 The specific weight values ​​are shown in Table 1:

[0061] Table 1 CIIP weight setting

[0062]

[0063]

[0064] There are three syntax elements related to the CIIP mode, as shown below:

[0065] (1) sps_ciip_enabled_flag: SPS level, indicating whether the current sequence is allowed to use CIIP mode, 1 means allowed, 0 means not allowed;

[0066] (2) ciip_flag: CU level, indicating whether the current CU uses CIIP mode, 1 means enabled, 0 means disabled;

[0067] (3) merge_idx: CU level, identifies the index of the current CU MVP in the Merge MVP list.

[0068] Whether the current CU adopts CIIP mode can be explicitly specified by ciip_flag as 1, or by implicitly inferring ciip_flag as 1. If the following four conditions are met, ciip_flag is inferred to be 1:

[0069] (1) SPS level syntax element sps_ciip_enabled_flag = 1;

[0070] (2) CU is not in Skip mode;

[0071] (3) CU belongs to Merge mode, but not the conventional Merge mode, SbTMVP mode, or affine Merge mode;

[0072] (4) The width and height of the CU are both less than 128, and the CU contains at least 64 brightness pixels (W*H>=64).

[0073] For the intra prediction mode of CIIP mode, it is proposed to replace the original Planar mode with the intra prediction mode obtained by the template-based intra mode derivation (TIMD) method, so as to refine the intra prediction mode of CIIP mode without increasing signaling transmission. TIMD is a technology based on template-based intra prediction mode derivation. Its principle is to first derive the candidate list of TIMD from the most probable mode (MPM) list and expand the number of intra prediction modes from 67 to 131; then for each candidate prediction mode in the candidate list, use its template reference pixels to generate the template prediction pixels of the current block in the corresponding mode; then calculate the minimum transform domain absolute error and cost (SATD cost) based on the reconstructed (or reference) pixels and predicted pixels of the current block template, select the mode corresponding to the minimum SATD cost as the intra prediction mode of the current block, and map the prediction mode to one of the original 67 conventional prediction modes through a table.

[0074] When the CIIP mode is combined with TIMD, that is, the CIIP-TIMD mode, the weight of the CIIP mode is also improved. If the intra-frame prediction mode derived by TIMD is Planar or DC mode, the weight coefficient remains unchanged, as shown in Table 1; otherwise, that is, the intra-frame prediction mode derived by TIMD is an angle mode, the current block needs to be equally divided into 4 sub-blocks, and different weight coefficients are used for each sub-block: For modes close to the horizontal direction, that is, near-horizontal modes (intra-frame mode index is less than 34), the current block is equally divided into 4 sub-blocks vertically, such as Figure 4 As shown; for modes close to the vertical direction, that is, near-vertical modes (intra-frame mode index greater than or equal to 34), the current block is horizontally divided into 4 sub-blocks, such as Figure 5 As shown, the weight coefficient of each sub-block is shown in Table 2. In addition, CIIP-TIMD requires that the current CU contains at most 1024 pixels, that is, the width*height of the CU is less than or equal to 1024.

[0075] Table 2 Weight settings for CIIP-TIMD angle mode

[0076] The sub-block index (wIntra,wInter) 0 (6,2) 1 (5,3) 2 (3,5) 3 (2,6)

[0077] Subsequently, the JVET-X conference proposed the Template Matching (TM) mode, that is, the inter-frame prediction part is based on the MV candidate after TM refinement, and the inter-frame prediction mode at this time is TM-Merge. The TM mode is a decoding-side MV derivation method that refines the current CU motion information. The MV is refined by searching for the best matching template (L-shape or Top or Left template) between the current frame and the reference frame, such as Figure 6 As shown, the best matching template is searched near the initial MV to obtain a better refined MV, and the search area is the [-8,8] pixel range of the current CU. In TM-Merge mode, the merge candidates indicated by the merge index also use a similar search method. When searching based on TM, 1 / 8 pixel MVD accuracy can be achieved or MVD accuracy of more than half a pixel can be skipped, depending on whether an alternative interpolation filter is used based on the merge motion information.

[0078] In CIIP-TM mode, it is necessary to construct a CIIP-TM Merge candidate list. The construction method is the same as that of a regular Merge list, with the difference that the CIIP-TM Merge candidate list has at most 2 candidates; the candidates in the TM-Merge list are MV candidates after TM refinement, and the reordering is achieved through adaptive reordering of merge candidates (ARMC).

[0079] If the current CU is encoded in CIIP mode, the SPS-level syntax element sps_ciip_enabled_flag is marked as 1, and the CU-level syntax element ciip_flag is marked as 1. Whether the current CU adopts the CIIP-TM mode depends on the other two CU-level syntax elements: pdpc_flag and tm_merge_flag. If pdpc_flag is 1, the intra-frame prediction mode of the CIIP-TM mode defaults to the Planar mode, and the PDPC process is applied, otherwise its intra-frame prediction mode is the mode derived from TIMD, and the PDPC process is not applied; if tm_merge_flag is 1, it indicates that the inter-frame prediction mode of the CIIP-TM mode is the TM-Merge mode, otherwise its inter-frame prediction mode is the conventional Merge mode.

[0080] The encoder determines the value of pdpc_flag through rate distortion optimization (RDO) decision at the encoding end, and determines the value of tm_merge_flag through the competition between the conventional Merge mode and the TM-Merge mode, or through the competition between the CIIP-merge and CIIP-TM-merge processes, and then transmits the above four syntax elements and the merge index or TM-merge index to the decoding end; after receiving the relevant syntax elements, the decoder performs corresponding correct decoding to obtain the reconstructed value of the current CU, and determines whether the current sequence allows CIIP mode encoding by sps_ciip_enabled_flag. If allowed, it determines whether the current CU adopts CIIP mode according to ciip_flag. If CIIP mode is adopted, The IIP mode further determines whether the intra-frame prediction mode of the CIIP-TM mode is the mode derived from TIMD according to pdpc_flag, and determines whether the inter-frame prediction mode of the CIIP-TM mode is the TM-Merge mode according to tm_merge_flag; and obtains the intra-frame prediction reconstruction value of the current CU according to the intra-frame prediction mode determined by decoding, and constructs the same merge list at the decoding end in the same way as the conventional Merge mode constructs the merge list at the encoding end, and then determines the MV candidate of the current CU by the merge index or TM-merge index, and obtains the inter-frame prediction reconstruction value of the current CU, and finally performs weighted averaging on the intra-frame and inter-frame prediction reconstruction values ​​to obtain the reconstruction value of the current CU, thereby completing correct decoding.

[0081] Figure 7 is a flowchart of an image coding prediction method provided by an embodiment of the present application. The present application can be applied to the case where predictive coding is performed during the image coding process. The method can be executed by an image coding prediction device, which is generally applied to the coding end. The device can be implemented by software and / or hardware. Figure 7 The method provided in the embodiment of the present application specifically includes the following steps:

[0082] Step 110: Obtain a coding unit of the video to be encoded, and determine a hybrid prediction mode according to an enabled coding mode of the coding unit.

[0083] Among them, the coding unit can be generated by dividing the video to be encoded into blocks, each frame of the video to be encoded can be divided into multiple coding units, the enabled coding mode can be the original coding mode of the coding unit, and can be the coding mode pre-configured or negotiated by the coding unit. The enabled coding mode may include an intra-block copy (Intra Block Copy, IBC) mode, an intra-frame template prediction mode, etc. The hybrid prediction mode can be an inter-frame and intra-frame joint prediction mode. The hybrid prediction mode can be implemented by two modes to respectively realize inter-frame prediction and intra-frame prediction.

[0084] In an embodiment of the present application, the video to be encoded can be divided into blocks to obtain coding units, the enabled coding mode of the coding unit can be determined, and it can be determined whether the enabled coding mode is a coding mode that triggers the coding unit to perform a hybrid prediction mode. If so, the coding unit can perform coding prediction in the hybrid prediction mode. If not, coding prediction in the hybrid prediction mode is not performed on the coding unit.

[0085] In an exemplary embodiment, the enabled coding modes may include the IBC mode and the IntraTMP mode. When determining whether the IBC mode or the IntraTMP mode is enabled for a coding unit CU, a hybrid prediction mode may be used for the coding unit CU, that is, if the coding unit adopts the IntraTMP mode, the IBC mode is added to the coding unit; if the coding unit adopts the IBC mode, the IntraTMP mode is added to it, and then the two modes are combined to obtain a new CIIP prediction method, that is, the IntraTMP mode is used as the intra-frame prediction part of the CIIP, and the IBC mode is used as the inter-frame prediction part of the CIIP.

[0086] Step 120: determine an inter-frame prediction signal and an intra-frame prediction signal of the coding unit according to the hybrid prediction mode, and determine a hybrid prediction signal of the inter-frame prediction signal and the intra-frame prediction signal.

[0087] Among them, the inter-frame prediction signal can be information generated by the coding unit for inter-frame prediction, and the intra-frame prediction signal can be information generated by the coding unit for intra-frame prediction. The prediction coding mode of the inter-frame prediction signal and the intra-frame prediction signal can be determined by the hybrid prediction mode. For example, the IntraTMP mode can be used as the intra-frame prediction part of the hybrid prediction mode, and the IBC mode can be used as the inter-frame prediction part of the hybrid prediction mode.

[0088] In an embodiment of the present application, when it is determined that the coding unit can perform a hybrid prediction mode, intra-frame prediction and inter-frame prediction can be performed respectively according to the intra-frame prediction part and the inter-frame prediction part of the hybrid prediction mode, so as to obtain the inter-frame prediction signal and the intra-frame prediction signal of the coding unit, and the inter-frame prediction signal and the intra-frame prediction signal can be merged to determine the hybrid prediction signal. The merging method of the hybrid prediction signal may include determining the weighted sum of the inter-frame prediction signal and the intra-frame prediction signal, or merging the inter-frame prediction signal and the intra-frame prediction signal through a prediction formula, etc.

[0089] Step 130: Transmit the syntax elements of the hybrid prediction mode to the decoding end.

[0090] The syntax element may be information used by the hybrid prediction mode for coding prediction, and the syntax element includes but is not limited to indication information indicating the intra-frame prediction part and the inter-frame prediction part in the hybrid prediction mode.

[0091] In an embodiment of the present application, the syntax elements of the hybrid prediction mode can be transmitted to the decoding end through the code stream corresponding to the video to be encoded, so that the decoding end can reconstruct the video to be encoded according to the syntax elements.

[0092] In an embodiment of the present application, a coding unit of a video to be encoded is obtained, a hybrid prediction mode is determined according to an enabled coding mode of the coding unit, an inter-frame prediction signal and an intra-frame prediction signal of the coding unit are generated according to the hybrid prediction mode, and a hybrid prediction signal is generated based on the inter-frame prediction signal and the intra-frame prediction signal, and the syntax elements of the hybrid prediction mode are transmitted to a decoding end, thereby realizing joint coding of inter-frame prediction and intra-frame prediction, which can improve image coding quality and enhance the display effect of video images.

[0093] In some application embodiments, obtaining a coding unit of a video to be encoded, and determining a hybrid prediction mode according to an enabled coding mode of the coding unit, includes:

[0094] At least one video frame of the video to be encoded is extracted, and the at least one video frame is divided into encoding units; the enabled encoding mode of the encoding unit is the IntraTMP mode or the IBC mode, which triggers the encoding unit to enable the hybrid prediction mode.

[0095] The video frame may be an image frame in a video to be encoded, and each video frame may be divided into a number of encoding units.

[0096] In an embodiment of the present application, each frame of the image can be extracted from the video to be encoded as a video frame, and the video frame can be divided into blocks to generate a number of coding units, and it is determined whether the enabled coding mode of each coding unit in the initial state is the IntraTMP mode or the IBC mode. If so, the hybrid prediction mode can be used to perform predictive coding on the coding unit. If not, the enabled coding mode in the initial state continues to be used for predictive coding.

[0097] Figure 8 is a flowchart of another image coding prediction method provided by an embodiment of the present application. The present application embodiment is a specific implementation based on the above application embodiment. Figure 8 The method provided in the embodiment of the present application specifically includes the following steps:

[0098] Step 210: Obtain a coding unit of the video to be encoded, and determine a hybrid prediction mode according to an enabled coding mode of the coding unit.

[0099] Step 220: Determine the best matching block vector of the coding unit as the inter-frame prediction signal according to the IBC mode in the hybrid prediction mode.

[0100] In an embodiment of the present application, the IBC mode in the hybrid prediction mode can be used as the inter-frame prediction part, the encoding unit can determine the best matching block vector (Block Vector, BV) according to the IBC mode, and the best matching block vector determined according to the IBC mode can be used as the inter-frame prediction signal of the encoding unit.

[0101] Step 230: Determine the best matching block vector of the coding unit as the intra-frame prediction signal according to the IntraTMP mode in the hybrid prediction mode.

[0102] In an embodiment of the present application, the IntraTMP mode in the hybrid prediction mode can be used as the intra prediction part, and the coding unit can determine the best matching block vector (or called IntraTMP BV) as the intra prediction signal of the coding unit according to the IntraTMP mode.

[0103] Step 240: Determine a mixed prediction value according to a prediction formula under the inter-frame prediction signal and the intra-frame prediction signal. The prediction formula includes:

[0104] P=(w ibc *P ibc +((1<<shift)-w ibc )*P intraTMP +(1<<(shift-1)))>>shift;

[0105] Among them, P ibc and P intraTMP They represent inter-frame prediction signals and intra-frame prediction signals respectively, shift represents the weight distribution value for controlling the IBC mode and the IntraTMP mode, and w ibc Indicates the weight of the IBC mode in the hybrid prediction mode.

[0106] In the embodiment of the present application, the hybrid prediction value may be generated by mixing the inter-frame prediction signal and the intra-frame prediction signal. The mixing may be implemented by a prediction formula, which may include:

[0107] P=(w ibc *P ibc +((1<<shift)-w ibc )*P intraTMP +(1<<(shift-1)))>>shift;

[0108] In the prediction formula, P ibc and P intraTMPThey represent inter-frame prediction signals and intra-frame prediction signals respectively, shift represents the weight distribution value for controlling the IBC mode and the IntraTMP mode, and w ibc Indicates the weight of the IBC mode in the hybrid prediction mode, << can represent the left shift symbol, and >> can identify the right shift symbol.

[0109] Step 250: Transmit the syntax elements of the hybrid prediction mode to the decoding end.

[0110] Based on the above application embodiment, the values ​​of w_ibc and shift include at least one of the following: the IBC mode is the IBC merge mode, w ibc =10, shift=4; IBC mode is IBC AMVP mode, w ibc =1, shift=1.

[0111] In the embodiment of the present application, the IBC mode may be an IBC merge mode or an IBC AMVP mode. When the IBC mode is an IBC merge mode, the IBC weight allocation value w in the prediction formula of the mixed intra-frame prediction signal and the inter-frame prediction signal is ibc =10, control the weight distribution value shift of IBC mode and IntraTMP mode = 4; when the IBC mode is IBC AMVP mode, the IBC weight distribution value w in the prediction formula of the mixed intra-frame prediction signal and the inter-frame prediction signal ibc =1, controls the weight distribution value of IBC mode and IntraTMP mode shift=1.

[0112] Fig. 9 is a flowchart of another image coding prediction method provided by an embodiment of the present application. The present application embodiment is a specific implementation based on the above application embodiment. Fig. 9 The method provided in the embodiment of the present application specifically includes the following steps:

[0113] Step 310: Obtain a coding unit of the video to be encoded, and determine a hybrid prediction mode according to an enabled coding mode of the coding unit.

[0114] Step 320: Determine the best matching block vector of the coding unit as the intra-frame prediction signal according to the IBC mode in the hybrid prediction mode.

[0115] In an embodiment of the present application, the IBC mode in the hybrid prediction mode can be used as the intra-frame prediction part, the encoding unit can determine the best matching block vector according to the IBC mode, and the best matching block vector determined according to the IBC mode can be used as the intra-frame prediction signal of the encoding unit.

[0116] Step 330: Determine the prediction value of the coding unit as the inter-frame prediction signal according to the intra-merge / TM merge mode of the hybrid prediction mode.

[0117] Specifically, the merge / TM merge mode in the hybrid prediction mode can be used as the inter-frame prediction part, and the coding unit can determine the prediction value according to the merge / TM merge mode, and the prediction value can be used as the inter-frame prediction signal of the coding unit.

[0118] Step 340: Determine a mixed prediction value according to a prediction formula under the intra-frame prediction signal and the inter-frame prediction signal. The prediction formula includes:

[0119] P=(w ibc *P ibc +((1<<shift)-w ibc )*P inter +(1<<(shift-1)))>>shift;

[0120] Among them, the P ibc And the P inter represent the intra-frame prediction signal and the inter-frame prediction signal respectively, the shift represents the weight distribution value for controlling the IBC mode and the merge / TM merge mode, the w ibc Represents the weight of the IBC mode in the hybrid prediction mode.

[0121] In the embodiment of the present application, the hybrid prediction value may be generated by mixing the inter-frame prediction signal and the intra-frame prediction signal. The mixing may be implemented by a prediction formula, which may include:

[0122] P=(w ibc *P ibc +((1<<shift)-w ibc )*P inter +(1<<(shift-1)))>>shift;

[0123] In the prediction formula, P ibc and P inter They represent intra-frame prediction signals and inter-frame prediction signals respectively, shift represents the weight distribution value for controlling the IBC mode and merge / TM merge mode, and w ibc Indicates the weight of the IBC mode in the hybrid prediction mode, << can represent the left shift symbol, and >> can identify the right shift symbol.

[0124] Step 350: Transmit the syntax elements of the hybrid prediction mode to the decoding end.

[0125] Based on the above application examples, w ibc The value of shift includes at least one of the following: IBC mode is IBC merge mode, w ibc =13, shift=4; IBC mode is IBC AMVP mode, w ibc =1, shift=1.

[0126] In the embodiment of the present application, the IBC mode may be an IBC merge mode or an IBC AMVP mode. When the IBC mode is an IBC merge mode, the IBC weight allocation value w in the prediction formula of the mixed intra-frame prediction signal and the inter-frame prediction signal is ibc =13, control the weight distribution value shift of IBC mode and IntraTMP mode = 4; when the IBC mode is IBC AMVP mode, the IBC weight distribution value w in the prediction formula of the mixed intra-frame prediction signal and the inter-frame prediction signal ibc =1, controls the weight distribution value of IBC mode and IntraTMP mode shift=1.

[0127] Based on the above application embodiment, the syntax element may include at least one of the following:

[0128] Mixed mode enabling flag at coding unit level; IBC BV candidate index at coding unit level; merge / TM merge mode candidate index at coding unit level.

[0129] In the embodiment of the present application, the syntax elements that need to be transmitted to the decoding end include: a CU-level hybrid mode enabling flag, a CU-level IBC BV candidate index, a CU-level merge / TM merge mode candidate index, and the like.

[0130] Fig.10 is a flowchart of another image coding prediction method provided by an embodiment of the present application. The present application embodiment is a specific implementation based on the above application embodiment. Fig.10 The method provided in the embodiment of the present application specifically includes the following steps:

[0131] Step 410: Obtain a coding unit of the video to be encoded, and determine a hybrid prediction mode according to an enabled coding mode of the coding unit.

[0132] Step 420: Determine the best matching block vector of the coding unit as the inter-frame prediction signal according to the IBC mode in the hybrid prediction mode.

[0133] In an embodiment of the present application, the IBC mode in the hybrid prediction mode can be used as the inter-frame prediction part, the encoding unit can determine the best matching block vector according to the IBC mode, and the best matching block vector determined according to the IBC mode can be used as the inter-frame prediction signal of the encoding unit.

[0134] Step 430: Determine the prediction value of the coding unit as an intra-frame prediction signal according to the TIMD mode within the hybrid prediction mode.

[0135] Specifically, the TIMD mode in the hybrid prediction mode may be used as the intra-frame prediction part, the coding unit may determine a prediction value according to the TIMD mode, and the prediction value may be used as the intra-frame prediction signal of the coding unit.

[0136] Step 440: Determine a mixed prediction value according to a prediction formula under the inter-frame prediction signal and the intra-frame prediction signal. The prediction formula includes:

[0137] P=(w ibc *P ibc +((1<<shift)-w ibc )*P intra +(1<<(shift-1)))>>shift;

[0138] Among them, P ibc and P intra They represent inter-frame prediction signals and intra-frame prediction signals respectively, shift represents the weight distribution value for controlling the IBC mode and TIMD mode, and w ibc Indicates the weight of the IBC mode in the hybrid prediction mode.

[0139] In the embodiment of the present application, the hybrid prediction value may be generated by mixing the inter-frame prediction signal and the intra-frame prediction signal. The mixing may be implemented by a prediction formula, which may include:

[0140] P=(w ibc *P ibc +((1<<shift)-w ibc )*P intra +(1<<(shift-1)))>>shift;

[0141] In the prediction formula, P ibc and P intra They represent inter-frame prediction signals and intra-frame prediction signals respectively, shift represents the weight distribution value for controlling the IBC mode and TIMD mode, and w ibc Indicates the weight of the IBC mode in the hybrid prediction mode, << can represent the left shift symbol, and >> can identify the right shift symbol.

[0142] Step 450: Transmit the syntax elements of the hybrid prediction mode to the decoding end.

[0143] Based on the above application examples, w ibc The value of shift includes at least one of the following: IBC mode is IBC merge mode, w ibc =9, shift=4; IBC mode is IBC AMVP mode, w ibc =1, shift=1.

[0144] In the embodiment of the present application, the IBC mode may be an IBC merge mode or an IBC AMVP mode. When the IBC mode is an IBC merge mode, the IBC weight allocation value w in the prediction formula of the mixed intra-frame prediction signal and the inter-frame prediction signal is ibc =9, control the weight distribution value shift of IBC mode and IntraTMP mode = 4; when the IBC mode is IBC AMVP mode, the IBC weight distribution value w in the prediction formula of the mixed intra-frame prediction signal and the inter-frame prediction signal ibc =1, controls the weight distribution value of IBC mode and IntraTMP mode shift=1.

[0145] Based on the above application embodiment, the syntax element includes at least one of the following:

[0146] Mixed mode enabling flag at coding unit level; IBC BV candidate index at coding unit level; TIMD mode candidate index at coding unit level.

[0147] In the embodiment of the present application, the syntax elements that need to be transmitted to the decoding end include: a CU-level hybrid mode enabling flag, a CU-level IBC BV candidate index, a TIMD mode candidate index, etc.

[0148] Further, based on the above application embodiment, before determining the inter-frame prediction signal and the intra-frame prediction signal of the coding unit according to the hybrid prediction mode and determining the hybrid prediction signal of the inter-frame prediction signal and the intra-frame prediction signal, the method further includes:

[0149] It is determined that the enabled coding mode is the IBC mode, and the best matching block vector of the coding unit is the IntraTMP BV of the adjacent coding unit.

[0150] In the embodiment of the present application, when the enabled coding mode is the IBC mode, if the adjacent coding unit adopts the IntraTMP mode, such as Fig.11As shown, the IntraTMP BV of this adjacent coding unit can be used as one of the candidates in the BV candidate list of the IBC mode of the current coding unit, and inserted into the second position of the IBC BV candidate list. Therefore, there are two possible situations for the BV of the IBC mode of the coding unit: one is that the BV is the IBC BV of the current coding unit, that is, the other candidate BVs in the BV candidate list except the second candidate; the second is that the BV is the IntraTMP BV of the adjacent coding unit, that is, the second candidate in the BV candidate list. Before predictive coding, the coding unit can judge the situation that the enabled coding mode is the IBC mode, and the best matching block vector of the coding unit is the IntraTMP BV of the adjacent coding unit, so as to select different BVs for different processing situations when the IBC mode is used in combination with other modes in the subsequent hybrid prediction mode.

[0151] In an exemplary embodiment, taking the joint intra-frame and inter-frame prediction of the IBC mode and IntraTMP as an example, when the coding unit enables the IntraTMP or IBC mode, a joint intra-frame and inter-frame prediction method is used: IBC-IntraTMPCIIP mode, that is, the intra-frame and inter-frame prediction parts of the CIIP mode obtain corresponding prediction signals by the IntraTMP and IBC modes respectively, and then the prediction signals of these two parts are weightedly fused to obtain the final prediction signal of the coding unit.

[0152] In the embodiment of the present application, a CU-level syntax element ibc_intratmpciip_flag can be added to indicate whether the current coding unit enables the IBC-IntraTMPCIIP mode; in addition, the SPS-level syntax element sps_ciip_enabled_flag that allows the CIIP mode to be enabled is included, as well as the original related syntax elements of the IBC and IntraTMP modes, such as: the BV candidate index of the IBC mode, the multiple candidate indexes and multiple template identifiers of the IntraTMP mode, etc. Fig.12 The specific implementation process of the IBC-IntraTMPCIIP mode on the encoding side is as follows:

[0153] Step 1: When the encoding unit enables the IntraTMP or IBC mode, the IBC-IntraTMPCIIP mode is used in conjunction with the IntraTMP and IBC modes;

[0154] Specifically, if the coding unit adopts the IntraTMP mode, the IBC mode is added to the coding unit; if the coding unit adopts the IBC mode, the IntraTMP mode is added to it, and then the two modes are combined to obtain a new CIIP prediction method, that is, the IntraTMP mode is used as the intra-frame prediction part of CIIP, and the IBC mode is used as the inter-frame prediction part of CIIP.

[0155] Step 2: Obtain prediction signals of two prediction parts of the IBC-IntraTMPCIIP mode of the coding unit;

[0156] According to the prediction mode adopted by the coding unit: IntraTMP or IBC mode, the best matching block vector of the corresponding mode is obtained. Since both IntraTMP and IBC modes obtain the prediction value of the coding unit by determining BV, recording the best BV of the two modes is the key point to obtain the prediction signal.

[0157] There may be two cases for BV in IBC mode. The intra-frame and inter-frame prediction parts of IBC-IntraTMPCIIP mode can be implemented in three different ways:

[0158] First, regardless of whether the IBC BV is the IntraTMP BV of the adjacent coding unit, the intra-frame and inter-frame prediction parts of the IBC-IntraTMPCIIP mode are still the prediction signals obtained by the IntraTMP and IBC modes;

[0159] Second, if the IBC BV is the IntraTMP BV of the adjacent coding unit, the intra prediction part of the IBC-IntraTMPCIIP mode is the prediction value of the IBC mode of the current coding unit, and the inter prediction part is the prediction value of the TM-merge or merge mode;

[0160] Third, if the IBC BV is the IntraTMP BV of the adjacent coding unit, the inter-frame prediction part of the IBC-IntraTMPCIIP mode is the prediction value of the IBC mode of the current coding unit, and the intra-frame prediction part is the prediction mode derived from TIMD.

[0161] For specific implementation, please refer to optional implementation modes 1-a, 1-b and 1-c.

[0162] Step 3: Determine the weight values ​​of the two prediction parts in the IBC-IntraTMPCIIP mode to obtain the final prediction signal of the coding unit;

[0163] The IBC-IntraTMPCIIP mode can be used in the IBC merge mode or the IBC AMVP mode, that is, the IBC mode in the IBC-IntraTMPCIIP mode can be the IBC merge mode or the IBC AMVP mode, then the weight value during fusion needs to determine different fixed values ​​according to the IBC merge mode or the IBC AMVP mode, and as described in step 2, there are three optional implementations of this mode, so the weight values ​​of different implementations are slightly modified; then the two modes are weightedly fused with fixed weight values ​​to obtain the final prediction value of the coding unit.

[0164] Step 4: Transmit the syntax elements related to the IBC-IntraCIIP mode to the decoding end.

[0165] The syntax elements related to this embodiment include: ibc_intratmpciip_enabled_flag, which is used to determine the IBC-IntraTMPCIIP mode enablement flag at the CU level, and syntax elements related to the IBC mode and the IntraTMP mode, such as: IBC merge / AMVP mode flag, candidate index of IBC BV, multiple templates and multiple candidate indexes of the IntraTMP mode, etc.

[0166] These syntax elements need to be transmitted to the decoding end in the bitstream. After receiving the relevant identifier, the decoding end can determine whether the coding unit enables the IBC-IntraTMPCIIP mode, and determine the prediction value based on the syntax elements such as the BV index of the IBC mode and the IntraTMP mode, and then fuse the two modes according to the same operation as the encoding end to obtain the reconstructed value of the coding unit to complete the correct decoding process.

[0167] In addition, additional high-level syntax elements may be added to any one or more of the video parameter set (Video Parameter Set, VPS), sequence parameter set (Sequence Parameter Set, SPS), picture parameter set (Picture Parameter Set, PPS), adaptive parameter set (Adaptive Parameter Set, APS), slice header (Slice Header, SH), picture header (Picture Header, PH) or supplementary enhancement information (Supplemental Enhancement Information, SEI) to indicate whether the IBC-IntraCIIP mode is enabled; if the above syntax elements exist, these syntax elements need to be transmitted to the decoding end in the bitstream.

[0168] In an exemplary implementation 1-a, the prediction signals of the two prediction parts of the IBC-IntraTMPCIIP mode of the coding unit are obtained, and the weight values ​​of the two prediction parts in the IBC-IntraTMPCIIP mode are determined. The process of obtaining the final prediction signal of the coding unit can be performed regardless of whether the BV of the IBC mode of the coding unit is the IntraTMPBV of the adjacent coding unit. The intra-frame prediction part and the inter-frame prediction part of the IBC-IntraTMPCIIP mode are the IntraTMP mode and the IBC mode, respectively. The intra-frame prediction signal of the coding unit can be determined by the IntraTMP mode, and the inter-frame prediction signal of the coding unit can be determined by the IBC mode. Then, the inter-frame prediction signal and the intra-frame prediction signal determined by the two modes are weighted and fused to obtain the final prediction value of the coding unit, as shown in the following formula: ibc and P intraTMP Represent the predicted signals of IBC and IntraTMP modes, respectively, w ibc It is the weight of the IBC mode during fusion. Shift is used to control the weight distribution of the IBC mode and the IntraTMP mode.

[0169] P=(w ibc *P ibc +((1<<shift)-w ibc )*P intraTMP +(1<<(shift-1)))>>shift

[0170] In IBC merge mode, the weight value is: (w ibc ,shift)=(10,4); In IBC AMVP mode, the weight value is: (w ibc ,shift)=(1,1).

[0171] In another exemplary implementation 1-b, the process of obtaining the prediction signals of the two prediction parts of the IBC-IntraTMPCIIP mode of the coding unit and determining the weight values ​​of the two prediction parts in the IBC-IntraTMPCIIP mode to obtain the final prediction signal of the coding unit can be implemented as follows:

[0172] If the BV of the IBC mode of the coding unit selects the IntraTMP BV of the adjacent coding unit, that is, the second candidate in the IBC BV candidate list, the intra-frame prediction part of the IBC-IntraTMPCIIP mode can be to determine the intra-frame prediction signal of the coding unit using the IBC mode, and the inter-frame prediction part of the coding unit can determine the inter-frame prediction signal of the coding unit through the merge / TM merge mode; then the inter-frame prediction signals of the two modes and the intra-frame prediction signals are weighted and fused to obtain the final prediction value of the coding unit, as shown in the following formula: ibc and P inter They represent the prediction signals obtained in IBC mode and merge / TM merge mode, respectively. ibc It is the weight of the IBC mode during fusion, and shift is used to control the weight distribution of the IBC mode and the inter-frame prediction mode.

[0173] P=(w ibc *P ibc +((1<<shift)-w ibc )*P inter +(1<<(shift-1)))>>shift

[0174] In IBC merge mode, the weight value is: (w ibc ,shift)=(13,4); In IBC AMVP mode, the weight value is: (w ibc ,shift)=(1,1).

[0175] The syntax elements that need to be transmitted to the decoding end in this implementation include: a CU-level IBC-IntraTMPCIIP mode enabling flag, a CU-level IBC BV candidate index, a CU-level merge / TM merge mode candidate index, and the like.

[0176] In another exemplary implementation 1-c, the process of obtaining the prediction signals of the two prediction parts of the IBC-IntraTMPCIIP mode of the coding unit and determining the weight values ​​of the two prediction parts in the IBC-IntraTMPCIIP mode to obtain the final prediction signal of the coding unit can be implemented as follows:

[0177] If the BV of the IBC mode of the coding unit selects the IntraTMP BV of the adjacent coding unit, that is, the second candidate in the IBC BV candidate list, the inter-frame prediction part of the IBC-IntraTMPCIIP mode may include the coding unit using the IBC mode to determine the inter-frame prediction signal, and the intra-frame prediction part is to use the TIMD mode to determine the intra-frame prediction signal of the coding unit; then the intra-frame prediction signals of the two modes and the inter-frame prediction signals are weightedly fused to obtain the final prediction value of the coding unit, as shown in the following formula: ibc and P intra They represent the prediction signals obtained by the intra-frame prediction mode derived from the IBC mode and TIMD, respectively, and w ibc It is the weight of the IBC mode during fusion, and shift is used to control the weight distribution of the IBC mode and the intra-frame prediction mode.

[0178] P=(w ibc *P ibc +((1<<shift)-w ibc )*P intra +(1<<(shift-1)))>>shift

[0179] In IBC merge mode, the weight value is: (w ibc ,shift)=(9,4); In IBC AMVP mode, the weight value is: (w ibc ,shift)=(1,1).

[0180] The syntax elements that need to be transmitted to the decoding end in this implementation include: a CU-level IBC-IntraTMPCIIP mode enabling flag, a CU-level IBC BV candidate index, a CU-level TIMD mode candidate index, etc.

[0181] Fig.13 is a flowchart of an image coding prediction method provided by an embodiment of the present application. The present application can be applied to the case where predictive coding is performed during image coding. The method can be executed by an image coding prediction device, which is generally applied to a decoding end. The device can be implemented by software and / or hardware. Fig.13 The method provided in the embodiment of the present application specifically includes the following steps:

[0182] Step 510: parse the syntax element indicating the hybrid prediction mode according to the bitstream transmitted by the encoder.

[0183] In the embodiment of the present application, a code stream transmitted by an encoding end may be received, and a syntax element indicating a hybrid prediction mode may be extracted from the code stream.

[0184] Step 520: Determine the hybrid prediction mode of the decoding unit according to the syntax element.

[0185] The decoding unit may be code stream information transmitted by the encoding end, and the decoding unit may include a device for reconstructing compressed video data into a video image.

[0186] In an embodiment of the present application, the syntax element may also include information indicating whether the hybrid prediction mode is enabled and information about the inter-frame prediction mode and the intra-frame prediction mode. The syntax element may be used to determine whether the decoding unit uses the hybrid prediction mode. When it is determined that the decoding unit uses the hybrid prediction mode, the modes corresponding to the inter-frame prediction part and the intra-frame prediction part in the hybrid prediction mode may be determined according to the information carried in the syntax element. It can be understood that the hybrid prediction mode may include an IBC mode, an IntraTMP mode, and the like.

[0187] Step 530: determine the inter-frame prediction signal and the intra-frame prediction signal of the decoding unit according to the hybrid prediction mode, and determine the hybrid reconstruction signal according to the inter-frame prediction signal and the intra-frame prediction signal.

[0188] In an embodiment of the present application, the inter-frame prediction information and the intra-frame prediction signal corresponding to the decoding unit can be determined through a hybrid prediction mode, and a hybrid reconstruction signal can be generated by mixing the inter-frame prediction signal and the intra-frame prediction signal. The mixing method of the hybrid reconstruction signal may include weighted summation, formula solving, and the like.

[0189] Fig.14 is a flowchart of another image coding prediction method provided by an embodiment of the present application. The embodiment of the present application is a specific implementation based on the above-mentioned embodiment of the application. Fig.14 The image coding prediction method provided in the embodiment of the present application comprises the following steps:

[0190] Step 610: Receive a code stream transmitted by the encoding end.

[0191] In an embodiment of the present application, a decoding end may receive a code stream transmitted by an encoding end, and the code stream may include syntax elements and encoded compressed data of a video.

[0192] Step 620, parse the first syntax element of the IBC mode and the second syntax element of the IntraTMP mode in the hybrid prediction mode of the bitstream; wherein the first syntax element includes an IBC merge / IBC AMVP mode identifier and an IBC BV candidate index, and the second syntax element includes at least one of an IntraTMP multi-template / multi-template fusion identifier, a multi-candidate index, and a filtering operation identifier.

[0193] In an embodiment of the present application, a first syntax element corresponding to the IBC mode and a second syntax element corresponding to the IntraTMP mode can be parsed in the bitstream, the first syntax element includes an IBC merge / IBC AMVP mode identifier and an IBC BV candidate index, the second syntax element includes at least one of an IntraTMP multi-template / multi-template fusion identifier, a multi-candidate index, and a filtering operation identifier, and the first syntax element and the second syntax element can indicate the inter-frame prediction part and the intra-frame prediction part of the hybrid prediction mode.

[0194] Step 630: Determine the hybrid prediction mode of the decoding unit according to the syntax element.

[0195] In the embodiment of the present application, it can be determined according to the syntax elements that the processing process of the decoding unit includes decoding prediction of the hybrid prediction mode.

[0196] Step 640: Determine the best matching block vector of the decoding unit in the IBC mode according to the first syntax element as the inter-frame prediction signal.

[0197] Specifically, the IBC mode in the hybrid prediction mode can be used as the intra-frame prediction part, and the decoding unit can determine the best matching block vector according to the IBC mode and the BC merge / IBC AMVP mode identifier and the IBC BV candidate index, and the best matching block vector determined according to the IBC mode can be used as the intra-frame prediction signal of the decoding unit.

[0198] Step 650: Determine the best matching block vector of the decoding unit in the IntraTMP mode according to the second syntax element as the intra prediction signal.

[0199] In an embodiment of the present application, the IntraTMP mode within the hybrid prediction mode can be used as the intra-frame prediction part, and the decoding unit can determine the best matching block vector (or called IntraTMP BV) as the intra-frame prediction signal of the decoding unit according to the IntraTMP mode and at least one of the IntraTMP multi-template / multi-template fusion identifier, multiple candidate indexes, and filtering operation identifiers.

[0200] Step 660: Determine a mixed reconstructed signal according to a prediction formula based on the inter-frame prediction signal and the intra-frame prediction signal. The prediction formula includes:

[0201] P=(w ibc *P ibc +((1<<shift)-w ibc )*P intraTMP +(1<<(shift-1)))>>shift;

[0202] Among them, P ibc and PintraTMP They represent inter-frame prediction signals and intra-frame prediction signals respectively, shift represents the weight distribution value for controlling the IBC mode and the IntraTMP mode, and w ibc Indicates the weight of the IBC mode in the hybrid prediction mode.

[0203] In an embodiment of the present application, the hybrid reconstruction signal may be generated by mixing the inter-frame prediction signal and the intra-frame prediction signal. The mixing may be implemented by a prediction formula, which may include:

[0204] P=(w ibc *P ibc +((1<<shift)-w ibc )*P intraTMP +(1<<(shift-1)))>>shift;

[0205] In the prediction formula, P ibc and P intraTMP They represent inter-frame prediction signals and intra-frame prediction signals respectively, shift represents the weight distribution value for controlling the IBC mode and the IntraTMP mode, and w ibc Indicates the weight of the IBC mode in the hybrid prediction mode, << can represent the left shift symbol, and >> can identify the right shift symbol.

[0206] Based on the above application examples, w ibc The value of shift includes at least one of the following: IBC mode is IBC merge mode, w ibc =10, shift=4; IBC mode is IBC AMVP mode, w ibc =1, shift=1.

[0207] Fig.15 is a flowchart of another image coding prediction method provided by an embodiment of the present application. The embodiment of the present application is a specific implementation based on the above-mentioned embodiment of the application. Fig.15 The image coding prediction method provided in the embodiment of the present application comprises the following steps:

[0208] Step 710: Receive a code stream transmitted by the encoding end.

[0209] In an embodiment of the present application, a decoding end may receive a code stream transmitted by an encoding end, and the code stream may include syntax elements and encoded compressed data of a video.

[0210] Step 720, parse the first syntax element of the IBC mode and the second syntax element of the merge / TMmerge mode in the hybrid prediction mode in the bitstream; wherein the first syntax element includes the IBC merge / IBC AMVP mode identifier and the IBCBV candidate index, and the second syntax element includes at least one of the merge / TM merge mode identifier and the best MV candidate index.

[0211] In an embodiment of the present application, a first syntax element corresponding to the IBC mode and a second syntax element corresponding to the IntraTMP mode can be parsed in the bitstream, the first syntax element includes an IBC merge / IBC AMVP mode identifier and an IBC BV candidate index, the second syntax element includes a merge / TM merge mode identifier and at least one of a best MV candidate index, and the first syntax element and the second syntax element can indicate the intra-frame prediction part and the inter-frame prediction part of the hybrid prediction mode.

[0212] Step 730: Determine the hybrid prediction mode of the decoding unit according to the syntax element.

[0213] In the embodiment of the present application, it can be determined according to the syntax elements that the processing process of the decoding unit includes decoding prediction of the hybrid prediction mode.

[0214] Step 740: Determine the best matching block vector of the decoding unit in the IBC mode according to the first syntax element as the intra prediction signal.

[0215] Specifically, the IBC mode in the hybrid prediction mode can be used as the intra-frame prediction part, and the decoding unit can determine the best matching block vector according to the IBC mode and the IBC merge / IBC AMVP mode identifier and the IBC BV candidate index, and the best matching block vector determined according to the IBC mode can be used as the intra-frame prediction signal of the decoding unit.

[0216] Step 750: Determine the prediction value of the decoding unit in the merge / TM merge mode according to the second syntax element as the inter-frame prediction signal.

[0217] In an embodiment of the present application, the merge / TM merge mode within the hybrid prediction mode can be used as the inter-frame prediction part, and the decoding unit can use the prediction value determined according to the merge / TM merge mode and the merge / TM merge mode identifier and at least one of the best MV candidate indexes as the inter-frame prediction signal of the decoding unit.

[0218] Step 760: Determine a mixed reconstructed signal according to a prediction formula under the intra-frame prediction signal and the inter-frame prediction signal. The prediction formula includes:

[0219] P=(w ibc *P ibc +((1<<shift)-w ibc )*P inter +(1<<(shift-1)))>>shift;

[0220] Among them, P ibc and P inter They represent intra-frame prediction signals and inter-frame prediction signals respectively, shift represents the weight distribution value for controlling the IBC mode and merge / TM merge mode, and w ibc Indicates the weight of the IBC mode in the hybrid prediction mode.

[0221] In an embodiment of the present application, the hybrid reconstruction signal may be generated by mixing the inter-frame prediction signal and the intra-frame prediction signal. The mixing may be implemented by a prediction formula, which may include:

[0222] P=(w ibc *P ibc +((1<<shift)-w ibc )*P inter +(1<<(shift-1)))>>shift;

[0223] In the prediction formula, P ibc and P inter They represent intra-frame prediction signals and inter-frame prediction signals respectively, shift represents the weight distribution value for controlling the IBC mode and merge / TM merge mode, and w ibc Indicates the weight of the IBC mode in the hybrid prediction mode, << can represent the left shift symbol, and >> can identify the right shift symbol.

[0224] In some application embodiments, w ibc The value of shift includes at least one of the following: IBC mode is IBCmerge mode, w ibc =10, shift=4; IBC mode is IBC AMVP mode, w ibc =1, shift=1.

[0225] Fig.16 is a flowchart of another image coding prediction method provided by an embodiment of the present application. The embodiment of the present application is a specific implementation based on the above-mentioned embodiment of the application. Fig.16 The image coding prediction method provided in the embodiment of the present application comprises the following steps:

[0226] Step 810: Receive a code stream transmitted by the encoding end.

[0227] Step 820, parse the first syntax element of the IBC mode and the second syntax element of the TIMD mode in the hybrid prediction mode in the bitstream; wherein the first syntax element includes the IBC merge / IBC AMVP mode identifier and the IBC BV candidate index, and the second syntax element includes the candidate index of the TIMD mode.

[0228] In an embodiment of the present application, a first syntax element corresponding to the IBC mode and a second syntax element corresponding to the TIMD mode can be parsed in the bitstream, the first syntax element includes an IBC merge / IBC AMVP mode identifier and an IBC BV candidate index, the second syntax element includes a candidate index of the TIMD mode, and the first syntax element and the second syntax element can indicate the inter-frame prediction part and the intra-frame prediction part of the hybrid prediction mode.

[0229] Step 830: Determine the hybrid prediction mode of the decoding unit according to the syntax element.

[0230] Step 840: Determine the best matching block vector of the decoding unit in the IBC mode according to the first syntax element as the inter-frame prediction signal.

[0231] Specifically, the IBC mode in the hybrid prediction mode can be used as the intra-frame prediction part, and the decoding unit can determine the best matching block vector according to the IBC mode and the BC merge / IBC AMVP mode identifier and the IBC BV candidate index, and the best matching block vector determined according to the IBC mode can be used as the intra-frame prediction signal of the decoding unit.

[0232] Step 850: Determine the prediction value of the decoding unit in TIMD mode according to the second syntax element as an intra-frame prediction signal.

[0233] In an embodiment of the present application, the TIMD mode in the hybrid prediction mode can be used as the inter-frame prediction part, and the decoding unit can determine the prediction value as the inter-frame prediction signal of the decoding unit according to the TIMD mode and the candidate index of the TIMD mode.

[0234] Step 860: Determine a mixed reconstructed signal according to a prediction formula based on the inter-frame prediction signal and the intra-frame prediction signal. The prediction formula includes:

[0235] P=(w ibc *P ibc +((1<<shift)-w ibc )*P intra +(1<<(shift-1)))>>shift;

[0236] Among them, P ibc and P intraThey represent inter-frame prediction signals and intra-frame prediction signals respectively, shift represents the weight distribution value for controlling the IBC mode and TIMD mode, and w ibc Indicates the weight of the IBC mode in the hybrid prediction mode.

[0237] In an embodiment of the present application, the hybrid reconstruction signal may be generated by mixing the inter-frame prediction signal and the intra-frame prediction signal. The mixing may be implemented by a prediction formula, which may include:

[0238] P=(w ibc *P ibc +((1<<shift)-w ibc )*P intra +(1<<(shift-1)))>>shift;

[0239] In the prediction formula, P ibc and P intra They represent inter-frame prediction signals and intra-frame prediction signals respectively, shift represents the weight distribution value for controlling the IBC mode and TIMD mode, and w ibc Indicates the weight of the IBC mode in the hybrid prediction mode, << can represent the left shift symbol, and >> can identify the right shift symbol.

[0240] Based on the above application examples, w ibc The value of shift includes at least one of the following: IBC mode is IBC merge mode, w ibc =9, shift=4; IBC mode is IBC AMVP mode, w ibc =1, shift=1.

[0241] In an exemplary implementation, taking the joint intra-frame and inter-frame prediction of the IBC mode and IntraTMP as an example, when the decoder parses the syntax elements in the bitstream, if the syntax element sps_ciip_enabled_flag at the SPS level indicates that the sequence of the current decoding unit allows the CIIP mode to be enabled, and the syntax element ibc_intratmpciip_flag at the CU level indicates that the current decoding unit uses the IBC-IntraTMPCIIP mode, then according to different optional implementations, the same operation as the encoder is used to perform weighted fusion of the two modes to obtain the reconstruction value of the decoding unit. Fig.17 The specific implementation process of IBC-IntraTMPCIIP mode at the decoding end is as follows:

[0242] Step 1: Parse the bitstream and determine whether the decoding unit has enabled the IBC-IntraTMPCIIP mode;

[0243] First, the high-level syntax element sps_ciip_enabled_flag at the SPS level needs to be marked as 1, which means that the sequence where the decoding unit is located allows the CIIP mode to be enabled; and the syntax element ibc_intratmpciip_flag at the CU level is marked as 1, indicating that the current decoding unit adopts the IBC-IntraTMPCIIP mode.

[0244] Step 2: Determine two prediction parts of the decoding unit IBC-IntraTMPCIIP mode and corresponding prediction values;

[0245] Since the IBC BV may select the IntraTMP BV of the adjacent coding unit, there are three different optional implementations of the IBC-IntraTMPCIIP mode. Then, the relevant syntax elements of the different implementations are further parsed to determine the two fusion modes and the corresponding prediction values. The specific implementation is shown in optional implementations 2-a, 2-b and 2-c.

[0246] Step 3: Determine the weight value of the weighted fusion of the IBC-IntraTMPCIIP mode to obtain the reconstruction value of the decoding unit.

[0247] According to three different optional implementations of the IBC-IntraTMPCIIP mode, and the IBC merge mode or the IBC AMVP mode, fixed weight values ​​are determined respectively, and then weighted fusion between the two modes is performed according to the fusion operation of the encoding end to obtain the reconstruction value of the decoding unit.

[0248] In addition, if any one or more of VPS, SPS, PPS, APS, SH, PH or SEI are parsed in the bitstream and contain high-level syntax elements, indicating whether the image where the current decoding unit is located allows the IBC-IntraTMPCIIP mode to be enabled, it is necessary to parse these syntax elements first, and then determine whether the decoding unit adopts this mode based on the parsing results.

[0249] In an exemplary implementation 2-a, determining two prediction parts of the decoding unit IBC-IntraTMPCIIP mode and corresponding prediction values ​​is implemented as follows:

[0250] First, the relevant syntax elements of the IBC mode are parsed in the bitstream, including: the IBC merge / AMVP mode identifier, the candidate index of the IBC BV, and the prediction value of the IBC mode is determined; then the relevant syntax elements of the IntraTMP mode are parsed, including: the IntraTMP multi-template / multi-template fusion identifier, multiple candidate indexes, filtering operation identifier, etc., to determine the prediction value of the IntraTMP mode.

[0251] Finally, the weight values ​​of the two prediction parts in the IBC-IntraTMPCIIP mode are determined according to the IBC merge or IBC AMVP mode, and the weighted fusion is completed according to the operation of the encoding end to obtain the reconstruction value of the decoding unit.

[0252] In an exemplary implementation 2-b, determining two prediction parts of the decoding unit IBC-IntraTMPCIIP mode and corresponding prediction values ​​is implemented as follows:

[0253] First, the relevant syntax elements of the IBC mode are parsed in the bitstream, including: the IBC merge / AMVP mode identifier, the candidate index of the IBC BV, and the prediction value of the IBC mode (that is, the IntraTMP BV of the adjacent coding unit); then the relevant syntax elements of the merge / TM merge mode are parsed, including: the merge / TM merge mode identifier, the best MV candidate index, etc., to determine the inter-frame prediction mode candidates and their prediction values ​​determined by the merge / TM merge mode.

[0254] Finally, the weight values ​​of the two prediction parts in the IBC-IntraTMPCIIP mode are determined according to the IBC merge or IBC AMVP mode, and the weighted fusion is completed according to the operation of the encoding end to obtain the reconstruction value of the decoding unit.

[0255] In an exemplary implementation 2-c, determining two prediction parts of the decoding unit IBC-IntraTMPCIIP mode and corresponding prediction values ​​is implemented as follows:

[0256] First, the relevant syntax elements of the IBC mode are parsed in the bitstream, including: the identifier of the IBC merge / AMVP mode, the candidate index of the IBC BV, and the prediction value of the IBC mode (that is, the IntraTMP BV of the adjacent coding unit); then the relevant syntax elements of the TIMD mode are parsed, such as: the candidate index of the TIMD mode, etc., to determine the intra-frame prediction mode and its prediction value derived from TIMD.

[0257] Finally, the weight values ​​of the two prediction parts in the IBC-IntraTMPCIIP mode are determined according to the IBC merge or IBC AMVP mode, and the weighted fusion is completed according to the operation of the encoding end to obtain the reconstruction value of the decoding unit.

[0258] Fig.18 1 is a schematic diagram of the structure of an image coding prediction device provided in an embodiment of the present application. The device can execute the video transmission method provided in any embodiment of the present application and has the corresponding functional modules and beneficial effects of the execution method. The device can be implemented by software and / or hardware. Fig.18As shown, the device provided in the embodiment of the present application specifically includes:

[0259] The hybrid mode determination module 901 is used to obtain a coding unit of a video to be encoded, and determine a hybrid prediction mode according to an enabled coding mode of the coding unit.

[0260] The mixed signal determination module 902 is used to determine the inter-frame prediction signal and the intra-frame prediction signal of the coding unit according to the mixed prediction mode, and determine a mixed prediction signal of the inter-frame prediction signal and the intra-frame prediction signal.

[0261] The syntax element transmission module 903 is used to transmit the syntax elements of the hybrid prediction mode to the decoding end.

[0262] Based on the above application embodiment, the in-device hybrid mode determination module 901 includes:

[0263] The coding extraction unit is used to extract at least one video frame of the video to be encoded and divide the at least one video frame into the coding units.

[0264] An enabling determination unit, wherein the enabled coding mode for the coding unit is the IntraTMP mode or the IBC mode, triggers the coding unit to enable the hybrid prediction mode.

[0265] Based on the above application embodiment, the mixed signal determination module 902 is specifically used for:

[0266] Determine the best matching block vector of the coding unit as the inter-frame prediction signal according to the IBC mode in the hybrid prediction mode;

[0267] Determine the best matching block vector of the coding unit as the intra-frame prediction signal according to the IntraTMP mode in the hybrid prediction mode;

[0268] The mixed prediction value is determined according to a prediction formula under the inter-frame prediction signal and the intra-frame prediction signal, and the prediction formula includes:

[0269] P=(w ibc *P ibc +((1<<shift)-w ibc )*P intraTMP +(1<<(shift-1)))>>shift;

[0270] Among them, the P ibc And the P intraTMP represent the inter-frame prediction signal and the intra-frame prediction signal respectively, the shift represents the weight distribution value for controlling the IBC mode and the IntraTMP mode, the wibc Represents the weight of the IBC mode in the hybrid prediction mode.

[0271] Based on the above application embodiment, the w in the device ibc And the value of shift includes at least one of the following:

[0272] The IBC mode is the IBC merge mode, and the w ibc =10, shift=4;

[0273] The IBC mode is the IBC AMVP mode, and the w ibc =1, the shift=1.

[0274] Based on the above application embodiment, the mixed signal determination module 902 is specifically used for:

[0275] Determine the best matching block vector of the coding unit as the intra-frame prediction signal according to the IBC mode in the hybrid prediction mode;

[0276] Determining a prediction value of the coding unit as the inter-frame prediction signal according to a merge / TM merge mode within the hybrid prediction mode;

[0277] The mixed prediction value is determined according to a prediction formula under the intra-frame prediction signal and the inter-frame prediction signal, and the prediction formula includes:

[0278] P=(w ibc *P ibc +((1<<shift)-w ibc )*P inter +(1<<(shift-1)))>>shift;

[0279] Among them, the P ibc And the P inter represent the intra-frame prediction signal and the inter-frame prediction signal respectively, the shift represents the weight distribution value for controlling the IBC mode and the merge / TM merge mode, the w ibc Represents the weight of the IBC mode in the hybrid prediction mode.

[0280] Based on the above application embodiment, the w in the device ibc And the value of shift includes at least one of the following:

[0281] The IBC mode is the IBC merge mode, and the w ibc =13, shift=4;

[0282] The IBC mode is the IBC AMVP mode, and the w ibc =1, the shift=1.

[0283] Based on the above application embodiment, the syntax element in the device includes at least one of the following:

[0284] The mixed mode enable flag at the coding unit level;

[0285] IBC BV candidate index at coding unit level;

[0286] Candidate index for merge / TM merge mode at coding unit level.

[0287] Based on the above application embodiment, the mixed signal determination module 902 is specifically used for:

[0288] Determine the best matching block vector of the coding unit as the inter-frame prediction signal according to the IBC mode in the hybrid prediction mode;

[0289] Determining a prediction value of the coding unit as the intra-frame prediction signal according to the TIMD mode within the hybrid prediction mode;

[0290] The mixed prediction value is determined according to a prediction formula under the inter-frame prediction signal and the intra-frame prediction signal, and the prediction formula includes:

[0291] P=(w ibc *P ibc +((1<<shift)-w ibc )*P intra +(1<<(shift-1)))>>shift;

[0292] Among them, the P ibc And the P intra represent the inter-frame prediction signal and the intra-frame prediction signal respectively, the shift represents the weight allocation value for controlling the IBC mode and the TIMD mode, and the w ibc Represents the weight of the IBC mode in the hybrid prediction mode.

[0293] Based on the above application embodiment, the w in the device ibc And the value of shift includes at least one of the following:

[0294] The IBC mode is the IBC merge mode, and the w ibc =9, shift=4;

[0295] The IBC mode is the IBC AMVP mode, and the w ibc =1, the shift=1.

[0296] Based on the above application embodiment, the syntax element in the device includes at least one of the following:

[0297] The mixed mode enable flag at the coding unit level;

[0298] IBC BV candidate index at coding unit level;

[0299] Candidate index for TIMD mode at coding unit level.

[0300] Based on the above application embodiment, the device also includes:

[0301] The monitoring module is used to determine that the enabled coding mode is the IBC mode, and the best matching block vector of the coding unit is the IntraTMP BV of the adjacent coding unit.

[0302] Fig.19 1 is a schematic diagram of the structure of another image coding prediction device provided in an embodiment of the present application. The device can execute the video transmission method provided in any embodiment of the present application and has the corresponding functional modules and beneficial effects of the execution method. The device can be implemented by software and / or hardware. Fig.19 As shown, the device provided in the embodiment of the present application specifically includes:

[0303] The syntax parsing module 1001 is used to parse the syntax elements indicating the hybrid prediction mode according to the bit stream transmitted by the encoding end.

[0304] The hybrid mode module 1002 is used to determine the hybrid prediction mode of the decoding unit according to the syntax element.

[0305] The hybrid reconstruction module 1003 is used to determine the inter-frame prediction signal and the intra-frame prediction signal of the decoding unit according to the hybrid prediction mode, and determine the hybrid reconstruction signal according to the inter-frame prediction signal and the intra-frame prediction signal.

[0306] Based on the above application embodiment, the syntax parsing module 1001 is specifically used for:

[0307] Receive the code stream transmitted by the encoding end;

[0308] Parsing the first syntax element of the IBC mode and the second syntax element of the IntraTMP mode in the bitstream hybrid prediction mode;

[0309] The first syntax element includes an IBC merge / IBC AMVP mode identifier and an IBC BV candidate index, and the second syntax element includes at least one of an IntraTMP multi-template / multi-template fusion identifier, a multi-candidate index, and a filtering operation identifier.

[0310] Based on the above application embodiment, the hybrid reconstruction module 1003 is specifically used for:

[0311] Determine the best matching block vector of the decoding unit in the IBC mode according to the first syntax element as the inter-frame prediction signal;

[0312] Determine the best matching block vector of the decoding unit as an intra prediction signal in the IntraTMP mode according to the second syntax element;

[0313] The mixed reconstructed signal is determined according to a prediction formula under the inter-frame prediction signal and the intra-frame prediction signal. The prediction formula includes:

[0314] P=(w ibc *P ibc +((1<<shift)-w ibc )*P intraTMP +(1<<(shift-1)))>>shift;

[0315] Among them, P ibc and P intraTMP They represent inter-frame prediction signals and intra-frame prediction signals respectively, shift represents the weight distribution value for controlling the IBC mode and the IntraTMP mode, and w ibc Indicates the weight of the IBC mode in the hybrid prediction mode.

[0316] Based on the above application embodiment, the device ibc And the value of shift includes at least one of the following:

[0317] The IBC mode is IBC merge mode, w ibc =10, shift=4;

[0318] The IBC mode is the IBC AMVP mode, w ibc =1, shift=1.

[0319] Based on the above application embodiment, the syntax parsing module 1001 is further specifically used for:

[0320] Receive the code stream transmitted by the encoding end;

[0321] Parsing the first syntax element of the IBC mode and the second syntax element of the merge / TM merge mode in the bitstream hybrid prediction mode;

[0322] The first syntax element includes an IBC merge / IBC AMVP mode identifier and an IBC BV candidate index, and the second syntax element includes at least one of a merge / TM merge mode identifier and a best MV candidate index.

[0323] Based on the above application embodiment, the hybrid reconstruction module 1003 is specifically used for:

[0324] Determining an inter-frame prediction signal and an intra-frame prediction signal of a decoding unit according to a hybrid prediction mode, and determining a hybrid reconstruction signal according to the inter-frame prediction signal and the intra-frame prediction signal, including:

[0325] Determine the best matching block vector of the decoding unit in the IBC mode according to the first syntax element as an intra prediction signal;

[0326] Determine a prediction value of a decoding unit as an inter-frame prediction signal in a merge / TM merge mode according to the second syntax element;

[0327] The mixed reconstructed signal is determined according to a prediction formula under the intra-frame prediction signal and the inter-frame prediction signal. The prediction formula includes:

[0328] P=(w ibc *P ibc +((1<<shift)-w ibc )*P inter +(1<<(shift-1)))>>shift;

[0329] Among them, P ibc and P inter They represent intra-frame prediction signals and inter-frame prediction signals respectively, shift represents the weight distribution value for controlling the IBC mode and merge / TM merge mode, and w ibc Indicates the weight of the IBC mode in the hybrid prediction mode.

[0330] Based on the above application embodiment, the device ibc And the value of shift includes at least one of the following:

[0331] The IBC mode is IBC merge mode, w ibc =10, shift=4;

[0332] The IBC mode is the IBC AMVP mode, w ibc =1, shift=1.

[0333] Based on the above application embodiment, the syntax parsing module 1001 is further specifically used for:

[0334] Receive the code stream transmitted by the encoding end;

[0335] Parsing the first syntax element of the IBC mode and the second syntax element of the TIMD mode in the bitstream hybrid prediction mode;

[0336] The first syntax element includes the IBC merge / IBC AMVP mode identifier and the IBC BV candidate index, and the second syntax element includes the candidate index of the TIMD mode.

[0337] Based on the above application embodiment, the hybrid reconstruction module 1003 is further used for:

[0338] Determine the best matching block vector of the decoding unit in the IBC mode according to the first syntax element as the inter-frame prediction signal;

[0339] Determining a prediction value of a decoding unit as an intra prediction signal in TIMD mode according to the second syntax element;

[0340] The mixed reconstructed signal is determined according to a prediction formula under the inter-frame prediction signal and the intra-frame prediction signal. The prediction formula includes:

[0341] P=(w ibc *P ibc +((1<<shift)-w ibc )*P intra +(1<<(shift-1)))>>shift;

[0342] Among them, P ibc and P intra They represent inter-frame prediction signals and intra-frame prediction signals respectively, shift represents the weight distribution value for controlling the IBC mode and TIMD mode, and w ibc Indicates the weight of the IBC mode in the hybrid prediction mode.

[0343] Based on the above application embodiment, the device ibc And the value of shift includes at least one of the following:

[0344] The IBC mode is IBC merge mode, w ibc =9, shift=4;

[0345] The IBC mode is the IBC AMVP mode, w ibc =1, shift=1.

[0346] Fig. 20 1 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. The electronic device includes a processor 10 and a memory 11. The number of processors 10 in the electronic device may be one or more. Fig. 20 A processor 10 is taken as an example; the processor 10 and the memory 11 in the electronic device can be connected by a bus or other means. Fig. 20 The example of connecting through bus is taken in the following.

[0347] The memory 11, as a computer-readable storage medium, can be used to store software programs, computer executable programs and modules, such as the modules corresponding to the device in the embodiment of the present application (mixed mode determination module 901, mixed signal determination module 902 and syntax element transmission module 903, or syntax parsing module 1001, mixed mode module 1002 and mixed reconstruction module 1003). The processor 10 executes various functional applications and data processing of the electronic device by running the software programs, instructions and modules stored in the memory 11, that is, realizes the above-mentioned image coding prediction method.

[0348] The memory 11 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system and an application required for at least one function; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 11 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 11 may further include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0349] The embodiment of the present application further provides a storage medium containing computer executable instructions, and when the computer executable instructions are executed by a computer processor, they are used to execute an image coding prediction method, the method comprising:

[0350] Obtaining a coding unit of a video to be encoded, and determining a hybrid prediction mode according to an enabled coding mode of the coding unit;

[0351] Determine an inter-frame prediction signal and an intra-frame prediction signal of the coding unit according to the hybrid prediction mode, and determine a hybrid prediction signal of the inter-frame prediction signal and the intra-frame prediction signal;

[0352] The syntax elements of the hybrid prediction mode are transmitted to a decoding end.

[0353] and / or,

[0354] The computer executable instructions are used to perform an image coding prediction method when executed by a computer processor, the method comprising:

[0355] Parsing a syntax element indicating a hybrid prediction mode according to a bitstream transmitted by an encoding end;

[0356] Determining a hybrid prediction mode of a decoding unit according to the syntax element;

[0357] An inter-frame prediction signal and an intra-frame prediction signal of the decoding unit are determined according to the hybrid prediction mode, and a hybrid reconstruction signal is determined according to the inter-frame prediction signal and the intra-frame prediction signal.

[0358] Through the above description of the implementation method, the technicians in the relevant field can clearly understand that the present application can be implemented with the help of software and necessary general hardware, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, read-only memory (ROM), random access memory (RAM), flash memory (FLASH), hard disk or optical disk, etc., including a number of instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform the carrier aggregation method described in each embodiment of the present application.

[0359] It is worth noting that in the embodiment of the above-mentioned device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application.

[0360] Those skilled in the art will appreciate that all or some of the steps, devices, and functional modules / units in the above disclosed methods may be implemented as software, firmware, hardware, or a suitable combination thereof.

[0361] In hardware implementation, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some physical components or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor or a microprocessor, or implemented as hardware, or implemented as an integrated circuit, such as an application-specific integrated circuit. The corresponding software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or temporary medium). As known to those of ordinary skill in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically embodies computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0362] The above content illustrates the preferred embodiments of the present application with reference to the accompanying drawings, but does not limit the scope of the present application. Any modification, equivalent substitution and improvement made by a person skilled in the art without departing from the scope and essence of the present application shall be within the scope of the present application.

Claims

1. An image coding prediction method, characterized in that: Applied to the encoding end, the method includes: Obtaining a coding unit of a video to be encoded, and determining a hybrid prediction mode according to an enabled coding mode of the coding unit; Determine an inter-frame prediction signal and an intra-frame prediction signal of the coding unit according to the hybrid prediction mode, and determine a hybrid prediction signal of the inter-frame prediction signal and the intra-frame prediction signal; The syntax elements of the hybrid prediction mode are transmitted to a decoding end.

2. The method according to claim 1, characterized in that: The obtaining of a coding unit of a to-be-coded video and determining a hybrid prediction mode according to an enabled coding mode of the coding unit comprises: Extracting at least one video frame of the video to be encoded, and dividing the at least one video frame into the encoding units; The enabled coding mode of the coding unit is IntraTMP mode or IBC mode, which triggers the coding unit to enable the hybrid prediction mode.

3. The method according to claim 1, characterized in that: The step of determining the inter-frame prediction signal and the intra-frame prediction signal of the coding unit according to the hybrid prediction mode, and determining a hybrid prediction signal of the inter-frame prediction signal and the intra-frame prediction signal, comprises: Determine the best matching block vector of the coding unit as the inter-frame prediction signal according to the IBC mode in the hybrid prediction mode; Determine the best matching block vector of the coding unit as the intra-frame prediction signal according to the IntraTMP mode in the hybrid prediction mode; The mixed prediction value is determined according to a prediction formula under the inter-frame prediction signal and the intra-frame prediction signal, and the prediction formula includes: P=(w ibc *P ibc +((1<<shift)-w ibc )*P intraTMP +(1<<(shift-1)))>>shift; Among them, the P ibc And the P intraTMP represent the inter-frame prediction signal and the intra-frame prediction signal respectively, the shift represents the weight distribution value for controlling the IBC mode and the IntraTMP mode, the w ibc Represents the weight of the IBC mode in the hybrid prediction mode.

4. The method according to claim 3, characterized in that: The w ibc And the value of shift includes at least one of the following: The IBC mode is the IBC merge mode, and the w ibc =10, shift=4; The IBC mode is the IBC AMVP mode, and the w ibc =1, the shift=1.

5. The method according to claim 1, characterized in that: The step of determining the inter-frame prediction signal and the intra-frame prediction signal of the coding unit according to the hybrid prediction mode, and determining a hybrid prediction signal of the inter-frame prediction signal and the intra-frame prediction signal, comprises: Determine the best matching block vector of the coding unit as the intra-frame prediction signal according to the IBC mode in the hybrid prediction mode; Determining a prediction value of the coding unit as the inter-frame prediction signal according to a merge / TM merge mode within the hybrid prediction mode; The mixed prediction value is determined according to a prediction formula under the intra-frame prediction signal and the inter-frame prediction signal, and the prediction formula includes: P=(w ibc *P ibc +((1<<shift)-w ibc )*P inter +(1<<(shift-1)))>>shift; Among them, the P ibc And the P inter represent the intra-frame prediction signal and the inter-frame prediction signal respectively, the shift represents the weight distribution value for controlling the IBC mode and the merge / TM merge mode, the w ibc Represents the weight of the IBC mode in the hybrid prediction mode.

6. The method according to claim 5, characterized in that: The w ibc And the value of shift includes at least one of the following: The IBC mode is the IBC merge mode, and the w ibc =13, shift=4; The IBC mode is the IBC AMVP mode, and the w ibc =1, the shift=1.

7. The method according to claim 5, characterized in that: The syntax elements include at least one of the following: The mixed mode enable flag at the coding unit level; IBC BV candidate index at coding unit level; Candidate index for merge / TM merge mode at coding unit level.

8. The method according to claim 1, characterized in that: The step of determining the inter-frame prediction signal and the intra-frame prediction signal of the coding unit according to the hybrid prediction mode, and determining a hybrid prediction signal of the inter-frame prediction signal and the intra-frame prediction signal, comprises: Determine the best matching block vector of the coding unit as the inter-frame prediction signal according to the IBC mode in the hybrid prediction mode; Determining a prediction value of the coding unit as the intra-frame prediction signal according to the TIMD mode within the hybrid prediction mode; The mixed prediction value is determined according to a prediction formula under the inter-frame prediction signal and the intra-frame prediction signal, and the prediction formula includes: P=(w ibc *P ibc +((1<<shift)-w ibc )*P intra +(1<<(shift-1)))>>shift; Among them, the P ibc And the P intra represent the inter-frame prediction signal and the intra-frame prediction signal respectively, the shift represents the weight allocation value for controlling the IBC mode and the TIMD mode, and the w ibc Represents the weight of the IBC mode in the hybrid prediction mode.

9. The method according to claim 8, characterized in that: The w ibc And the value of shift includes at least one of the following: The IBC mode is the IBC merge mode, and the w ibc =9, shift=4; The IBC mode is the IBC AMVP mode, and the w ibc =1, the shift=1.

10. The method according to claim 8, characterized in that: The syntax elements include at least one of the following: The mixed mode enable flag at the coding unit level; IBC BV candidate index at coding unit level; Candidate index for TIMD mode at coding unit level.

11. The method according to claim 5 or 8, characterized in that: Before determining the inter-frame prediction signal and the intra-frame prediction signal of the coding unit according to the hybrid prediction mode and determining a hybrid prediction signal of the inter-frame prediction signal and the intra-frame prediction signal, the method further includes: It is determined that the enabled coding mode is the IBC mode, and the best matching block vector of the coding unit is the IntraTMP BV of the adjacent coding unit.

12. An image coding prediction method, characterized in that: Applied to a decoding end, the method comprises: Parsing a syntax element indicating a hybrid prediction mode according to a bitstream transmitted by an encoding end; Determining a hybrid prediction mode of a decoding unit according to the syntax element; An inter-frame prediction signal and an intra-frame prediction signal of the decoding unit are determined according to the hybrid prediction mode, and a hybrid reconstruction signal is determined according to the inter-frame prediction signal and the intra-frame prediction signal.

13. The method according to claim 12, characterized in that: The syntax element indicating the hybrid prediction mode according to the code stream transmitted by the encoding end includes: Receiving the code stream transmitted by the encoding end; Parsing the first syntax element of the IBC mode and the second syntax element of the IntraTMP mode in the hybrid prediction mode in the bitstream; The first syntax element includes an IBC merge / IBC AMVP mode identifier and an IBC BV candidate index, and the second syntax element includes at least one of an IntraTMP multi-template / multi-template fusion identifier, a multi-candidate index, and a filtering operation identifier.

14. The method according to claim 13, characterized in that: The step of determining the inter-frame prediction signal and the intra-frame prediction signal of the decoding unit according to the hybrid prediction mode, and determining the hybrid prediction signal according to the inter-frame prediction signal and the intra-frame prediction signal, comprises: Determine, in the IBC mode, the best matching block vector of the decoding unit as the inter-frame prediction signal according to the first syntax element; Determine the best matching block vector of the decoding unit in the IntraTMP mode according to the second syntax element as the intra prediction signal; The mixed reconstructed signal is determined according to a prediction formula under the inter-frame prediction signal and the intra-frame prediction signal, and the prediction formula includes: P=(w ibc *P ibc +((1<<shift)-w ibc )*P intraTMP +(1<<(shift-1)))>>shift; Among them, the P ibc And the P intraTMP represent the inter-frame prediction signal and the intra-frame prediction signal respectively, the shift represents the weight distribution value for controlling the IBC mode and the IntraTMP mode, the w ibc Represents the weight of the IBC mode in the hybrid prediction mode.

15. The method according to claim 14, characterized in that: The w ibc And the value of shift includes at least one of the following: The IBC mode is the IBC merge mode, and the w ibc =10, shift=4; The IBC mode is the IBC AMVP mode, and the w ibc =1, the shift=1.

16. The method according to claim 12, characterized in that: The syntax element indicating the hybrid prediction mode according to the code stream transmitted by the encoding end includes: Receiving the code stream transmitted by the encoding end; Parsing, in the bitstream, a first syntax element of the IBC mode and a second syntax element of the merge / TM merge mode in the hybrid prediction mode; The first syntax element includes an IBC merge / IBC AMVP mode identifier and an IBC BV candidate index, and the second syntax element includes at least one of a merge / TM merge mode identifier and a best MV candidate index.

17. The method according to claim 16, characterized in that: The step of determining the inter-frame prediction signal and the intra-frame prediction signal of the decoding unit according to the hybrid prediction mode, and determining the hybrid reconstruction signal according to the inter-frame prediction signal and the intra-frame prediction signal, comprises: Determine the best matching block vector of the decoding unit as the intra prediction signal in the IBC mode according to the first syntax element; Determining a prediction value of the decoding unit as the inter-frame prediction signal in a merge / TM merge mode according to the second syntax element; The mixed reconstructed signal is determined according to a prediction formula under the intra-frame prediction signal and the inter-frame prediction signal, and the prediction formula includes: P=(w ibc *P ibc +((1<<shift)-w ibc )*P inter +(1<<(shift-1)))>>shift; Among them, the P ibc And the P inter represent the intra-frame prediction signal and the inter-frame prediction signal respectively, the shift represents the weight distribution value for controlling the IBC mode and the merge / TM merge mode, the w ibc Represents the weight of the IBC mode in the hybrid prediction mode.

18. The method according to claim 17, characterized in that: The w ibc And the value of shift includes at least one of the following: The IBC mode is the IBC merge mode, and the w ibc =10, shift=4; The IBC mode is the IBC AMVP mode, and the w ibc =1, the shift=1.

19. The method according to claim 12, characterized in that: The syntax element indicating the hybrid prediction mode according to the code stream transmitted by the encoding end includes: Receiving the code stream transmitted by the encoding end; Parsing the first syntax element of the IBC mode and the second syntax element of the TIMD mode in the hybrid prediction mode in the bitstream; The first syntax element includes an IBC merge / IBC AMVP mode identifier and an IBC BV candidate index, and the second syntax element includes a candidate index of the TIMD mode.

20. The method according to claim 19, characterized in that: The step of determining the inter-frame prediction signal and the intra-frame prediction signal of the decoding unit according to the hybrid prediction mode, and determining the hybrid reconstruction signal according to the inter-frame prediction signal and the intra-frame prediction signal, comprises: Determine, in the IBC mode, the best matching block vector of the decoding unit as the inter-frame prediction signal according to the first syntax element; Determine, in the TIMD mode, a prediction value of the decoding unit as the intra prediction signal according to the second syntax element; The mixed reconstructed signal is determined according to a prediction formula under the inter-frame prediction signal and the intra-frame prediction signal, and the prediction formula includes: P=(w ibc *P ibc +((1<<shift)-w ibc )*P intra +(1<<(shift-1)))>>shift; Among them, the P ibc And the P intra represent the inter-frame prediction signal and the intra-frame prediction signal respectively, the shift represents the weight allocation value for controlling the IBC mode and the TIMD mode, and the w ibc Represents the weight of the IBC mode in the hybrid prediction mode.

21. The method according to claim 20, characterized in that: The w ibc And the value of shift includes at least one of the following: The IBC mode is the IBC merge mode, and the w ibc =9, shift=4; The IBC mode is the IBC AMVP mode, and the w ibc =1, the shift=1.

22. An electronic device, characterized in that: The electronic device comprises: one or more processors; A memory for storing one or more programs; When the first one or more programs are executed by the one or more processors, the one or more processors implement the image coding prediction method as described in any one of claims 1-21.

23. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores one or more programs, and the one or more programs are executed by one or more processors to implement the image coding prediction method as described in any one of claims 1-21.

Images