Method, device and storage medium for video encoding
By determining the signaling of the merge mode in video encoding and decoding, and selecting the appropriate merge mode to obtain motion information, the problem of insufficient encoding efficiency of high-resolution videos is solved, and more efficient video encoding and decoding is achieved.
Patent Information
- Application Number
- CN202311365138.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2019-04-09
- Filing Date
- 2020-04-09
- Publication Date
- 2024-11-08
- Estimated Expiration
- 2040-04-09
AI Technical Summary
Existing video encoding and decoding technologies are difficult to achieve more efficient encoding and decoding while maintaining video quality, especially when processing high-resolution video data, the encoding efficiency is insufficient, resulting in an exponential increase in the amount of data.
Using a video encoding and decoding method, by determining the signaling of the merge mode, including the conventional merge mode and the extended merge mode, selecting the appropriate merge mode according to the enabled state to obtain the motion information of the current block, and using the conventional merge mode in the signaling or other merge modes to optimize the acquisition of motion information.
It improves the efficiency and quality of video encoding and decoding, reduces the amount of data, and is suitable for encoding and decoding of high-resolution videos, improving encoding efficiency.
Smart Images

Figure CN117425015B_ABST
Abstract
Description
[0001] CROSS-REFERENCE TO RELATED APPLICATIONS
[0002] This application is a divisional application of Chinese patent application No. 202080025033.5, which is a Chinese national phase application of international patent application PCT / US2020 / 027525 filed on April 9, 2020, and which claims priority to U.S. patent application No. 62 / 831,716 filed on April 9, 2019. Technical Field
[0003] The present application relates generally to video coding and compression, and in particular, but not limited to, methods and apparatus for signaling motion merging modes in video codecs. Background Art
[0004] Various electronic devices such as digital televisions, laptop or desktop computers, tablet computers, digital cameras, digital recording devices, digital media players, video game consoles, smart phones, video teleconferencing devices, video streaming devices, etc. support digital video. Electronic devices transmit, receive, encode, decode and / or store digital video data by implementing video compression / decompression. Digital video devices implement video codec technologies such as those described in the standards defined by the following and extensions of such standards: Versatile Video Codec (VVC), Joint Exploration Test Model (JEM), MPEG-2, MPEG-4, ITU-T H.263, Part 10 of ITU-T H.264 / MPEG-4, Advanced Video Codec (AVC), ITU-T H.265 / High Efficiency Video Codec (HEVC).
[0005] Video coding and decoding are usually performed using prediction methods (e.g., inter-frame prediction, intra-frame prediction) that exploit the redundancy present in video images or sequences. An important goal of video coding and decoding technology is to compress video data into a form that uses a lower bit rate while avoiding or minimizing video quality degradation. With the emergence of evolving video services, coding techniques with better codec efficiency are needed.
[0006] Video compression typically includes performing spatial (intra-frame) prediction and / or temporal (inter-frame) prediction to reduce or remove the redundancy inherent in video data. For block-based video codecs, a video frame is divided into one or more slices, each with multiple video blocks, which may also be referred to as coding tree units (CTUs). Each CTU may contain a coding unit (CU) or may be recursively divided into smaller CUs until a predefined minimum CU size is reached. Each CU (also referred to as a leaf CU) contains one or more transform units (TUs), and each CU also contains one or more prediction units (PUs). Each CU may be encoded and decoded in intra-frame, inter-frame, or IBC mode. Video blocks in intra-coded (I) slices of a video frame are encoded using spatial prediction relative to reference samples in neighboring blocks within the same video frame. Video blocks in inter-coded (P or B) slices of a video frame may use spatial prediction relative to reference samples in neighboring blocks within the same video frame or use temporal prediction relative to reference samples in other previous and / or future reference video frames.
[0007] A prediction block for the current video block to be encoded is generated based on spatial or temporal prediction of previously encoded reference blocks (e.g., neighboring blocks). The process of finding a reference block can be accomplished by a block matching algorithm. The residual data representing the pixel difference between the current block to be encoded and the prediction block is called a residual block or prediction error. The inter-coded block is encoded according to the motion vector pointing to the reference block in the reference frame forming the prediction block, and the residual block. The process of determining the motion vector is typically called motion estimation. The intra-coded block is encoded according to the intra-frame prediction mode and the residual block. For further compression, the residual block is transformed from the pixel domain to the transform domain, such as the frequency domain, thereby generating residual transform coefficients, which can then be quantized. The quantized transform coefficients, which are initially arranged as a two-dimensional array, can be scanned to generate a one-dimensional vector of transform coefficients, and then entropy encoded into a video bitstream to achieve more compression.
[0008] The encoded video bitstream is then stored in a computer-readable storage medium (e.g., a flash memory) to be accessed by another electronic device with digital video capabilities, or directly transmitted to the electronic device in a wired or wireless manner. The electronic device then performs video decompression (which is the reverse process of the video compression described above) by, for example, parsing the encoded video bitstream to obtain syntax elements from the bitstream and reconstructing digital video data from the encoded video bitstream to its original format based at least in part on the syntax elements obtained from the bitstream, and rendering the reconstructed digital video data on a display of the electronic device.
[0009] As the quality of digital video increases from HD to 4K×2K or even 8K×4K, the amount of video data to be encoded / decoded increases exponentially. There has always been a challenge in how to encode / decode video data more efficiently while maintaining the image quality of the decoded video data.
[0010] At the Joint Video Experts Team (JVET) meeting, JVET defined the first draft of the Universal Video Codec (VVC) and VVC Test Model 1 (VTM1) coding methods. It was decided to include quadtrees with nested multi-type trees using binary and ternary partitioned coding block structures as the initial new coding features of VVC. Since then, the reference software VTM for implementing the coding method and the draft VVC decoding process has been developed during the JVET meetings. Summary of the invention
[0011] In general, this disclosure describes examples of techniques related to signaling of motion merging modes for video codecs.
[0012] According to a first aspect of the present disclosure, a method for video encoding and decoding is provided, the method comprising: determining a signaling of a merge mode for obtaining motion information of a current block, wherein the merge mode comprises: a normal merge mode and an extended merge mode; and when the signaling for determining the merge mode comprises a positive signal of the normal merge mode, obtaining the motion information of the current block using the normal merge mode.
[0013] According to a second aspect of the present disclosure, a method for video encoding and decoding is provided, the method comprising: providing a signal sending order of merge modes, the merge modes comprising a first group of merge modes and a second group of merge modes; obtaining a variable indicating an enabled state of at least one merge mode in the second group of merge modes; when it is determined that the variable indicates that a merge mode in the second group of merge modes is not enabled, using a merge mode in the first group of merge modes to obtain motion information of a current block; and when it is determined that the variable indicates that at least one merge mode in the second group of merge modes is enabled, determining the signaling of the merge mode for obtaining the motion information of the current block.
[0014] According to a third aspect of the present disclosure, a device for video encoding and decoding is provided, the device comprising: one or more processors; and a memory, the memory being configured to store instructions executable by the one or more processors; wherein the one or more processors are configured, when executing the instructions, to: determine a signaling of a merge mode for obtaining motion information of a current block, wherein the merge mode comprises: a normal merge mode and an extended merge mode; and when the signaling for determining the merge mode comprises a positive signal of the normal merge mode, use the normal merge mode to obtain the motion information of the current block.
[0015] According to a fourth aspect of the present disclosure, a device for video encoding and decoding is provided, the device comprising: one or more processors; and a memory, the memory being configured to store instructions executable by the one or more processors; wherein the one or more processors are configured, when executing the instructions, to: provide a signal sending order of merge modes, the merge modes comprising a first group of merge modes and a second group of merge modes; obtain a variable indicating an enabled state of at least one merge mode in the second group of merge modes; when it is determined that the variable indicates that a merge mode in the second group of merge modes is not enabled, use a merge mode in the first group of merge modes to obtain motion information of a current block; and when it is determined that the variable indicates that at least one merge mode in the second group of merge modes is enabled, determine the signaling of the merge mode for obtaining the motion information of the current block. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] A more specific description of examples of the present disclosure will be presented by reference to specific examples illustrated in the accompanying drawings. Given that these drawings depict only some examples and are therefore not to be considered limiting of scope, these examples will be described and explained in more detail and detail through the use of the accompanying drawings.
[0017] Figure 1 is a block diagram illustrating an exemplary video encoder according to some embodiments of the present disclosure.
[0018] Figure 2 is a block diagram illustrating an exemplary video decoder according to some embodiments of the present disclosure.
[0019] Figure 3 is a schematic diagram illustrating a merge mode with a motion vector difference (MMVD) search point according to some embodiments of the present disclosure.
[0020] Figure 4 is a schematic diagram illustrating an example of a control point-based affine motion model according to some embodiments of the present disclosure.
[0021] Figure 5 is a diagram illustrating an example of an affine motion vector field (MVF) for each sub-block of a block according to some embodiments of the present disclosure.
[0022] Figure 6 is a schematic diagram illustrating the location of inherited affine motion prediction values according to some embodiments of the present disclosure.
[0023] Figure 7 is a schematic diagram illustrating inheritance of control point motion vectors according to some embodiments of the present disclosure.
[0024] Figure 8 is a schematic diagram illustrating the positioning of candidate positions of constructed affine merge mode candidates according to some embodiments of the present disclosure.
[0025] Fig. 9A is a schematic diagram illustrating spatial neighboring blocks used by sub-block based temporal motion vector prediction (SbTMVP) according to some embodiments of the present disclosure.
[0026] Fig. 9B is a schematic diagram illustrating a SbTMVP process for obtaining a sub-CU motion field according to some embodiments of the present disclosure.
[0027] Fig.10 is a schematic diagram illustrating an example of inter-frame prediction based on triangular partitions according to some embodiments of the present disclosure.
[0028] Fig.11 is a block diagram illustrating an exemplary apparatus for video encoding and decoding according to some embodiments of the present disclosure.
[0029] Fig.12 is a flow chart illustrating an exemplary process of signaling a merge mode of a video codec according to some embodiments of the present disclosure.
[0030] Fig.13 is a flow chart illustrating another exemplary process for signaling a merge mode of a video codec according to some embodiments of the present disclosure. DETAILED DESCRIPTION
[0031] Reference will now be made in detail to specific embodiments, examples of which are illustrated in the accompanying drawings. In the following detailed description, many non-limiting specific details are set forth to aid in understanding the subject matter presented herein. However, it will be apparent to one of ordinary skill in the art that various alternatives may be used. For example, it will be apparent to one of ordinary skill in the art that the subject matter presented herein may be implemented on many types of electronic devices having digital video capabilities.
[0032] References throughout this specification to "one embodiment," "an embodiment," "an example," "some embodiments," "some examples," or similar language mean that the particular feature, structure, or characteristic being described is included in at least one embodiment or example. Unless explicitly stated otherwise, features, structures, elements, or characteristics described in conjunction with one or some embodiments are also applicable to other embodiments.
[0033] Throughout this disclosure, the terms "first", "second", "third", etc. are used as nomenclatures and are only used to refer to related elements, such as devices, parts, components, steps, etc., and do not imply any spatial or temporal order unless otherwise explicitly stated. For example, "first device" and "second device" may refer to two separately formed devices, or two parts, components or working states of the same device, and may be named arbitrarily.
[0034] As used herein, the terms "if" or "when" may be understood to mean "upon" or "in response to," depending on the context. These terms, if appearing in a claim, may not indicate that the associated limitation or feature is conditional or optional.
[0035] The terms "module", "sub-module", "circuit", "sub-circuit", "circuitry", "sub-circuitry", "unit", or "sub-unit" may include memory (shared, dedicated, or group) that stores code or instructions that can be executed by one or more processors. A module may include one or more circuits with or without stored code or instructions. A module or circuit may include one or more components that are directly or indirectly connected. These components may or may not be physically attached to or adjacent to each other.
[0036] The unit or module may be implemented in pure software, pure hardware, or a combination of hardware and software. For example, in a pure software implementation, the unit or module may include functionally related code blocks or software components that are linked together directly or indirectly to perform a specific function.
[0037] Figure 1 A block diagram illustrating an exemplary block-based hybrid video encoder 100 that can be used in conjunction with many video codec standards that use block-based processing is shown. In the encoder 100, a video frame is segmented into multiple video blocks for processing. For each given video block, a prediction is formed based on an inter-frame prediction method or an intra-frame prediction method. In inter-frame prediction, one or more prediction values are formed through motion estimation and motion compensation based on pixels from a previously reconstructed frame. In intra-frame prediction, prediction values are formed based on reconstructed pixels in the current frame. Through mode decision, the best prediction value can be selected to predict the current block.
[0038] The prediction residual representing the difference between the current video block and its predicted value is sent to the transform circuit 102. The transform coefficients are then sent from the transform circuit 102 to the quantization circuit 104 for entropy reduction. The quantized coefficients are then fed to the entropy coding circuit 106 to generate a compressed video bitstream. Figure 1 As shown, prediction related information 110 (such as video block partition information, motion vector, reference picture index and intra prediction mode) from the inter-frame prediction circuit and / or the intra-frame prediction circuit 112 is also fed through the entropy encoding circuit 106 and saved into the compressed video bitstream 114.
[0039] In the encoder 100, decoder related circuits are also required to reconstruct pixels for prediction purposes. First, the prediction residual is reconstructed by inverse quantization 116 and inverse transform circuit 118. The reconstructed prediction residual is combined with the block prediction value 120 to generate unfiltered reconstructed pixels for the current video block.
[0040] Spatial prediction (or "intra-prediction") uses pixels from samples of already coded neighboring blocks (called reference samples) in the same video frame as the current video block to predict the current video block.
[0041] Temporal prediction (also known as "inter prediction") uses reconstructed pixels from already coded video pictures to predict the current video block. Temporal prediction reduces temporal redundancy inherent in video signals. The temporal prediction signal for a given coding unit (CU) or coding block is typically signaled by one or more motion vectors (MVs) that indicate the amount and direction of motion between the current CU and its temporal reference. Furthermore, if multiple reference pictures are supported, a reference picture index is additionally sent that identifies which reference picture in the reference picture storage device the temporal prediction signal comes from.
[0042] After performing spatial prediction and / or temporal prediction, the intra / inter mode decision circuit 121 in the encoder 100 selects the best prediction mode, for example based on a rate-distortion optimization method. Then, the block prediction value 120 is subtracted from the current video block; and the resulting prediction residual is decorrelated using the transform circuit 102 and the quantization circuit 104. The resulting quantized residual coefficients are dequantized by the dequantization circuit 116, and the quantized residual coefficients are inversely transformed by the inverse transform circuit 118 to form a reconstructed residual, which is then added back to the prediction block to form a reconstructed signal for the CU. Before the reconstructed CU is placed in the reference picture storage device of the picture buffer 117 and used for encoding and decoding future video blocks, further loop filtering 115, such as a deblocking filter, sample adaptive offset (SAO) and / or adaptive loop filter (ALF), can be applied to the reconstructed CU. To form the output video bitstream 114, the coding mode (inter or intra), prediction mode information, motion information, and quantized residual coefficients are sent to the entropy coding unit 106 for further compression and packing to form a bitstream.
[0043] For example, deblocking filters are available in the current version of VVC as well as AVC and HEVC. In HEVC, an additional loop filter called SAO (sample adaptive offset) is defined to further improve the coding efficiency. In the current version of the VVC standard, another loop filter called ALF (adaptive loop filter) is being actively studied and is likely to be included in the final standard.
[0044] These loop filter operations are optional. Performing these operations helps improve coding efficiency and visual quality. These operations can also be cancelled according to the decision made by the encoder 100 to save computational complexity.
[0045] It should be noted that if these filter options are turned on by the encoder 100, intra prediction is typically based on unfiltered reconstructed pixels, while inter prediction is based on filtered reconstructed pixels.
[0046] Figure 2 is a block diagram illustrating an exemplary block-based video decoder 200 that can be used in conjunction with many video codec standards. The decoder 200 is similar to the one residing in Figure 1 The decoder 200 is a decoder that decodes the input video bitstream 201 to obtain the quantization coefficient levels and prediction related information. The quantization coefficient levels are then processed by inverse quantization 204 and inverse transform 206 to obtain the reconstructed prediction residuals. The block prediction value mechanism implemented in the intra / inter mode selector 212 is configured to perform intra prediction 208 or motion compensation 210 based on the decoded prediction information. A set of unfiltered reconstructed pixels is obtained by summing the reconstructed prediction residuals from the inverse transform 206 with the prediction output generated by the block prediction value mechanism using an adder 214.
[0047] The reconstructed block may further pass through a loop filter 209 before it is stored in a picture buffer 213 used as a reference picture storage device. The reconstructed video in the picture buffer 213 may be sent to drive a display device and used to predict future video blocks. When the loop filter 209 is turned on, a filtering operation is performed on these reconstructed pixels, thereby obtaining a final reconstructed video output 222.
[0048] The video coding / decoding standards mentioned above, such as VVC, JEM, HEVC, and Part 10 of MPEG-4, are conceptually similar. For example, all of these standards use block-based processing. At the Joint Video Experts Group (JVET) meeting, JVET defined the first draft of the generic video codec (VVC) and VVC Test Model 1 (VTM1) coding methods. It was decided to include a quadtree with nested multi-type trees that use binary and ternary partitioned coding block structures as the initial new codec feature of VVC.
[0049] In VVC, the picture partitioning structure divides the input video into blocks, which are called coding tree units (CTUs). Using a quadtree with a nested multi-type tree structure, CTUs are divided into coding units (CUs), with leaf coding units (CUs) defining areas that share the same prediction mode (e.g., intra or inter). In this disclosure, the term "unit" defines an image area covering all components; the term "block" is used to define an area covering a specific component (e.g., luma), and may differ in spatial location when considering chroma sample formats (such as 4:2:0).
[0050] Normal merge mode
[0051] In VVC Test Model 4 (VTM4), the merge candidate list is constructed by including five types of candidates in the following order:
[0052] 1) Spatial motion vector prediction value (MVP) from spatially neighboring CUs;
[0053] 2) Time MVP from the same CU;
[0054] 3) History-based MVP from FIFO (First-In-First-Out) table;
[0055] 4) Pairwise average MVP; and
[0056] 5) Zero MV.
[0057] The size of the merge list is signaled in the slice header, and the maximum allowed size of the merge list in VTM4 is 6. For each CU codec in merge mode, the index of the best merge candidate is encoded using truncated unary binarization (TU). The first binary bit of the merge index is coded using context, and the other binary bits are coded using bypass. In the following context of the present disclosure, this merge mode is also referred to as normal merge mode because the concept is the same as the merge mode used in HEVC.
[0058] Merge mode with motion vector difference (MMVD)
[0059] In addition to the conventional merge mode in which the implicitly obtained motion information is directly used for prediction sample generation of the current CU, a merge mode with motion vector difference (MMVD) is introduced in VVC. In some examples, the MMVD flag can be signaled immediately after the skip flag and the merge flag are sent to specify whether MMVD is used for the CU.
[0060] Figure 3 is a schematic diagram illustrating a merge mode with a motion vector difference (MMVD) search point according to some embodiments of the present disclosure.
[0061] In MMVD, after a merge candidate is selected, the merge candidate is further refined by the motion vector difference (MVD) information sent by the signal. This information includes a merge candidate flag, an index for specifying the magnitude of motion (i.e., distance index), and an index for indicating the direction of motion (i.e., direction index). In MMVD, one of the first two candidates in the merge list is selected to be used as the MV basis (or starting point). The merge candidate flag is signaled to specify which one to use.
[0062] The distance index specifies the motion magnitude information and indicates a predefined offset from the starting point. Figure 3 As shown, the offset is added to the horizontal component or the vertical component of the starting MV to obtain the MMVD search point. The relationship between the distance index and the predefined offset is specified in Table 1.
[0063] Table 1: Relationship between distance index and predefined offsets
[0064] Distance from IDX 0 1 2 3 4 5 6 7 Offset (in luma samples) 1 / 4 1 / 2 1 2 4 8 16 32
[0065] The direction index indicates the direction of the MVD relative to the starting point. The direction index can represent four directions shown in Table 2. Depending on the information of the starting MV, the meaning of the MVD sign (e.g., + or -) may be different. When the starting MV is a unidirectional prediction MV or a bidirectional prediction MV in which both MVs point to the same side of the current picture (i.e., the picture order counts (POCs) of both references are greater than the POC of the current picture, or the POCs of both references are less than the POC of the current picture), the sign in Table 2 specifies the sign of the MV offset added to the starting MV. When the starting MV is a bidirectional prediction MV and the two MVs point to different sides of the current picture, respectively (i.e., the POC of one reference is greater than the POC of the current picture and the POC of the other reference is less than the POC of the current picture), the sign in Table 2 specifies the sign of the MV offset added to the list 0 MV component of the starting MV, and the sign of the list 1 MV has the opposite value.
[0066] Table 2: Sign of MV offset specified by direction index
[0067] Directions to IDX 00 01 10 11 x-axis + - N / A N / A y-axis N / A N / A + -
[0068] Affine Motion Compensated Prediction
[0069] Figure 4 is a schematic diagram illustrating an example of a control point-based affine motion model according to some embodiments of the present disclosure.
[0070] In HEVC, only the translational motion model is applied to motion compensated prediction (MCP). However, in the real world, there are many kinds of motion, such as zooming in / out, rotation, perspective motion, and other irregular motions. In VVC, especially in VTM4, block-based affine transformation motion compensated prediction is applied. Figure 4 As shown, the affine motion field of a block is described by motion information of two control point motion vectors (ie, 4 parameters) or three control point motion vectors (ie, 6 parameters).
[0071] For the 4-parameter affine motion model 410, the motion vector at the sample point position (x, y) in the block is obtained as:
[0072]
[0073] For the 6-parameter affine motion model 420, the motion vector at the sample point position (x, y) in the block is obtained as:
[0074]
[0075] Among them, (mc 0x ,mv 0y ) is the motion vector of the upper left control point, (mc 1x ,mv 1y ) is the motion vector of the upper right control point, and (mc 2x ,mv 2y ) is the motion vector of the lower left control point.
[0076] Figure 5 is a schematic diagram illustrating an affine motion vector field (MVF) 501 of each sub-block of a block according to some embodiments of the present disclosure. In order to simplify motion compensation prediction, block-based affine transformation prediction is applied. Figure 5 As shown, in order to obtain the motion vector of each 4×4 luminance subblock, the motion vector of the center sample of each subblock is calculated according to equations (1) and (2) above and rounded to 1 / 16 fractional precision. Then, a motion compensation interpolation filter is applied to generate a prediction for each subblock using the obtained motion vector. The subblock size of the chrominance component can also be set to 4×4. The motion vector (MV) of the 4×4 chrominance subblock can be calculated as the average of the MVs of the four corresponding 4×4 luminance subblocks.
[0077] Similar to translational motion inter prediction, there are also two affine motion inter prediction modes: affine merge mode and affine AMVP (Advanced Motion Vector Prediction) mode.
[0078] Affine merge mode (AF_MERGE mode) can be applied to CUs whose width and height are both greater than or equal to 8. In this mode, the control point motion vector (CPMV) of the current CU is generated based on the motion information of the spatially neighboring CUs. There can be up to five CPMV prediction value (CPMVP) candidates, and the index indicating which CPMVP candidate to be used for the current CU is signaled. The following three types of CPMVP candidates are used to form the affine merge candidate list:
[0079] 1) Inherited affine merge candidates derived from the CPMV of neighboring CUs;
[0080] 2) constructing affine merge candidates using the translated MVs of neighboring CUs; and
[0081] 3) Zero MV.
[0082] Figure 6 is a schematic diagram illustrating the location of inherited affine motion prediction values according to some embodiments of the present disclosure. In VTM4, there are at most two inherited affine candidates derived from the affine motion model of the neighboring blocks, one from the neighboring CU on the left and one from the neighboring CU above. The candidate blocks of the current CU are as follows: Figure 6 As shown in . For the left prediction value, the scanning order is A0->A1, and for the top prediction value, the scanning order is B0->B1->B2. Only the first inherited candidate on each side is selected. No pruning check is performed between two inherited candidates.
[0083] Figure 7 is a schematic diagram illustrating the inheritance of control point motion vectors according to some embodiments of the present disclosure. When a neighboring affine CU 720 is identified, its control point motion vector is used to obtain a CPMVP candidate in the affine merge list of the current CU 710. Figure 7 As shown, if the adjacent lower left block A is encoded in affine mode, motion vectors v2, v3, and v4 of the upper left corner, upper right corner, and lower left corner of the CU 720 containing block A are obtained. When block A is encoded using a 4-parameter affine model, two CPMVs of the current CU 710 are calculated based on v2 and v3. In the case where block A is encoded using a 6-parameter affine model, three CPMVs of the current CU 710 are calculated based on v2, v3, and v4.
[0084] Figure 8 is a schematic diagram illustrating the positioning of candidate positions of constructed affine merge mode candidates according to some embodiments of the present disclosure. The constructed affine candidate is constructed by combining adjacent translation motion information of each control point. The motion information of the control point is obtained from Figure 8 The CPMV is obtained from the specified spatial and temporal neighbors shown.k (k=1, 2, 3, 4) represents the kth control point of the current block 810. For CPMV1, blocks B2->B3->A2 are checked in sequence, and the MV of the first available block is used. Similarly, CPMV2 is obtained by checking blocks B1->B0 in sequence. CPMV3 is obtained by checking blocks A1->A0 in sequence. If the temporal motion vector predictor T is available, the temporal motion vector predictor T is used as CPMV4.
[0085] After obtaining the MVs of the four control points, an affine merge candidate is constructed based on the motion information. The following combinations of control point MVs are used to construct in sequence:
[0086] {CPMV1, CPMV2, CPMV3},
[0087] {CPMV1, CPMV2, CPMV4},
[0088] {CPMV1, CPMV3, CPMV4},
[0089] {CPMV2, CPMV3, CPMV4},
[0090] {CPMV1, CPMV2}, and
[0091] {CPMV1, CPMV3}.
[0092] The combination of 3 CPMVs constructs a 6-parameter affine merge candidate, and the combination of 2 CPMVs constructs a 4-parameter affine merge candidate. To avoid the motion scaling process, if the reference index of the control points is different, the related control point MV combination is discarded.
[0093] After checking the inherited affine merge candidates and the constructed affine merge candidates, if the affine merge candidate list is still not full, a zero MV is inserted to the end of the list.
[0094] Sub-block based temporal motion vector prediction (SbTMVP)
[0095] The reference software VTM supports a sub-block based temporal motion vector prediction (SbTMVP) method. In some examples, the sub-block merge mode or sub-block based merge mode may include a sub-block based temporal motion vector prediction (SbTMVP) and an affine merge prediction (AMP) candidate. Fig. 9A and Fig. 9B The SbTMVP process is illustrated in FIG.
[0096] Fig. 9A is a schematic diagram illustrating spatial neighboring blocks used by sub-block based temporal motion vector prediction (SbTMVP) according to some embodiments of the present disclosure.
[0097] Fig. 9B is a schematic diagram illustrating a SbTMVP process for deriving a sub-CU motion field by applying motion shifts from spatial neighbors and scaled motion information of corresponding co-located sub-CUs according to some embodiments of the present disclosure.
[0098] Similar to the temporal motion vector prediction (TMVP) in HEVC, SbTMVP uses the motion field in the co-located picture to improve the motion vector prediction and merge mode of the CU in the current picture. The same co-located pictures used by TMVP are used for SbTMVP. SbTMVP differs from TMVP in two main aspects:
[0099] 1) TMVP predicts motion at CU level, but SbTMVP predicts motion at sub-CU level; and
[0100] 2) TMVP obtains the temporal motion vector from the co-located block in the co-located picture (the co-located block is the lower right corner block or the center block relative to the current CU), while SbTMVP applies motion shift before obtaining the temporal motion information from the co-located picture, where the motion shift is obtained from the motion vector of one of the spatial neighboring blocks of the current CU.
[0101] SbTMVP predicts the motion vector of a sub-CU within the current CU in two steps. In the first step, the motion vectors of the sub-CUs within the current CU are calculated in the following order: A1, B1, B0, and A0. Fig. 9A The spatial neighbors in the . Once the first spatial neighboring block with a motion vector that uses the co-located picture as its reference picture is identified, this motion vector is selected as the motion shift to be applied. If no such motion is identified from the spatial neighbors, the motion shift is set to (0, 0).
[0102] In the second step, if Fig. 9B As shown, the motion shift identified in the first step is applied (i.e., added to the coordinates of the current block) to Fig. 9B Sub-CU level motion information (e.g., motion vector and reference index) is obtained in the co-located picture shown in . Fig. 9B The example in assumes that the motion shift is set to the motion of the A1 block. Then, for each sub-CU, the motion information of the sub-CU is obtained using the motion information of the corresponding block of the sub-CU in the co-located picture (the minimum motion grid covering the center sample point). After identifying the motion information of the co-located sub-CU, the motion information is converted into the motion vector and reference index of the current sub-CU in a manner similar to the TMVP process of HEVC, where temporal motion scaling is applied to align the reference picture of the temporal motion vector with the reference picture of the current CU.
[0103] In VTM4, a combined sub-block based merge list is used to signal the sub-block based merge mode, which combined sub-block based merge list contains SbTMVP candidates and affine merge candidates, and the sub-block based merge mode can be referred to as sub-block merge mode or SbTMVP mode. The SbTMVP mode is enabled / disabled by a sequence parameter set (SPS) flag. If the SbTMVP mode is enabled, the SbTMVP prediction value is added as the first entry of the sub-block based merge candidate list, followed by the affine merge candidate. The size of the sub-block based merge list is signaled in the SPS, and the maximum allowed size of the sub-block based merge list in VTM4 is 5.
[0104] The sub-CU size used in SbTMVP is fixed to 8×8, and similar to the affine merge mode, the SbTMVP mode is only applicable to CUs whose width and height are both greater than or equal to 8.
[0105] The encoding logic for the additional SbTMVP merge candidate is the same as that for other merge candidates, ie, for each CU in a P or B slice, an additional RD check is performed to decide whether to use the SbTMVP candidate.
[0106] Combined Inter and Intra Prediction (CIIP)
[0107] In VTM4, when a CU is encoded and decoded in merge mode, and if the CU contains at least 64 luma samples (i.e., the CU width multiplied by the CU height is equal to or greater than 64), an additional flag is signaled to indicate whether the combined inter / intra prediction (CIIP) mode is applied to the current CU.
[0108] To form the CIIP prediction, the intra prediction mode is first obtained from two additional syntax elements. Up to four possible intra prediction modes can be used: DC, plane, horizontal or vertical. Then, the inter prediction and intra prediction signals are obtained using conventional intra and inter decoding processes. Finally, the inter and intra prediction signals are weighted averaged to obtain the CIIP prediction. The CIIP mode can be simplified by using only the plane mode for CIIP; for example, for CIIP, the intra prediction mode described in the following paragraphs can be removed. Additionally, in some CIIP designs, the weights of the intra and inter prediction samples are adaptively selected based on the number of adjacent intra-coded blocks. Specifically, the weights (wIntra, wInter) are adaptively set as follows. If both the top and left neighbors are intra-coded, (wIntra, wInter) are set equal to (3,1). Otherwise, if one of these blocks is intra-coded, the weights are the same, i.e. (2,2). If no block is intra-coded, the weights are set equal to (1,3). In these CIIP designs, the weight derivation method described in the following paragraphs may not be used.
[0109] In some examples, for intra prediction mode derivation, up to 4 intra prediction modes can be used to predict the luma component in CIIP mode, including DC, PLANAR, HORIZONTAL, and VERTICAL modes. If the CU shape is wide (i.e., its width is more than twice its height), the HORIZONTAL mode is not allowed. If the CU shape is narrow (i.e., its height is more than twice its width), the VERTICAL mode is not allowed. In these cases, only 3 intra prediction modes are allowed.
[0110] CIIP mode uses 3 most probable modes (MPM) for intra prediction. The CIIP MPM candidate list is formed as follows:
[0111] i. Set the left and top adjacent blocks to A and B respectively;
[0112] ii. The intra prediction modes of blocks A and B, denoted as intraModeA and intraModeB respectively, are derived as follows:
[0113] a. Let X be A or B;
[0114] b. If 1) block X is not available; or 2) block X is not predicted using CIIP mode or intra mode; or 3) block B is outside the current CTU, set intraModeX to DC;
[0115] c. Otherwise, set intraModeX to: 1) DC or PLANAR, if the intra prediction mode of block X is DC or PLANAR; or 2) VERTICAL, if the intra prediction mode of block X is a "vertical-like" angular mode (greater than 34), or 3) HORIZONTAL, if the intra prediction mode of block X is a "horizontal-like" angular mode (less than or equal to 34);
[0116] iii. If intraModeA is the same as intraModeB:
[0117] a. If intraModeA is PLANAR or DC, the three MPMs will be set to {PLANAR, DC, VERTICAL} in order;
[0118] b. Otherwise, the three MPMs are set to {intraModeA, PLANAR, DC} in order;
[0119] iv. Otherwise (intraModeA is different from intraModeB):
[0120] a. The first two MPMs are set to {intraModeA, intraModeB} in order;
[0121] b. Check PLANAR, DC, and VERTICAL for uniqueness in the order in which they were checked for the first two MPM candidate patterns; once a unique pattern is found, it is added as the third MPM.
[0122] If the CU shape is very wide or very narrow as defined above, the MPM flag is inferred to be 1 and need not be signaled. Otherwise, the MPM flag indicating whether the CIIP intra prediction mode is one of the CIIP MPM candidate modes is signaled.
[0123] If the MPM flag is 1, an MPM index is further signaled to indicate which of the MPM candidate modes is used in CIIP intra prediction. Otherwise, if the MPM flag is 0, the intra prediction mode is set to the "missing" mode in the MPM candidate list. For example, if the PLANAR mode is not in the MPM candidate list, PLANAR is the missing mode and the intra prediction mode is set to PLANAR. Since 4 possible intra prediction modes are allowed in CIIP and the MPM candidate list contains only 3 intra prediction modes, one of the 4 possible modes must be a missing mode.
[0124] For chroma components, DM mode is always applied without additional signaling; that is, for a CU, chroma uses the same prediction mode as luma.
[0125] The intra prediction mode of a CIIP-coded CU is saved and used for intra mode coding of future neighboring CUs.
[0126] In order to combine the inter-frame and intra-frame prediction signals, the same inter-frame prediction process applied to the normal merge mode is used to obtain the inter-frame prediction signal P in CIIP mode. inter ; and according to the conventional intra-frame prediction process, the intra-frame prediction signal P is obtained using the CIIP intra-frame prediction mode intra Then, the intra and inter prediction signals are combined using a weighted average. In some examples, the weight values depend on the intra prediction mode and where the sample is located in the coding block, as follows:
[0127] 1) If the intra prediction mode is DC or planar mode, or if the block width or height is less than 4, equal weights are applied to the intra prediction signal and the inter prediction signal;
[0128] 2) Otherwise, the weights are determined based on the intra prediction mode (horizontal mode or vertical mode in this case) and the sample position in the block. Take the horizontal prediction mode as an example (the weights for the vertical mode are obtained similarly but in the orthogonal direction). Denote W as the width of the block and H as the height of the block. First, divide the coding block into four equal-area parts, each of size (W / 4)×H. Starting from the part closest to the intra prediction reference sample and ending with the part farthest from the intra prediction reference sample, the weight wt of each of the 4 regions is set to 6, 5, 3 and 2 respectively. The final CIIP prediction signal is derived using the following:
[0129] P CIIP =((8-wt)*P inter +wt*P intra )>>3 (3)
[0130] Triangular partitioning for inter prediction
[0131] In VTM4, a new triangle partition mode is introduced for inter prediction. Triangle partition mode (TPM) is only applicable to CUs that are 8×8 or larger and are encoded or decoded in skip mode or merge mode. For CUs that meet these conditions with the merge flag turned on, a CU-level flag is signaled to indicate whether the triangle partition mode is applied. Triangle partition mode (TPM) can also be called triangle merge mode.
[0132] Fig.10 is a schematic diagram illustrating an example of inter-frame prediction based on triangular partitions according to some embodiments of the present disclosure.
[0133] When the triangle merging mode is used, the CU is evenly divided into the following Fig.10 Two triangular partitions are shown. Each triangular partition in the CU uses its own motion for inter-frame prediction; and each partition only allows unidirectional prediction, that is, each partition has one motion vector and one reference index. Unidirectional prediction motion constraints are applied to ensure that in triangular prediction mode, the CU only requires two motion compensation predictions, which is the same as traditional bidirectional prediction.
[0134] If the CU level flag indicates that the current CU is encoded using the triangle partition mode, a flag is signaled to indicate the direction of the triangle partition (i.e., diagonal or anti-diagonal). Then, an index is signaled for each of the two partitions, respectively, and the index is used to indicate the merged motion vector candidate used for each triangle partition. After predicting each triangle partition in the triangle partition, a hybrid process with adaptive weights is used to adjust the sample values along the diagonal or anti-diagonal edges. After the prediction process, the transformation and quantization process is applied to the entire CU. It is worth mentioning that the motion field of the CU predicted using the triangle partition mode is stored in 4×4 units.
[0135] Context-Adaptive Binary Arithmetic Codec (CABAC)
[0136] Context-Adaptive Binary Arithmetic Codec (CABAC) is a form of entropy coding used in many video codec standards, such as H.264 / MPEG-4 AVC, High Efficiency Video Codec (HEVC), and VVC. CABAC is based on arithmetic coding with some innovations and changes to adapt it to the needs of video codec standards:
[0137] i. CABAC encodes and decodes binary symbols, keeping complexity low and allowing probabilistic modeling of the more common bits of any symbol.
[0138] ii. The probability model is adaptively selected based on the local context, allowing for better modeling of the probabilities since coding patterns are often well correlated locally.
[0139] iii. CABAC uses multiplication-free range partitioning by using quantized probability ranges and probability states.
[0140] CABAC has multiple probability modes for different contexts. CABAC first converts all non-binary symbols to binary. Then, for each binary digit (or bit), the codec selects the probability model to use and uses information from nearby elements to optimize the probability estimate. Finally, arithmetic coding and decoding are applied to compress the data.
[0141] Context modeling provides an estimate of the conditional probability of the coded and decoded symbols. With a suitable context model, a given inter-symbol redundancy can be exploited by switching between different probability models depending on the coded and decoded symbols near the current symbol to be encoded.
[0142] Encoding and decoding data symbols involves the following stages:
[0143] i. Binarization: CABAC uses binary arithmetic coding, which means that only binary decisions (1 or 0) are encoded. Before arithmetic coding, non-binary valued symbols (such as transform coefficients or motion vectors) are "binarized" or converted to binary codes. This process is similar to the process of converting data symbols to variable length codes, but the binary codes are further encoded (by the arithmetic codec) before transmission.
[0144] ii. Repeat stage for each binary digit (or "bit") of the binarized symbol.
[0145] iii. Context model selection: A "context model" is a probability model for one or more bits of a binarized symbol. The model can be selected from a selection of available models based on statistics of recently encoded and decoded data symbols. The context model stores the probability of each bit being "1" or "0".
[0146] iv. Arithmetic Coding: The arithmetic codec encodes each binary bit according to the chosen probability model. Note that each binary bit has only two sub-ranges (corresponding to "0" and "1").
[0147] v. Probability update: Update the selected context model based on the actual decoded value (e.g., if the binary bit value is "1", increase the frequency count of "1").
[0148] In summary, the merge modes in VTM-4.0 are divided into five categories, including normal merge mode, merge mode with MVD (MMVD), sub-block merge mode (including affine merge and sub-block-based temporal motion vector prediction), combined inter-frame and intra-frame prediction (CIIP) merge mode, and triangle partition merge mode. The syntax for merging mode signaling in the current VVC is described in Table 3. The symbol ae(v) indicates a context-adaptive arithmetic entropy coding syntax element.
[0149] Table 3: Syntax of merge-related modes in current VVC
[0150]
[0151]
[0152] That is, in the current VVC, the syntax (associated flag) for indicating the corresponding merge mode transmitted by signal is shown in Table 4.
[0153] Table 4: Signaling of merge-related modes in current VVC
[0154] MMVD logo Sub-block flag CIIP logo Triangle logo MMVD 1 - - - sub-block 0 1 - - CIIP 0 0 1 - Triangle 0 0 0 1 conventional 0 0 0 0
[0155] It is observed that more than 50% of the merge modes are regular merge modes. However, in VTM-4.0, the codeword of the regular merge mode is the longest one among the five different merge modes (as shown in Table 4), which is not an efficient design in terms of syntax parsing. In the current VVC, the skip mode has a syntax design similar to that of the merge mode, except that there is no CIIP mode for skipping. The same problem is observed in the skip mode. Several methods are proposed to improve the efficiency of syntax signaling of merge-related modes. These methods can be applied independently or jointly.
[0156] In some examples, it is proposed to signal an explicit flag for normal merge mode, the explicit flag being used to indicate whether normal merge mode is used. As shown in the examples of Tables 5 and 6 below, a normal flag (which may be referred to as a normal merge flag) is explicitly signaled to the bitstream, and all signaling of related flags is modified accordingly. The normal merge flag is context-encoded using CABAC. In one embodiment, only one context is used to encode and decode the normal merge flag. In yet another embodiment, multiple context models are used to encode and decode the normal merge flag, and the selection of the context model is based on codec information, such as the normal merge flag of an adjacent block, the size of the current CU, or the skip flag of the current CU.
[0157] Table 5: Example of signaling of combined correlation mode
[0158] General signs MMVD logo Sub-block flag CIIP logo conventional 1 - - - MMVD 0 1 - - sub-block 0 0 1 - CIIP 0 0 0 1 Triangle 0 0 0 0
[0159] Table 6: Example of signaling of skip-related patterns
[0160] General signs MMVD logo Sub-block flag conventional 1 - - MMVD 0 1 - sub-block 0 0 1 Triangle 0 0 0
[0161] In the example shown in Table 5, if a conventional flag 1 is signaled (i.e., the signaling of the merge mode includes a positive signal of the conventional merge mode), it indicates that the conventional merge mode is used and the motion information of the current block is obtained using the conventional merge mode. If a conventional flag 0 is signaled (i.e., the signaling of the merge mode does not include a positive signal of the conventional merge mode), it indicates that other merge modes are used. Depending on the value(s) of the other merge mode flag(s), one of these other merge modes including MMVD, sub-block, CIIP, and triangle merge modes may be indicated for obtaining the motion information of the current block. For example, if the MMVD flag is signaled positively, it indicates that the motion information of the current block is obtained using MMVD.
[0162] In the current VVC, the constraints and / or conditions for enabling each different merge-related mode are different, which are summarized in Table 7 below. In addition, a flag for indicating the on / off of each merge mode is signaled in the sequence parameter set (SPS), and the merge modes include MMVD, sub-block (including affine and sub-block TMVP), CIIP, and triangle mode. Therefore, a variable indicating the enabled state of the merge mode can be obtained.
[0163] Table 7: Constraints for enabling or signaling merge / skip correlation mode
[0164]
[0165] Tables 5 and 6 show the binarized codewords, assuming that all merge-related modes are available (i.e., enabled). However, due to the constraints on merge mode enablement described above, there are situations where only a subset of merge-related modes, rather than all merge-related modes, are available. That is, specific merge modes may not be allowed or enabled as mode options. In this case, the corresponding codeword binarization may also be adjusted accordingly to improve encoding and decoding efficiency. For example, in certain cases where only the conventional merge mode and the MMVD mode are enabled for a given CU, only one binary bit needs to be signaled, which is used to indicate which of the two merge modes is used for a given CU, as shown in Table 8. In this case, there is no need to signal flags such as MMVD, sub-block, or CIIP flags.
[0166] Table 8: Special cases of merge mode signaling
[0167] General signs MMVD logo Sub-block flag CIIP logo conventional 1 - - - MMVD 0 - - -
[0168] There are other examples where only a subset of merge-related patterns are available due to merge-mode enabled constraints. Whenever only a subset of merge-related patterns are available, the corresponding codeword binarization can be adjusted accordingly to save signaling overhead.
[0169] According to the present disclosure, the necessity of signaling each merge mode flag in different merge mode flags can be adjusted accordingly according to the enable / disable status (or enablement status) of each merge mode. More specifically, based on the signal sending order (for example, the signal sending order can be a conventional merge flag->MMVD flag->sub-block flag->CIIP flag), the given merge mode flag is only signaled when, in the signal sending order, there is at least one other type of merge mode after the given merge mode and the at least one other type of merge mode is also allowed (i.e., enabled as a mode option) for the current CU. Based on this concept, an exemplary signal sending method is provided. Taking the above signal sending order as an example, if the CIIP and triangle merge modes are not enabled, the sub-block flag may not be signaled.
[0170] The above-described signal transmission order is merely an example, and other signal transmission orders may be possible.
[0171] In some examples, when a certain merge mode is allowed (enabled) but the corresponding flag is not signaled as described above, the value of the corresponding flag can be inferred or obtained. If all the merge mode flags explicitly signaled for the current CU are false, the value of the corresponding flag is inferred to be true. Otherwise, the inferred value is false. In the example shown in Table 8, where the MMVD mode is allowed but the MMVD flag is not signaled, if the conventional flag signaled is 0, the value of the MMVD flag is inferred to be true (or 1). Otherwise, the value of the inferred MMVD flag is false (or 0).
[0172] Table 9 is a syntax table illustrating an example of the syntax of the merge mode signal transmission according to the present disclosure. In the example, the values of four Boolean variables MMVDAllowed, MergeSubblockAllowed, MergeCIIPAllowed, and MergeTriangleAllowed are first obtained, specifying whether each mode is allowed for the current coding block according to the following items:
[0173] i.MMVDAllowed=sps_MMVD_enabled_flag;
[0174] ii.MergeSubblockAllowed = (sps_affine_enabled_flag | sps_sbtmvp_enabled_flag) &&block width >= 8 &&block height >= 8;
[0175] iii.MergeCIIPAllowed=sps_ciip_enabled_flag&&The current codec block is not encoded in skip mode&&(block width x block height)>=64&&block width !=128&&block height !=128;
[0176] iv.MergeTriangleAllowed=sps_triangle_enabled_flag&&(block width x block height)>=64.
[0177] In the above expressions, the symbols "||" and "&&" represent logical OR and AND operations, respectively. It is worth noting that the conditions of each Boolean variable listed above correspond to the current VVC. In the future, if the associated constraints of a certain merge mode change, the corresponding equation or expression can be updated accordingly, and the method described in the present disclosure still applies. The Boolean variable is used to determine whether each merge mode flag needs to be signaled, as illustrated in Table 9. In the syntax table, the bold variables represent the syntax that is signaled. The symbol ae(v) indicates a context adaptive arithmetic entropy coding syntax element.
[0178] Table 9: Syntax for merging related patterns in an example
[0179]
[0180]
[0181]
[0182] In some examples, different merge modes may be considered in groups. For example, a first group of merge modes may include a conventional merge mode, and a second group of merge modes may include a CIIP merge mode and a triangle merge mode (TPM). Alternatively, or in addition, a sub-block based merge mode may be considered as a third group. A signaling order may be provided based on the merge mode group. According to different embodiments, the merge mode groups may be in any predefined order.
[0183] In one example, the first group may be listed before the second group in the signaling order, and then the regular merge mode flag may be signaled only if there is at least one merge mode in the second group of merge modes that is allowed or enabled as a mode option (i.e., one or both of CIIP and TPM are allowed).
[0184] Fig.11 1 is a block diagram illustrating an apparatus for video encoding and decoding according to some embodiments of the present disclosure. The apparatus 1100 may be a terminal such as a mobile phone, a tablet computer, a digital broadcast terminal, a tablet device, or a personal digital assistant.
[0185] like Fig.11 As shown in , device 1100 may include one or more of the following components: a processing component 1102, a memory 1104, a power component 1106, a multimedia component 1108, an audio component 1110, an input / output (I / O) interface 1112, a sensor component 1114, and a communication component 1116.
[0186] The processing component 1102 generally controls the overall operation of the device 1100, such as operations related to display, phone calls, data communications, camera operations, and recording operations. The processing component 1102 may include one or more processors 1120 for executing instructions to perform all or part of the steps of the above method. Further, the processing component 1102 may include one or more modules to facilitate interaction between the processing component 1102 and other components. For example, the processing component 1102 may include a multimedia module to facilitate interaction between the multimedia component 1108 and the processing component 1102.
[0187] The memory 1104 is configured to store different types of data to support the operation of the device 1100. Examples of such data include instructions for any application or method running on the device 1100, contact data, phone book data, messages, pictures, videos, etc. The memory 1104 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, and the memory 1104 can be a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic memory, a flash memory, a magnetic disk, or an optical disk.
[0188] The power supply component 1106 provides power to the various components of the device 1100. The power supply component 1106 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device 1100.
[0189] Multimedia component 1108 includes a screen that provides an output interface between device 1100 and a user. In some examples, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen that receives input signals from a user. The touch panel may include one or more touch sensors for sensing touch, sliding, and gestures on the touch panel. The touch sensor may not only sense the boundaries of a touch action or a sliding action, but may also detect the duration and pressure associated with a touch or sliding operation. In some examples, multimedia component 1108 may include a front camera and / or a rear camera. When device 1100 is in an operating mode (such as a shooting mode or a video mode), the front camera and / or the rear camera may receive external multimedia data.
[0190] The audio component 1110 is configured to output and / or input audio signals. For example, the audio component 1110 includes a microphone (MIC). When the device 1100 is in a working mode (such as a call mode, a recording mode, and a speech recognition mode), the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 1104 or sent via the communication component 1116. In some examples, the audio component 1110 further includes a speaker for outputting an audio signal.
[0191] The I / O interface 1112 provides an interface between the processing component 1102 and a peripheral interface module. The peripheral interface module may be a keyboard, a click wheel, buttons, etc. These buttons may include but are not limited to a home button, a volume button, a start button, and a lock button.
[0192] The sensor component 1114 includes one or more sensors for providing different aspects of status evaluation for the device 1100. For example, the sensor component 1114 can detect the on / off state of the device 1100 and the relative position of the components. For example, the components are the display and keyboard of the device 1100. The sensor component 1114 can also detect the position change of the device 1100 or the components of the device 1100, the presence or absence of the user's contact with the device 1100, the orientation or acceleration / deceleration of the device 1100, and the temperature change of the device 1100. The sensor component 1114 may include a proximity sensor, which is configured to detect the presence of nearby objects without any physical contact. The sensor component 1114 may further include an optical sensor, such as a CMOS or CCD image sensor used in imaging applications. In some examples, the sensor component 1114 may further include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.
[0193] Communication component 1116 is configured to promote wired or wireless communication between device 1100 and other equipment. Device 1100 can access a wireless network based on communication standards such as WiFi, 4G or its combination. In an example, communication component 1116 receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel. In an example, communication component 1116 may further include a near field communication (NFC) module for promoting short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.
[0194] In an example, the apparatus 1100 may be implemented by one or more of the following to perform the above method: an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a controller, a microcontroller, a microprocessor, or other electronic components.
[0195] The non-transitory computer-readable storage medium may be, for example, a hard disk drive (HDD), a solid-state drive (SSD), a flash memory, a hybrid drive or a solid-state hybrid drive (SSHD), a read-only memory (ROM), a compact disk-read-only memory (CD-ROM), a magnetic tape, a floppy disk, etc.
[0196] Fig.12 is a flow chart illustrating an exemplary process of signaling a merge mode of a video codec according to some embodiments of the present disclosure.
[0197] In step 1202 , the processor 1120 determines signaling of a merge mode for obtaining motion information of a current block, wherein the merge mode includes: a normal merge mode and an extended merge mode.
[0198] In step 1204 , when it is determined that the signaling of the merge mode includes a positive signal of the normal merge mode, the processor 1120 obtains the motion information of the current block using the normal merge mode.
[0199] The extended merge mode may include a merge mode MMVD with motion vector difference, a subblock merge mode, a combined inter and intra prediction CIIP merge mode, and a triangle merge mode.
[0200] Fig.13 is a flow chart illustrating another exemplary process for signaling a merge mode of a video codec according to some embodiments of the present disclosure.
[0201] In step 1302 , the processor 1120 provides a signal transmission order of a merging mode, where the merging mode includes a first group merging mode and a second group merging mode.
[0202] In step 1304 , the processor 1120 obtains a variable indicating an activation state of at least one merging mode in the second group of merging modes.
[0203] In step 1306 , when it is determined that the variable indicates that the merge mode in the second group of merge modes is not enabled, the processor 1120 obtains the motion information of the current block using the merge mode in the first group of merge modes.
[0204] In step 1308 , when it is determined that the variable indicates that at least one merge mode in the second group of merge modes is enabled, the processor 1120 determines signaling of a merge mode for obtaining motion information of the current block.
[0205] In step 1310 , when it is determined that at least one other type of merge mode exists after the given merge mode in the signaling order for the current block and the at least one other type of merge mode is enabled, the processor 1120 signals a given merge mode flag.
[0206] In step 1312 , when it is determined that there are no other types of enabled merge modes after the given merge mode in the signaling order for the current block, the processor 1120 does not signal the given merge mode flag.
[0207] At step 1314 , when it is determined that a certain merge mode is enabled but the corresponding flag is not signaled, the processor 1120 infers that the value of the corresponding flag is opposite to the explicitly signaled merge mode flag for the current block.
[0208] In some examples, a device for video encoding and decoding is provided. The device includes one or more processors 1120; and a memory 1104, the memory is configured to store instructions executable by the one or more processors; wherein the processor is configured to perform the following when executing the instructions: Fig.12 or Fig.13 The method shown in the figure.
[0209] In some other examples, a non-transitory computer-readable storage medium 1104 having instructions stored therein is provided. When the instructions are executed by one or more processors 1120, the instructions cause the processor to perform the following steps: Fig.12 or Fig.13 The method shown in the figure.
[0210] The description of the present disclosure has been presented for illustrative purposes, and the description is not intended to be exhaustive or limited to the present disclosure. Many modifications, variations, and alternative embodiments will be apparent to one of ordinary skill in the art having the benefit of the teachings presented in the foregoing description and the associated drawings.
[0211] The embodiments are chosen and described in order to explain the principles of the present disclosure and to enable other persons skilled in the art to understand the various embodiments of the present disclosure and to best utilize the basic principles and various embodiments with various modifications suitable for the specific use contemplated. Therefore, it should be understood that the scope of the present disclosure should not be limited to the specific examples of the disclosed embodiments, and modifications and other embodiments are intended to be included within the scope of the present disclosure.
Claims
1. A method for video encoding, the method comprising: determining an enabled state of at least one merging mode in the first group of merging modes; When it is determined that the merge mode in the first group of merge modes is not enabled, determining whether to apply the merge mode in the second group of merge modes to obtain motion information of the current block; as well as When it is determined to enable at least one merge mode in the first group of merge modes, determining whether to apply the merge mode in the first group of merge modes to obtain the motion information of the current block; wherein a variable for determining the enabling state of the at least one merge mode in the first group of merge modes is encoded into a bitstream, and when determining to enable a merge mode in the first group of merge modes, merge mode information is encoded into the bitstream, the merge mode information comprising a flag indicating whether to apply a merge mode in the first group of merge modes, the flag being different from the variable, and The first group of merge modes includes a combined inter-frame and intra-frame prediction CIIP merge mode and a triangular partition TPM mode, and the second group of merge modes includes a normal merge mode.
2. The method of claim 1, further comprising: When it is determined to apply a merge mode in the second group of merge modes, the motion information of the current block is obtained using the merge mode in the second group of merge modes.
3. The method of claim 1, further comprising: When it is determined that the merge mode in the second group of merge modes is not to be applied, the motion information of the current block is obtained by using the merge mode in the first group of merge modes.
4. The method of claim 1, wherein: The flag indicating whether to apply a merging mode in the first group of merging modes is obtained according to a sequence parameter set SPS flag and a size constraint for a corresponding merging mode.
5. The method of claim 1, wherein: When it is determined that a particular merge mode is enabled but a corresponding flag is not present in the bitstream, the value of the corresponding flag is inferred to be opposite to the merge mode flag explicitly encoded into the bitstream for the current block.
6. The method of claim 1, wherein: The normal merge mode is signaled using context adaptive binary arithmetic coding (CABAC) with multiple context models, and selection of the multiple context models is based on a skip flag of the current block.
7. A device for video encoding, the device comprising: one or more processors; as well as a memory configured to store instructions executable by the one or more processors; Wherein, when executing the instructions, the one or more processors are configured to perform the method according to any one of claims 1-6 to generate a bit stream.
8. A non-transitory computer-readable storage medium storing instructions, wherein: The instructions, when executed by one or more processors of a computing device, cause the computing device to perform the method according to any one of claims 1-6.
9. A computer program product, comprising instructions for being executed by a computing device having one or more processors, wherein when the instructions are executed by the one or more processors, the computing device performs the method according to any one of claims 1 to 6 to generate a bitstream.
