Encoding method, decoding method, encoder, decoder, and storage medium

Abstract

The invention relates to an encoding method, a decoding method, an encoder, a decoder, and a storage medium. The invention also relates to signaling affine modes in the encoded video stream; specifically, a list of merge candidates corresponding to adjacent blocks of a current block is determined; and signaling an affine mode of the current block; wherein signaling the affine mode comprises decoding a context coded flag from the data stream, and wherein a context variable of the flag is determined based on whether the neighboring block uses the affine mode. Related encoding and decoding methods and apparatuses are also disclosed.

Description

[0001] (This application is a divisional application of an application filed on September 18, 2019, with application number 2019800621169 and invention name “Video Encoding and Decoding”.) Technical Field

[0002] The present invention relates to video encoding and decoding. Background Art

[0003] Recently, the Joint Video Experts Group (JVET) (a collaborative team consisting of MPEG and ITU-T Study Group 16 VCEG) began work on a new video coding standard called Versatile Video Coding (VVC). The goal of VVC is to provide significant improvements in compression performance over the existing HEVC standard (i.e., typically twice as good as before) and to be completed in 2020. Key target applications and services include, but are not limited to, 360-degree and high dynamic range (HDR) video. In total, JVET evaluated feedback from 32 organizations using formal subjective tests conducted by independent test labs. Some recommendations showed that compression efficiency was generally improved by 40% or more when compared to using HEVC. Particular results were shown on ultra-high definition (UHD) video test material. Therefore, for the final standard, we can expect compression efficiency improvements far exceeding the 50% that is targeted.

[0004] The JVET Exploration Model (JEM) uses all HEVC tools. Another tool that is not present in HEVC is the use of "affine motion mode" when applying motion compensation. Motion compensation in HEVC is limited to translation, but in reality there are many kinds of motion, such as zooming in / out, rotation, perspective motion, and other irregular motions. When using affine motion mode, more complex transforms are applied to try to more accurately predict these forms of motion. However, the use of affine motion mode may increase the complexity of the encoding / decoding process and may also increase signal overhead.

[0005] Therefore, a solution to at least one of the above-mentioned problems is desired. Summary of the invention

[0006] In a first aspect of the present invention, a method of signaling a motion prediction mode for a portion of a bitstream is provided, the method comprising: determining an inter-frame prediction mode for the portion of the bitstream; and signaling an affine motion mode based on the inter-frame prediction mode used in the portion of the bitstream.

[0007] Optionally, the inter prediction mode used is determined based on a state of a skip flag in said portion of said bitstream.

[0008] Optionally, if the skip flag is present, affine mode is not enabled.

[0009] Optionally, the method further comprises enabling a merge mode when the affine mode is enabled.

[0010] Optionally, if the inter prediction mode is Advanced Motion Vector Predictor (AMVP), affine mode is enabled.

[0011] Optionally, the determining is performed based on a high-level syntax flag, wherein the high-level syntax flag indicates processing at least one of: a slice level, a frame level, a sequence level, and a coding tree unit (CTU) level.

[0012] Optionally, determining the inter prediction mode includes determining a mode of one or more neighboring blocks of the current block.

[0013] In a second aspect of the present invention, a method of signaling a motion prediction mode in a bitstream is provided, the method comprising: determining a mode of one or more neighboring blocks of a current block; and signaling an affine motion mode for the current block based on the mode.

[0014] Optionally, the adjacent blocks consist of only block A1 and block B1.

[0015] Optionally, the adjacent blocks include block A2 and block B3; preferably, the adjacent blocks consist of only block A2 and block B3.

[0016] Optionally, the method comprises enabling the affine motion mode if one or both of the neighboring blocks use the affine motion mode.

[0017] Optionally, the adjacent blocks further include B0, A0 and B2.

[0018] Optionally, the use of affine mode in the neighboring blocks is determined in series, and if one of the neighboring blocks uses affine mode, affine mode is enabled for the current block. Preferably, the series of neighboring blocks is A2, B3, B0, A0, B2.

[0019] In a third aspect of the present invention, a method for signaling a motion prediction mode for a portion of a bitstream is provided, the method comprising: determining a list of merge candidates corresponding to neighboring blocks of a current block; and enabling the affine mode for the current block if one or more of the merge candidates use an affine mode.

[0020] Optionally, the list starts with a block that has been used to determine context variables related to the block.

[0021] Optionally, the list starts with blocks A2 and B3 in that order.

[0022] Optionally, the list is in the order A2, B3, B0 or ​​A0 or B2.

[0023] Optionally, if the neighboring block does not use the merge mode, the affine mode is enabled for the current block.

[0024] Optionally, if the neighboring block does not use the merge skip mode, the affine mode is enabled for the current block.

[0025] Optionally, signaling the affine mode comprises inserting a context encoding flag into the data stream, and wherein a context variable for the flag is determined based on whether a neighboring block uses the affine mode.

[0026] In another aspect of the present invention, a method of signaling a motion prediction mode for a block encoded in a bitstream is provided, the method comprising: determining whether neighboring blocks of the block encoded in the bitstream use an affine mode; and inserting a context coding flag into the bitstream; wherein a context variable for the context coding flag depends on the determination of whether the neighboring blocks of the block encoded in the bitstream use an affine mode.

[0027] Optionally, the adjacent blocks include blocks A1 and B1.

[0028] Alternatively, if the mode of the block for which the motion prediction mode is being enabled is the merge mode, the neighboring blocks include blocks A1 and B1.

[0029] Optionally, the context of the affine flag is obtained according to the following formula: Ctx=IsAffine(A1)+IsAffine(B1), where Ctx is the context variable of the affine flag, and IsAffine is a function that returns 0 if the block is not an affine block and returns 1 if the block is affine.

[0030] In a fourth aspect of the present invention, a method of signaling a motion prediction mode of a block for encoding within a bitstream according to whether a neighboring block uses a merge mode and / or a merge skip mode.

[0031] In a fifth aspect of the present invention, there is provided a method of signaling a motion prediction mode in a bitstream, the method comprising: compiling a list of candidate motion predictors; and inserting an affine merge mode as a merge candidate.

[0032] Optionally, the affine merge mode candidates follow the neighboring block motion vectors in the merge candidate list.

[0033] Optionally, affine merge mode candidates precede alternative temporal motion vector predictor (ATMVP) candidates in the merge candidate list.

[0034] Optionally, the position (merge index) of the affine merge mode candidate in the candidate list is fixed.

[0035] Optionally, the position of the affine merge mode candidate in the candidate list is variable.

[0036] Optionally, the position of the affine merge mode candidate is determined based on one or more of: a) the state of the skip flag; b) motion information of the neighboring block; c) alternative temporal motion vector predictor (ATMVP) candidates; and d) whether the neighboring block uses the affine mode.

[0037] Optionally, the affine merge mode is placed lower in the candidate list (assigned a higher merge index) if one or more of the following conditions are met: a) a skip flag exists; b) the motion information of adjacent blocks is equal; c) the ATMVP candidate contains only one motion information; and d) more than one adjacent block uses the affine mode.

[0038] Optionally, the adjacent blocks include blocks A1 and B1.

[0039] Optionally, if one or more of the above conditions a) to d) are met, the affine merge mode is placed at a lower position than the spatial motion vector candidate in the candidate list (assigned a higher merge index).

[0040] Optionally, if one or more of the above conditions a) to d) are met, the affine merge mode is placed at a lower position than the temporal motion vector candidate (assigned a higher merge index).

[0041] Optionally, the affine merge mode is assigned a merge index related to the number of neighboring blocks using the affine mode.

[0042] Optionally, the affine merge mode is assigned a value equal to 5 minus the number of neighboring blocks using the affine mode among A1, B1, B0, A0, B2.

[0043] According to other aspects of the present invention, a method for signaling an affine motion mode of a bitstream is provided, the method comprising: determining the likelihood of whether the affine mode is used for a current block; compiling a list of candidate motion predictors; and inserting an affine merge mode as a merge candidate based on the likelihood of the affine mode determined for the current block.

[0044] Optionally, the likelihood is determined based on at least one of: a) a state of a skip flag; b) motion information of a neighboring block; and c) an ATMVP candidate.

[0045] Optionally, the affine merge mode is not inserted as a merge candidate if one or more of the following conditions are met: a) the state of the skip flag; b) the motion information of adjacent blocks are equal; and c) the ATMVP candidate contains only one motion information.

[0046] Optionally, the adjacent blocks include blocks A1 and B1.

[0047] Optionally, the affine mode is signaled according to characteristics of the apparatus used to record the video corresponding to the bitstream being encoded.

[0048] Compared with existing coding standards or recommendations, aspects of the present invention provide improvements in coding efficiency and / or reductions in coding complexity. In this way, more efficient and faster video encoding and / or decoding methods and systems are provided.

[0049] Another aspect of the invention relates to a coding and decoding method using the method of any of the above aspects.

[0050] Yet another aspect of the invention relates to an apparatus for signaling the use of an affine mode in a bitstream representing encoded video as defined in claim 14.

[0051] A further aspect of the invention relates to an encoder and a decoder as defined in claims 17 and 18, respectively.

[0052] A further aspect of the invention relates to a program as defined in claim 19. The program may be provided separately or may be carried by or in a carrier medium. The carrier medium may be non-transitory, such as a storage medium, in particular a computer-readable storage medium. The carrier medium may also be transient, such as a signal or other transmission medium. The signal may be transmitted via any suitable network including the Internet.

[0053] A further aspect of the invention relates to a peripheral device such as a camera or a mobile device as defined in claims 15 and 16.

[0054] Optionally, the camera further comprises a zoom component and may be adapted to indicate when the zoom component is operable and to signal the affine mode in dependence on the indication that the zoom component is operable.

[0055] Optionally, the camera further comprises a pan component and may be adapted to indicate when the pan component is operable and to signal the affine mode in dependence on the indication that the pan component is operable.

[0056] Optionally, the mobile device further comprises at least one position sensor adapted to sense a change in orientation of the mobile device, and may be adapted to signal the affine mode in dependence on the sensed change in orientation of the mobile device.

[0057] Further characteristics of the invention are characterized by the other independent and dependent claims.

[0058] Any features in one aspect of the invention may be applied to other aspects of the invention in any appropriate combination. In particular, method aspects may be applied to apparatus aspects, and vice versa.

[0059] Furthermore, features implemented in hardware may be implemented in software and vice versa. Any references herein to software and hardware features should be interpreted accordingly.

[0060] Any apparatus features as described herein may also be provided as method features, and vice versa.As used herein, means-plus-function features may alternatively be expressed in terms of their corresponding structure (such as a suitably programmed processor and associated memory, etc.).

[0061] It will also be appreciated that specific combinations of the various features described and defined in any aspect of the present invention may be independently implemented, provided and / or used. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Reference will now be made by way of example to the accompanying drawings, in which:

[0063] Figure 1 is a diagram for explaining the coding structure used in HEVC;

[0064] Figure 2 is a block diagram schematically illustrating a data communication system in which one or more embodiments of the present invention may be implemented;

[0065] Figure 3 is a block diagram illustrating components of a processing device that may implement one or more embodiments of the present invention;

[0066] Figure 4 is a flowchart illustrating the steps of an encoding method according to an embodiment of the present invention;

[0067] Figure 5 is a flowchart illustrating the steps of a decoding method according to an embodiment of the present invention;

[0068] Figure 6a and 6b Illustrate spatial and temporal blocks that can be used to generate motion vector predictors;

[0069] Figure 7 Simplified steps of the AMVP predictor subset derivation process are shown;

[0070] Figure 8 is a schematic diagram of the motion vector derivation process for merge mode;

[0071] Fig. 9 Illustrate segmentation and temporal motion vector prediction of the current block;

[0072] Fig.10 (a) illustrates the encoding of the merge index for HEVC, or when ATMVP is not enabled at the SPS level;

[0073] Fig.10 (b) illustrates the encoding of the merge index when ATMVP is enabled at the SPS level;

[0074] Fig.11 (a) illustrates a simple affine motion field;

[0075] Fig.11 (b) illustrates a more complex affine motion field;

[0076] Fig.12 is a flowchart of a partial decoding process of some syntactic elements related to the coding mode;

[0077] Fig.13 is a flow chart illustrating the derivation of merge candidates;

[0078] Fig.14 is a flow chart illustrating a first embodiment of the present invention;

[0079] Fig.15 is a flow chart illustrating a second embodiment of the present invention;

[0080] Fig.16 is a flow chart illustrating a third embodiment of the present invention;

[0081] Fig.17 is a flow chart illustrating a fourth embodiment of the present invention;

[0082] Fig.18 is a flow chart illustrating a fifth embodiment of the present invention;

[0083] Fig.19 is a schematic block diagram of a computing device for implementing one or more embodiments of the present invention;

[0084] Fig. 20 is a schematic block diagram of a computing device;

[0085] Fig.21 is a diagram illustrating a network camera system; and

[0086] Fig. 22 The figure illustrates an example of a smart phone. DETAILED DESCRIPTION

[0087] The present invention relates to improved signalling of affine motion modes - in particular determining situations where an affine mode may result in an improvement in coding efficiency, and ensuring that the affine mode is used and / or given priority accordingly.

[0088] Figure 1 The invention relates to a coding structure used in the High Efficiency Video Coding (HEVC) video standard. A video sequence 1 consists of a series of digital images i. Each of these digital images is represented by one or more matrices. The matrix coefficients represent pixels.

[0089] The images 2 of the sequence may be partitioned into slices 3. In some cases, one slice may constitute the entire image. The slices are partitioned into non-overlapping coding tree units (CTUs). The coding tree unit (CTU) is the basic processing unit of the High Efficiency Video Coding (HEVC) video standard and conceptually corresponds in structure to the macroblock unit used in several previous video standards. The CTU is sometimes also referred to as the largest coding unit (LCU). The CTU has luma and chroma component parts, each of which is called a coding tree block (CTB). These different color components are not in the same Figure 1 Shown in.

[0090] For HEVC, a CTU is typically 64 pixels by 64 pixels in size, but for VVC it ​​can be 128 pixels by 128 pixels. Each CTU can be iteratively partitioned into smaller, variable-sized coding units (CUs) using a quadtree decomposition5.

[0091] The coding unit is the basic coding element and consists of two subunits called prediction units (PUs) and transform units (TUs). The maximum size of a PU or TU is equal to the CU size. A prediction unit corresponds to a partition of a CU for prediction of pixel values. Various different partitions of a CU into PUs are possible, as shown in 606, including a partition into 4 square PUs, and two different partitions into 2 rectangular PUs. A transform unit is a basic unit for spatial transformation using DCT. A CU can be partitioned into TUs based on a quadtree representation 607.

[0092] Each slice is embedded in a network abstraction layer (NAL) unit. In addition, the coding parameters of the video sequence are stored in a dedicated NAL unit called a parameter set. In HEVC and H.264 / AVC, two parameter set NAL units are used: first, the sequence parameter set (SPS) NAL unit, which collects all parameters that remain unchanged during the entire video sequence. Generally, it handles the encoding profile, the size of the video frame, and other parameters. Second, the picture parameter set (PPS) NAL unit, which includes parameters that can be changed from one image (or frame) of the sequence to other images (or frames). HEVC also includes a video parameter set (VPS) NAL unit, which contains parameters describing the overall structure of the bitstream. VPS is a new type of parameter set defined in HEVC and is applied to all layers of the bitstream. A layer can contain multiple temporal sublayers, and all version 1 bitstreams are limited to a single layer. HEVC has certain hierarchical extensions for scalability and multi-view, and these extensions will allow multiple layers with a backward compatible version 1 base layer.

[0093] Figure 2 The data communication system in which one or more embodiments of the present invention can be implemented is illustrated. The data communication system includes a transmission device (in this case, a server 201) that is operable to transmit data packets of a data stream to a receiving device (in this case, a client terminal 202) via a data communication network 200. The data communication network 200 can be a wide area network (WAN) or a local area network (LAN). Such a network can be, for example, a wireless network (Wifi / 802.11a or b or g), an Ethernet network, an Internet network, or a hybrid network consisting of several different networks. In a specific embodiment of the present invention, the data communication system can be a digital television broadcasting system, in which the server 201 sends the same data content to multiple clients.

[0094] The data stream 204 provided by the server 201 may consist of multimedia data representing video and audio data. In some embodiments of the present invention, the audio and video data streams may be captured by the server 201 using a microphone and a camera, respectively. In some embodiments, the data streams may be stored on the server 201 or received by the server 201 from other data providers, or generated at the server 201. The server 201 is provided with an encoder for encoding the video and audio streams, in particular for providing a compressed bit stream for transmission, which is a more compact representation of the data presented as input to the encoder.

[0095] In order to obtain a better ratio of quality of transmitted data to the amount of transmitted data, the video data may be compressed, for example, according to the HEVC format or the H.264 / AVC format.

[0096] The client 202 receives the transmitted bitstream and decodes the reconstructed bitstream to reproduce the video image on the display device and reproduce the audio data using the speaker.

[0097] Despite Figure 2 A streaming scenario is considered in the example of , but it will be appreciated that in some embodiments of the invention, data communication between the encoder and the decoder may be carried out using, for example, a media storage device (such as an optical disc).

[0098] In one or more embodiments of the invention, a video image is transmitted along with data representing a compensating offset to be applied to reconstructed pixels of the image to provide filtered pixels in the final image.

[0099] Figure 3 A processing device 300 configured to implement at least one embodiment of the present invention is schematically illustrated. The processing device 300 may be a device such as a microcomputer, a workstation, or a light portable device. The device 300 includes a communication bus 313, which is connected to:

[0100] - a central processing unit 311 denoted as CPU, such as a microprocessor or the like;

[0101] - a read-only memory 306, denoted ROM, for storing a computer program implementing the invention;

[0102] - a random access memory 312 represented as a RAM for storing executable codes of the method of the embodiment of the present invention, and registers suitable for recording variables and parameters required for implementing the method for encoding a digital image sequence and / or the method for decoding a bit stream according to the embodiment of the present invention; and

[0103] A communication interface 302 connected to a communication network 303, via which digital data to be processed are transmitted or received.

[0104] Optionally, the device 300 may further include the following components:

[0105] - a data storage component 304, such as a hard disk, for storing a computer program for implementing the method of one or more embodiments of the present invention and data used or generated during the implementation of one or more embodiments of the present invention;

[0106] a disk drive 305 for a disk 306, which is suitable for reading data from the disk 306 or writing data to said disk;

[0107] - A screen 309 for displaying data and / or serving as a graphical interface for interaction with a user by means of a keyboard 310 or any other pointing means.

[0108] Device 300 may be connected to various peripheral devices such as digital camera 320 or microphone 308 , each of which is connected to an input / output card (not shown) to provide multimedia data to device 300 .

[0109] The communication bus provides communication and interoperability between the various elements included in or connected to the device 300. The representation of the bus is not limiting, and in particular, the central processing unit is operable to communicate instructions to any element of the device 300 directly or via other elements of the device 300.

[0110] The disk 306 may be replaced by any information medium, such as a rewritable or non-rewritable compact disk (CD-ROM), a ZIP disk or a memory card, and in general, by an information storage component that can be read by a microcomputer or a microprocessor, the disk 306 being integrated into the device or not, possibly removable and suitable for storing one or more programs whose execution enables the implementation of the method for encoding a digital image sequence and / or the method for decoding a bit stream according to the present invention.

[0111] The executable code may be stored in a read-only memory 306, on a hard disk 304 or on a removable digital medium such as, for example, a disk 306 as previously described, etc. According to a variant, the executable code of the program may be received via the interface 302 by means of a communication network 303 to be stored in one of the storage means of the device 300, such as a hard disk 304, etc., before being executed.

[0112] The central processing unit 311 is adapted to control and direct the execution of instructions or parts of software code of one or more programs according to the present invention, the execution of instructions stored in one of the above-mentioned storage means. At power-on, one or more programs stored in non-volatile memory (e.g., on hard disk 304 or in read-only memory 306) are transferred to random access memory 312 (which then contains the executable code of one or more programs) and registers for storing variables and parameters necessary to implement the present invention.

[0113] In this embodiment, the device is a programmable device that implements the invention using software. Alternatively, however, the invention may be implemented in hardware (for example in the form of an application specific integrated circuit or ASIC).

[0114] Figure 4 Block diagram illustrating an encoder according to at least one embodiment of the invention. The encoder is represented by connected modules, each module being adapted to implement, for example in the form of programming instructions executed by the CPU 311 of the apparatus 300, at least one corresponding step of a method for implementing at least one embodiment of encoding an image in a sequence of images according to one or more embodiments of the invention.

[0115] The encoder 400 receives as input a raw sequence 401 of digital images i0 to in. Each digital image is represented by a set of samples, called pixels.

[0116] After implementing the encoding process, the encoder 400 outputs a bitstream 410. The bitstream 410 includes a plurality of coding units or slices, each slice including a slice header for transmitting the encoded values ​​of the encoding parameters used for encoding the slice, and a slice body including the encoded video data.

[0117] Module 402 divides the input digital images i0 to in 401 into blocks of pixels. Blocks correspond to image parts and can have variable sizes (e.g., 4×4, 8×8, 16×16, 32×32, 64×64, 128×128 pixels, and several rectangular block sizes can also be considered). A coding mode is selected for each input block. Two families of coding modes are provided: coding modes based on spatial prediction coding (intra-frame prediction) and coding modes based on temporal prediction (inter-frame coding, merge, skip). Possible coding modes are tested.

[0118] Module 403 implements an intra prediction process in which a given block to be encoded is predicted by a predictor calculated from neighboring pixels of the block to be encoded. If intra coding is selected, the selected intra predictor and an indication of the difference between the given block and its predictor are encoded to provide a residual.

[0119] Temporal prediction is implemented by a motion estimation module 404 and a motion compensation module 405. First, a reference image from a reference image set 416 is selected, and a portion of the reference image (also referred to as a reference region or image portion) is selected by the motion estimation module 404, which is the region closest to a given block to be encoded. The motion compensation module 405 then uses the selected region to predict the block to be encoded. The difference between the selected reference region and the given block (also referred to as a residual block) is calculated by the motion compensation module 405. The selected reference region is indicated by a motion vector.

[0120] Thus, in both cases (spatial and temporal prediction), the residual is calculated by subtracting the prediction from the original block.

[0121] In intra prediction implemented by module 403, the prediction direction is encoded. In temporal prediction, at least one motion vector is encoded.

[0122] If inter prediction is selected, information about the motion vector and the residual block is encoded. To further reduce the bit rate, the motion is assumed to be homogeneous and the motion vector is encoded by the difference relative to the motion vector predictor. The motion vector predictor from a set of motion information predictors is obtained from the motion vector field 418 by the motion vector prediction and encoding module 417.

[0123] The encoder 400 further comprises a selection module 406 for selecting a coding mode by applying a coding cost criterion such as a rate-distortion criterion, etc. To further reduce redundancy, a transform such as DCT is applied to the residual block by a transform module 407, and then the obtained transform data is quantized by a quantization module 408 and entropy encoded by an entropy encoding module 409. Finally, the encoded residual block of the current block being encoded is inserted into a bitstream 410.

[0124] The encoder 400 also performs decoding of the encoded images to produce reference images for motion estimation of subsequent images. This enables the encoder and decoder receiving the bitstream to have the same reference frame. The inverse quantization module 411 performs inverse quantization of the quantized data, followed by an inverse transform by the inverse transform module 412. The inverse intra prediction module 413 uses the prediction information to determine which predictor to use for a given block, and the inverse motion compensation module 414 actually adds the residual obtained by module 412 to the reference area obtained from the reference image set 416.

[0125] Post filtering is then applied by module 415 to filter the reconstructed pixel frame. In an embodiment of the invention, a SAO loop filter is used, wherein a compensating offset is added to the pixel values ​​of the reconstructed pixels of the reconstructed image.

[0126] Figure 5 A block diagram of a decoder 60 according to an embodiment of the present invention is shown, which can be used to receive data from an encoder. The decoder is represented by connected modules, each module being suitable for implementing the corresponding steps of the method implemented by the decoder 60, for example in the form of programming instructions to be executed by the CPU 311 of the device 300.

[0127] The decoder 60 receives a bitstream 61 comprising coding units, each coding unit consisting of a header containing information about the encoded parameters and a body containing the encoded video data. Figure 4 As illustrated, for a given block, the coded video data is entropy encoded on a predetermined number of bits and the index of the motion vector predictor is encoded. The received coded video data is entropy decoded by module 62. The residual data is then dequantized by module 63, after which an inverse transform is applied by module 64 to obtain pixel values.

[0128] Mode data indicating an encoding mode is also entropy-decoded, and based on the mode, an encoding block of the image data is intra-type decoded or inter-type decoded.

[0129] In the case of intra mode, the intra inverse prediction module 65 determines the intra predictor based on the intra prediction mode specified in the bitstream.

[0130] If the mode is inter, motion prediction information is extracted from the bitstream to find the reference region used by the encoder. The motion prediction information consists of a reference frame index and a motion vector residual. The motion vector predictor is added to the motion vector residual to obtain the motion vector by the motion vector decoding module 70.

[0131] The motion vector decoding module 70 applies motion vector decoding to each current block encoded by motion prediction. Once the index of the motion vector predictor for the current block has been obtained, the actual value of the motion vector associated with the current block can be decoded and used to apply inverse motion compensation by module 66. The reference image portion indicated by the decoded motion vector is extracted from the reference image 68 to apply inverse motion compensation 66. The motion vector field data 71 is updated with the decoded motion vector for inverse prediction of subsequently decoded motion vectors.

[0132] Finally, a decoded block is obtained. Post filtering is applied by a post filtering module 67. The decoder 60 finally provides a decoded video signal 69.

[0133] CABAC

[0134] HEVC uses several types of entropy coding, such as context-based adaptive binary arithmetic coding (CABAC), Golomb-Rice Code, or a simple binary representation called fixed-length coding. Most of the time, a binary coding process is performed to represent different syntactic elements. This binary coding process is also very specific and depends on the different syntactic elements. Arithmetic coding represents syntactic elements according to their current probabilities. CABAC is an extension of arithmetic coding that separates the probabilities of syntactic elements according to a "context" defined by a context variable. This corresponds to conditional probabilities. The context variable can be obtained from the decoded top left block (such as Figure 6b A2 in, as described in more detail below) and the upper left block ( Figure 6b The value of the current syntax of B3) in is derived.

[0135] Interframe Coding

[0136] HEVC uses 3 different inter-frame modes: inter-frame mode, merge mode, and merge skip mode. The main difference between these modes is the data signaled in the bitstream. For motion vector coding, the current HEVC standard includes a competitive-based scheme for motion vector prediction, which does not exist in earlier versions of the standard. This means that several candidates are competing on the encoder side with a rate-distortion criterion to find the best motion vector predictor or best motion information for inter-frame or merge mode, respectively. The index corresponding to the best predictor or best candidate for motion information is inserted into the bitstream. The decoder can derive the same set of predictors or candidates based on the decoded index and use the best predictor or candidate. In the picture content extension of HEVC, a new coding tool called intra-frame block copy (IBC) is signaled as any of the three inter-frame modes, and the difference between IBC and the equivalent inter-frame mode is made by checking whether the reference frame is the current frame. This can be achieved, for example, by checking the reference index of list L0, and if it is the last frame in the list, it is inferred to be an intra-frame block copy. Another approach is to compare the picture order counts of the current frame and the reference frame: if they are equal, it is an intra block copy.

[0137] The design of the derivation of predictors and candidates is important in achieving optimal coding efficiency without a disproportionate impact on complexity. In HEVC, two motion vector derivations are used: one for inter mode (Advanced Motion Vector Prediction (AMVP)) and one for merge mode (Merge Derivation Process). These processes are described below.

[0138] Figure 6a and 6b Illustrate spatial and temporal blocks that can be used to generate motion vector predictors in Advanced Motion Vector Prediction (AMVP) and merge modes of HEVC encoding and decoding systems, and Figure 7 Simplified steps of the process of AMVP predictor subset derivation are shown.

[0139] like Figure 6a As represented in the figure, two predictors (i.e., two spatial motion vectors of the AMVP mode) are selected in the top block (indicated by the letter "B") and the left block (indicated by the letter "A") including the top corner block (block B2) and the left corner block (block A0), and one predictor is selected in the bottom right block (H) and the center block (Center) of the collocated block.

[0140] Table 1 below summarizes the Figure 6a and 6b The nomenclature used when referring to a block relative to the current block shown in . This nomenclature is used as a shorthand, but it should be understood that other labeling systems may be used, particularly in future versions of the standard.

[0141] Block Tags Description of the relative position of adjacent blocks A0 "Left Corner" - The diagonal of the current block is downward and to the left A1 "Left" or "BottomLeft" - the left side of the bottom of the current block A2 "Top Left" - left side of the top of the current block B0 "Above Right" - the current block's diagonal upward and to the right B1 "Above" - ​​above the top right of the current block B2 "Above Left" - the current block's diagonal upwards and to the left B3 "Top" - Above the top left of the current block H Bottom right side of the concatenated block in the reference frame Center Blocks within collocated blocks in the reference frame

[0142] Table 1

[0143] It should be noted that the "current block" can be of variable size, such as 4×4, 16×16, 32×32, 64×64, 128×128, or any size in between. The size of the block is preferably a factor of 2 (i.e., 2^n×2^m, where n and m are positive integers), as this results in a more efficient use of bits when binary encoding is used. The current block does not need to be square, but this is generally a preferred embodiment for coding complexity.

[0144] Go to Figure 7 , the first step aims at selecting the first spatial predictor (Cand 1, 706) among the bottom left blocks A0 and A1 (spatial positions are illustrated in FIG6 ). To this end, these blocks are selected one after another in a given order (700, 702), and for each selected block, the following conditions are evaluated in a given order (704), and the first block that satisfies the conditions is set as the predictor:

[0145] - Motion vectors from the same reference list and the same reference picture;

[0146] - Motion vectors from other reference lists and the same reference picture;

[0147] - scaled motion vectors from the same reference list and a different reference picture; or

[0148] - Scaled motion vectors from other reference lists and different reference pictures.

[0149] If no value is found, the left predictor is considered unavailable. In this case, it indicates that the relevant blocks are intra-coded or that these blocks do not exist.

[0150] The following steps aim to select the second spatial predictor (Cand 2, 716) from the upper right block B0, the upper block B1 and the upper left block B2 (spatial positions are illustrated in FIG6 ). To this end, these blocks are selected one by one in a given order (708, 710, 712), and for each selected block, the above conditions are evaluated in a given order (714), and the first block that satisfies the above conditions is set as the predictor.

[0151] Likewise, if no value is found, the top predictor is considered unavailable. In this case, it indicates that the relevant blocks are intra-coded or that these blocks do not exist.

[0152] In the next step (718), if both predictors are available, the two predictors are compared to each other to remove one of the two predictors if the two predictors are equal (i.e., same motion vector value, same reference list, same reference index and same direction type). If only one spatial predictor is available, the algorithm looks for a temporal predictor in the next step.

[0153] The temporal motion predictor (Cand 3, 726) is derived as follows: The bottom right (H, 720) position of the collocated block in the previous frame is first considered in the availability check module 722. If not present or if the motion vector predictor is unavailable, the center (Centre, 724) of the collocated block is selected for checking. These temporal positions (Centre and H) are depicted in Figure 6. In any case, scaling 723 is applied to these candidates to match the temporal distance between the current frame and the first frame in the reference list.

[0154] The motion predictor value is then added to the set of predictors. Next, the number of predictors (Nb_Cand) is compared to the maximum number of predictors (Max_Cand) (728). As mentioned above, in the current version of the HEVC standard, the maximum number of predictors (Max_Cand) that the AMVP derivation process needs to generate motion vector predictors is two.

[0155] If the maximum number is reached, a final list or set of AMVP predictors is constructed (732). Otherwise, a zero predictor is added to the list (730). A zero predictor is a motion vector equal to (0, 0).

[0156] like Figure 7 As illustrated in , a final list or set ( 732 ) of AMVP predictors is constructed from a subset of spatial motion predictors ( 700 to 712 ) and from a subset of temporal motion predictors ( 720 , 724 ).

[0157] As mentioned above, the motion prediction sub-candidates for merge mode or merge skip mode represent all the required motion information: direction, list, reference frame index and motion vector. A list with indices of several candidates is generated by the merge derivation process. In the current HEVC design, the maximum number of candidates for these two merge modes is equal to five (4 spatial candidates and 1 temporal candidate).

[0158] Figure 8 800 to 808. Figure 3The spatial positions are depicted with reference numerals A1, B1, B0, A0 and B2 in FIG. In the next step, the availability of spatial motion vectors is checked and up to five motion vectors are selected (810). The predictor is considered available if it exists and the block is not intra-coded. Therefore, the motion vectors corresponding to the five blocks are selected as candidates according to the following conditions:

[0159] If the "left" A1 motion vector (800) is available (810), i.e., if it exists and if the block is not intra-coded, the motion vector of the "left" block is selected and used as the first candidate in the candidate list (814);

[0160] If the "above" B1 motion vector (802) is available (810), the candidate "above" block motion vector is compared to the "left" A1 motion vector (if present) (812). If the B1 motion vector is equal to the A1 motion vector, B1 is not added to the list of spatial candidates (814). Conversely, if the B1 motion vector is not equal to the A1 motion vector, B1 is added to the list of spatial candidates (814);

[0161] If the "above right" B0 motion vector (804) is available (810), the "above right" motion vector is compared to the B1 motion vector (812). If the B0 motion vector is equal to the B1 motion vector, the B0 motion vector is not added to the list of spatial candidates (814). Conversely, if the B0 motion vector is not equal to the B1 motion vector, the B0 motion vector is added to the list of spatial candidates (814);

[0162] If the "below left" A0 motion vector (806) is available (810), the "below left" motion vector is compared to the A1 motion vector (812). If the A0 motion vector is equal to the A1 motion vector, the A0 motion vector is not added to the list of spatial candidates (814). Conversely, if the A0 motion vector is not equal to the A1 motion vector, the A0 motion vector is added to the list of spatial candidates (814); and

[0163] If the list of spatial candidates does not contain four candidates, the availability of the "upper left" B2 motion vector (808) is checked (810). If available, it is compared to the A1 motion vector and the B1 motion vector. If the B2 motion vector is equal to the A1 motion vector or the B1 motion vector, the B2 motion vector is not added to the list of spatial candidates (814). Conversely, if the B2 motion vector is not equal to the A1 motion vector or the B1 motion vector, the B2 motion vector is added to the list of spatial candidates (814).

[0164] At the end of this phase, the list of spatial candidates includes up to four candidates.

[0165] For the temporal candidate, two positions can be used: the bottom right position of the collocated block (816, denoted as H in FIG6) and the center of the collocated block (818). These positions are depicted in FIG6.

[0166] Regarding the AMVP motion vector derivation process, the first step aims to check the availability of the block at the H position (820). Next, if the block is not available, the availability of the block at the central position is checked (820). If at least one motion vector for these positions is available, the temporal motion vector can be scaled relative to the reference frame with index 0 for both lists L0 and L1 as needed (822) to create a temporal candidate (824) that is added to the list of merged motion vector predictor candidates. The temporal candidate is placed after the spatial candidate in the list. Lists L0 and L1 are 2 reference frame lists containing zero, one or more reference frames.

[0167] If the number of candidates (Nb_Cand) is strictly less than the maximum number of candidates (Max_Cand, which is signaled in the bitstream slice header and is equal to five in the current HEVC design) (826), and if the current frame is of type B, a combined candidate is generated (828). The combined candidate is generated based on the available candidates of the list of merged motion vector predictor candidates. It mainly consists in combining the motion vector of one candidate of list L0 with the motion vector of one candidate of list L1.

[0168] If the number of candidates (Nb_Cand) remains strictly less than the maximum number of candidates (Max_Cand) (830), zero motion candidates are generated (832) until the number of candidates of the merged motion vector predictor candidate list reaches the maximum number of candidates.

[0169] At the end of this process, a list or set of merged motion vector predictor candidates is constructed (834). Figure 8 As illustrated in , a list or set of merged motion vector predictor candidates is constructed ( 834 ) from a subset of spatial candidates ( 800 to 808 ) and from a subset of temporal candidates ( 816 , 818 ).

[0170] Alternative Temporal Motion Vector Prediction (ATMVP)

[0171] Alternative Temporal Motion Vector Prediction (ATMVP) is a special case of motion compensation. Instead of considering only one motion information for the current block from the temporal reference frame, individual motion information for each collocated block is considered. Fig. 9As depicted in , the temporal motion vector prediction utilizes the relative motion information of each sub-block to give a partition of the current block.

[0172] In the current VTM reference software, ATMVP is signaled as a merge candidate inserted into the merge candidate list. When ATMVP is enabled at the SPS level, the maximum number of merge candidates increases by one. Thus, 6 candidates are considered instead of 5 candidates when this mode is disabled.

[0173] In addition, when the prediction is enabled at the SPS level, all bits of the merge index are context coded by CABAC. When in HEVC or when ATMVP is not enabled at the SPS level, only the first bit is context coded and the remaining bits are context bypass coded. Fig.10 (a) illustrates the encoding of the merge index for HEVC or when ATMVP is not enabled at the SPS level. This corresponds to unary maximum encoding. In addition, the first bit is CABAC encoded and the other bits are bypass CABAC encoded.

[0174] Fig.10 (b) illustrates the encoding of the merged index when ATMVP is enabled at the SPS level. In addition, all bits are CABAC encoded (from bit 1 to bit 5). It should be noted that each index has its own context - in other words, their probabilities are separate.

[0175] Affine Mode

[0176] In HEVC, only the translational motion model is applied to motion compensated prediction (MCP). In the real world, there are many kinds of motions, such as zoom-in / zoom-out, rotation, perspective motion, and other irregular motions.

[0177] In JEM, a simplified affine transform motion compensated prediction is applied and the general principles of the affine mode are described below based on an extract from the document JVET-G1001 presented at the JVET conference in Torino, July 13-21, 2017. This document is incorporated herein by reference in its entirety to the extent that it describes other algorithms used in JEM.

[0178] like Fig.11 As shown in (a), the affine motion field of a block is described by two control point motion vectors.

[0179] The motion vector field (MVF) of a block is described by the following equation:

[0180]

[0181] Where (v 0x ,v 0y) is the motion vector of the control point at the top left corner, and (v 1x ,v 1y ) is the motion vector of the control point of the top right corner.

[0182] To further simplify motion compensated prediction, sub-block based affine transform prediction is applied. The sub-block size M×N is derived as in Equation 2, where MvPre is the motion vector fractional precision (1 / 16 in JEM), (v 2x ,v 2y ) is the motion vector of the bottom left control point calculated according to Equation 1.

[0183]

[0184] After being derived by Equation 2, M and N may be adjusted downward, if necessary, to be divisors of w and h, respectively.

[0185] In order to derive the motion vector of each M×N sub-block, the following equation is used to calculate: Figure 6a The motion vector of the central sample of each sub-block shown in is calculated and rounded to 1 / 16 fractional precision. A motion compensated interpolation filter is then applied to generate a prediction of each sub-block with the derived motion vector.

[0186] Affine mode is a motion compensation mode that is used as an inter-frame mode (AMVP, Merge, Merge Skip). Its principle is to generate a motion information for each pixel based on 2 or 3 adjacent motion information. In the current VTM reference software, Fig.11 As depicted in (a), the affine mode derives one motion information for each 4×4 block. This mode is available for AMVP and two merge modes are enabled using a flag. The flag is CABAC coded. In an embodiment, the context depends on the left block ( Figure 6b Position A2) and the upper left block ( Figure 6b The sum of the affine markers of position B3).

[0187] Therefore, for an affine flag, three context variables (0, 1, or 2) can be given by the following formula in JEM:

[0188] Ctx=IsAffine(A2)+IsAffine(B3)

[0189] Where IsAffine(block) is a function that returns 0 if the block is not an affine block and returns 1 if the block is affine.

[0190] Affine merge candidate extraction

[0191] In JEM, the affine merge mode (merge or merge-skip) is derived based on the first neighboring block that is affine in the blocks at positions A1, B1, B0, A0, B2. These positions are in Figure 6a and 6b However, how to derive the affine parameters is not fully defined, and the present invention aims to improve at least this aspect.

[0192] Signaling Affine Merges

[0193] Fig.12 1 is a flowchart of a partial decoding process of some syntax elements related to the coding mode. In this figure, the skip flag (1201), prediction mode (1211), merge flag (1203), merge index (1208) and affine flag (1207) can be decoded.

[0194] For all CUs in the inter slice, decode the skip flag (1201). If the CU is not skip (1202), decode the pred mode (prediction mode) (1211). This syntax element indicates whether the current CU is inter mode or intra mode. Note that if the CU is skip (1202), its current mode is inter mode. If the CU is inter mode (1212), the CU is encoded in AMVP or merge mode. If the CU is inter (1212), decode the merge flag (1203). If the CU is merge (1204) or if the CU is skip (1202), verify (1205) whether the affine flag (1206) needs to be decoded. If the current CU is a 2N×2N CU, decode this flag, which means that the height and width of the CU in the current VVC should be equal. In addition, at least one neighboring CU A1 or B1 or B0 or ​​A0 or B2 must be encoded using affine mode (merge or AMVP). Finally, the current CU should not be a 4×4 CU, but CU 4×4 is disabled by default in the VTM reference software. If the condition (1205) is false, it is determined that the current CU is encoded in classic merge mode or merge skip mode, and the merge index is decoded (1208). If the affine flag (1206) is set equal to 1 (1207), the CU is a merge affine CU or a merge skip affine CU, and the merge index (1208) does not need to be decoded. Otherwise, the current CU is a classic (basic) merge or merge skip CU, and the merge index candidate is decoded (1208).

[0195] In this specification, "signaling" may refer to inserting into or extracting from a bitstream one or more syntax elements representing other information of enabling or disabling a mode.

[0196] Merge candidate export

[0197] Fig.13 is a flowchart illustrating the derivation of merge candidates. Figure 8 The merge list export of HEVC represented in is built on the basis of . The main changes compared to HEVC are the addition of ATMVP candidates (1319, 1321, 1323), the full redundancy check of the candidates (1320, 1325), and the new order of the candidates. The ATMVP prediction is set to a specific candidate because it represents some motion information of the current CU. The value of the first sub-block (top left) is compared with the temporal candidate, and if they are equal, the temporal candidate is not added to the merged list (1320). The ATMVP candidate is not compared with other spatial candidates. Instead, the temporal candidate is compared with each spatial candidate already in the list (1325) and is not added to the merge candidate list if it is a redundant candidate.

[0198] When a spatial candidate is added to the list, it is compared with the other spatial candidates in the list (1310), which is not the case in the final version of HEVC.

[0199] In the current VTM version, the list of merge candidates is set to the following order, as it has been determined to provide the best results under encoding test conditions:

[0200] A1

[0201] B1

[0202] ·B0

[0203] ·A0

[0204] ATMVP

[0205] B2

[0206] ·time

[0207] ·combination

[0208] ·Zero_MV

[0209] It is important to note that the spatial candidate B2 is placed after the ATMVP candidate.

[0210] Additionally, when ATMVP is enabled at the slice level, the maximum number in the candidate list is 6 instead of 5.

[0211] It is an object of the present invention to signal an affine mode in a part of a bitstream in an efficient manner taking into account coding efficiency and complexity.It is a further object of the present invention to signal an affine mode in a manner requiring a minimum amount of structural modifications to the existing video coding framework.

[0212] Now refer to Figures 13 to 21Exemplary embodiments of the present invention are described. It should be noted that, unless explicitly stated otherwise, the embodiments may be combined; for example, certain combinations of embodiments may improve coding efficiency at the expense of increased complexity, but this may be acceptable in certain use cases.

[0213] In summary, by modifying the syntax for signaling motion prediction sub-modes to take advantage of affine mode, when affine mode is more likely to provide improved motion compensation, coding efficiency can be improved with an acceptable increase in coding complexity.

[0214] First embodiment

[0215] In a first embodiment, an affine motion prediction mode may be signaled (e.g., enabled or disabled) for a portion of a bitstream of at least one inter mode. An inter prediction mode for the portion of the bitstream is determined, and the affine motion mode is signaled (enabled or disabled) according to the inter prediction mode used in the portion of the bitstream.

[0216] The advantage of this embodiment is that better coding efficiency is achieved by removing unused syntax. In addition, the encoder complexity is reduced by avoiding some inter-frame coding probabilities that do not need to be evaluated. Finally, at the decoder side, some CABAC-encoded affine flags do not need to be extracted from the bit stream, which improves the efficiency of the decoding process.

[0217] In the example of the first embodiment, the skip mode is not enabled for the affine mode. This means that when the CU is a skip CU (based on the state or presence of the skip flag in the data stream), the affine flag does not need to be extracted from the bitstream. Fig.14 (Its sharing and Fig.12 The same structure as in the previous embodiment and the corresponding description applies here) is used to illustrate this example. Fig.14 In , when the CU is skip (1402), the affine flag is not decoded (1406) and the condition in 1405 is not evaluated. If the CU is skip, the merge index is decoded (1406).

[0218] The advantage of this example is that the coding efficiency is improved for sequences with little motion, and the coding efficiency is not reduced for sequences with more motion. This is because the skip mode is usually used when there is little or no motion, and therefore the affine mode will be less likely to be suitable. As mentioned above, the complexity of the encoding and decoding process is also reduced.

[0219] In an additional example, affine merge skip mode may be enabled or disabled at a high level (e.g., at a slice, frame, sequence, or CTU level). This may be determined based on a high-level syntax flag. In such a case, affine merge skip may be disabled for sequences or frames with a small amount of motion, and enabled when the amount of motion increases. An advantage of this additional example is flexibility in using affine merge skip.

[0220] In one embodiment, the affine merge skip mode is never evaluated at the encoder side, so no bitstream contains the affine merge skip mode.An advantage is that a coding efficiency can be observed but is less than for the first embodiment.

[0221] Merge and skip modes may not be enabled for affine (or affine may be enabled only for AMVP). In a further example, merge and merge skip modes are not enabled for affine mode. This means that when a CU is skipped or merged, there is no need to extract the affine flag from the bitstream. Fig.14 In contrast, in this embodiment, modules 1405, 1406, and 1407 are removed.

[0222] The advantages of this example are similar to the example immediately above. The advantages are improved coding efficiency for sequences with little motion and the same coding efficiency for sequences with more motion. As described above, the complexity of the encoding and decoding processes is reduced.

[0223] A high-level syntax element signals that affine merging can be enabled. In yet another example, affine merge mode and merge skip mode can be enabled or disabled at a high level (such as slice, frame, sequence, or CTU level) using a flag. In this case, affine merging can be disabled for sequences or frames with a small amount of motion, and enabled when the amount of motion increases. An advantage of this additional embodiment is flexibility in using affine skip.

[0224] In an alternative example, one flag is signaled for merge skip mode and one flag is signaled for merge mode.

[0225] In other examples, affine merge skip mode and merge mode are never evaluated on the encoder side. Therefore, no bitstream contains affine merge skip mode. The advantage is that coding efficiency can be observed.

[0226] Second embodiment

[0227] In a second embodiment, the affine mode for the current block is signaled to depend on the mode of one or more neighboring blocks. There may be correlations in how neighboring blocks are encoded, which can be exploited to improve coding efficiency. In particular, if one or more of some neighboring blocks use an affine mode, the affine mode is more likely to be suitable for the current mode.

[0228] In one embodiment, the number of candidates for affine merge or affine merge skip mode is reduced to only 2 candidates. The advantage of this embodiment is that the complexity on the decoder side is reduced because fewer affine flags are decoded for merge modes and fewer comparisons and memory buffer accesses are required for the affine merge check condition (1205). On the encoder side, fewer affine merge modes need to be evaluated.

[0229] In an example of the second embodiment, one neighboring block to the left and one neighboring block above the current block (e.g., candidates A1 and B1 depicted in FIG6 ) are evaluated to see if the affine flag needs to be decoded and used to derive the affine merge candidate. The advantage of using only these 2 positions for affine merging is a coding efficiency similar to keeping 5 candidates as in the current VTM implementation with reduced complexity. Fig.15 This embodiment is illustrated. In this figure, Fig.12 In comparison, module 1505 has been changed by checking only the A1 and B1 positions.

[0230] In another example of the second embodiment, only the evaluation Figure 6b The candidates A2 and B3 depicted in FIG. 1 are used to determine whether the affine flag needs to be decoded and used for the derivation of the affine merge candidate. This example has the same advantages as the previous example, but also reduces the "worst case" memory accesses compared to the previous example. In fact, for positions A2 and B3, these positions are the same as those used for the affine flag context derivation. In fact, for the affine flag, the context derivation depends on Figure 6b Therefore, if the affine flag needs to be decoded, the affine flag values ​​of A2 and B3 are already in the memory for context derivation of the current affine flag, and thus no further memory access is required.

[0231] Third embodiment

[0232] In a third embodiment, signaling the affine mode for the current block depends on a list of merging candidates corresponding to neighboring blocks of the current block.

[0233] In an example of the third embodiment, the list starts with blocks that have been used to determine context variables related to the block, because the affine flag values ​​of such blocks are already stored in memory for context derivation of the current affine flag, and therefore no further memory access is required. For example, possible affine merge candidates are A2 or B3 or B0 or ​​A0 or B2 in this order (rather than A1 or B1 or B0 or ​​A0 or B2), such as Figure 6bAs depicted in . This gives an improvement in coding efficiency compared to the current VTM. And also limits the amount of affine flags that need to be accessed for decoding of the affine flags for the worst case scenario. For the current version, it is 5 for module 1205 and 2 for the affine flag context derivation, and for this embodiment, it is only 5 because the affine flag values ​​for A2 and B3 are already in memory for the current affine flag context derivation, and therefore no further memory access is required.

[0234] A variation of the third embodiment relates to context alignment.Signaling the affine mode may include inserting a context encoding flag into the data stream, wherein a context variable of the flag is determined based on whether a neighboring block uses the affine mode.

[0235] In an alternative example of the third embodiment, the locations considered for context derivation of the affine markers are locations A1 and B1, rather than locations A2 and B3, as shown in FIG. Figure 6b As depicted in . In this case, the same advantages as in the previous example are obtained. This is another alignment between the context and the affine merge derivation. In this case, the context variable of the affine sign is obtained according to the following formula:

[0236] Ctx=IsAffine(A1)+IsAffine(B1)

[0237] Where Ctx is the context of the affine flag, and IsAffine is a function that returns 0 if the block is not an affine block and 1 if the block is affine. In this example, the affine flag values ​​of A1 and B1 are stored in memory for context derivation of the current affine flag, and therefore no further memory access is required.

[0238] In another alternative example, when the current block is in merge mode (both merge modes), the positions considered for context derivation of the affine flag are positions A1 and B1 instead of positions A2 and B3. An additional advantage over the previous example is better coding efficiency. In fact, for AMVP, there is no need to align the context derivation with the derivation of motion vectors, because for AMVP, affine blocks are not considered for this derivation.

[0239] Fourth embodiment

[0240] In a fourth embodiment, the affine mode is signaled according to whether the neighboring block is in merge mode or not.

[0241] In one example of the fourth embodiment, the candidates for affine merging (merge and skip) may be only affine AMVP candidates. Fig.17This embodiment is illustrated. An advantage of this embodiment is that the coding complexity is reduced without affecting the coding efficiency, because fewer affine markers need to be decoded.

[0242] In yet another example of the fourth embodiment, the candidate for affine merge (merge and skip) may be only an affine AMVP candidate or a merged affine candidate, but not an affine merge skip.

[0243] As with the previous example, this example has the advantage of reducing coding complexity without affecting coding efficiency, since fewer affine markers need to be decoded.

[0244] Fifth embodiment

[0245] In a fifth embodiment, signaling the affine mode comprises inserting the affine mode as a candidate motion predictor.

[0246] In one example of the fifth embodiment, affine merging (and merging skipping) is signaled as a merging candidate. In this case, Fig.12 Modules 1205, 1206 and 1207 are removed. In addition, in order not to affect the coding efficiency of the merge mode, the maximum possible number of merge candidates is incremented. For example, in the current VTM version, this value is set to be equal to 6, so if this embodiment is applied to the current version of VTM, the value will be 7.

[0247] The advantage is a simplified design of the syntax elements of the merge mode, since fewer syntax elements need to be decoded. In some cases, coding efficiencies may be observed.

[0248] Two possible ways of implementing this example will now be described:

[0249] The affine merge index always has the same position within the list regardless of the values ​​of the other merged MVs. The position of a candidate motion predictor indicates its likelihood of being selected, and therefore, if it is placed higher in the list (lower index value), it is more likely to be selected.

[0250] In the first example, the affine merge index always has the same position within the merge candidate list. This means that it has a fixed merge idx value. For example, the value can be set to be equal to 5 because the affine merge mode should represent complex motion that is not the most likely content. An additional advantage of this embodiment is that when parsing the current block (decoding / reading only syntax elements without decoding the data itself), the current block can be set to an affine block. Therefore, this value can be used to determine the CABAC context of the affine flag for AMVP. Therefore, the conditional probability should be improved for the affine flag and the coding efficiency should be better.

[0251] In a second example, an affine merge candidate is derived together with other merge candidates. In this example, a new affine merge candidate is added to the list of merge candidates. Fig.18 This example is shown below. Fig.13 By comparison, the affine candidate is the first affine neighboring block A1, B1, B0, A0, B2 (1917). Fig.12 If the same conditions as 1205 are valid (1927), a motion vector field generated using affine parameters is generated to obtain affine candidates (1929). Depending on the ATMVP, time, and use of affine candidates, the list of initial candidates can have 4, 5, 6, or 7 candidates.

[0252] The order between all these candidates is important because the more likely candidates should be processed first to ensure they are more likely to advance to motion vector candidates - the preferred order is as follows:

[0253] A1

[0254] B1

[0255] B0

[0256] A0

[0257] Affine Merge

[0258] ATMVP

[0259] B2

[0260] time

[0261] combination

[0262] Zero_MV

[0263] It is important to note that the affine merge is before the ATMVP mode but after the four primary neighbors. The advantage of placing the affine merge before the ATMVP candidate is increased coding efficiency compared to placing the affine merge after the ATMVP and the temporal predictor. This increase in coding efficiency depends on the GOP (Group of Pictures) structure and the quantization parameter (QP) settings for each picture in the GOP. But for the most commonly used GOP and QP settings, this order gives an increase in coding efficiency.

[0264] Another advantage of this solution is the concise design of merge and merge skip for both syntax and derivation. In addition, the affine candidate merge index can be changed according to the availability or value of the previous candidate in the list (redundancy check). Therefore, efficient signaling can be obtained.

[0265] In another example, the affine merge index may vary according to one or several conditions.

[0266] For example, the merge index or position within the list associated with an affine candidate changes depending on the criteria. The principle is to set a low value for the merge index corresponding to the affine merge when the affine merge has a high probability of being selected (and set a higher value when there is a low probability of being selected).

[0267] An advantage of this example is that the coding efficiency is increased, benefiting from the best adaptation of the merge index when it is most likely to be used.

[0268] The criteria used to select the position of the affine pattern in the list of merge candidates include:

[0269] a) If skip mode is enabled (state of the skip flag)

[0270] In one example of applying this criterion, if the current merge is merge skip mode, then the affine merge index has a value set equal to a high value (eg, 5), or is set after the spatial and temporal MVs.

[0271] As explained for the first embodiment, the likelihood of affine mode being selected for skip mode is low since there is unlikely to be any significant (or complex) motion.

[0272] b) Motion information of adjacent blocks

[0273] In one example of applying this criterion, if the motion information of a block on the left and a block above (e.g., block A1 and block B1) are similar or equal, the affine merge index has a value set equal to a high value, or is set after the spatial and temporal MVs.

[0274] When A1 has the same motion information as B1, the probability that the motion information is constant for the current block is high. Therefore, the probability that affine merging is selected is low.

[0275] c) ATMVP Candidates

[0276] In one example of applying this criterion, if the ATMVP candidate contains only one motion information, the affine merge index has a value set equal to a high value, or is set after the spatial and temporal MVs. In this case, there is no subdivision in the previous frame of the collocated block.

[0277] Therefore, the probability that the current block content is inside a non-constant motion is small, so it is preferred not to set the affine at a high position inside the merge list.

[0278] d) If the adjacent block uses affine mode

[0279] In one example of applying this criterion, if more than one neighboring block is affine, then the affine merge index has a value set equal to a low value, or is set ahead of the temporal predictor and away from the spatial predictor.

[0280] In an additional example applying this criterion, the affine merge index or affine position (idx) is set equal to idx=PN, where P is the lowest possible position of the affine merge index and N is the number of affine neighboring blocks. In one example, P is 5 and N is 5, and the neighboring blocks are A1, B1, B0, A0, B2. It should be noted that in this notation, the index value of the highest position is zero.

[0281] By this example, the affine merge index of the merge candidate position is set according to the probability related to the neighboring blocks of the merge candidate position. Therefore, it is at the first position when all neighboring positions are affine, and at the fourth position when only one neighboring block is affine.

[0282] It should be understood that the example value "5" can be set to 6 or 7 to obtain similar encoding efficiency.

[0283] It will also be appreciated that combinations of these criteria will be possible.

[0284] In other examples of the fifth embodiment, the affine mode is signaled based on the likelihood of determining the affine mode for the current block. In a specific example, the affine merge candidate is not added to the candidate list based on the criteria, or no index corresponds to the affine merge. The principle of this example is to disable the affine mode in the case where the affine mode is unlikely to be useful.

[0285] An advantage of this example is that the coding efficiency is increased, benefiting from the optimal use of the merge index bits.

[0286] Criteria used to determine the likelihood that an affine pattern is useful include:

[0287] a) The state of the skip flag

[0288] In one example of applying this criterion, when the current merge is the merge skip mode, no affine merge candidate is added. As described for the first embodiment, the possibility of selecting the affine mode for the skip mode is low.

[0289] b) Motion information of adjacent blocks

[0290] In one embodiment of applying this criterion, if the motion information of a block on the left and a block above (eg, blocks A1 and B1) are similar or equal, no affine merge candidate is added.

[0291] When a block on the left and a block above (eg, blocks A1 and B1) have the same motion information, there is a high probability that the motion information is constant for the current block. Therefore, affine merging should be disabled.

[0292] c) ATMVP Candidates

[0293] In one embodiment of applying this criterion, if the ATMVP candidate contains only one motion information, no affine merge candidate is added.

[0294] In such an example, the likelihood that the current block content is inside a non-constant motion is small, so it is preferred to disable affine higher inside the merge list.

[0295] It will be appreciated that combinations of these criteria will be possible.

[0296] Implementation of the embodiments of the present invention

[0297] Fig. 2013 is a schematic block diagram of a computing device 1300 for implementing one or more embodiments of the present invention. The computing device 1300 may be a device such as a microcomputer, a workstation, or a light portable device. The computing device 1300 includes a communication bus connected to the following: - a central processing unit (CPU) 1301, such as a microprocessor; - a random access memory (RAM) 1302 for storing executable code of the method of the embodiment of the present invention and registers suitable for recording variables and parameters required to implement the method for encoding or decoding at least a portion of an image according to the embodiment of the present invention, and its storage capacity can be expanded, for example, by an optional RAM connected to an expansion port; - a read-only memory (ROM) 1303 for storing a computer program for implementing an embodiment of the present invention; - a network interface (NET) 1304, which is usually connected to a communication network through which the digital data to be processed is transmitted or received, and the network interface (NET) 1304 may be a single network interface, or composed of a group of different network interfaces (for example, wired and wireless interfaces, or different kinds of wired or wireless interfaces), which are connected to the CPU when running. 1301, data packets are written to the network interface for transmission or read from the network interface for reception; - a user interface (UI) 1305, which can be used to receive input from a user or display information to a user; - a hard disk (HD) 1306, which can be provided as a mass storage device; - an input / output module (IO) 1307, which can be used to receive / send data from / to an external device (such as a video source or a display, etc.). The executable code can be stored in ROM 1303, on HD 1306, or on a removable digital medium such as a disk. According to a variant, the executable code of the program can be received via NET 1304 by means of a communication network to be stored in one of the storage components (such as HD 1306, etc.) of the communication device 1300 before being executed. CPU 1301 is suitable for controlling and directing the execution of instructions or parts of the software code of one or more programs according to an embodiment of the present invention, which instructions are stored in one of the aforementioned storage components. For example, after power-up, CPU 1301 is capable of executing those instructions relating to a software application from main RAM memory 1302 after loading instructions from program ROM 1303 or HD 1306. Such a software application, when executed by CPU 1301, causes the steps of the method according to the invention to be performed.

[0298] It should also be understood that according to other embodiments of the present invention, a decoder according to the above embodiments is provided in a user terminal such as a computer, a mobile phone (cellular phone), a tablet or any other type of device (e.g., a display device) capable of providing / displaying content to a user. According to yet another embodiment, an encoder according to the above embodiments is provided in an image capture device, which also includes a camera, a video camera or a webcam (e.g., a closed-circuit television or video surveillance camera) for capturing and providing content for encoding by the encoder. See below Fig.21 and 22 Two such examples are provided.

[0299] Fig.21 21 is a diagram illustrating a network camera system 2100 including a network camera 2102 and a client device 2104 .

[0300] The network camera 2102 includes an imaging unit 2106 , an encoding unit 2108 , a communication unit 2110 , and a control unit 2112 .

[0301] The network camera 2102 and the client device 2104 are connected to each other via the network 200 to be able to communicate with each other.

[0302] The camera unit 2106 includes a lens and an image sensor (e.g., a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS)), and captures an image of an object and generates image data based on the image. The image may be a still image or a video image. The camera unit may also include a zoom component and / or a pan component adapted for zooming or panning (optically or digitally), respectively.

[0303] The encoding unit 2108 encodes the image data by using the encoding method described in the first to fifth embodiments. The encoding unit 2108 uses at least one of the encoding methods described in the first to fifth embodiments. For other examples, the encoding unit 2108 may use a combination of the encoding methods described in the first to fifth embodiments.

[0304] The communication unit 2110 of the network camera 2102 transmits the encoded image data encoded by the encoding unit 2108 to the client device 2104 .

[0305] In addition, the communication unit 2110 receives commands from the client device 2104. The commands include commands for setting parameters for encoding by the encoding unit 2108.

[0306] The control unit 2112 controls other units in the network camera 2102 according to the command received by the communication unit 2110 .

[0307] The client device 2104 includes a communication unit 2114 , a decoding unit 2116 , and a control unit 2118 .

[0308] The communication unit 2118 of the client device 2104 transmits a command to the network camera 2102 .

[0309] Furthermore, the communication unit 2118 of the client device 2104 receives the encoded image data from the network camera 2102 .

[0310] The decoding unit 2116 decodes the encoded image data by using the decoding method described in any one of the first embodiment to the fifth embodiment. For other examples, the decoding unit 2116 may use a combination of the decoding methods described in the first embodiment to the fifth embodiment.

[0311] The control unit 2118 of the client device 2104 controls other units in the client device 2104 according to a user operation or command received by the communication unit 2114 .

[0312] The control unit 2118 of the client device 2104 controls the display device 2120 to display the image decoded by the decoding unit 2116 .

[0313] The control unit 2118 of the client device 2104 also controls the display device 2120 to display a GUI (Graphical User Interface) for specifying values ​​of parameters of the network camera 2102 (including parameters for encoding by the encoding unit 2108 ).

[0314] The control unit 2118 of the client device 2104 also controls other units in the client device 2104 according to user operation input to the GUI displayed by the display device 2120 .

[0315] The control unit 2118 of the client device 2104 controls the communication unit 2114 of the client device 2104 according to the user operation input to the GUI displayed by the display device 2120 to transmit a command for specifying the value of the parameter of the network camera 2102 to the network camera 2102 .

[0316] The network camera system 2100 can determine whether the camera 2102 utilized zooming or panning during recording of the video, and can use such information when encoding the video stream, since zooming or panning during capture can benefit from the use of affine modes, which are well suited for encoding complex motions such as zooming, rotating, and / or stretching (which can be a side effect of panning, particularly if the lens is a "fisheye" lens).

[0317] Fig. 22 is a diagram illustrating a smartphone 2200 .

[0318] The smartphone 2200 includes a communication unit 2202 , a decoding / encoding unit 2204 , a control unit 2206 , and a display unit 2208 .

[0319] The communication unit 2202 receives the encoded image data via the network.

[0320] The decoding unit 2204 decodes the encoded image data received by the communication unit 2202 .

[0321] The decoding unit 2204 decodes the encoded image data by using the decoding method described in the first to fifth embodiments. The decoding unit 2204 may use at least one of the decoding methods described in the first to fifth embodiments. For other examples, the encoding unit 2202 may use a combination of the decoding methods described in the first to fifth embodiments.

[0322] The control unit 2206 controls other units in the smartphone 2200 according to user operations or commands received by the communication unit 2202 .

[0323] For example, the control unit 2206 controls the display device 2208 to display the image decoded by the decoding unit 2204 .

[0324] The smartphone may also include an image recording device 2210 (eg, a digital camera and associated circuitry) for recording images or videos. Such recorded images or videos may be encoded by the decoding / encoding unit 2204 under the instruction of the control unit 2206.

[0325] The smartphone may also include a sensor 2212 adapted to sense the orientation of the mobile device. Such a sensor may include an accelerometer, a gyroscope, a compass, a global positioning (GPS) unit, or a similar position sensor. Such a sensor 2212 may determine whether the smartphone changes orientation, and such information may be used when encoding the video stream, since changes in orientation during capture may benefit from the use of an affine pattern, which is well suited for encoding complex motions such as rotations.

[0326] Replacements and modifications

[0327] It will be appreciated that the purpose of the present invention is to ensure that affine modes are utilized in the most efficient manner, and that some of the examples discussed above involve signaling the use of affine modes based on the perceived likelihood that the affine mode is useful. Further examples of the present invention may be applied to encoders when it is known that complex motion is being encoded, where affine transforms may be particularly effective. Examples of such situations include:

[0328] a) Camera zoom in / out

[0329] b) A portable camera (eg, a mobile phone) changes orientation (ie, rotational movement) during capture.

[0330] c) "Fisheye" lens camera pan (e.g. stretching / distortion of part of the image)

[0331] In this way, indications of complex motion may be raised during the recording process so that an affine pattern may be given a higher probability of being used for a slice, a sequence of frames, or indeed the video stream as a whole.

[0332] In a further example, affine mode may be given a higher likelihood of being used depending on the features or functionality of the device used to record the video. For example, a mobile device may be more likely to change orientation than, for example, a fixed security camera, so affine mode may be more suitable for encoding video from the former. Examples of features or functionality include: the presence / use of a zoom component, the presence / use of a position sensor, the presence / use of a pan component, whether the device is portable, or a user selection on the device.

[0333] Although the present invention has been described with reference to the embodiments, it should be understood that the present invention is not limited to the disclosed embodiments. It will be appreciated by those skilled in the art that various changes and modifications may be made without departing from the scope of the present invention as defined in the appended claims. All features disclosed in this specification (including any appended claims, abstracts and drawings) and / or all steps of any disclosed method or process may be combined in any combination, except for at least some mutually exclusive combinations of such features and / or steps. Unless otherwise expressly stated, each feature disclosed in this specification (including any appended claims, abstracts and drawings) may be replaced by alternative features for the same, equivalent or similar purposes. Therefore, unless otherwise expressly stated, each feature disclosed is only an example of a general series of equivalent or similar features.

[0334] It should also be understood that any results of the above-described comparison, determination, evaluation, selection, execution, proceeding, or consideration (e.g., selections made during an encoding or filtering process) may be indicated in data in the bitstream (e.g., a flag or data indicating the result) or may be determined / inferred from data in the bitstream, such that the indicated or determined / inferred result may be used for processing rather than actually being compared, determined, evaluated, selected, executed, performed, or considered, for example, during a decoding process.

[0335] In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. The mere fact that different features are recited in mutually different dependent claims does not indicate that a combination of these features cannot be used to advantage.

[0336] Reference signs appearing in the claims are by way of illustration only and shall have no limiting effect on the scope of the claims.

Claims

1. A method for decoding an image from a bit stream, the method comprising: determining a prediction mode for decoding a current block of the image from a plurality of prediction modes including an intra mode and an inter mode; as well as In a case where the inter mode is determined as the prediction mode for the current block, generating a list of motion predictor candidates, wherein a candidate for subblock affine prediction is located at a lower position than the temporal motion vector candidate in the list as a merge candidate, the subblock affine prediction deriving at least one motion vector for each subblock in the current block by using two or three motion vectors of blocks from the same frame as the frame of the current block, and The position of the candidate for sub-block affine prediction in the list is determined based on whether a block of the same frame as the current block uses sub-block affine prediction, and the block of the same frame corresponds to the block at position A1 on the bottom left of the current block.

2. The method according to claim 1, further comprising: selecting a sub-block merge mode with sub-block affine prediction for the current block, wherein selecting the sub-block merge mode with sub-block affine prediction comprises decoding a flag from the bitstream using CABAC decoding, and The context variable of the flag is determined based on whether a first block adjacent to the current block uses sub-block affine prediction and whether a second block adjacent to the current block uses sub-block affine prediction.

3. The method according to claim 2, wherein: The first block is located on the left side of the current block, and the second block is located above the current block.

4. The method according to claim 1, wherein: In a state where the current block has a size of 16×16, the number of subblocks in the current block is 16, and one motion vector for each subblock in the current block is derived by using the two or three motion vectors in the subblock affine prediction.

5. The method according to claim 1, wherein: The temporal motion vector candidate uses a motion vector in a block that is part of a different picture from the picture including the current block.

6. A method of encoding an image into a bitstream using motion prediction, the method comprising: determining a prediction mode for encoding a current block of the image from a plurality of prediction modes including an intra mode and an inter mode; as well as In a case where the inter mode is determined as the prediction mode for the current block, generating a list of motion predictor candidates, wherein a candidate for subblock affine prediction is located at a lower position than the temporal motion vector candidate in the list as a merge candidate, the subblock affine prediction deriving at least one motion vector for each subblock in the current block by using two or three motion vectors of blocks from the same frame as the frame of the current block, and The position of the candidate for sub-block affine prediction in the list is determined based on whether a block of the same frame as the current block uses sub-block affine prediction, and the block of the same frame corresponds to the block at position A1 on the bottom left of the current block.

7. The method according to claim 6, further comprising: selecting a sub-block merge mode with sub-block affine prediction for the current block, wherein selecting the sub-block merge mode with sub-block affine prediction comprises encoding a flag into the bitstream using CABAC encoding, and The context variable of the flag is determined based on whether a first block adjacent to the current block uses sub-block affine prediction and whether a second block adjacent to the current block uses sub-block affine prediction.

8. The method according to claim 7, wherein: The first block is located on the left side of the current block, and the second block is located above the current block.

9. The method according to claim 6, wherein: In a state where the current block has a size of 16×16, the number of subblocks in the current block is 16, and one motion vector for each subblock in the current block is derived by using the two or three motion vectors in the subblock affine prediction.

10. The method according to claim 6, wherein: The temporal motion vector candidate uses a motion vector in a block that is part of a different picture from the picture including the current block.

11. An encoder for encoding an image into a bitstream using motion prediction, the encoder comprising: means for determining a prediction mode for encoding a current block of said image from a plurality of prediction modes including an intra mode and an inter mode; as well as means for generating a list of motion prediction factor candidates if the inter mode is determined as the prediction mode for the current block, wherein a candidate for subblock affine prediction is located at a lower position than the temporal motion vector candidate in the list as a merge candidate, the subblock affine prediction deriving at least one motion vector for each subblock in the current block by using two or three motion vectors of blocks from the same frame as the frame of the current block, and The position of the candidate for sub-block affine prediction in the list is determined based on whether a block of the same frame as the current block uses sub-block affine prediction, and the block of the same frame corresponds to the block at position A1 on the bottom left of the current block.

12. A decoder for decoding an image from a bit stream, the decoder comprising: means for determining a prediction mode for decoding a current block of said image from a plurality of prediction modes including intra mode and inter mode; as well as means for generating a list of motion predictor candidates in a case where the inter mode is determined as the prediction mode for the current block, wherein a candidate for subblock affine prediction is located at a lower position than the temporal motion vector candidate in the list as a merge candidate, the subblock affine prediction deriving at least one motion vector for each subblock in the current block by using two or three motion vectors of blocks from the same frame as the frame of the current block, and The position of the candidate for sub-block affine prediction in the list is determined based on whether a block of the same frame as the current block uses sub-block affine prediction, and the block of the same frame corresponds to the block at position A1 on the bottom left of the current block.

13. A computer-readable storage medium storing a program for causing a computer to execute a method for decoding an image from a bit stream, the method comprising: determining a prediction mode for decoding a current block of the image from a plurality of prediction modes including an intra mode and an inter mode; as well as In a case where the inter mode is determined as the prediction mode for the current block, generating a list of motion predictor candidates, wherein a candidate for subblock affine prediction is located at a lower position than the temporal motion vector candidate in the list as a merge candidate, the subblock affine prediction deriving at least one motion vector for each subblock in the current block by using two or three motion vectors of blocks from the same frame as the frame of the current block, and The position of the candidate for sub-block affine prediction in the list is determined based on whether a block of the same frame as the current block uses sub-block affine prediction, and the block of the same frame corresponds to the block at position A1 on the bottom left of the current block.

14. A computer-readable storage medium storing a program for causing a computer to execute a method for encoding an image into a bitstream using motion prediction, the method comprising: determining a prediction mode for encoding a current block of the image from a plurality of prediction modes including an intra mode and an inter mode; as well as In a case where the inter mode is determined as the prediction mode for the current block, generating a list of motion predictor candidates, wherein a candidate for subblock affine prediction is located at a lower position than the temporal motion vector candidate in the list as a merge candidate, the subblock affine prediction deriving at least one motion vector for each subblock in the current block by using two or three motion vectors of blocks from the same frame as the frame of the current block, and The position of the candidate for sub-block affine prediction in the list is determined based on whether a block of the same frame as the current block uses sub-block affine prediction, and the block of the same frame corresponds to the block at position A1 on the bottom left of the current block.

Images