Efficient coding of global motion vectors

Abstract

This application relates to efficient encoding of global motion vectors. The decoder includes circuitry configured to receive a bitstream, extract residuals of control point motion vectors for the current frame from the bitstream, and combine the residuals of the control point motion vectors for the current frame with predictions of the control point motion vectors. Related apparatus, systems, techniques, and articles are also described.

Description

[0001] Information related to divisional application This case is a divisional application of the invention patent application filed on April 24, 2020, with application number 202080045886.5 and invention title "Efficient Encoding of Global Motion Vectors". Cross-references to related applications

[0002] This application claims priority to U.S. Provisional Patent Application Serial No. 62 / 838,521, filed April 25, 2019, entitled “Efficient Encoding of Global Motion Vectors,” the entire contents of which are incorporated herein by reference. Technical Field

[0003] This invention generally relates to the field of video compression. In particular, this invention is for the efficient encoding of global motion vectors. Background Technology

[0004] A video codec can include electronic circuitry or software that compresses or decompresses digital video. It can convert uncompressed video to a compressed format and vice versa. In the context of video compression, the device that compresses video (and / or performs some of its functions) is often referred to as an encoder, and the device that decompresses video (and / or performs some of its functions) is often referred to as a decoder.

[0005] The format of compressed data can conform to standard video compression specifications. Compression may be lossy because compressed video lacks some information present in the original video. One consequence of this is that the quality of the decompressed video may be lower than the original uncompressed video because there is not enough information to accurately reconstruct the original video.

[0006] There can be complex relationships between video quality, the amount of data used to represent the video (e.g., determined by bit rate), the complexity of encoding and decoding algorithms, sensitivity to data loss and errors, ease of editing, random access, end-to-end latency (e.g., delay), and so on.

[0007] Motion compensation can include a method that, given a reference frame (e.g., a previous frame and / or a future frame), predicts a video frame or a portion thereof by calculating the motion of objects in the camera and / or video. It can be used for encoding and decoding video data in video compression, such as encoding and decoding using the Moving Picture Experts Group (MPEG)-2 (also known as High-Level Video Coding (AVC) and H.264) standards. Motion compensation can describe an image based on the transformation from a reference image to the current image. The reference image can be earlier in time when compared to the current image, and can be future when compared to the current image. Compression efficiency can be improved when images can be accurately synthesized from previously transmitted and / or stored images. Summary of the Invention

[0008] In one aspect, the decoder includes circuitry configured to receive a bitstream, extract a residual of the control point motion vector of the current frame from the bitstream, and combine the residual of the control point motion vector of the current frame with a prediction of the control point motion vector.

[0009] In another approach, one method involves receiving a bitstream by a decoder. This method includes extracting a residual of the control point motion vectors of the current frame from the bitstream. The method further includes combining the residual of the control point motion vectors of the current frame with a prediction of the control point motion vectors.

[0010] Details of one or more variations of the subject matter described herein are set forth in the accompanying drawings and the description below. Other features and advantages of the subject matter described herein will be apparent from the specification, the accompanying drawings, and the claims. Attached Figure Description

[0012] To illustrate the invention, the accompanying drawings show aspects of one or more embodiments of the invention. However, it should be understood that the invention is not limited to the precise arrangements and means shown in the drawings, wherein:

[0013] Figure 1 This is a graph showing the motion vectors of an example frame with global and local motion;

[0014] Figure 2 Three example motion models that can be used for global motion are shown, including their index values ​​(0, 1, or 2);

[0015] Figure 3 This is a process flowchart based on some example implementations of the current topic;

[0016] Figure 4 This is a system block diagram of an example decoder based on some example implementations of the current topic;

[0017] Figure 5 This is a process flowchart based on some example implementations of the current topic;

[0018] Figure 6 This is a system block diagram of an example encoder based on some example implementations of the current topic; and

[0019] Figure 7 It is a block diagram of a computational system that can be used to implement any one or more methods disclosed herein and any one or more parts thereof.

[0020] The accompanying drawings are not necessarily drawn to scale and may use dashed lines, schematic diagrams, and partial views for illustration. In some cases, details that are not necessary for understanding the embodiments or that make other details difficult to perceive may have been omitted. The same reference numerals in different drawings denote the same elements. Detailed Implementation

[0021] Global motion in a video refers to motion that occurs throughout the entire frame. Global motion may be caused by camera movement; for example, but not limited to, camera panning and zooming can produce motion within a frame that typically affects the entire frame. Motion present in parts of the video can be referred to as local motion. Local motion can be caused by moving objects in the scene, for example, but not limited to objects moving from left to right in the scene. Video may contain a combination of local and global motion. Some implementations of the current topic can provide efficient methods for passing global motion to the decoder, as well as improving compression efficiency by using global motion vectors.

[0022] Figure 1 This is a diagram illustrating an exemplary embodiment of motion vectors for an exemplary frame 100 having global and local motion. Frame 100 may include a plurality of pixel blocks shown as squares, and their associated motion vectors shown as arrows. Squares (e.g., pixel blocks) with arrows pointing upwards and to the left may indicate blocks with motion that can be considered global motion, and squares with arrows pointing in other directions (indicated by 104) indicate blocks with local motion. Figure 1 In the example shown, many blocks share the same global motion. Sending global motion signals, such as a Picture Parameter Set (PPS) or Sequence Parameter Set (SPS), in the header, and using signaled global motion, can reduce the amount of motion vector information required for a block and can lead to improved prediction. Although the examples described below for illustrative purposes determine and / or apply global or local motion vectors at the block level, global motion vectors can be determined and / or applied to any region of a frame and / or image, including regions consisting of multiple blocks, regions bounded by any geometry, such as, but not limited to, regions defined by geometric and / or exponential coding, where one or more lines and / or curves enclosing the boundary can be angled and / or curved, and / or the entire frame and / or image. Although signaling is described herein as being performed at the frame level and / or within the frame's header and / or parameter set, signaling can alternatively or additionally be performed at the sub-image level, where a sub-image can include any region of a frame and / or image as described above.

[0023] As an example, and still for reference Figure 1Simple translational motion can be described using motion vectors (MVs), which have two components, MVx and MVy, describing the displacement of blocks and / or pixels in the current frame. More complex motions, such as rotation, scaling, and warping, can be described using affine motion vectors, where "affine motion vector" as used in this disclosure is a vector describing the uniform displacement of a set of pixels or points represented in a video image and / or image, for example, showing a set of pixels that move in a video view during motion without changing their appearance shape. Some methods of video encoding and / or decoding can use four-parameter or six-parameter affine models for motion compensation in inter-image coding.

[0024] For example, six-parameter affine motion can be described as:

[0025] x' = ax + by + c

[0026] y' = dx + ey + f

[0027] Four-parameter affine motion can be described as:

[0028] x' = ax + by + c

[0029] y' = -bx + ay + f

[0030] Where (x, y) and (x', y') are the pixel positions in the current image and the reference image, respectively; a, b, c, d, e, and f are the parameters of the affine motion model.

[0031] Continue to refer to Figure 1 The parameters describing affine motion can be sent to the decoder via signals so that affine motion compensation can be applied at the decoder. In some methods, motion parameters can be explicitly sent via signals, or translational control point motion vectors (CPMVs) can be sent via signals, and then the affine motion parameters can be derived from the translational motion vectors. Affine motion parameters for a four-parameter affine motion model can be derived using two control point motion vectors (CPMVs), and parameters for a six-parameter motion model can be obtained using three control point translational motion vectors (CPMVs). Sending affine motion parameters via signals using control point motion vectors allows for the use of efficient motion vector encoding methods to send affine motion parameters via signals.

[0032] In some implementations, and still referencing Figure 1Global motion signaling can be included in the header, such as PPS or SPS. Global motion may vary from image to image. Motion vectors signaled in the image header can describe motion relative to previously decoded frames. In some implementations, global motion can be translational or affine. The motion model used (e.g., the number of parameters, whether the model is affine, translational, or other) can also be signaled in the image header. Figure 2 Three example motion models 200 that can be used for global motion are shown, including their index values ​​(0, 1, or 2).

[0033] Continue to refer to Figure 2 PPSs can be used to send parameters via signals, which can be changed between images in a sequence. Parameters that remain the same across the image sequence can be sent using signals from a sequence parameter set, reducing the size of the PPS and lowering the video bitrate. Table 1 shows an example set of image parameter sets (PPS):

[0034]

[0035]

[0036]

[0037]

[0038]

[0039] Still referencing Figure 2 Additional fields can be added to the PPS to signal global motion. In the case of global motion, the presence of global motion parameters in the image sequence can be signaled in the SPS, and the PPS references the SPS by the SPS ID. In some decoding methods, the SPS can be modified to add a field to signal the presence of global motion parameters in the SPS. For example, a single bit can be added to the SPS. If the global_motion_present bit is 1, global motion-related parameters can be expected in the PPS; if the global_motion_present bit is 0, the field related to global motion parameters may not exist in the PPS. For example, the PPS in Table 1 can be extended to include a global_motion_present field, as shown in Table 2.

[0040]

[0041] Similarly, PPS can include the pps_global_motion_parameters field for a single frame, as shown in Table 3:

[0042]

[0043] More specifically, the PPS may include fields that use control point motion vectors to characterize global motion parameters, as shown in Table 4, for example:

[0044]

[0045] As a further non-limiting example, Table 5 below can represent exemplary SPS:

[0046]

[0047]

[0048]

[0049]

[0050]

[0051]

[0052]

[0053]

[0054] As mentioned above, the SPS table can be expanded to include global motion rendering indicators as shown in Table 6:

[0055]

[0056] Additional fields may be incorporated into the SPS to reflect the further indicators described in this disclosure.

[0057] In one embodiment, still referencing Figure 2 In PPS and / or SPS, the `sps_affine_enabled_flag` specifies whether affine-based motion compensation can be used for inter-frame prediction. If `sps_affine_enabled_flag` equals 0, the syntax is constrained so that affine-based motion compensation is not used in later encoded video sequences (CLVS), and `inter_affine_flag` and `cu_affine_type_flag` may not exist in the CLVS coding unit syntax. Otherwise (`sps_affine_enabled_flag` equals 1), affine-based motion compensation can be used in CLVS.

[0058] Continue to refer to Figure 2The `sps_affine_type_flag` in PPS and / or SPS specifies whether motion compensation based on a six-parameter affine model can be used for inter-frame prediction. If `sps_affine_type_flag` equals 0, the syntax may be constrained so that motion compensation based on a six-parameter affine model is not used in CLVS, and `cu_affine_type_flag` may not exist in the coding unit syntax in CLVS. Otherwise (`sps_affine_type_flag` equals 1), motion compensation based on a six-parameter affine model can be used in CLVS. When it does not exist, the value of `sps_affine_type_flag` can be inferred to be equal to 0.

[0059] Still refer to Figure 2 Translational CPMVs can be sent in the PPS via signals. Control points can be predefined. For example, control point MV0 can be relative to the top left corner of the image, MV1 can be relative to the top right corner of the image, and MV3 can be relative to the bottom left corner of the image. Table 4 illustrates an example method for sending CPMV data via signals based on the motion model used.

[0060] In an exemplary embodiment, and still referring to Figure 2The array amvr_precision_idx, which can be transmitted via signals in coding units, coding trees, etc., can specify the resolution AmvrShift of the motion vector difference. As a non-limiting example, it can be defined as shown in Table 7 below. The array indices x0 and y0 can specify the position (x0, y0) of the top-left luminance sample point of the coded block under consideration relative to the top-left luminance sample point of the image; when amvr_precision_idx[x0][y0] does not exist, it can be inferred that it is equal to 0. When inter_affine_flag[x0][y0] equals 0, the variables MvdL0[x0][y0][0], MvdL0[x0][y0][1], MvdL1[x0][y0][0], and MvdL1[x0][y0][1] represent the modulation vector difference corresponding to the block under consideration. These values ​​can be modified by offsetting them by AmvrShift, for example, by using MvdL0[x0][y0][0] = MvdL0[x0][y0][0] < <AmvrShift;MvdL0[x0][y0][1]= MvdL0[x0][y0][1]<<AmvrShift;MvdL1[x0][y0][0]= MvdL1[x0][y0][0]<<AmvrShift;MvdL1[x0][y0][1]= MvdL1[x0][y0][1]<<AmvrShift。where inter_affine_flag[x0][y0] is equal to 1, the variables MvdCpL0[x0][y0][0][0], MvdCpL0[x0][y0][0][1], MvdCpL0[x0][y0][1][0], MvdCpL0[x0][y0][1][1], MvdCpL0[x0][y0][2][0] and MvdCpL0[x0][y0][2][1] can be modified by an offset, for example as follows: MvdCpL0[x0][y0][0][0]= MvdCpL0[x0][y0][0][0 ]<<AmvrShift; MvdCpL1[x0][y0][0][1]= MvdCpL1[x0][y0][0][1]<<AmvrShift; MvdCpL0[x0][y0][1][0]= MvdCpL0[x0][y0][1][0]<<AmvrShift; MvdCpL1[x0][y0][1][1]= MvdCpL1[x0][y0][1][1]<<AmvrShift; MvdCpL0[x0][y0][2][0]= MvdCpL0[x0][y0][2][0]<<AmvrShift; and MvdCpL1[x0][y0][2][1]= MvdCpL1[x0][y0][2][1]<<AmvrShift.

[0061]

[0062] Continuing reference Figure 2 , the global motion may be related to a previously encoded frame. When there is only one set of global motion parameters, the motion may be related to the frame presented immediately before the current frame.

[0063] Still referring Figure 2 . Some embodiments of the current subject matter may include predicting the global motion vector in the current frame based on the previously encoded global motion vectors of previous frames to improve compression.

[0064] Continuing reference Figure 2 , the current image encoded as an inter-frame image may use motion estimation to improve compression. The global motion vector of the current image may be signaled in the PPS. In some methods of video compression, when encoding the global motion parameters (e.g., control point motion vectors) in the current frame, the following information may already have been decoded and be available: 1) the global motion parameters from the previous frame; 2) the global motion parameters relative to the available reference images already encoded in the current frame in List0; and 3) the control point motion vectors among the encoded global motion parameters.

[0065] Still referencing Figure 2 The predicted motion vector (PMV) of the control point motion vector (CPMV) can be determined from the previously encoded motion vector, and the difference between the CPMV and PMV can be encoded to reduce bits and improve compression efficiency.

[0066] For example, continue to refer to Figure 2 CPMV0 i CPMV1 i and CPMV2 i These can be the motion vectors of the three control points of the frame 'i' to be encoded. In an exemplary method, this represents the CPMV0 of the vector components and / or vectors determined for frames preceding the current frame (including but not limited to the reference frame and / or the immediately preceding frame). i-1 , can be used for prediction or CPMVO i Furthermore, it can encode the differences between motion vectors. Specifically, it can encode the differences between the x and y components of a motion vector and their predictions.

[0067] Still referencing Figure 2 For CPMV(j,i), j in the range 0 <= j < 3 can be the number of motion vectors, and i in the range 0 <= i <= ref_pic_count can be the index of the reference image. ref_pic_count = 0 can refer to the current image. CPMV(j,1) can be used as a prediction of CPMV(j,0). The control points of global motion in a frame can be located at the corners of the frame, and the CPMVs at the corresponding corners of the frame are likely to be similar and thus provide a better prediction.

[0068] Continue to refer to Figure 2 As a non-limiting example, more complex motion vector predictions can be made using CPMV for all available reference images in the list. In this exemplary approach, CPMV(j,i) can be used as a prediction of CPMV(j,0). In this case, index i can also be encoded along with the motion vector difference.

[0069] Still referencing Figure 2 The previously encoded CPMV can be used as a prediction for encoding a subsequent CPMV, which can be the next CPMV. For example, CPMV(j, i-1) can be used as a prediction for CPMV(j, i). In this case, index i can also be encoded along with the motion vector difference. When only one set of CPMVs is encoded, CPMV0 can be a prediction for CPMV1 and CPMV2.

[0070] Further reference Figure 2 As a non-limiting example, Table 5 shows an example PPS with global motion parameters using control point motion vectors.

[0071]

[0072] Table 6 shows another example of PPS for global motion parameters with differential coding for one or more frames in the reference image list.

[0073]

[0074] Continue to refer to Figure 2 The following is example pseudocode for deriving the predicted CPMV according to the example implementation:

[0075] get_cpmv_pred(i, j){

[0076] if(i == 0){

[0077] if(j == 0){

[0078] pmv.x = 0;

[0079] pmv.y = 0;

[0080] }else{

[0081] pmv.x = mv0_x;

[0082] pmv.y = mv0_y;

[0083] }

[0084] }else{

[0085] pmv[j] = cpmv(j, i-1)

[0086] }

[0087] return pmv;

[0088] }

[0089] In many cases, still refer to Figure 2 Global motion may occur within a set of frames. Global motion may terminate when the scene changes or the camera stops moving. Therefore, global motion in consecutive frames may be similar. In some implementations, if global motion is used in the previous frame, the CPMV of the previous frame is likely to be a good predictor, thereby reducing motion vector differences and the bits required to encode motion vectors.

[0090] In some implementations, and continuing to refer to Figure 2Zero residuals can be implicitly encoded for global motion vector residuals by using global motion information from previous frames. For example, if global motion is enabled and if a global motion skip mode is enabled (e.g., the global_motion_skip flag), the current frame can use the global motion information from the previous frame as the global motion for the current frame. In some implementations, an index can be provided to a reference list (e.g., list()) indicating from which reference frame motion information can be taken.

[0091] Figure 3 This is a flowchart illustrating an example process 300 for predicting global motion vectors in the current frame based on previously encoded global motion vectors from previous frames. In step 305, the decoder receives the current block. The current block may be contained in the bitstream received by the decoder. The bitstream may include data found, for example, in the bitstream that is used as input to the decoder when data compression is employed. The bitstream may include information required for decoding the video. Receiving may include extracting and / or parsing the block and associated signaling information from the bitstream. In some implementations, the current block may include a coding tree unit (CTU), a coding unit (CU), or a prediction unit (PU).

[0092] In step 310, still refer to Figure 3 The residual of the control point motion vector of the current frame can be extracted from the bitstream. In step 315, the residual of the control point motion vector can be combined with the prediction of the control point motion vector of the current frame.

[0093] Figure 4 This is a system block diagram illustrating an example decoder 400, which is capable of decoding bitstream 428 by predicting global motion vectors in the current frame from previously encoded global motion vectors of previous frames. Decoder 400 may include an entropy decoder processor 404, an inverse quantization and inverse transform processor 408, a deblocking filter 412, a frame buffer 416, a motion compensation processor 420, and / or an intra-frame prediction processor 424.

[0094] During operation, still refer to Figure 4 The bitstream 428 can be received by the decoder 400 and input to the entropy decoder processor 404, which decodes a portion of the bitstream's entropy into quantization coefficients. These quantization coefficients can be provided to the inverse quantization and inverse transform processor 408, which performs inverse quantization and inverse transform to create a residual signal. This residual signal can be added to the output of the motion compensation processor 420 or the intra-prediction processor 424, depending on the processing mode. The outputs of the motion compensation processor 420 and the intra-prediction processor 424 can include block predictions based on previously decoded blocks. The sum of the predictions and residuals can be processed by the deblocking filter 630 and stored in the frame buffer 640.

[0095] Figure 5 This is a flowchart illustrating an exemplary embodiment of a process 200 for encoding video based on some aspects of the current subject. This process 200 can reduce encoding complexity while improving compression efficiency by predicting global motion vectors in the current frame from previously encoded global motion vectors of previous frames. In step 505, the video frame may undergo initial block segmentation, for example, using a tree-structured macroblock segmentation scheme, which may include segmenting the image frame into CTUs and CUs. In step 510, residuals of the control point motion vectors of the current frame may be determined. In step 515, the blocks may be encoded and included in the bitstream. For example, encoding may include utilizing inter-frame prediction and intra-frame prediction modes.

[0096] Figure 6 This is a system block diagram illustrating an example video encoder 600 based on some aspects of the current topic, which is capable of predicting global motion vectors in the current frame based on previously encoded global motion vectors from previous frames. The example video encoder 600 may receive input video 604, which may undergo initial segmentation or partitioning according to a processing scheme such as a tree-structured macroblock segmentation scheme (e.g., quadtree plus binary tree). Examples of tree-structured macroblock segmentation schemes may include segmenting image frames into large blocks called coding tree units (CTUs). In some implementations, each CTU may be further divided into multiple sub-blocks called coding units one or more times. The final result of such segmentation may include a set of sub-blocks called prediction units. Transform units may also be used.

[0097] The example video encoder 600 may include an intra-frame prediction processor 415, a motion estimation / compensation processor 612 (also known as an inter-frame prediction processor) capable of predicting global motion vectors in the current frame from previously encoded global motion vectors of previous frames based on some aspect of the current subject, a transform / quantization processor 616, an inverse quantization / inverse transform processor 620, a loop filter 624, a decoded image buffer 628, and / or an entropy coding processor 632. Bitstream parameters may be input to the entropy coding processor 632 to be included in the output bitstream 636.

[0098] During operation, still refer to Figure 6For each block of the input video 604, it can be determined whether the block is processed via intra-frame prediction or using motion estimation / compensation. The block can be provided to either the intra-frame prediction processor 608 or the motion estimation / compensation processor 612. If the block is to be processed via intra-frame prediction, the intra-frame prediction processor 608 can perform processing to output a predictor. If the block is to be processed via motion estimation / compensation, the motion estimation / compensation processor 612 can perform processing (if applicable) including predicting global motion vectors in the current frame based on previously encoded global motion vectors from previous frames, according to some aspects of the current topic.

[0099] Further reference Figure 6 The residual can be formed by subtracting the predicted value from the input video. The residual can be received by a transform / quantization processor 616, which can perform transform processing (e.g., Discrete Cosine Transform (DCT)) to produce coefficients that can be quantized. The quantized coefficients and any associated signaling information can be provided to an entropy coding processor 632 for entropy coding and included in the output bitstream 636. The entropy coding processor 632 can support the encoding of signaling information associated with encoding the current block. Furthermore, the quantized coefficients can be provided to an inverse quantization / inverse transform processor 620, which can reproduce pixels. These pixels can be combined with the predictor and processed by a loop filter 624. The output of the loop filter 624 can be stored in a decoded image buffer 628 for use by a motion estimation / compensation processor 612, which is capable of predicting global motion vectors in the current frame from previously encoded global motion vectors in previous frames based on some aspects of the current subject.

[0100] Still referencing Figure 6 Although some changes have been described in detail above, other modifications or additions are possible. For example, in some implementations, the current block may include any symmetric block (8×8, 16×16, 32×32, 64×64, 128×128, etc.) and any asymmetric block (8×4, 16×8, etc.).

[0101] In some implementations, and continuing to refer to Figure 6 This allows for the implementation of a quadtree plus binary decision tree (QTBT). In QTBT, at the coding tree unit level, the partitioning parameters of the QTBT can be dynamically derived to adapt to local characteristics without any transmission overhead. Subsequently, at the coding unit level, the joint decision tree classifier structure can eliminate unnecessary iterations and control the risk of mispredictions. In some implementations, the LTR frame block update mode can be used as an additional option available at each leaf node of the QTBT.

[0102] In some implementations, and still referencing Figure 6Additional syntax elements can be signaled at different levels of the bitstream. For example, the entire sequence can be enabled by including an encoded enable flag in the Sequence Parameter Set (SPS). Furthermore, CTU flags can be encoded at the Code Tree Unit (CTU) level.

[0103] It should be noted that any one or more aspects and embodiments described herein can be readily implemented using digital electronic circuits, integrated circuits, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), computer hardware, firmware, software, and / or combinations thereof, as will be apparent to those skilled in the art of computers, as implemented in one or more machines programmed according to the teachings of this specification (e.g., one or more computing devices used as user computing devices for electronic documents, one or more server devices such as document servers, etc.). These different aspects or features may include implementations in one or more computer programs and / or software executable and / or interpretable on a programmable system including at least one programmable processor, which may be dedicated or general-purpose, coupled to receive data and instructions from a storage system, at least one input device, and at least one output device, and to send data and instructions to the storage system, at least one input device, and at least one output device. It will be apparent to those skilled in the art of software that a skilled programmer can readily prepare appropriate software code based on the teachings of this disclosure. The aspects and implementations employing software and / or software modules discussed above may also include appropriate hardware for assisting in the implementation of machine-executable instructions for the software and / or software modules.

[0104] Such software can be a computer program product employing a machine-readable storage medium. A machine-readable storage medium can be any medium capable of storing and / or encoding sequences of instructions for execution by a machine (e.g., a computing device) and causing the machine to perform any of the methods and / or embodiments described herein. Examples of machine-readable storage media include, but are not limited to, magnetic disks, optical disks (e.g., CDs, CD-Rs, DVDs, DVD-Rs, etc.), magneto-optical disks, read-only memory "ROM" devices, random access memory "RAM" devices, magnetic cards, optical cards, solid-state storage devices, EPROMs, EEPROMs, programmable logic devices (PLDs), and / or any combination thereof. The use of "machine-readable medium" herein is intended to include both single media and collections of physically separate media, such as collections of optical disks or one or more hard disk drives combined with computer memory. As used herein, machine-readable storage media do not include temporary forms of signal transmission.

[0105] Such software may also include information (e.g., data) carried as a data signal on a data carrier (e.g., a carrier wave). For example, machine-executable information may be included as a data-bearing signal contained in a data carrier, wherein the signal encodes a sequence of instructions or a portion thereof for execution by a machine (e.g., a computing device), and any relevant information (e.g., data structures and data) that causes the machine to perform the methods and / or any of the embodiments described herein.

[0106] Examples of computing devices include, but are not limited to, e-book readers, computer workstations, terminal computers, server computers, handheld devices (e.g., tablets, smartphones, etc.), network devices, network routers, network switches, bridges, any machine capable of executing a sequence of instructions (which specifies the action the machine should take), and any combination thereof. In one example, a computing device may include and / or be included in a kiosk.

[0107] Figure 7 The illustration shows an embodiment of a computing device in an exemplary form of a computer system 700, wherein a set of instructions for causing a control system to perform any one or more aspects and / or methods of this disclosure can be executed. It is also contemplated that a specially configured set of instructions can be implemented using multiple computing devices to cause one or more devices to perform any one or more aspects and / or methods of this disclosure. The computer system 700 includes a processor 704 and a memory 708, which communicate with each other and with other components via a bus 712. The bus 712 may include any of several types of bus architectures, including but not limited to a memory bus, memory controller, peripheral bus, local bus, and any combination thereof using any of various bus architectures.

[0108] Memory 708 may include various components (e.g., machine-readable media), including but not limited to random access memory components, read-only components, and any combination thereof. In one example, a basic input / output system 716 (BIOS), including basic routines that facilitate the transfer of information between elements within computer system 700, may be stored in memory 708, for example, during startup. Memory 708 may also include (e.g., stored on one or more machine-readable media) instructions (e.g., software) 720 embodying any one or more aspects and / or methods of this disclosure. In another example, memory 708 may further include any number of program modules, including but not limited to an operating system, one or more application programs, other program modules, program data, and any combination thereof.

[0109] Computer system 700 may also include storage device 724. Examples of storage devices (e.g., storage device 724) include, but are not limited to, hard disk drives, disk drives, optical disk drives combined with optical media, solid-state storage devices, and any combination thereof. Storage device 724 may be connected to bus 712 via a suitable interface (not shown). Example interfaces include, but are not limited to, SCSI, Advanced Technology Attachment (ATA), Serial ATA, Universal Serial Bus (USB), IEEE 1394 (FireWire), and any combination thereof. In one example, storage device 724 (or one or more components thereof) may be removably interfaced with computer system 700 (e.g., via an external port connector (not shown)). In particular, storage device 724 and associated machine-readable medium 728 may provide non-volatile and / or volatile storage of machine-readable instructions, data structures, program modules, and / or other data for computer system 700. In one example, software 720 may reside wholly or partially in machine-readable medium 728. In another example, software 720 may reside wholly or partially within processor 704.

[0110] Computer system 700 may also include input device 732. In one example, a user of computer system 700 may input commands and / or other information to computer system 700 via input device 732. Examples of input device 732 include, but are not limited to, alphanumeric input devices (e.g., keyboards), pointing devices, joysticks, game controllers, audio input devices (e.g., microphones, voice response systems, etc.), cursor control devices (e.g., mice), touchpads, optical scanners, video capture devices (e.g., still cameras, video cameras), touchscreens, and any combination thereof. Input device 732 may be connected to bus 712 via any of a variety of interfaces (not shown), including but not limited to serial interfaces, parallel interfaces, game ports, USB interfaces, firewire interfaces, direct interfaces to bus 712, and any combination thereof. Input device 732 may include a touchscreen interface, which may be part of or separate from display 736, as will be discussed further below. Input device 732 may be used as a user selection device for selecting one or more graphical representations in the graphical interface described above.

[0111] Users can also input commands and / or other information to computer system 700 via storage device 724 (e.g., removable disk drive, flash drive, etc.) and / or network interface device 740. Network interface devices, such as network interface device 740, can be used to connect computer system 700 to one or more of various networks, such as network 744, and one or more remote devices 748 connected thereto. Examples of network interface devices include, but are not limited to, network interface cards (e.g., mobile network interface cards, LAN cards), modems, and any combination thereof. Examples of networks include, but are not limited to, wide area networks (e.g., the Internet, corporate networks), local area networks (e.g., networks associated with offices, buildings, campuses, or other relatively small geographical spaces), telephone networks, data networks associated with telephone / voice providers (e.g., mobile communication provider data and / or voice networks), direct connections between two computing devices, and any combination thereof. Networks, such as network 744, can employ wired and / or wireless communication modes. Typically, any network topology can be used. Information (e.g., data, software 720, etc.) can be transferred to and / or from computer system 700 via network interface device 740.

[0112] Computer system 700 may also include a video display adapter 752 for transmitting displayable images to a display device, such as display device 736. Examples of display devices include, but are not limited to, liquid crystal displays (LCDs), cathode ray tubes (CRTs), plasma displays, light-emitting diode displays (LEDs), and any combination thereof. Display adapter 752 and display device 736 may be used in conjunction with processor 704 to provide graphical representations of various aspects of this disclosure. In addition to display devices, computer system 700 may include one or more other peripheral output devices, including but not limited to audio speakers, printers, and any combination thereof. Such peripheral output devices may be connected to bus 712 via peripheral interface 756. Examples of peripheral interfaces include, but are not limited to, serial ports, USB connections, firewire connections, parallel connections, and any combination thereof.

[0113] The illustrative embodiments of the invention have been described in detail above. Various modifications and additions can be made without departing from the spirit and scope of the invention. To provide multiple combinations of features in related new embodiments, features of the various embodiments described above can be suitably combined with features of other described embodiments. Furthermore, while many individual embodiments have been described above, what is described herein is merely an illustration of the application of the principles of the invention. Moreover, although specific methods herein may be shown and / or described as being performed in a particular order, this order is highly variable within the art to implement the embodiments disclosed herein. Therefore, this description is intended only as an example and not as limiting the scope of the invention.

[0114] In the above description and claims, phrases such as “at least one” or “one or more” may appear after a list of connected elements or features. The term “and / or” may also appear in a list of two or more elements or features. Unless implied or explicitly contradicted by the context in which it is used, such phrases are intended to mean any element or feature listed individually or any of the listed elements or features combined with any other listed element or feature. For example, the phrases “at least one of A and B,” “one or more of A and B,” and “A and / or B” are each intended to mean “A alone, B alone, or A and B together.” Similar interpretations apply to lists containing three or more items. For example, the phrases “at least one of A, B, and C,” “one or more of A, B, and C,” and “A, B, and / or C” are each intended to mean “A alone, B alone, C alone, A and B together, A and C together, B and C together, or A and B and C together.” Furthermore, the term “based on” as used above and in the claims is intended to mean “at least partially based on,” allowing for the inclusion of uncited features or elements.

[0115] Depending on the desired configuration, the subject matter described herein can be embodied in systems, apparatus, methods, and / or articles. The embodiments set forth in the foregoing description do not represent all embodiments consistent with the subject matter described herein. Rather, they are merely some examples consistent with aspects related to the described subject matter. Although some variations have been described in detail above, other modifications or additions are possible. In particular, further features and / or variations may be provided in addition to the features and / or variations set forth herein. For example, the above embodiments may be applicable to various combinations and sub-combinations of the disclosed features and / or combinations and sub-combinations of several further features disclosed above. Furthermore, the logical flows depicted in the drawings and / or described herein do not necessarily require the specific order or sequence shown to achieve the desired results. Other embodiments may be within the scope of the following claims.

Claims

1. A decoder, the decoder comprising circuitry configured to: Receive bitstream, including encoded image data containing an encoded image sequence of the current image, the current image including a first region and a second region, the first region including only a first consecutive plurality of encoded units with global motion, the second region having a second consecutive plurality of encoded units with local motion, in the first region, information for determining the first plurality of control point motion vector predictions for affine motion compensation, including the index of a reference image selected in a reference image list and motion vector difference information of the three control point motion vectors for six-parameter affine motion compensation; The motion vector prediction of the first plurality of control points is obtained by decoding the reference image of the reference image list indicated by the index; The motion vector difference information from the bit stream is combined with the obtained motion vector prediction to form three current control point motion vectors; and The coded image data of the current image in the coded image sequence is decoded using the motion vectors of the three current control points and the six-parameter affine motion compensation.

Images