Coding device

The encoding device optimizes motion vector coding through nested entropy encoding and trimming techniques, addressing high bit rate requirements in video transmission systems by improving coding efficiency and reducing bandwidth usage.

JP2024161262AActive Publication Date: 2024-11-15DOLBY INTERNATIONAL AB
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
JP2024153800
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Priority Date
2010-10-01
Filing Date
2024-09-06
Publication Date
2024-11-15
Estimated Expiration
2031-09-30

AI Technical Summary

Technical Problem

Existing video transmission systems require high bit rates for maintaining desired video quality due to inefficiencies in motion vector coding, particularly in low bit rate applications, leading to excessive bandwidth usage.

Method used

An encoding device that trims and optimizes motion vector candidate sets using nested entropy encoding structures, incorporating spatial and temporal dependencies to reduce overhead and improve coding efficiency.

Benefits of technology

The proposed method reduces the bit rate required for video transmission by optimizing motion vector coding, maintaining desired video quality while minimizing bandwidth usage.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024161262000001_ABST
    Figure 2024161262000001_ABST
Patent Text Reader

Abstract

To solve the problem that the necessary bit rate to preserve a desired quality is often too high for the transmission medium used to transmit the video to a decoder.SOLUTION: A coding device is configured to execute a step of generating a motion vector candidate set comprising a first motion vector and a second motion vector. The first motion vector is a motion vector of an adjacent block that is adjacent to an objective block of a current frame. The second motion vector is a co-located motion vector of a first block in a preceding frame that precedes the current frame. The first block is larger in block size than a minimum size block.SELECTED DRAWING: Figure 1A
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] There are no related applications to which this case is cross-referenced.

[0002] The present invention relates to a method for generating a set of motion vector candidates. [Background technology]

[0003] Modern video transmission and display systems, especially those supporting high resolution content, require significant data compression to produce visually acceptable moving images because the transmission medium cannot transmit uncompressed sequences of video frames fast enough to allow the human eye to perceive the displayed images as continuous moving images. At the same time, to produce visually acceptable moving images, the compression techniques used should not unduly sacrifice image quality by discarding too much frame data.

[0004] To achieve these two competing goals, video compression coding standards such as MPEG and H.264 exploit the temporal redundancy of video frame sequences. In other words, in most video sequences of human interest, adjacent frames often show the same objects or features, which move slightly from frame to frame, either due to the movement of the objects in the captured scene (resulting in local motion within the frame), or the movement of the camera capturing the scene (resulting in global motion), or both.

[0005] Video compression standards use motion estimation to define regions in an image (which may correspond to objects) and associate with them motion vectors that indicate the frame-to-frame motion of the content of each region. This avoids redundant coding and transmission of objects or patterns that appear in two or more successive frames, even though their location in the successive frames differs only slightly. Motion vectors can be represented by a translational model or many other models that approximate the motion of a real video camera, such as rotation, translation, or magnification. Motion estimation is thus the process of calculating and coding motion vectors instead of repeatedly coding similar information in successive frames.

[0006] A motion vector may refer to the whole image, but more often it refers to a small region in the image, such as a rectangular block, any shape, the boundary of an object, or even individual pixels. There are various detection methods for detecting motion vectors. One well-known method is the block matching method. In the block matching method, the current image is divided into rectangular pixel blocks (4x4 pixels, 4x8 pixels, 8x8 pixels, 16x16 pixels, etc.), and the motion vector (or displacement vector) of each pixel block is predicted by searching for the pixel block of a reference image that best matches the pixel block from a given search area of ​​a subsequent frame.

[0007] As implied by the above description, the use of motion vectors improves the coding efficiency of a particular block in an image by allowing the block to be coded only with respect to the motion vector pointing to the corresponding block in another frame and the "residual" or difference between the current block and the reference block. Therefore, the goal is to determine the motion vector of the block so that the difference that needs to be coded is minimized. Thus, there are many variations of block matching, which differ in the definition of the size and location of the block, the search method, the matching criteria between the blocks of the current frame and the reference frame, or some other aspect.

[0008] In conventional motion compensation, an encoder performs motion prediction and signals motion vectors as part of the bitstream. The bits spent on transmitting motion vectors can take up a significant portion of the overall bit budget, especially in low bitrate applications. Recently, Motion Vector Competition (MVC) techniques have been proposed to reduce the amount of motion information in the compressed bitstream. MVC improves the coding of motion vector data by differentially coding the motion vector itself with respect to a motion vector predictor and a differential motion vector. The predicted motion vector is typically selected by the encoder from a number of candidates in a rate-distortion-optimizing manner, where the motion vector candidates consist of already coded motion vectors of adjacent blocks in the same frame and / or a subset of motion vectors in a previous frame. In other words, the use of motion vectors and differentials improves the coding efficiency of block data by removing information redundancy between successive frames, while the coding of the motion vectors can exploit redundancy in situations where the motion vector does not change significantly between successive frames by identifying an optimal prediction from a limited set of already coded candidates, thereby minimizing the bit length of the differential. A prediction set typically includes both spatial motion vector neighbors and temporally co-located motion vectors, and may also include spatiotemporal vectors.

[0009] However, even when vector competition techniques are used to code video, the bit rate required to maintain a desired quality is often too high for the transmission medium used to transmit the video to a decoder. Therefore, there is a need for improved coding systems for video transmission.

[0010] The above and other objects, features, and advantages of the present invention will be more readily understood by considering the following detailed description of the invention in conjunction with the accompanying drawings. [Prior art documents] [Non-patent literature]

[0011] [Non-Patent Document 1] Frank Bossen, Philipp Kosse, "Simplified motion vector coding method", Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO / IEC JTC1 / SC29 / WG11 2nd Meeting: Geneva, CH, 21-28 July, 2010, [JCTVC-B094] Summary of the Invention [Problem to be solved by the invention]

[0012] Even when vector competition techniques are used to code video, the bit rate required to maintain a desired quality is often too high for the transmission medium used to transmit the video to a decoder. Therefore, there is a need for improved coding systems for video transmission. [Means for solving the problem]

[0013] A preferred embodiment is an encoding device for encoding a motion vector of a target block in an image, the encoding device including a process for identifying a first block adjacent to the target block, a process for identifying a second block adjacent to the target block, a process for determining a motion vector of the first block and a motion vector of the second block, and a process for generating a motion vector predictor candidate set including at least one of the motion vector of the first block or the motion vector of the second block, wherein when the motion vector of the first block is equal to the motion vector of the second block, the motion vector predictor candidate set includes the motion vector of the first block and the motion vector of the second block. the prediction motion vector for the target block; a process of determining whether a motion vector located in the time direction can be used as a prediction motion vector for the target block; a process of including a motion vector of a block in another image in the prediction motion vector candidate set as a result of determining that a motion vector located in the time direction can be used as a prediction motion vector for the target block; a process of generating a flag indicating whether a motion vector located in the time direction can be used as a prediction motion vector; and a process of selecting a motion vector from the prediction motion vector candidate set as the prediction motion vector for the target block. Effect of the Invention

[0014] The improvement of the coding system for video transmission ensures that even when vector competition techniques are used to code the video, the bit rate required to maintain a desired quality is not too high for the transmission medium used to transmit the video to a decoder. [Brief description of the drawings]

[0015] [Figure 1A] 1 illustrates a schematic diagram of motion vector competition. [Figure 1B] 1 illustrates a schematic diagram of motion vector competition. [Diagram 2]1 shows an example of a motion vector encoding / decoding system. [Diagram 3] 1 shows a nested entropy coding structure. [Figure 4] A system utilizing the nested entropy coding structure shown in FIG. [Figure 5A] 1 shows an example of an encoder capable of trimming a set of motion vector candidates. [Figure 5B] 6 illustrates an example of a method for trimming the set of motion vector candidates used by the encoder of FIG. [Figure 6] 1 illustrates an alternative embodiment for encoding a temporally co-located motion vector in a motion vector candidate set. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] 1A and 1B, the motion vector of a candidate block (shown as a shaded area) in a current frame (T=0) points to a shaded area in a subsequent frame (T=1). The motion vector is the motion vector V a , V x , V y , and V z In this example, the motion vector V a is a co-located motion vector in the previous frame (T=-1) that points to block A in the current frame. x , V y , and V z are pre-encoded motion vectors in the current frame, pointing to blocks X, Y, and Z in the subsequent frame (T=1), respectively. Figure 1A also shows blocks A', X', Y', and Z', which are the blocks that would be pointed to by the motion vectors if they were used in encoding the candidate blocks.

[0017] As shown in FIG. 1B, when the motion vector competition (MVC) method is used, the difference V d The motion vector V is calculated so as to minimize the code length of z is selected. In this case, only a single component (lower) of the vector needs to have a value of "1". All other differential motion vectors need to code two components or need a larger value for the single component.

[0018] The above description is a simplification. Therefore, blocks of different sizes may be used, each block may represent a single pixel, and more motion vectors may be included in the candidate set. For example, all previously calculated motion vectors in the current frame may be included in the candidate set, as well as any motion vectors calculated in the previous frame. Furthermore, the candidate set may include any desired number of motion vectors that are useful for capturing large and rapid motion in a scene.

[0019] Continuing with the above example, let us consider FIG. z In a straightforward approach, the encoder 10 assigns a value to each candidate motion vector in a table 14 of symbols. This relies on a variable length entropy coding method such as Huffman coding or arithmetic coding, and takes the following form: Motion Vector Candidate Symbols V a 0 V x 10 V y 110 V z 1110 Note that none of the above symbols are prefixes of any other symbols. Thus, in this example, the decoder 12 can correctly analyze the received bitstream by pausing when it receives a "0" in the received bitstream and refer to the corresponding table 16 to decode the received bitstream. Furthermore, the encoder and decoder preferably collect statistics while the bitstream is being encoded and decoded, and rearrange the assignment of symbols to potential motion vectors in each of the tables 14 and 16, e.g., so that the shortest symbols are always assigned to the most frequent motion vectors. This process is commonly referred to as entropy coding, and typically results in a dramatic lossless compression of the bitstream. The encoder 10 and decoder 12 use the same technique to build and update the tables 14 and 16, respectively, which are initialized at the beginning of the bitstream. This ensures that for all symbols, the table 16 used to encode the symbol is the same table as the table used to decode the symbol.

[0020] Even with entropy coding, the system shown in FIG. 2 may incur significant overhead when transmitting signals indicating which prediction is selected from a set of motion vector candidates. This is especially true when the number of predictions is large. On the other hand, the more predictions used, the more efficient the coding of the differential motion vectors can be. Further techniques may be employed to further reduce the overhead of transmitting signals indicating which prediction is selected.

[0021] First, the set of motion vector predictor candidates may be trimmed to remove duplicated vectors. If the horizontal values, vertical values, and reference indices between two motion vectors are equal, the two motion vectors are duplicated vectors. The term "duplicated vector" is a term having an equivalent meaning to "duplicated motion vector", "identical vector", or "identical motion vector". For example, in FIG. 1A, vector V x , Vy Since V is the same vector, we can trim one of these motion vectors. This allows us to remove the largest symbol in the table, “1110”. Secondly, knowing the size of the trimmed motion vector predictor set means that we can omit the last bit of the last symbol in the set. For example, in the above example, V x , V y The result is that the trimming is done on one side of the The last symbol is “110”. For this symbol “110”, the bit arrangement is If the sequence "11" is distinct from all other preceding symbols stored in the table, then "110" may simply be encoded as the bit sequence "11" and the decoder will determine that there are no more symbols based on the size of the trimmed set.

[0022] The addition of these two techniques may significantly reduce the overhead incurred when transmitting signals indicating the selected predicted motion vector. However, the use of these techniques makes the entropy decoding of the predicted motion vector dependent on the motion prediction set. That is, the bitstream cannot be analyzed correctly until a complete set of motion prediction sets is available and correctly constructed. Such a constraint seriously affects the error resilience of the decoder, resulting in two disadvantages. The first disadvantage is time dependency: if an image is damaged or lost, the decoding of subsequent images may fail at the analysis stage. The second disadvantage is spatial dependency: if a certain region of an image is damaged, the decoding of subsequent regions in that image may fail at the analysis stage.

[0023] This can be a serious problem: if motion vector data for the previous or current frame is lost and a complete set of candidate motion vectors is required to be reconstructed, the decoder cannot even parse the bitstream until an independently coded frame arrives. This is a more serious problem than simply being unable to correctly decode the parsed data due to the loss of motion vectors, differential motion vectors and the information used to code the differentials, since in the latter case it would be possible to decode any parsed data in the subsequently received bitstream that is not dependent on the lost data. However, once the decoder is no longer able to parse the bitstream, it has no way to decode subsequent symbols.

[0024] Although counterintuitive, the tradeoff between error resilience and overhead is not an insurmountable problem. The inventors have further found that, in theory, coding efficiency can be achieved by signaling one of a set of ordered candidate motion vectors, similar to the coding efficiency achieved by signaling one of a set of candidate motion vectors. This coding efficiency can be achieved not only by trimming motion vectors and using them in series with truncated unary codes, but in fact as an alternative to these techniques. That is, spatial and temporal independence can be maintained when performing bitstream analysis without trimming duplicate motion vector candidates and truncating the longest bit length symbols.

[0025] Specifically, referring to FIG. 3, an encoder or decoder utilizes a nested entropy coding structure, in which one of a number of coding symbols 18 is assigned to each of a number of entropy coded candidate sets of motion vectors, such as individual VLC tables 20. Note that a particular one of the VLC tables 20 may contain a different set of motion vectors than the other VLC tables 20. That is, a particular motion vector contained in one VLC table 20 need not be contained in all VLC tables 20. The encoder may signal one of the symbols 18 corresponding to one of the VLC tables 20 (candidate sets) in which the signaled motion vector has the highest frequency and, therefore, the shortest code length. The coded symbols 18 each identifying a candidate set may themselves be entropy coded, if desired, or may be coded by a fixed length code or any other coding technique.

[0026] An underlying assumption in the above explanation is that there is a certain non-random distribution of all possible motion vector candidate sets. For example, if each individual candidate set contains all permutations of symbols that are randomly distributed with respect to each other, there is no reason to expect an improvement in coding efficiency, because the number of motion vector candidate sets required to ensure that there are enough motion vector candidates in the candidate set in the table to benefit from the reduced code length would be too large. In essence, the efficiency gain in coding a selected one of the motion vector candidates is lost in the overhead incurred when coding the symbols associated with a particular candidate set. This can be seen from the following: Just as entropy coding of motion vectors works by relying on predictable spatial and temporal relationships between motion vectors, causing some motion vector candidates to occur more frequently than others, the nested entropy coding structure of this disclosure aims to perform further bitstream compression only when some of the possible permutations of symbols in a candidate set occur more frequently than others, such that candidate sets with longer code lengths are used less frequently than candidate sets with shorter code lengths.

[0027] As a result of research, the inventors have found that not only does the nested entropy coding structure of the present disclosure actually improve coding efficiency, but also that the syntax elements of neighboring pixels or neighboring pixel blocks are correlated with the probability of ordering of motion vector candidates in a set. For example, referring to FIG. 4, the encoder 10 may obtain syntax symbols from a syntax model 24. The syntax model 24 defines a set of syntax elements in the coded data that are used to distinguish between multiple VLC tables of a motion vector candidate set, and thus also defines a set of syntax elements that the encoder and decoder use to determine the VLC table to be used in coding the selected motion vector candidate with the code symbol. These syntax elements can, for example, relate to selected motion vector candidates in temporally or spatially neighboring pixel blocks, or to combinations of such selected motion vector candidates, or to any elements determined to have a relationship to the probability distribution of the selected motion vector in the candidate set. In one embodiment, the encoder 10 (and therefore the decoder 12) includes a learning factor that tests different combinations of syntax elements to intelligently maximize coding efficiency. In other words, the encoder 10 intelligently optimizes coding efficiency by iteratively selecting different combinations of available syntax elements, measuring the change in coding efficiency associated with each selected combination, and reacting accordingly by substituting one or more syntax elements in each combination.

[0028] Using the syntax symbols obtained from the syntax model 24, the encoder 10 may then use the motion vector symbols available in the VLC tables 28a, 28b, 28c, 28d, etc. for the motion vector selected for the current block and encode the motion vector symbols in the bitstream sent to the decoder 12. The encoder 10 also updates the order of the motion vector symbols in the VLC tables used based on the selected symbols. In one embodiment, the symbols are reordered when a change occurs in the frequency distribution of the symbols contained in the tables. In another embodiment, the encoder 10 (and the decoder 12) keeps track of the most frequently occurring symbol in the set before the reordering occurs and ensures that the symbol is ordered highest in the table (i.e., ensures that the code length of the symbol is shortest). Note that in this example, since the syntax symbols are determined solely by the syntax of the already encoded data, there is no need for the encoder to encode the syntax symbols along with the motion vector symbols, so long as the decoder 12 uses the same syntax model to determine the particular VLC table 30a, 30b, 30c, 30d from which to extract the received motion vector symbols. In other words, very high coding efficiency can be achieved if the encoder 10 uses the syntax of the already encoded data to distinguish between VLC tables, and in the process updates the ordering of symbols in those VLC tables.

[0029] When decoder 12 receives the encoded bitstream from encoder 10, it parses the bitstream, determines the appropriate VLC table for the received symbols (using syntax model 26 if available), decodes the received symbols, and identifies a motion vector selected from the candidate set. The decoder also updates each VLC table in the same manner as encoder 10.

[0030] The predicted motion vector set may include motion vector candidates that spatially predict the selected motion vector (i.e., candidates in the same frame as the current block), motion vector candidates that temporally predict the selected motion vector (i.e., candidates in a collocated block in a frame preceding the current frame), and motion vector candidates that spatio-temporally predict the selected motion vector (i.e., candidates in a frame preceding the current block that is spatially offset from the collocated block). As mentioned above, the nested entropy coding structure of the present disclosure allows a decoder to parse the bitstream without trimming the motion vector candidates or shortening the code symbols, thereby maintaining spatial and temporal independence in the parsing process and maintaining error resilience while achieving very high coding efficiency. Alternatively, the nested entropy coding structure may be used in conjunction with a trimming technique or a reduction technique for the motion vector candidates while at least partially maintaining error resilience.

[0031] For example, as shown in FIG. 5A, the encoder 10 may include a motion vector candidate set construction module 40. The motion vector candidate set construction module 40 reads from one or more buffers 28 the entire set of motion vector candidates applicable to the current block being encoded. The motion vector candidate set trimming module 42 then selectively trims the motion vector candidate set according to a predefined rule by applying the syntax model 24 to the motion vector candidate set before the selected motion vector is encoded by the encoding module 44. The encoding module 44 selects symbols in turn based on the trimmed candidate set. One of the possible predefined rules may, for example, prevent the motion vector candidate set module 42 from trimming a predicted motion vector derived from an already reconstructed / transmitted frame. In other words, if two predicted motion vectors have the same value, one predicted motion vector corresponds to data of the current frame and the other predicted motion vector corresponds to data of another frame, both of these two predicted motion vectors are included in the trimmed set. This maintains the time independence. According to another possible predetermined rule, for example, the motion vector candidate set module 42 may prevent the predicted motion vector from being trimmed by different reference indexes. In other words, if two predicted motion vectors have the same horizontal and vertical values, and the reference index of one predicted motion vector is different from the reference index of the other predicted motion vector, both of these two predicted motion vectors are included in the trimmed set. According to another possible predetermined rule, for example, two motion vectors with the same reference index but different horizontal and / or vertical values ​​are included in the trimmed set. According to yet another possible predetermined rule, for example, if two motion vectors with the same reference index value, horizontal and vertical values ​​are present in the set, one motion vector is deleted by the motion vector candidate set module 42.

[0032] As another example, a predetermined rule may be set to prevent predicted motion vectors derived from regions located in different slices from being trimmed by the motion vector candidate set trimming module 42 to maintain spatial independence. As an additional embodiment, a predetermined rule may be set to prevent predicted motion vectors derived from regions located in different entropy slices from being trimmed by the motion vector candidate set trimming module 42. An entropy slice is a unit of a bitstream that can be analyzed without reference to other data in the current frame.

[0033] These two rules are provided for illustrative purposes only, as examples of rules that can be added if necessary. FIG. 5B shows a general technique for applying various trimming rule sets, which are transmitted using a novel flag, as an example. For example, in step 50, the encoder 10 receives a candidate set of predicted motion vectors from the buffer 28. In step 52, a flag is transmitted by the encoder (or received by the decoder) that is used to determine whether to apply trimming in decision step 53. Optionally, a set of trimming rules may also be transmitted that can be used to define which vectors to trim. If the flag indicates that no trimming is to be performed, the process proceeds to step 60, where the entire set of candidate motion vectors is used to encode the selected motion vector. However, if the flag indicates that trimming is to be performed according to a predefined set of rules, then in step 54, a subset of overlapping motion vectors is identified. Thus, in one embodiment, the subset of overlapping motion vectors can be considered as the maximal collection of motion vectors, each of which has an identical motion vector outside the subset. In other words, the subset may be obtained by removing from the subset those motion vectors that do not have any duplicate motion vectors in the motion vector candidate set, and by removing exactly one from the set of identical duplicate motion vectors.

[0034] In step 56, the selected candidate motion vector may be selectively removed from the overlapping subset according to a predefined rule in the rule set. This step allows for spatial and / or temporal independence to be ensured. Optionally, the candidate motion vector may be added to the subset of overlapping motion vectors, for reasons that will be explained in more detail below. At a conceptual level, the purpose of steps 54 and 56 is simply to apply the rule set to identify motion vectors to be trimmed from the complete set of candidates. Once this subset is identified, the candidate motion vectors in this subset are trimmed in step 58, after which the encoder encodes a selected one of the remaining candidate motion vectors based on the size of the trimmed candidate set in step 60.

[0035] To explain the functionality of the general method shown in Fig. 5A, consider the example of a temporal MVP flag (temporal_mvp_flag). The temporal MVP flag is used by the encoder to transmit in the bitstream a true / false status indicating whether the motion vector selected from the candidate set is a motion vector located in the temporal direction. Also, the premise is that the rule set applicable for this flag is intended to preserve temporal independence. If the temporal MVP flag indicates that temporal prediction is selected by the encoder, the temporal prediction subset in the candidate set is not trimmed, because trimming the temporal prediction subset would result in a temporal dependency. However, the spatial prediction subset in the candidate set can be trimmed, because the size of the temporal prediction subset is predicted in advance by the decoder 12.

[0036] Conversely, if temporal_mvp_flag indicates that temporal prediction is not selected by the encoder, in some embodiments, it is possible to trim not only overlapping motion vectors but also temporal predictions to the candidate set, in which case the candidate set that needs to be encoded can be significantly smaller. It should be further recognized that if temporal and spatial dependencies are allowed by the applicable rule set, temporal_mvp_flag, regardless of its value, can be used to trim (1) temporally or spatially overlapping subsets that are transmitted using the flag, or (2) entire subsets that are not transmitted using the flag.

[0037] The present inventors have found by chance that there is a definite correlation between the value of temporal_mvp_flag in the present disclosure and the constrained intra prediction flag (constrained_intra_pred_flag). The constrained_intra_pred_flag is associated with a frame and is often used in a coded video bitstream. Specifically, the present inventors have found that there is a strong correlation between these two flags when the value of constrained_intra_pred_flag is 1, and a somewhat weak correlation when the value of constrained_intra_pred_flag is 0. Therefore, in order to reduce the overhead in notifying the selected motion vector, the encoder may be optionally configured as follows. That is, if constrained_intra_pred_flag is set to 1 for the frame of the current pixel, the decoder may be configured not to code the temporal_mvp_flag of the present disclosure, but to simply insert or assume an equal value for the temporal_mvp_flag, and to code the temporal_mvp_flag if the constrained_intra_pred_flag is not set to 1. Alternatively, the temporal_mvp_flag of the present disclosure may simply be assigned the same value as the constrained_intra_pred_flag. In this case, however, it is preferable for the prescribed rule set to associate a value of 0 with the result of simply trimming overlapping vectors in the candidate set.

[0038] The nested entropy coding structure of the present disclosure can be additionally applied to this temporal_mvp_flag syntax. In one embodiment, the upper neighbor flag and the left neighbor flag are used to determine the prediction set template used in the entropy coding of the temporal_mvp_flag. This is usually effective when the encoder and the decoder exclusively allocate entropy symbols for the coding value and the temporal_mvp_flag has multiple values. In another embodiment, the prediction set template used to code the selected motion vector in the candidate set is created according to the temporal_mvp_flag of the current block.

[0039] Also, as another embodiment, the predicted motion vector may be equal to the motion vector derived from the current frame, or equal to the motion vector derived from a previously reconstructed / transmitted frame, as described above for temporal_mvp_flag. However, in this particular embodiment, a flag is transmitted, indexed by the number of unique predicted motion vectors derived from the current frame. For example, the prediction set template in this embodiment can distinguish all possible combinations of a first coding value reflecting the combination of flags in the blocks to the left and above the current block (e.g., 00, 01, 10, 11 (entropy coded as 0, 10, 110, 1110)) indexed by a second coding value reflecting the number of unique motion vectors in the candidate set. Alternatively, the context template in this embodiment can identify all possible combinations of a first coding value reflecting whether the flags in the block to the left and above the current block are identical (e.g., 00 and 11 (entropy coded as 0) and 01 and 10 (entropy coded as 10)) and a second coding value reflecting the number of unique motion vectors in the candidate set.

[0040] The encoding scheme may include a motion vector candidate set that includes a large number of temporally collocated motion vectors (e.g., the motion vectors shown in FIG. 6) for each of a number of frames. This means that to encode a block 64 of the current frame, the encoder may need to access one or more buffers that contain the history of all motion vectors selected in each of the previous frames from which the motion vector candidates are extracted. This requires a significant amount of memory. Alternatively, the small size pixel blocks (e.g., 2×2 pixel blocks) used in the encoding scheme may be grouped into larger blocks 62 (where the motion vectors are stored in a buffer and then used as collocated motion vectors when encoding subsequent blocks) and instead use an average motion vector 66 of all selected vectors in each group. This arrangement trades memory requirements for lower coding efficiency, since the averaging process tends to result in larger differences being coded whenever a collocated motion vector is selected. Nevertheless, the reduction in coding efficiency is not significant if the average collocated vector is selected only when it is more efficient to use the average collocated vector than other motion vectors in the candidate set. In addition to using the average value of the neighboring blocks, vector median processing or component-wise median processing may be performed, as well as other standard processing, such as upper limit processing, lower limit processing, or a combination of these, or so-called dilate, erode, open, close, etc. Furthermore, motion vectors at a predetermined position may be used. For example, in the latter usage described above, the motion vector corresponding to the Nth small size pixel block in the large block may be recorded in a buffer as a collocated motion vector for the large block (N is an integer corresponding to the position of the small size block in the large block in the raster scan order).

[0041] In some embodiments, the operation used to group small sized pixel blocks into larger sized blocks may be signaled in the bitstream from the encoder to the decoder. For example, the operation may be signaled in a sequence parameter set. Alternatively, the operation may be signaled in a picture parameter set, a slice header, or any predefined group of pixels. Furthermore, the operation may be determined from a level or profile identifier signaled in the bitstream.

[0042] In some embodiments, the number of small size blocks that have been grouped into larger blocks may be signaled in the bitstream from the encoder to the decoder. For example, the number of small size blocks may be signaled in a sequence parameter set. Alternatively, the number of small size blocks may be signaled in a picture parameter set, a slice header, or any predefined group of pixels. The number of small size blocks may be determined from a level or profile identifier signaled in the bitstream. In some embodiments, the number of small size blocks may be expressed as the number of rows and columns of small size blocks.

[0043] It should be understood that the above-described embodiments of the encoder and / or decoder can be used in any of a number of hardware, firmware, or software implementations. For example, the encoder may be used in a set-top recorder, a server, a desktop computer, etc., and the decoder may be used in a display device, a set-top cable box, a set-top recorder, a server, a desktop computer, etc. These examples are illustrative and not limiting. When the encoders and decoders of the present disclosure are implemented in firmware and / or software, the various components of the encoders and decoders of the present disclosure may access any available processing devices and storage to implement the techniques described above.

[0044] The terms and expressions used in the above specification are terms and expressions for describing the present invention, and are not intended to limit the present invention. Therefore, the use of such terms and expressions is not intended to exclude the features illustrated and described above or equivalents to any part thereof. It should be recognized that the scope of the invention is defined and limited only by the following claims.

[0045] (Another expression of the invention) The present invention can also be expressed as follows. That is, a preferred embodiment is a decoding method for decoding a predicted motion vector of a target block in an image of a sequence of images, comprising: a first identification step of identifying a first adjacent block adjacent to the target block in the image; a second identification step of identifying a second adjacent block adjacent to the target block in the image; a first generation step of generating a candidate set of predicted motion vectors including both the motion vectors of the first adjacent block and the second adjacent block when the motion vector of the first adjacent block and the motion vector of the second adjacent block are not equal; and a second generation step of generating a candidate set of predicted motion vectors including only one of the motion vector of the first adjacent block and the motion vector of the second adjacent block when the motion vector of the first adjacent block and the motion vector of the second adjacent block are equal. the step of acquiring from the bitstream a flag indicating whether a motion vector located in the time direction can be used as a predictive motion vector; the step of including a motion vector of a block in another image in the predictive motion vector candidate set if the flag indicates that a motion vector located in the time direction can be used as a predictive motion vector; the step of excluding the motion vector of the block in another image from the predictive motion vector candidate set if the flag indicates that a motion vector located in the time direction cannot be used as a predictive motion vector; and the step of selecting a motion vector from the predictive motion vector candidate set as the predictive motion vector for the target block, wherein the motion vector of the target block is derived based on the selected predictive motion vector and a differential motion vector.

[0046] Also, a preferred embodiment is an apparatus comprising a decoder for decoding a predictive motion vector for a current block in an image of a sequence of images, the decoder identifying a first neighboring block adjacent to the current block in the image, identifying a second neighboring block adjacent to the current block in the image, generating a candidate set of predictive motion vectors including the motion vectors of both the first and second neighboring blocks when the motion vectors of the first and second neighboring blocks are not equal, and generating a candidate set of predictive motion vectors including only one of the motion vectors of the first and second neighboring blocks when the motion vectors of the first and second neighboring blocks are equal. a flag indicating whether a motion vector located in the time direction can be used as a predictive motion vector is obtained from the bitstream, and if the flag indicates that a motion vector located in the time direction can be used as a predictive motion vector, a motion vector of a block in another image is included in the predictive motion vector candidate set, and if the flag indicates that a motion vector located in the time direction cannot be used as a predictive motion vector, the motion vector of the block in the other image is excluded from the predictive motion vector candidate set, and a motion vector is selected from the predictive motion vector candidate set as the predictive motion vector for the target block, the motion vector of the target block is derived based on the selected predictive motion vector and a differential motion vector, and the differential motion vector is obtained from the bitstream.

[0047] Also, a preferred embodiment is a method for decoding a predictive motion vector of a target block in an image of a sequence of images, comprising the steps of: accessing a target block in an image; identifying a first block and a second block, each of which is adjacent to the target block in the image; including the motion vector of the first block and the motion vector of the second block in a candidate predictive motion vector set for the target block, if it is determined that the motion vector of the first block is not equal to the motion vector of the second block; including either the motion vector of the first block or the motion vector of the second block in the candidate predictive motion vector set for the target block, if it is determined that the motion vector of the first block is equal to the motion vector of the second block; The method includes the steps of receiving a prediction motion vector from a stream, including a motion vector of a block in another image in the prediction motion vector candidate set of the target block, on condition that it is determined that the flag indicates that a motion vector located in the time direction can be used as a prediction motion vector, excluding the motion vector of the block in the other image from the prediction motion vector candidate set of the target block, on condition that it is determined that the flag indicates that a motion vector located in the time direction cannot be used as a prediction motion vector, selecting a motion vector from the prediction motion vector candidate set as the prediction motion vector of the target block, deriving a motion vector of the target block based on the selected prediction motion vector and a differential motion vector, and generating a residual block as the difference between the target block and a reference block identified by the motion vector of the target block.

[0048] In addition, a preferred embodiment is an encoding device that encodes a predictive motion vector of a target block in an image, and performs the following processes: identifying a first block adjacent to the target block; identifying a second block adjacent to the target block; determining a motion vector of the first block and a motion vector of the second block; generating a predictive motion vector candidate set including at least one of the motion vector of the first block or the motion vector of the second block; determining whether a motion vector located in the time direction can be used as a predictive motion vector for the target block; and, as a result of determining that a motion vector located in the time direction can be used as a predictive motion vector for the target block, including a motion vector of a block in another image in the predictive motion vector candidate set; and selecting a motion vector from the predictive motion vector candidate set as the predictive motion vector for the target block.

[0049] Also, a preferred embodiment is a bitstream storing one or more encoded images of a sequence of images, comprising: a first block adjacent to a target block in an image of the one or more encoded images; a second block adjacent to the target block in the image; and a flag indicating whether a motion vector located in a time direction can be used as a predictive motion vector, wherein if the motion vector of the first block is not equal to the motion vector of the second block, a predictive motion vector candidate set for the target block includes the motion vector of the first block and the motion vector of the second block; and if the motion vector of the first block is equal to the motion vector of the second block and the flag indicates that a motion vector located in the time direction can be used as a predictive motion vector, the predictive motion vector candidate set for the target block includes the motion vector of the first block and the motion vector located in the time direction.

Claims

1. 1. An encoding device for encoding a motion vector of a current block in an image, comprising: identifying a first block adjacent to the target block; identifying a second block adjacent to the target block; determining a motion vector of the first block and a motion vector of the second block; a process of generating a motion vector predictor candidate set including at least one of the motion vector of the first block or the motion vector of the second block, and if the motion vector of the first block is equal to the motion vector of the second block, a process of excluding one motion vector from the motion vector predictor candidate set such that the motion vector predictor candidate set includes only one of the motion vectors of the first block and the motion vector of the second block, but not both; determining whether a motion vector located in a time direction can be used as a predicted motion vector for the current block; as a result of determining that a motion vector located in a time direction can be used as a motion vector predictor for the current block, a motion vector of a block in another image is included in the motion vector predictor candidate set; A process of generating a flag indicating whether a motion vector located in a time direction can be used as a predicted motion vector; selecting a motion vector from the motion vector predictor candidate set as the motion vector predictor for the current block.

2. The encoding device according to claim 1 , further comprising a process of generating a differential motion vector for the current block based on the selected predicted motion vector for the current block.

Citation Information

Patent Citations

  • Apparatus and method for image processing, recording medium, and program

    JP2004241880A