Method and apparatus for signaling and constructing video coding reference picture lists

Abstract

To provide an improved method and an apparatus for signaling of reference pictures used for temporal prediction, where signaling schemes and a construction process for various reference picture lists in HEVC Working Draft 5 (WD5) are improved.SOLUTION: In accordance with one embodiment, a method of generating reference picture lists L0 and L1 for decoding a predicted picture within video data generates a first ordered list of reference pictures from a decoded picture buffer (DPB). The list is ordered so that when the reference pictures that are temporally before a current picture are in the DPB, the reference pictures are listed in order according to temporal distance from the current picture, when the reference pictures that are temporally after the current picture are in the DPB, the reference pictures are listed in order according to temporal distance from the current picture, and when long term reference pictures are in the DPB, the reference pictures are listed in order by which the reference pictures are stored in the DPB.SELECTED DRAWING: Figure 10

Description

[Background technology]

[0001] Video coding systems are widely used to compress digital video signals to reduce their storage requirements and / or transmission bandwidth. Among various video coding systems, such as block-based, wavelet-based, and object-based systems, block-based hybrid video coding systems are currently the most widely used and deployed. Examples of block-based video coding systems include international video coding standards such as MPEG1 / 2 / 4 part 2, H.264 / MPEG-4 part 10 AVC (Non-Patent Documents 1, 3), and VC-1 (Non-Patent Document 2) standards.

[0002] Figure 1 is a block diagram of a general-purpose block-based hybrid video coding system. An input video signal 102 is processed in blocks. In all existing video coding standards, a video block unit consists of 16x16 pixels, commonly referred to as a "macroblock" or "MB." The ITU-T / SG16 / Q.6 / VCEG and ISO / IEC / MPEG Joint Collaborative Team on Video Coding (JCT-VC) are currently developing a next-generation video coding standard called High Efficiency Video Coding (HEVC) (Non-Patent Document 4). HEVC uses an expanded block size (called a "coding unit" or "CU") to efficiently compress high-resolution (1080p and above) video signals. In HEVC, a CU can be up to 64x64 pixels in size. CUs can be further divided into prediction units, or PUs, and individual prediction methods are applied to the PUs. Spatial prediction (160) and / or temporal prediction (162) can be performed for each input video block (MB or CU). Spatial prediction (or "intra prediction") predicts a current video block using pixels from already-encoded neighboring blocks in the same video picture / slice. Spatial prediction reduces spatial redundancy inherent in video signals. Temporal prediction (also called "inter prediction" or "motion-compensated prediction") predicts a current video block using pixels from already-encoded video pictures (commonly referred to as "reference pictures"). Temporal prediction reduces temporal redundancy inherent in video signals. The temporal prediction for a given video block is typically signaled by one or more motion vectors that indicate the amount and direction of motion between the current block and its predicted block in the reference picture. Additionally, when multiple reference pictures are supported (such as in modern video coding standards such as H.264 / AVC or HEVC), the index of each reference picture is also transmitted for each video block.The reference picture index identifies a reference picture in a reference picture store (164) (also called a "decoded picture buffer" or "DPB") from which a temporal prediction signal should be obtained to generate a prediction of the current video block to be reconstructed. After spatial prediction and / or temporal prediction, a mode decision block (180) of the encoder selects the best prediction mode based, for example, on a rate-distortion optimization method. The prediction block is then subtracted (116) from the current video block, and the prediction residual is transformed (104) and quantized (106). The quantized residual coefficients are inverse quantized (110) and inverse transformed (112) to form a reconstructed residual, which is placed into a prediction block (126) to form a reconstructed video block. Further, in-loop filtering, such as a deblocking filter, a sample adaptive offset, and an adaptive loop filter, is applied (166) to the reconstructed video block before the reconstructed video block is placed into the reference picture store (164) and used to encode future video blocks. To form the output video bitstream 120, the coding mode (inter or intra), prediction mode information, motion information, and quantized residual coefficients are all sent to an entropy coding unit (108) for further compression and packing to form the bitstream.

[0003] Figure 2 is a schematic block diagram of a block-based video decoder. A video bitstream 202 is first unpacked and entropy decoded by an entropy decoder 208. Coding mode and prediction information are sent to either a spatial predictor 260 (for intra-coding) or a temporal predictor 262 (for inter-coding) to form a prediction block. For inter-coding, the prediction information includes the size of the prediction block, one or more motion vectors (indicating the direction and amount of motion), and one or more reference indices (indicating the reference picture from which the prediction signal should be derived). Motion-compensated prediction is then applied by the temporal predictor 262 to form a temporal prediction block. Residual transform coefficients are sent to an inverse quantizer 210 and an inverse transformer 212 to reconstruct a residual block. The prediction block and residual block are then summed together at 226. The reconstructed block may undergo further in-loop filtering before being stored in a reference picture store 264. The reconstructed video in the reference picture store is then sent to drive a display device and is used to predict future video blocks. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] ITU-T Rec H.264 and ISO / IEC / MPEG 4 part 10, Advanced video coding for generic audiovisual services, November 2007 [Non-patent document 2] SMPTE 421M, “VC-1 Compressed Video Bitstream Format and Decoding Process,” April 2006 [Non-patent document 3] JM reference software JM18.2, located at hypertext transfer protocol, colon, slash-slash iphome.hhi.de / suehring / tml / download / jml8.2.zip,November,2011 [Non-patent document 4] B.Bross,W.-J.Han,J.-R.Ohm,GJSullivan,T.Wiegand.WD5: Working Draft 5 of High-Efficiency Video Coding.Document no CTVC-Gl 103,November 2011 [Non-Patent Document 5] K.McCann,S.Sekiguci,B.Bross,W.-J.Han,HM5: HEVC Test Model 5 Encoder Description.Document no JCTVC-G1102,December 2011 [Non-patent document 6] J.Boyce, R.Sjoberg, YKWang, BoG report: Reference picture buffering and list construction.Document no JCTVC-G1002,November 2011 [Non-Patent Document 7] D.Flynn,R.Sjoberg,et al,JCTVC AhG report: Reference picture buffering and list construction.Document no JCTVC-G021,November 2011 [Non-patent document 8] Y.Suzuki,et al,Extension of uni-prediction simplification in B slices.Document no JCTVC-D421 ,January 2011 [Non-Patent Document 9] B.Bross,W.-J.Han,J.-R.Ohm,GJSullivan,T.Wiegand.WD9: Working Draft 9 of High-Efficiency Video Coding.Document no JCTVC-K1103,October 2012 Summary of the Invention [Problem to be solved by the invention]

[0005] This specification describes a method and system that provides flexibility for improving the signaling of reference pictures used for temporal prediction (see block 162 in FIG. 1 and block 262 in FIG. 2). In particular, the signaling scheme and construction process for various reference picture lists of WD5 (HEVC Working Draft 5) (Non-Patent Documents 4, 5) are improved. [Means for solving the problem]

[0006] According to one embodiment, a method for generating reference picture lists L0 and L1 for decoding predicted pictures in video data comprises the steps of generating a first ordered list of reference pictures from a decoding picture buffer (DPB), the list being ordered such that, if there are reference pictures in the DPB that are temporally earlier than a current picture, those reference pictures are listed in order according to their temporal distance from the current picture, if there are reference pictures in the DPB that are temporally later than the current picture, those reference pictures are listed in order according to their temporal distance from the current picture, and if there are long-term reference pictures in the DPB, those reference pictures are listed in the order in which they are stored in the DPB; generating a second ordered list of reference pictures from the DPB, the list being ordered so that if there are reference pictures in the DPB that are temporally later than the current picture, those reference pictures are first listed in order according to their temporal distance from the current picture, if there are reference pictures temporally later than the current picture in the DPB, those reference pictures are listed in order according to their temporal distance from the current picture, and if there are long-term reference pictures in the DPB, those reference pictures are listed in the order in which they are stored in the DPB; and generating at least one of lists L0 and L1 by selecting reference pictures from the first ordered list and the second ordered list, respectively.

[0007] According to another embodiment, a method for initializing a reference picture list decoder for decoding a P or B slice header comprises: cIdx=0 NumRpsCurrTempList=NumRpsStCurr0+NumRpsStCurr1+NumRpsLtCurr for(i=0;i <NumRpsStCurr0;cIdx++,i++) RefPicSetCurrTempList0[cIdx]=RefPicSetStCurr0[i] for(i=0;i <NumRpsStCurr1;cIdx++,i++) RefPicSetCurrTempList0[cIdx]=RefPicSetStCurr1[i] for(i=0;i <NumRpsLtCurr;cIdx++,i++) RefPicSetCurrTempList0[cIdx]=RefPicSetLtCurr[i] The method includes constructing a first temporary list RefPicSetCurrTempList0 by:

[0008] According to yet another embodiment, a method for signaling a change in multiple reference picture lists comprises signaling a change in multiple reference picture lists using a unified signaling syntax.

[0009] According to another embodiment, the method comprises the steps of determining the number of entries in a reference picture list; and generating a message including a value identifying an entry in the reference picture list, the value being represented by one bit if the number of entries in the reference picture list is two, the value being represented by multiple bits if the number of entries in the reference picture list is more than two, and the message omitting the value if the number of entries in the reference picture list is one.

[0010] According to another embodiment, a method for creating a combined list of reference pictures LC used for decoding a B slice from a list of first reference pictures L0 and a list of second reference pictures L1 comprises the steps of determining whether L0 contains two or more entries, determining whether L1 contains two or more entries, and if either L0 or L1 contains two or more entries, indicating using the syntax element ref_idx_list_curr which entries of at least one of L0 and L1 should be added to the LC, setting ref_idx_list_curr to 0 if L0 contains only one entry, setting ref_idx_list_curr to 0 if L1 contains only one entry, and creating the LC using the value of ref_idx_list_curr.

[0011] A more detailed understanding may be had from the following description, given by way of example in conjunction with the accompanying drawings, in which: [Effects of the Invention]

[0012] A method and system are provided that provides flexibility for improving the signaling of reference pictures used in temporal prediction. [Brief explanation of the drawings]

[0013] [Figure 1] 1 is a block diagram of a block-based hybrid video coding scheme that can incorporate embodiments of the present invention; [Figure 2] FIG. 1 is a block diagram of a block-based video decoding scheme that can incorporate embodiments of the present invention; [Figure 3] FIG. 1 illustrates temporal unidirectional prediction from a reference picture store holding multiple reference pictures according to the prior art. [Figure 4] FIG. 1 illustrates prior art temporal bi-prediction using multiple reference pictures. [Figure 5]1 is a flowchart of a process for building a combined reference picture list according to the prior art; [Figure 6] FIG. 6 illustrates an example process for constructing a combined reference picture list according to the process described in connection with FIG. 5. [Figure 7] FIG. 1 illustrates an example of a modified process for constructing a combined reference picture list according to the prior art. [Figure 8] 1 is a flowchart of L0 and L1 reference picture list modification according to the prior art, using L0 as an example; [Figure 9] FIG. 9 illustrates an example of ref_pic_list_modification processing for L0 according to the processing described in connection with FIG. 8. [Figure 10] 1 is a flowchart of a reference picture list modification using L0 as an example according to an embodiment of the present invention; [Figure 11] FIG. 10 is an illustration of ref_pic_list_modification processing for the same example as FIG. 9, in accordance with the principles of one embodiment of the present invention. [Figure 12A] FIG. 1 is a system diagram of an example communication system in which one or more disclosed embodiments may be implemented. [Figure 12B] 12B is a system diagram of an example wireless transmit / receive unit (WTRU) that may be used within the communication system shown in FIG. 12A. [Figure 12C] 12B is a system diagram of an example radio access network and an example core network that may be used within the communication system shown in FIG. 12A. [Figure 12D] 12B is a system diagram of an example radio access network and an example core network that may be used within the communication system shown in FIG. 12A. [Figure 12E] 12B is a system diagram of an example radio access network and an example core network that may be used within the communication system shown in FIG. 12A. DETAILED DESCRIPTION OF THE INVENTION

[0014] As used in this specification, the terms "temporal prediction," "motion prediction," "motion compensated prediction," and "inter prediction" are used interchangeably, and the terms "reference picture store," "decoded picture buffer," and "DPB" are used interchangeably.

[0015] According to known techniques adopted in H.264 and HEVC WD 5, temporal prediction of video blocks can be performed using unidirectional or bidirectional prediction techniques. To perform such predictions, reference picture lists are signaled and constructed. In the case of unidirectional prediction, only one list of reference pictures is required from which blocks of the current picture are predicted. In the case of bidirectional prediction, there are two lists, L0 and L1, and one reference picture is selected from each list to form the prediction of blocks of the current picture. Furthermore, a bidirectional prediction technique has been proposed (though, as of the time of writing, it is no longer incorporated into the latest HEVC WD9 (Non-Patent Document 9)) that combines the first two lists, L0 and L1, and uses a third list, referred to herein as the "list LC." Described herein are methods and systems for an efficient and unified technique for signaling the syntax of modifications to all reference picture lists L0, L1, and / or LC, as well as a technique for signaling the combined reference picture list LC.

[0016] FIG. 3 is a diagram that diagrammatically illustrates unidirectional prediction with a single reference picture list 301, which may be performed by an inter-prediction processing unit (e.g., block 162 of FIG. 1). According to the unidirectional prediction technique, reference picture list 301 predicts a current video block by including links to video blocks, e.g., block 304, in neighboring, already-encoded video frames, thus exploiting temporal correlation and eliminating temporal redundancy inherent in video signals. Such already-encoded video frames are stored in a decoded picture buffer (DPB, e.g., reference picture store 164 of FIG. 1). In H.264 / AVC and HEVC WD5, more than one reference picture may be used. In FIG. 3, ref n A list of N reference pictures 303, denoted (n=0...N-1), can be used to predict a video block 307 of a current picture 305. A motion vector (mvx, mvy) is used as a reference to predict the current block 307. m is selected. The time prediction is performed as follows. P(x,y)=ref m (x-mvx, y-mvy) Equation (1) ref m (x,y) is the reference picture ref m where P(x,y) is the pixel value at position (x,y) in the block to be predicted. Existing video coding systems can support inter prediction with fractional pixel accuracy (Non-Patent Documents 1, 2, 4). When a motion vector (mvx,mvy) has fractional pixel values, an interpolation filter is applied to obtain pixel values ​​at the fractional pixel positions.

[0017] In equation (1), the time prediction is based on one source (i.e., ref n ), which is commonly called unidirectional prediction. A picture or slice (a group of video blocks) in which all blocks in that picture or slice are predicted using unidirectional prediction is commonly called a P picture or P slice.

[0018] To improve the accuracy of temporal prediction, newer block-based video coding systems also support multi-hypothesis prediction, in which multiple prediction signals from different reference pictures are combined to form a prediction signal. A commonly used form of multi-hypothesis prediction is called bidirectional prediction, in which two prediction signals, each from a picture in a different reference picture list, are combined to form a prediction for the current block. Figure 4 helps explain bidirectional prediction. In particular, two reference picture lists, List 0 401 and List 1 403, are used to predict video blocks in the current picture. List 0 contains a total of N0 pictures 404, and List 1 contains a total of N1 pictures 404. In Figure 4, ref 0 401 in List 0 401 with motion vector (mvx0,mvy0) is used to predict a video block in the current picture. m0 and the ref in list 1 403 with motion vectors (mvx1,mvy1) m1 are selected to form the bidirectional prediction of the prediction block 410 of the current picture 412 as in equation (2).

[0019]

number

[0020] where P0(x,y) and P1(x,y) are the first and second predicted blocks 407 and 408, respectively. A picture or slice is typically called a B-picture or B-slice if at least some of the blocks in that picture or slice are predicted using bidirectional prediction (other blocks can be predicted using unidirectional prediction). Bidirectional prediction is supported in all modern video coding standards, including MPEG2 / 4, VC1, H.264, and HEVC.

[0021] After prediction, the prediction block P(x,y) is subtracted from the original video block in a first adder (see 116 in FIG. 1 ) to form a prediction residual block. The prediction residual block is transformed in a transform unit 104 and quantized in a quantizer 106. The quantized residual transform coefficient block is then sent to an entropy coder 108 for entropy coding, thereby further reducing the bit rate. The entropy-coded residual coefficients are then packed to form part of an output video bitstream 120.

[0022] The structure of the reference picture list for P pictures / slices is relatively simple because all blocks are predicted using unidirectional prediction, i.e., only one reference picture list is required. Meanwhile, in B pictures / slices, some blocks may be predicted using bidirectional prediction, while other blocks may be predicted using unidirectional prediction. In HEVC, the reference picture lists for bidirectional prediction, i.e., list 0 (or L0) 401 and list 1 (or L1) 403 in FIG. 4, are the same as those in H.264 / AVC. However, HEVC differs from H.264 / AVC in the way in which reference picture lists for unidirectional prediction are formed for B pictures / slices. In H.264 / AVC, unidirectional prediction for a video block in a B picture / slice must first indicate whether the prediction is made from L0 or L1, and then indicate the ref_idx of that particular list. In HEVC, the concept of a joint reference picture list was presented at the 4th JCT-VC meeting (Non-Patent Document 8). A combined list, referred to in this disclosure as "LC," is formed by combining L0 and L1 together, and LC serves as the only reference picture list for all blocks predicted using unidirectional prediction within a B picture / slice.

[0023] Currently, in HEVC, by default, the combined list LC is formed by alternately taking non-overlapping pictures from L0 and L1, thereby ensuring minimal redundancy in the combined list. A flowchart of the default combined list generation is shown in FIG. 5. Specifically, indices i, j, and k, which refer to lists L0, L1, and LC, respectively, are initialized in 501, and two lists L0 and L1 are also initialized. In decision block 503, it is determined whether all reference pictures in L0 have been examined. If not, the process proceeds to decision block 505, where it is determined whether the reference picture at index i in L0 is already in the combined list LC. If it is not already in the combined list LC, it is added to list LC, and the index of the combined list LC is incremented (507). Index i is also incremented (509). On the other hand, if the reference picture at index i in L0 is already in the combined list LC, the process proceeds directly from 505 to 509. Essentially the same process is then performed for the reference picture at index j in list L1. Specifically, in decision block 511, it is determined whether all reference pictures in L1 have been examined. If not, the process proceeds to decision block 513, which determines whether the reference picture at index j in L1 is already in the combined list LC. If it is not already in list LC, it is added, and the LC index is incremented (515). Index j in L1 is also incremented (517). However, if the reference picture at index j in L1 is already in LC, the process proceeds directly from 513 to 517. As can be seen in decision block 519, this process is repeated by alternately examining the next reference picture in each of lists L0 and L1 until the end of the two lists is reached.

[0024] FIG. 6 shows an example of a combined list LC created by the process described in the flowchart of FIG. 5. In this example, the current picture to be coded is temporally between reference pictures 2 and 4. L0 includes reference pictures Ref2, Ref1, and Ref4, in that order, and L1 includes reference pictures Ref4, Ref5, and Ref2, in that order. Following the flow of FIG. 5, the example of FIG. 6 forms a combined list LC by alternately checking whether each of the three reference pictures in L0 and L1 is already in LC and adding all reference pictures that are not already in LC. As a result, in the example of FIG. 6, a combined list LC including four reference pictures is formed by sequentially adding the first reference picture in L0 (Ref2), the first reference picture in L1 (Ref4), the second reference picture in L0 (Ref1), and the second reference picture in L1 (Ref5) to LC. The third reference picture in L0 (Ref4) is skipped because it is the same picture as the first reference picture in L1 and has already been added to the LC, and the third reference picture in L1 (Ref2) is skipped because it is the same picture as the first reference picture in L0 and has already been added to the LC.

[0025] Note that the coding order of the reference pictures in each list L0, L1, and LC in Figure 6 differs from display order because reference pictures Ref4 and Ref5 (which are later in display order) are coded before the current picture. This default process of constructing LC by traversing lists L0 and L1 ensures that each entry in LC represents a unique picture in the coded video sequence, thus minimizing redundancy.

[0026] Since this default handling does not support reordering of reference pictures (i.e., having a different list size than the default list size, ordering the list entries differently than the default handling, repeating some entries in the list, and / or removing some entries from the list, etc.), additional syntax elements are used in HEVC WD5 (see Table 1 below) to support modification of the combined list LC. Figure 7 shows two examples of combined list modification, the first showing a reordered LC and the second showing an LC with repeated entries and a modified LC size (3 entries) different from the default LC size (4 entries). In HEVC WD5, the combined reference picture list LC is signaled using the syntax table in Table 1.

[0027] [Table 1]

[0028] The meaning of the reference picture list combination is as follows. When ref_pic_list_combination_flag is equal to 1, it means that reference picture list 0 and reference picture list 1 are combined to generate an additional combined reference picture list to be used for a unidirectionally predicted block or other prediction unit. When this flag is equal to 0, it means that reference picture list 0 and reference picture list 1 are identical, and therefore reference picture list 0 can be used as the combined reference picture list. The combined reference picture list is set to empty at the start of the loop defined in Table 1. num_ref_idx_lc_active_minus1+1 specifies the number of reference pictures selected from reference picture list 0 or reference picture list 1 in the combined reference picture list. ref_pic_list_modification_flag_lc equal to 1 indicates the presence of syntax elements pic_from_list_0_flag and ref_idx_list_curr to specify the correspondence between entries in the spliced ​​reference picture list and entries in reference picture list 0 and reference picture list 1. ref_pic_list_modification_flag_lc equal to 0 indicates the absence of these syntax elements. The combined reference picture list is initialized as specified in subclause 8.2.2.4 of HEVC WD 5. pic_from_list_0_flag indicates whether the current reference picture being added to the combined reference picture list is a picture from reference picture list 0 or reference picture list 1. If this flag is equal to 1, it is a picture from reference picture list 0 and CurrRefPicList is reference picture list 0. If this flag is equal to 0, it is a picture from reference picture list 1 and CurrRefPicList is reference picture list 1. ref_idx_list_curr indicates the reference index of the picture in CurrRefPicList that should be appended to the end of the concatenation of reference picture lists.

[0029] Modifications can be made to the reference picture lists L0 and L1. To allow flexibility in the use of reference picture lists L0 and L1, default and modified construction processes are also supported in HEVC. The current reference picture list construction and modification process for L0 and L1 was presented at the 7th JCT-VC meeting in November 2011 (Non-Patent Documents 6 and 7) and adopted in HEVC WD5 (Non-Patent Document 4). The syntax for reference picture list modification of List 0 and List 1 in HEVC WD5 is shown in Table 2 below and in flowchart form in Figure 8.

[0030] [Table 2]

[0031] The meaning of the reference picture list change is as follows:

[0032] The syntax elements list_modification_idc and ref_pic_set_idx specify the modifications from the initial reference picture list to the reference picture list used to decode the slice.

[0033] ref_pic_list_modification_flag_l0 equal to 1 indicates the presence of the syntax element list_modification_idc specifying reference picture list 0, and ref_pic_list_modification_flag_l0 equal to 0 indicates the absence of that syntax element. When ref_pic_list_modification_flag_l0 equals 1, the number of times list_modification_idc is not equal to 3 after ref_pic_list_modification_fiag_l0 must not exceed num_ref_idx_l0_active_minus1+1.

[0034] ref_pic_list_modification_flag_l1 equal to 1 indicates the presence of the syntax element list_modification_idc specifying reference picture list 1, while ref_pic_list_modification_flag_l1 equal to 0 indicates the absence of that syntax element. If ref_pic_list_modification_flag_l1 equals 1, the number of times list_modification_idc is not equal to 3 after ref_pic_list_modification_flag_l1 must not exceed num_ref_idx_l1_active_minus1+1.

[0035] list_modification_idc, together with ref_pic_set_idx, specifies which reference picture to re-map. Values ​​of list_modification_idc are specified in Table 3. The value of the first list_modification_idc immediately following ref_pic_list_modification_flag_l0 or ref_pic_list_modification_flag_l1 must not be equal to 3.

[0036] [Table 3]

[0037] ref_pic_set_idx specifies the index of RefPicSetStCurr0, RefPicSetStCurr1, or RefPicSetLtCurr of the reference picture referenced by the current index in the reference picture list. The value of ref_pic_set_idx is in the range from 0 to max_num_ref_frames.

[0038] Figure 8 shows a flowchart of the process of changing the reference picture lists of L0 and L1, using L0 as an example. Detailed change processes for L0 and L1, including definitions of the reference picture sets (RefPicSetStCurr0, RefPicSetStCurr1, and RefPicSetLtCurr), can be found in the working draft sections of HEVC WD5 (Non-Patent Document 4) and Non-Patent Documents 6 and 7. The reference picture sets in Figure 8 are briefly defined below. RefPicSetStCurr0: Short-term reference pictures that are earlier in display order, i.e., before the current picture (e.g., Ref1 and Ref2 in Figure 6) RefPicSetStCurr1: Short-term reference pictures that are later in display order, i.e., later than the current picture (e.g., Ref4 and Ref5 in Figure 6) RefPicSetLtCurr: Long-term reference picture (not shown in Figure 6)

[0039] In 801, the index of list L0 is initialized to zero. In 803, ref_modification_idc is read. ref_modification_idc can have four values: 0, 1, 2, and 3. A value of 3 means that no further modifications will be made and the modification process may end. (The intended modifications signaled in ref_modification_idc having a value of 0, 1, or 2 are described below in relation to steps 811, 813, and 815.) Thus, in decision step 805, if ref_modification_idc is set to 3, no further syntax is read. Otherwise, in 807, ref_pic_set_idx is read. This is an index that points to one of the three picture sets in the DPB (i.e., the set of pictures "before" the current picture being decoded, the set of pictures "after," or the long-term set of pictures). (The selection of a particular one of the three sets occurs in steps 811, 813, and 815, as described below.) In decision step 809, it is determined whether ref_modification_idc is 0, 1, or 2. If so, then in 811 the entry of list L0 at RefIdxL0, the current index into list L0, is set to the earlier short-term reference picture in display order at position ref_pic_set_idc in the set of short-term temporally previous reference pictures in the DPB (i.e., RefPicSetStCurr0). Otherwise, if so, then in 813 the entry of list L0 at RefIdxL0, the current index into list L0, is set to the later short-term reference picture in display order than the current picture to be coded, at position ref_pic_set_idc in the set of short-term temporally later reference pictures in the DPB (i.e., RefPicSetStCurr1). Finally, for case 2, at 815, the entry in list L0 at RefIdxL0, the current index into list L0, is set to the long-term reference picture at position ref_pic_set_idc in the set of long-term reference pictures in the DPB (i.e., RefPicSetLtCurr).

[0040] In all three cases, the process then proceeds to 817, where all entries in list L0 that follow the just-modified entry and that refer to the same picture as the just-modified entry are removed from L0. In 819, the index of list L0 is incremented and the process returns to 803. This process continues until ref_modification_idc has a value of 3, indicating that no further modifications are to be made.

[0041] Again using L0 as an example, Figure 9 illustrates the results of the reference picture list modification process outlined in the flowchart of Figure 8 for a DPB holding (1) a short-term temporally earlier reference picture set, i.e., reference pictures Ref2 and Ref1 (in that order) in RefPicSetStCurr0, and (2) a short-term temporally later reference picture set, i.e., reference pictures Ref4 and Ref5 (in that order) in RefPicSetStCurr1. For simplicity and without loss of generality, the example of Figure 9 does not consider RefPicSetLtCurr, which is related to the use of long-term reference pictures, but only considers the use of short-term reference pictures indicated by RefPicSetStCurr1 and RefPicSetStCurr1.

[0042] As shown in Figure 9, the default list L0 consists of reference pictures Ref2, Ref1, Ref4, and Ref5 (in that order). In the example of Figure 9, a simple modification of the last entry of L0 is desired. The processing of Figure 8 requires looping through steps 803-819 once for each entry of L0, including the first three entries that do not require modification, and signaling ref_modification_idc and ref_pic_set_idx for each entry, and then further signaling that the processing is finished by further signaling another ref_modification_idc with a value of 3. Thus, five steps are used to arrive at the target modification list L0. At each step except the last, two syntax elements (list_modification_idc and ref_pic_set_idx) are signaled, and an additional variable RefIdx is maintained and incremented.

[0043] Furthermore, comparing the handling of reference picture list modification for LC (Table 1 above) with the handling of L0 / L1 modification (Tables 2 and 3 above), it should be noted that the handling of LC modification in HEVC WD5 is different from that of L0 and L1. In particular, the handling of LC modification is simpler because each entry of the modified LC is explicitly signaled, instead of signaling two syntax elements (list_modification_idc and ref_pic_set_idx) for each entry in that particular list.

[0044] A method is described herein that unifies these list modification processes to provide a simpler modification process for L0 and L1 that requires less signaling.

[0045] In one embodiment, a method is provided for improving the efficiency of the process of combining reference picture lists. Table 4 shows pseudocode for forming a combined reference picture list according to one embodiment of the present invention. Changes from Table 1 (pseudocode for the HEVC WD5 method of forming the combined list LC) are indicated with an asterisk.

[0046] [Table 4]

[0047] Note that the syntax ref_idx_list_curr is only signaled when L0 (if pic_from_list_0_flag is 1) or L1 (if pic_from_list_1_flag is 0) contains more than one entry. This is because nothing needs to be sent if the corresponding list (L0 or L1) contains only one entry. Thus, the amount of signaling is reduced.

[0048] Also, instead of using ue(v), te(v) is a more efficient means of signaling ref_idx_list_curr. This is because the entropy coding method te(v) (subclause 9.1 of H.264) was specifically designed for encoding syntax elements like ref_idx. Ue(v) (known as Exponential-Golomb coding) can send the value 1 using 3 bits. On the other hand, te(v) is used to first determine the number of possible values ​​in ref_idx_list_curr (by examining L0 and L1), and if there are only two values, the syntax element can be sent using 1 bit. If there are more than two values, ue(v) can be used.

[0049] That is, if a syntax element is coded as te(v), the range of possible values ​​for the syntax element is first determined. If the range of possible values ​​for the syntax element is 0 to 1, then the syntax element is coded using only one bit, thereby saving signaling overhead. Otherwise, if the range of the syntax element is between 0 and x (x>1), then the syntax element is coded using ue(v).

[0050] Thus, the system makes the decision based on the possible values ​​of ref_idx_list_curr. If the syntax element ref_idx_list_curr has only one possible value, nothing is transmitted because both the encoder and decoder can determine its value based on the other values. If the syntax element ref_idx_list_curr has two possible values, one bit is transmitted. Otherwise, if the syntax element ref_idx_list_curr has more than two possible values, ue(v) is used to encode ref_idx_list_curr.

[0051] Therefore, signaling overhead savings are realized compared to HEVC WD5.

[0052] In yet another embodiment, a single unified reference picture list modification process is disclosed that can be used to modify L0 and L1. According to this embodiment, the L0 and L1 reference picture list modification process uses the syntax shown in Table 5. Changes compared to the pseudocode in Table 2 (i.e., the reference picture list modification syntax for list 0 and list 1 in HEVC WD5) are indicated with an asterisk.

[0053] [Table 5]

[0054] The meaning of the reference picture list change is as follows: The syntax element ref_pic_set_idx is used to specify the change from the initial reference picture list to the modified reference picture list. When ref_pic_list_modification_flag_l0 is equal to 1, it indicates that the syntax element ref_pic_set_idx is present to specify reference picture list 0. When ref_pic_list_modification_flag_l0 is equal to 0, it indicates that this syntax element is not present. When ref_pic_list_modification_flag_l1 is equal to 1, it indicates the presence of the syntax element ref_pic_set_idx that specifies reference picture list 1. When ref_pic_list_modification_flag_l1 is equal to 0, it indicates that this syntax element is not present.

[0055] ref_pic_set_idx specifies that the index of the picture in RefPicSetCurrTempListX should be placed at the current position in the reference picture list LX (X is 0 if it relates to list L0, 1 if it relates to list L1). The syntax ref_pic_set_idx must be in the range 0 to max_num_ref_frames-1 inclusive within list LX. If the syntax element ref_pic_set_idx is not present, it is set to 0.

[0056] This new process significantly reduces signaling in some cases (and in fact probably in most cases). Briefly, instead of signaling for each list entry the type of change to make and the DPB index of the reference picture to use, as in the syntax of Table 2 and the flowchart of Figure 8, the process of the present invention signals only the index into the DPB and does not require an additional signal to indicate the end of the list change process.

[0057] The process disclosed in Table 5 above uses an intermediate list of reference pictures for L0 and / or L1, respectively, RefPicSetCurrTempListX, where X represents 0 or 1 depending on which modified list is considered. This scheme provides a modified initialization process for the reference picture list. This initialization process is invoked when decoding the header of a P or B slice. When decoding a P or B slice, there may be at least one reference picture in RefPicSetStCurr0, RefPicSetStCurr1, or RefPicSetLtCurr.

[0058] The following steps are taken to construct RefPicSetCurrTempList0: cIdx=0 NumRpsCurrTempList=NumRpsStCurr0+NumRpsStCurr1+NumRpsLtCurr for(i=0;i <NumRpsStCurr0;cIdx++,i++) RefPicSetCurrTempList0[cIdx]=RefPicSetStCurr0[i] for(i=0;i <NumRpsStCurr1;cIdx++,i++) RefPicSetCurrTempList0[cIdx]=RefPicSetStCurr1[i] for(i=0;i <NumRpsLtCurr;cIdx++,i++) RefPicSetCurrTempList0[cIdx]=RefPicSetLtCurr[i]

[0059] If ref_pic_list_modification_flag_l0 is 0, no changes are made to the default list L0, and the default RefPicList0 is constructed by sequentially taking the first num_ref_idx_l0_active_minus1+1 entries from RefPicSetCurrTempList0. On the other hand, if ref_pic_list_modification_flag_l0 is 1, the operation in Table 5 is called to modify the reference picture list L0, with RefPicSetCurrTempList0 and num_ref_idx_l0_active_minus1 as inputs and RefPicList0(L0) as output.

[0060] Briefly, the above pseudocode finds the number of reference pictures in the DPB (i.e., NumRpsCurrTempList) by adding the number of "previous" pictures, "after" pictures, and long-term pictures, and then arranges the "previous" pictures first (in order of closest to farthest temporal distance from the current picture), then the "after" pictures (also in order of closest to farthest temporal distance from the current picture), then the long-term reference pictures.

[0061] The following steps are performed to construct RefPicSetCurrTempList1: cIdx=0 NumRpsCurrTempList=NumRpsStCurr0+NumRpsStCurr1+NumRpsLtCurr for(i=0;i <NumRpsStCurr1;cIdx++,i++) RefPicSetCurrTempList1[cIdx]=RefPicSetCurr1[i] for(i=0;i <NumRpsStCurr0;cIdx++,i++) RefPicSetCurrTempList1[cIdx]=RefPicSetCurr0[i] for(i=0;i <NumRpsLtCurr;cIdx++,i++) RefPicSetCurrTempList1[cIdx]=RefPicSetLtCurr[i]

[0062] If ref_pic_list_modification_flag_l1 is 0, no modifications are made to the default list L1, and a default RefPicList1 is constructed by taking the first num_ref_idx_l1_active_minus1+1 entries from RefPicSetCurrTempList1, whereas if ref_pic_list_modification_flag_l1 is 1, the modification process in Table 5 is called to modify the reference picture list L1 with RefPicSetCurrTempList1 and num_ref_idx_l1_active_minus1 as inputs and RefPicList1 as output.

[0063] Briefly, the above pseudocode finds the number of reference pictures in the DPB (i.e., NumRpsCurrTempList) by adding the number of "previous" pictures, "later" pictures, and long-term pictures, and then arranges the "later" pictures first (in order of closest to furthest temporal distance from the current picture), then the "previous" pictures (also in order of closest to furthest temporal distance from the current picture), then the long-term reference pictures.

[0064] Note that creating two lists RpsCurrTempLX is beneficial even if no changes are made to the reference picture lists L0 and L1, because in such a case the first few entries of RpsCurrTempLX are already in the default order for lists L0 and L1, respectively, and so the reference picture lists L0 and L1 can be created very easily by simply taking those entries.

[0065] The reference picture list modification process reflected in Table 5 accepts as input the above reference picture array RefPicSetCurrTempLX and the reference picture list size num_ref_idx_lX_active_minus1 (where X is 0 or 1 depending on the list being modified). The output of this process is an array containing the modified reference picture list RefPicListX.

[0066] FIG. 10 is a flowchart illustrating the list modification process of Table 5 for an exemplary list L0. The process is similar for list L1. At 1001, the index of list L0 is initialized to zero. At 1003, it is determined whether the temporary list RefPicSetCurrTempL0 contains more than one entry. This is because signaling ref_pic_set_idx is unnecessary if the list contains only one entry. If the list contains only one entry, proceed to 1004, where ref_pic_set_idx is not signaled, but instead defaults to 0. Otherwise, proceed to 1005, where the index into the intermediate list RefPicSetCurrTempList0, ref_pic_set_idx, is read. At 1007, the entry at the current index of modification list L0 is set to the value at the signaled index position ref_pic_set_idx in the RefPicSetCurrTempList0 list. The index of the modified list L0 is then incremented 1009. At 1011 it is determined whether the end of L0 has been reached. If not, the process returns to 1003. If so, the process ends.

[0067] As mentioned above, if no list change is requested, the process of FIG. 10 is not performed, and the first num_ref_idx_lx_active_minus1+1 entries of RefPicSetCurrTempListX simply become the corresponding list LX.

[0068] Figure 11 illustrates how the proposed reference picture list scheme of the present invention works using the same example as Figure 9. Comparing Figure 11 with Figure 9, the modification process in Figure 11 uses half the number of syntax elements as in Figure 9, i.e., instead of signaling ref_pic_set_idx and list_modification_idc, only ref_pic_set_idx is signaled for each entry in list L0. Furthermore, the process described in the flowchart of Figure 10 is simpler than the process in the flowchart of Figure 8 in that it explicitly signals each entry in the list and does not require the complex process of Figure 8.

[0069] The systems and methods described herein are suitable for communicating video streams over both wired and wireless networks. Wired networks are well known. An overview of various wireless devices and infrastructures is provided in connection with Figures 12A-12B, where various elements of the network may utilize the systems and methods described herein. More specifically, base stations such as base transceiver stations (BTSs), Node-Bs, eNodeBs, Home NodeBs, Home eNodeBs, site controllers, access points (APs), wireless routers, media aware network elements (MANEs), and wireless transmit / receive units (WTRUs) may generate and / or process the signaling described above to convey encoded video data from one entity to another.

[0070] 12A is a diagram of an example communication system 100 in which one or more disclosed embodiments may be implemented. The communication system 100 may be a multiple-access system that provides content, such as voice, data, video, messaging, broadcast, etc., to multiple wireless users. The communication system 100 may enable the multiple wireless users to access such content through the sharing of system resources, such as radio bandwidth. For example, the communication system 100 may use one or more channel access methods, such as code division multiple access (CDMA), time division multiple access (TDMA), frequency division multiple access (FDMA), orthogonal FDMA (OFDMA), single-carrier FDMA (SC-FDMA), etc.

[0071] 12A , communications system 100 includes wireless transmit / receive units (WTRUs) 102a, 102b, 102c, and / or 102d, a radio access network (RAN) 104, a core network 106, a public switched telephone network (PSTN) 108, the Internet 110, and other networks 112, although it will be understood that the disclosed embodiments contemplate any number of WTRUs, base stations, networks, and / or network elements. Each WTRU 102a, 102b, 102c, 102d may be any type of device configured to operate and / or communicate in a wireless environment. For example, WTRUs 102a, 102b, 102c, 102d may be configured to transmit and / or receive wireless signals and may include user equipment (UE), mobile stations, fixed or mobile subscriber units, pagers, mobile phones, personal digital assistants (PDAs), smartphones, laptops, netbooks, personal computers, wireless sensors, consumer electronics devices, etc.

[0072] The communications system 100 may also include a base station 114a and a base station 114b. Each base station 114a, 114b may be any type of device configured to wirelessly interface with at least one of the WTRUs 102a, 102b, 102c, 102d to facilitate access to one or more communications networks, such as the core network 106, the Internet 110, and / or the network 112. For example, the base stations 114a, 114b may be a base transceiver station (BTS), a NodeB, an eNodeB, a Home NodeB, a Home eNodeB, a site controller, an access point (AP), a wireless router, etc. Although the base stations 114a, 114b are illustrated as single elements, it will be understood that the base stations 114a, 114b may include any number of interconnected base stations and / or network elements.

[0073] The base station 114a may be part of the RAN 104, which may also include other base stations and / or network elements (not shown), such as a base station controller (BSC), a radio network controller (RNC), relay nodes, etc. The base station 114a and / or base station 114b may be configured to transmit and / or receive wireless signals within a particular geographic area, also referred to as a cell (not shown). A cell may be further divided into cell sectors. For example, the cell associated with the base station 114a is divided into three sectors. Thus, in one embodiment, the base station 114a may include three transceivers, i.e., one transceiver for each sector of the cell. In another embodiment, the base station 114a may employ multiple-input multiple-output (MIMO) technology and utilize multiple transceivers for each sector of the cell.

[0074] The base stations 114a, 114b may communicate with one or more of the WTRUs 102a, 102b, 102c, 102d over an air interface 116, which may be any suitable wireless communication link (e.g., radio frequency (RF), microwave, infrared (IR), ultraviolet (UV), visible light, etc.). The air interface 116 may be established using any suitable radio access technology (RAT).

[0075] More specifically, as noted above, the communication system 100 may be a multiple-access system and may use one or more channel access schemes, such as CDMA, TDMA, FDMA, OFDMA, SC-FDMA, etc. For example, the base station 114a and the WTRUs 102a, 102b, 102c in the RAN 104 may implement a radio technology such as Integrated Mobile Telecommunications System (UMTS) Terrestrial Radio Access (UTRA), in which case the air interface 116 may be established using Wideband CDMA (WCDMA). WCDMA may include communication protocols such as High Speed ​​Packet Access (HSPA) and / or Evolved HSPA (HSPA+). HSPA may include High Speed ​​Downlink Packet Access (HSDPA) and / or High Speed ​​Uplink Packet Access (HSUPA).

[0076] In another embodiment, the base station 114a and the WTRUs 102a, 102b, 102c may implement a radio technology such as Evolved UMTS Terrestrial Radio Access (E-UTRA), in which case the air interface 116 may be established using LTE and / or LTE-Advanced (LTE-A).

[0077] In other embodiments, the base station 114a and the WTRUs 102a, 102b, 102c may implement a radio technology such as IEEE 802.16 (i.e., WiMAX), CDMA2000, CDMA2000 1X, CDMA2000EV-DO, Interim Standard 2000 (IS-2000), Interim Standard 95 (IS-95), Interim Standard 856 (IS-856), Global System for Mobile Communications (GSM), Enhanced Data rates for GSM Evolution (EDGE), GSM EDGE (GERAN), or the like.

[0078] The base station 114b in FIG. 12A may be, for example, a wireless router, a Home NodeB, a Home eNodeB, or an access point, and may utilize an appropriate RAT that facilitates wireless connectivity within a limited area, such as a workplace, a residence, a vehicle, a campus, or the like. In one embodiment, the base station 114b and the WTRUs 102c, 102d may implement a wireless technology such as IEEE 802.11 to establish a wireless local area network (WLAN). In another embodiment, the base station 114b and the WTRUs 102c, 102d may implement a wireless technology such as IEEE 802.15 to establish a wireless personal area network (WPAN). In yet another embodiment, the base station 114b and the WTRUs 102c, 102d may establish a picocell or femtocell utilizing a cellular RAT (e.g., WCDMA, CDMA2000, GSM, LTE, LTE-A, etc.). As shown in FIG. 12A, the base station 114b may have a direct connection to the Internet 110. Therefore, the base station 114 b may not need to access the Internet 110 via the core network 106 .

[0079] The RAN 104 is in communication with a core network 106, which may be any type of network configured to provide voice, data, application, and / or Voice over Internet Protocol (VoIP) services to one or more of the WTRUs 102a, 102b, 102c, 102d. For example, the core network 106 may provide call control, billing services, mobile location services, prepaid calling, Internet connectivity, video distribution, etc., and / or perform high-level security functions such as user authentication. Although not shown in FIG. 12A , it will be understood that the RAN 104 and / or core network 106 may communicate directly or indirectly with other RANs that use the same RAT as the RAN 104 or a different RAT. For example, in addition to being connected to the RAN 104 that utilizes E-UTRA radio technology, the core network 106 may also be in communication with another RAN (not shown) that uses GSM radio technology.

[0080] The core network 106 may also serve as a gateway for the WTRUs 102a, 102b, 102c, 102d to access the PSTN 108, the Internet 110, and / or other networks 112. The PSTN 108 may include a circuit-switched telephone network providing plain old telephone service (POTS). The Internet 110 may include a worldwide system of interconnected computer networks and devices that use common communication protocols, such as TCP, UDP, and IP of the TCP / IP Internet protocol suite. The network 112 may include wired or wireless communication networks owned and / or operated by other service providers. For example, the network 112 may include another core network connected to one or more RANs that use the same RAT as the RAN 104 or a different RAT.

[0081] Some or all of the WTRUs 102a, 102b, 102c, 102d in the communication system 100 may have multi-mode capabilities. That is, the WTRUs 102a, 102b, 102c, 102d may include multiple transceivers for communicating with different wireless networks over different wireless links. For example, the WTRU 102c shown in FIG. 12A may be configured to communicate with a base station 114a that uses cellular wireless technology and a base station 114b that uses IEEE 802 wireless technology.

[0082] 12B is a system diagram of an example WTRU 102. As shown in FIG. 12B, the WTRU 102 may include a processor 118, a transceiver 120, a transmit / receive element 122, a speaker / microphone 124, a keypad 126, a display / touchpad 128, non-removable memory 130, removable memory 132, a power source 134, a GPS chipset 136, and other peripheral functions 138. It will be understood that the WTRU 102 may include any sub-combination of the above elements while remaining consistent with an embodiment.

[0083] The processor 118 may be a general-purpose processor, a special-purpose processor, a conventional processor, a digital signal processor (DSP), multiple microprocessors, one or more microprocessors in conjunction with a DSP core, a controller, a microcontroller, an ASIC, an FPGA circuit, any other type of integrated circuit (IC), a state machine, etc. The processor 118 may perform signal coding, data processing, power control, input / output processing, and / or other functions that enable the WTRU 102 to operate in a wireless environment. The processor 118 may be coupled to the transceiver 120, which may be coupled to the transmit / receive element 122. While the processor 118 and the transceiver 120 are shown in FIG. 12B as separate components, it will be understood that the processor 118 and the transceiver 120 may be integrated together in an electronic package or chip.

[0084] The transmit / receive element 122 can be configured to transmit or receive signals to or from a base station (e.g., base station 114a) over the air interface 116. For example, in one embodiment, the transmit / receive element 122 can be an antenna configured to transmit and / or receive RF signals. In another embodiment, the transmit / receive element 122 is an emitter / detector configured to transmit and / or receive, for example, IR, UV, or visible light signals. In yet another embodiment, the transmit / receive element 122 can be configured to transmit and / or receive both RF and light signals. It will be understood that the transmit / receive element 122 may be configured to transmit and / or receive any combination of various wireless signals.

[0085] 12B depicts the transmit / receive element 122 as a single element, the WTRU 102 may include any number of transmit / receive elements 122. More specifically, the WTRU 102 may employ MIMO techniques. As such, in one embodiment, the WTRU 102 may include two or more transmit / receive elements 122 (e.g., multiple antennas) for transmitting and receiving wireless signals over the air interface 116.

[0086] The transceiver 120 may be configured to modulate signals to be transmitted from the transmit / receive element 122 and demodulate signals received by the transmit / receive element 122. As mentioned above, the WTRU 102 may have multi-mode capabilities. As such, the transceiver 120 may include multiple transceivers that enable the WTRU 102 to communicate via multiple types of RATs, such as UTRA and IEEE 802.11.

[0087] The processor 118 of the WTRU 102 is coupled to and may receive user input data from a speaker / microphone 124, a keypad 126, and / or a display / touchpad 128 (e.g., a liquid crystal display (LCD) or an organic light emitting diode (OLED) display). The processor 118 may also output user data to the speaker / microphone 124, the keypad 126, and / or the display / touchpad 128. The processor 118 may also access information and store data in any type of suitable memory, such as non-removable memory 130 and / or removable memory 132. The non-removable memory 130 may include RAM, ROM, a hard disk, or any other type of memory storage device. The removable memory 132 may include a subscriber identity module (SIM) card, a memory stick, a secure digital (SD) memory card, etc. In other embodiments, the processor 118 may access information and store data in memory that is not physically located in the WTRU 102, such as a server or home computer (not shown).

[0088] The processor 118 may be configured to receive power from the power source 134 and distribute and / or control the power to other components in the WTRU 102. The power source 134 may be any device suitable for providing power to the WTRU 102. For example, the power source 134 may include one or more dry cell batteries (e.g., nickel-cadmium (NiCd), nickel-zinc (NiZn), nickel-metal hydride (NiMH), lithium-ion (Li-ion), etc.), solar cells, fuel cells, etc.

[0089] The processor 118 is also coupled to the GPS chipset 136, which may be configured to provide location information (e.g., longitude and latitude) regarding the current location of the WTRU 102. In addition to or instead of information from the GPS chipset 136, the WTRU 102 may receive location information from a base station (e.g., base stations 114a, 114b) over the air interface 116 and / or determine its location based on the timing of signals received from two or more nearby base stations. It will be appreciated that the WTRU 102 may obtain location information by any suitable location-determination method while remaining consistent with an embodiment.

[0090] The processor 118 may further be coupled to other peripheral functions 138, which may include one or more software and / or hardware modules that provide additional features, functionality, and / or wired or wireless connectivity. For example, the peripheral functions 138 may include an accelerometer, an electronic compass, a satellite transceiver, a digital camera (for photos or videos), a Universal Serial Bus (USB) port, a vibration device, a television transceiver, a hands-free headset, a Bluetooth module, a frequency modulation (FM) radio unit, a digital music player, a media player, a video game player module, an internet browser, etc.

[0091] FIG. 12C is a system diagram of the RAN 104 and the core network 106 according to an embodiment. As described above, the RAN 104 may communicate with the WTRUs 102a, 102b, and 102c over the air interface 116 using UTRA radio technology. The RAN 104 may also be in communication with the core network 106. As shown in FIG. 12C, the RAN 104 includes NodeBs 140a, 140b, and 140c, each of which may include one or more transceivers for communicating with the WTRUs 102a, 102b, and 102c over the air interface 116. The NodeBs 140a, 140b, and 140c may each be associated with a particular cell (not shown) within the RAN 104. The RAN 104 may also include RNCs 142a and 142b. It will be understood that the RAN 104 may include any number of NodeBs and RNCs while remaining consistent with an embodiment.

[0092] As shown in FIG. 12C, NodeBs 140a and 140b are in communication with RNC 142a. NodeB 140c may also be in communication with RNC 142b. NodeBs 140a, 140b, and 140c may communicate with their respective RNCs 142a and 142b via an Iub interface. RNCs 142a and 142b may communicate with each other via an Iur interface. Each RNC 142a and 142b may be configured to control its associated NodeB 140a, 140b, and 140c. Each RNC 142a and 142b may also be configured to perform or support other functions, such as outer loop power control, load control, admission control, packet scheduling, handover control, macro diversity, security functions, data encryption, etc.

[0093] 12C may include a media gateway (MGW) 144, a mobile switching center (MSC) 146, a serving GPRS support node (SGSN) 148, and / or a gateway GPRS support node (GGSN) 150. Although each of the above elements is illustrated as part of the core network 106, it will be understood that any one of these elements may be owned and / or operated by an entity other than the operator of the core network.

[0094] The RNC 142a in the RAN 104 may be connected to an MSC 146 in the core network 106 via an IuCS interface. The MSC 146 may be connected to the MGW 144. The MSC 146 and MGW 144 may provide the WTRUs 102a, 102b, 102c with access to circuit-switched networks, such as the PSTN 108, to facilitate communications between the WTRUs 102a, 102b, 102c and traditional landline communications equipment.

[0095] The RNC 142a in the RAN 104 may also be connected to an SGSN 148 in the core network 106 via an IuPS interface. The SGSN 148 may be connected to a GGSN 150. The SGSN 148 and GGSN 150 may provide the WTRUs 102a, 102b, 102c with access to packet-switched networks, such as the Internet 110, to facilitate communications between the WTRUs 102a, 102b, 102c and IP-enabled devices.

[0096] As noted above, the core network 106 is also connected to networks 112, which may include other wired or wireless networks owned and / or operated by other service providers.

[0097] 12D is a system diagram of the RAN 104 and the core network 106 according to another embodiment. As mentioned above, the RAN 104 may communicate with the WTRUs 102a, 102b, 102c over the air interface 116 using E-UTRA radio technology. The RAN 104 may also be in communication with the core network 106.

[0098] The RAN 104 includes eNodeBs 160a, 160b, and 160c, although it will be appreciated that the RAN 104 may include any number of eNodeBs while remaining consistent with an embodiment. The eNodeBs 160a, 160b, and 160c may each include one or more transceivers for communicating with the WTRUs 102a, 102b, and 102c over the air interface 116. In one embodiment, the eNodeBs 160a, 160b, and 160c may implement MIMO technology. Thus, for example, the eNodeB 160a may use multiple antennas to transmit and receive wireless signals to and from the WTRU 102a.

[0099] Each eNodeB 160a, 160b, 160c is associated with a particular cell (not shown) and may be configured to handle radio resource management decisions, handover decisions, uplink and / or downlink user scheduling, etc. As shown in FIG. 12D, the eNodeBs 160a, 160b, 160c may communicate with each other over an X2 interface.

[0100] The core network 106 shown in Figure 12D may include a mobility management gateway (MME) 162, a serving gateway 164, and a packet data network (PDN) gateway 166. Although each of the above elements is shown as part of the core network 106, it will be understood that any one of the elements may be owned and / or operated by an entity other than the operator of the core network.

[0101] The MME 162 is connected to each of the eNodeBs 160a, 160b, 160c in the RAN 104 via an S1 interface and may act as a control node. For example, the MME 162 is responsible for authenticating users of the WTRUs 102a, 102b, 102c, activating / deactivating bearers, selecting a specific serving gateway when the WTRUs 102a, 102b, 102c initially attach, etc. The MME 162 may also provide a control plane function for switching between the RAN 104 and other RANs (not shown) that employ other radio technologies, such as GSM or WCDMA.

[0102] The serving gateway 164 may be connected to each eNodeB 160a, 160b, 160c in the RAN 104 via an S1 interface. The serving gateway 164 may generally transmit and forward user data packets to and from the WTRUs 102a, 102b, 102c. The serving gateway 164 may also perform other functions, such as fixing the user plane during handovers between eNodeBs, triggering paging when there is downlink data available for the WTRUs 102a, 102b, 102c, managing and storing the context of the WTRUs 102a, 102b, 102c, etc.

[0103] The serving gateway 164 is also connected to a PDN gateway 166 to facilitate communications between the WTRUs 102a, 102b, 102c and IP-enabled devices. The PDN gateway 166 may provide the WTRUs 102a, 102b, 102c with access to packet-switched networks, such as the Internet 110.

[0104] The core network 106 may facilitate communications with other networks. For example, the core network 106 may provide the WTRUs 102a, 102b, 102c with access to circuit-switched networks, such as the PSTN 108, to facilitate communications between the WTRUs 102a, 102b, 102c and traditional landline communications equipment. For example, the core network 106 may include or communicate with an IP gateway (e.g., an IP Multimedia Subsystem (IMS) server) that serves as an interface between the core network 106 and the PSTN 108. The core network 106 may also provide the WTRUs 102a, 102b, 102c with access to the network 112, which may include other wired or wireless networks owned and / or operated by other service providers.

[0105] 12E is a system diagram of the RAN 104 and the core network 106 according to another embodiment. The RAN 104 may be an access service network (ASN) that communicates with the WTRUs 102a, 102b, 102c over the air interface 116 using IEEE 802.16 radio technology. As described further below, communication links between different functional entities of the WTRUs 102a, 102b, 102c, the RAN 104, and the core network 106 may be defined as reference points.

[0106] As shown in FIG. 12E, the RAN 104 may include base stations 170a, 170b, and 170c and an ASN gateway 172, although it will be understood that the RAN 104 may include any number of base stations and ASN gateways while remaining consistent with an embodiment. The base stations 170a, 170b, and 170c are each associated with a particular cell (not shown) within the RAN 104 and each may include one or more transceivers for communicating with the WTRUs 102a, 102b, and 102c over the air interface 116. In one embodiment, the base stations 170a, 170b, and 170c may implement MIMO technology. Thus, for example, the base station 170a may use multiple antennas to transmit and receive wireless signals to and from the WTRU 102a. The base stations 170a, 170b, and 170c may also provide mobility management functions such as handoff triggering, tunnel establishment, radio resource management, traffic classification, and quality of service (QoS) policy enforcement. The ASN gateway 172 acts as a traffic aggregation point and may be responsible for paging, caching of subscriber profiles, routing to the core network 106, etc.

[0107] The air interface 116 between the WTRUs 102a, 102b, 102c and the RAN 104 may be defined as an R1 reference point that implements the IEEE 802.16 specification. Additionally, the WTRUs 102a, 102b, 102c may each establish a logical interface (not shown) with the core network 106. The logical interface between the WTRUs 102a, 102b, 102c and the core network 106 may be defined as an R2 reference point that is used for authentication, authorization, IP host configuration management, and / or mobility management.

[0108] The communication link between each of the base stations 170a, 170b, 170c may be defined as an R8 reference point that includes protocols that facilitate WTRU handover and data transfer between the base stations. The communication link between the base stations 170a, 170b, 170c and the ASN gateway 172 may be defined as an R6 reference point that includes protocols that facilitate mobility management based on mobility events associated with each of the WTRUs 102a, 102b, 102c.

[0109] As shown in Figure 12E, the RAN 104 is connected to a core network 106. The communication link between the RAN 104 and the core network 106 may be defined as an R3 reference point, including protocols that facilitate data forwarding and mobility management functions, for example. The core network 106 may include a Mobile IP Home Agent (MIP-HA) 174, an Authentication, Authorization, and Accounting (AAA) server 176, and a gateway 178. While each of the above elements is illustrated as part of the core network 106, it will be understood that any one of these elements may be owned and / or operated by an entity other than the operator of the core network.

[0110] The MIP-HA 174 is responsible for IP address management, enabling the WTRUs 102a, 102b, 102c to move between different ASNs and / or different core networks. The MIP-HA 174 may provide the WTRUs 102a, 102b, 102c with access to packet-switched networks, such as the Internet 110, to facilitate communications between the WTRUs 102a, 102b, 102c and IP-enabled devices. The AAA server 176 may be responsible for user authentication and user service assistance. The gateway 178 may facilitate interworking with other networks. For example, the gateway 178 may provide the WTRUs 102a, 102b, 102c with access to circuit-switched networks, such as the PSTN 108, to facilitate communications between the WTRUs 102a, 102b, 102c and traditional landline communications equipment. The gateway 178 may also provide the WTRUs 102a, 102b, 102c with access to the networks 112, including other wired or wireless networks owned and / or operated by other service providers.

[0111] 12E, it will be understood that the RAN 104 may be connected to other ASNs and the core network 106 may be connected to other core networks. The communication link between the RAN 104 and the other ASNs is defined as an R4 reference point and may include protocols that govern mobility of the WTRUs 102a, 102b, 102c between the RAN 104 and the other ASNs. The communication link between the core network 106 and the other core networks is defined as an R5 reference point and may include protocols that facilitate interoperation between a home core network and a visited core network.

[0112] Embodiment In one embodiment, a method for generating reference picture lists L0 and L1 for decoding predicted pictures in video data is implemented, the method comprising the steps of: generating a first ordered list of reference pictures RefPicSetCurrTempList0 from a decoded picture buffer (DPB), the list being ordered such that, if there are reference pictures in the DPB that are temporally earlier than the current picture, the reference pictures are listed in order according to their temporal distance from the current picture, then if there are reference pictures in the DPB that are temporally later than the current picture, the reference pictures are listed in order according to their temporal distance from the current picture, and then if there are long-term reference pictures in the DPB, the reference pictures are listed in the order in which they are stored in the DPB; generating a second ordered list RefPicSetCurrTempList1, the list being ordered so that if there are reference pictures in the DPB that are temporally later than the current picture, then those reference pictures are first listed in order according to their temporal distance from the current picture, then if there are reference pictures in the DPB that are temporally later than the current picture, then those reference pictures are listed in order according to their temporal distance from the current picture, and then if there are long-term reference pictures in the DPB, then those reference pictures are listed in the order in which they are stored in the DPB; and generating at least one of lists L0 and L1 by selecting reference pictures from RefPicSetCurrTempList0 and RefPicSetCurrTempList1, respectively.

[0113] According to this embodiment, the method may further include a step of determining whether either of lists L0 and L1 should be a modified list, and if list L0 should be a modified list, the step of generating list L0 includes a step of receiving, for each reference picture in reference picture list L0, a first index pointing to a first ordering list and listing the reference picture identified by that index in the first ordering list in a corresponding entry in L0, and if list L0 should be a modified list, the step of generating list L1 includes a step of receiving, for each reference picture entry in reference picture list L1, a second index pointing to a second ordering list and listing the reference picture identified by that index in the second ordering list in a corresponding entry in L1.

[0114] One or more of the above embodiments may further include: if list L0 is not a modified list, generating list L0 includes sequentially retrieving entries from RefPicSetCurrTempList0 up to a first specified number of entries; and if list L1 is not a modified list, generating list L1 includes sequentially retrieving entries from RefPicSetCurrTempList1 up to a second specified number of entries.

[0115] One or more of the above embodiments may further include that the determining step includes reading a syntax element ref_pic_list_modification_flag_l0 for list L0 and reading a syntax element ref_pic_list_modification_flag_l1 for list L1.

[0116] One or more of the above embodiments may further include, wherein the first index and the second index range from zero to the number of pictures in the DPB.

[0117] One or more of the above embodiments may further include a syntax element ref_pic_set_idx being used to specify the first index and the second index.

[0118] One or more of the above embodiments may further include reading a syntax element ref_pic_list_modification_flag_l1, where ref_pic_list_modification_flag_l1 equal to a first value indicates that a syntax element ref_pic_set_idx specifying L1 is present, and where ref_pic_list_modification_flag_l1 equal to a second value indicates that this syntax element specifying L1 is not present.

[0119] One or more of the above embodiments may further include reading a syntax element ref_pic_list_modification_flag_l0, where ref_pic_list_modification_flag_l0 equal to a first value indicates that a syntax element ref_pic_set_idx specifying L0 is present, and where ref_pic_list_modification_flag_l0 equal to a second value indicates that this syntax element specifying L0 is not present.

[0120] One or more of the above embodiments further include reading a syntax element ref_pic_list_modification_flag_l1, where ref_pic_list_modification_flag_l1 equal to a first value indicates that a syntax element ref_pic_set_idx specifying L1 is present, and ref_pic_list_modification_flag_l1 equal to a second value indicates that this syntax element specifying L1 is not present.

[0121] One or more of the above embodiments may further include: if the first index is not present, the first index is set to zero; and if the second index is not present, the second index is set to zero.

[0122] In another embodiment, or in conjunction with any of the above embodiments, a method that includes initializing a reference picture list decoder for decoding a P or B slice header comprises: cIdx=0 NumRpsCurrTempList=NumRpsStCurr0+NumRpsStCurr1+NumRpsLtCurr for(i=0;i <NumRpsStCurr0;cIdx++,i++) RefPicSetCurrTempList0[cIdx]=RefPicSetStCurr0[i] for(i=0;i <NumRpsStCurr1;cIdx++,i++) RefPicSetCurrTempList0[cIdx]=RefPicSetStCurr1[i] for(i=0;i <NumRpsLtCurr;cIdx++,i++) RefPicSetCurrTempList0[cIdx]=RefPicSetLtCurr[i] The method may include constructing a first temporary list RefPicSetCurrTempList0 by:

[0123] One or more of the above embodiments may further include, if flag ref_pic_list_modification_flag_l0 is 0, constructing list L0 (RefPicList0) by taking the first num_ref_idx — 10_active_minus1+1 entries of RefPicSetCurrTempList0.

[0124] One or more of the above embodiments may further include, if the flag ref_pic_list_modification_flag_l0 is 1, building list L0 by invoking a picture list modification process with RefPicSetCurrTempList0 and num_ref_idx_l0_active_minus1 as inputs.

[0125] One or more of the above embodiments may, when decoding a B slice header, cIdx=0 NumRpsCurrTempList=NumRpsStCurr0+NumRpsStCurr1+NumRpsLtCurr for(i=0;i <NumRpsStCurr1;cIdx++,i++) RefPicSetCurrTempList1[cIdx]=RefPicSetCurr1[i] for(i=0;i <NumRpsStCurr0;cIdx++,i++) RefPicSetCurrTempList1[cIdx]=RefPicSetCurr0[i] for(i=0;i <NumRpsLtCurr;cIdx++,i++) RefPicSetCurrTempList1[cIdx]=RefPicSetLtCurr[i] The method may further include constructing a second temporary list RefPicSetCurrTempList1 by:

[0126] One or more of the above embodiments may further include constructing list L1 (RefPicList1) by taking the first num_ref_idx_ll_active_minus1+1 entries of RefPicSetCurrTempList1 if flag (ref_pic_list_modification_flag_l1) is 0.

[0127] One or more of the above embodiments may further include, if the flag (ref_pic_list_modification_flag_l1) is 1, constructing the list L1 (RefPicList1) by invoking a reference picture list modification process with RefPicSetCurrTempList1 and num_ref_idx_ll_active_minus1 as inputs.

[0128] One or more of the above embodiments use a reference picture list modification process to generate RefPicListX, where X specifies the corresponding list 0 or 1; Set refIdxLX as an index into the reference picture list RefPicListLX; Until refIdxLX becomes greater than num_ref_idx_lX_active_minus1+1 RefPicListX[refIdxLX++]=RefPicSetCurrTempLX[ref_pic_set_idx] The method may further include repeating:

[0129] In another embodiment, or in relation to any of the above embodiments, a method for signaling a change to multiple reference picture lists may include signaling a change to multiple reference picture lists using a unified signaling syntax.

[0130] Further in one or more of the above embodiments, the multiple reference picture lists may include L0, L1 and a combined list LC.

[0131] Further in one or more of the above embodiments, the unified signaling syntax may include encoding the index of the reference picture using an entropy coding method.

[0132] Further in one or more of the above embodiments, the unified signaling syntax may include encoding an index of a reference picture using te(v).

[0133] In another embodiment, or in relation to any of the above embodiments, the method may include determining the number of entries in the reference picture list; and generating a message including a value identifying the entries in the reference picture list, the value being represented by one bit if the number of entries in the reference picture list is two, the value being represented by multiple bits if the number of entries in the reference picture list is three or more, and the message omitting the value if the number of entries in the reference picture list is one.

[0134] In one or more of the above embodiments, the value may be ue(v) when the number of entries is a value greater than or equal to three.

[0135] In one or more of the above embodiments, the index may be specified in the syntax element ref_idx_list_curr.

[0136] In another embodiment, or in relation to any of the above embodiments, a method for creating a combined list of reference pictures LC used to decode a B slice from a list of first reference pictures L0 and a list of second reference pictures L1 may include the steps of determining whether L0 contains two or more entries, determining whether L1 contains two or more entries, and if either L0 or L1 contains two or more entries, indicating the entries of at least one of L0 and L1 to be added to the LC using a syntax element ref_idx_list_curr, if L0 contains only one entry, setting ref_idx_list_curr to 0, if L1 contains only one entry, setting ref_idx_list_curr to 0, and creating the LC using the value of ref_idx_list_curr.

[0137] In one or more of the above embodiments, determining whether L0 contains two or more entries may include determining whether a syntax element num_ref_idx_l0_active_minus1 is greater than or equal to zero, and determining whether L1 contains two or more entries may include determining whether a syntax element num_ref_idx_l1_active_minus1 is greater than or equal to zero.

[0138] Conclusion Although features and elements have been described above in particular combinations, those skilled in the art will understand that each feature and element can be used alone or in combination with other features and elements. The methods described herein are implemented as a computer program, software, or firmware embodied in a computer-readable medium for execution by a computer or processor. Examples of non-transitory computer-readable storage media include, but are not limited to, ROM, RAM, registers, cache memory, semiconductor memory devices, magnetic media such as internal hard disks and removable disks, magneto-optical media, and optical media such as CD-ROM disks and digital versatile disks (DVDs). A processor in association with software may be used to implement a radio frequency transceiver for use in a WTRU, UE, terminal, base station, RNC, or host computer.

[0139] Furthermore, in the above embodiments, processing platforms, computing systems, controllers, and other devices incorporating processors are described. These devices may incorporate at least one central processing unit ("CPU") and memory. References to operations and symbolic representations of operations or instructions may be made by various CPUs and memories in accordance with the practices of those skilled in the computer programming arts. Such operations and instructions may be referred to as being "executed," "computer-executed," or "CPU-executed."

[0140] Those skilled in the art will understand that the operations and symbolically represented operations or instructions include the manipulation of electrical signals by a CPU. The electrical system reconfigures or otherwise alters the operation of the CPU as well as other processing of signals by resulting in the transformation or degradation of the electrical signals and representing data bits that can be maintained in storage locations within a memory system. The storage locations where data bits are maintained are physical locations that have particular electrical, magnetic, optical, or organic properties that correspond to or represent that data bit.

[0141] Data bits may also be stored on computer-readable media, including magnetic disks, optical disks, and other volatile (e.g., RAM) or non-volatile (e.g., ROM) mass storage systems readable by a CPU. Computer-readable media includes cooperating or interconnected computer-readable media, where such media reside exclusively on a processing system or are distributed across multiple interconnected processing systems that are local or remote to the processing system. It will be appreciated that exemplary embodiments are not limited to the above memories, and that other platforms and memories may be compatible with the described methods.

[0142] No element, act, or instruction used in the description of this application should be construed as essential or required unless explicitly stated to be so. Also, as used herein, the article "a" is intended to include one or more items. Where only one item is intended, the term "one" or similar language is used. Furthermore, the term "any of," followed by a list of multiple items and / or a grouping of multiple items, as used herein, is intended to include "any," "any combination of," "any plurality of," and / or "any combination of" of that item and / or grouping of items, individually or in conjunction with other items and / or groups of items. Furthermore, as used herein, the term "set" is intended to include any number of items, including zero. Furthermore, as used herein, the term "number" is intended to include any number, including zero.

[0143] Furthermore, the claims should not be limited to the described order or elements unless so stated. Also, the use of the term "means for" in the claims is intended to invoke 35 U.S.C. 112, paragraph 6, and any claim that does not contain the word "means for" does not intend such invocation.

[0144] Although the systems and methods are described herein in the context of a UWB multi-band communication system, it is contemplated that they may also be implemented as software on a microprocessor / general-purpose computer (not shown). In certain embodiments, one or more of the functions of the various components may be implemented as software controlling a general-purpose computer. [Industrial Applicability]

[0145] The present invention can be used in digital video coding. [Explanation of symbols]

[0146] 100 Communication Systems 102a, 102b, 102c, 102d wireless transmitting / receiving units 104 RAN 106 Core Network 108 PSTN 110 Internet 112 other networks 118 processors 120 Transceiver (transmitter / receiver)

Claims

1. A processor configured to generate a temporally ordered list of reference pictures from a decoded picture buffer (DPB), the DPB including four short-term reference pictures Ref1, Ref2, Ref4, Ref5, reference pictures Ref1 and Ref2 being earlier in display order, reference pictures Ref4 and Ref5 being later in display order, and a current picture being temporally between reference picture R2 and reference picture R4; the temporal ordering list is reference pictures in the DPB that are temporally earlier than the current picture and that are relative to the current picture being decoded, i.e., the short-term reference pictures R1 and R2, ordered by temporal distance from the current picture such that the short-term reference picture R2 is ordered before the short-term reference picture R1; followed by reference pictures in the DPB that are temporally later than the current picture and are reference pictures for the current picture, namely the short-term reference pictures R4 and R5, ordered by temporal distance from the current picture such that the short-term reference picture R4 is ordered before the short-term reference picture R5; and subsequently, a processor for ordering long-term reference pictures relative to the current picture in the DPB; the processor is further configured to generate a reference picture list by selecting a reference picture from the temporal ordering list of reference pictures; If the reference picture list is to be a modified list, generating the reference picture list includes at least, for each entry in the reference picture list: reading an index into said temporally ordered list of reference pictures; and by selecting, for said entry in said reference picture list, a reference picture identified by said index from said temporally ordered list of reference pictures. Video decoder device.

2. A method for generating a reference picture list, comprising: generating a temporally ordered list of reference pictures from a decoded picture buffer (DPB), the DPB including four short-term reference pictures Ref1, Ref2, Ref4, Ref5, where reference pictures Ref1 and Ref2 are earlier in display order and reference pictures Ref4 and Ref5 are later in display order, and the current picture is temporally between reference picture R2 and reference picture R4; the temporal ordering list is reference pictures in the DPB that are temporally earlier than the current picture and that are relative to the current picture being decoded, i.e., the short-term reference pictures R1 and R2, ordered by temporal distance from the current picture such that the short-term reference picture R2 is ordered before the short-term reference picture R1; followed by reference pictures in the DPB for the current picture that are temporally later than the current picture, namely the short-term reference pictures R4 and R5, ordered by temporal distance from the current picture such that the short-term reference picture R4 is ordered temporally earlier than the short-term reference picture R5; followed by ordering long-term reference pictures relative to the current picture in the DPB; generating a reference picture list by selecting reference pictures from said temporally ordered list of reference pictures, If the reference picture list is to be a modified list, generating a reference picture list includes at least, for each entry in the reference picture list: reading an index into said temporally ordered list of reference pictures; and by selecting, for said entry in said reference picture list, a reference picture identified by said index from said temporally ordered list of reference pictures; A method for providing the above.

Images