High level syntax headers in video coding

Abstract

Various embodiments provide methods, apparatuses, and computer program products. An example apparatus includes at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to: selecting a first frame to be a random access point (RAP) frame; setting an open bitstream unit (OBU) RAP flag equal to a first value for OBUs in a temporal unit of the first frame; setting a frame type as a key frame in a frame header of the first frame; and coding the first frame as an intra frame.

Description

HIGH LEVEL SYNTAX HEADERS IN VIDEO CODINGTECHNICAL FIELD

[0001] The examples and non-limiting embodiments relate generally to multimedia coding and, more particularly to, implementing high level syntax headers in video coding.BACKGROUND

[0002] It is known to provide standardized formats for encoding, signaling, or decoding of media data.SUMMARY

[0003] Example 1 : An apparatus comprising: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to: selecting a first frame to be a random access point (RAP) frame; setting an open bitstream unit (OBU) RAP flag equal to a first value for OBUs in a temporal unit of the first frame; setting a frame type as a key frame in a frame header of the first frame; and coding the first frame as an intra frame.

[0004] Example 2: The apparatus of example 1, wherein the apparatus is further caused to perform: selecting a second frame, following the first frame in a coding order, to be a non-RAP frame; setting the open bitstream unit RAP flag equal to a second value for OBUs in a temporal unit of the second frame, wherein the second value is different from the first value; setting the frame type as an inter frame in a frame header of the second frame; and coding the second frame as the inter frame.

[0005] Example 3: The apparatus of example 2, wherein, for coding the second frame, the apparatus is further caused to perform: setting at least one reference frame to the first frame or another frame that follows the first frame in the coding order, wherein the another frame is different from the second frame.

[0006] Example 4: The apparatus of any of the previous examples, wherein the temporal unit comprises one or more of a temporal delimiter OBU, a sequence header OBU, an optional frame header OBU, one or more tile groups OBUs, or an optional metadata OBU.

[0007] Example 5: The apparatus of any of the previous examples, wherein the RAP frame comprises a key frame.

[0008] Example 6: The apparatus of example 5, wherein the apparatus is further caused to perform: setting a show frame flag in the frame header of the first frame to the first value.

[0009] Example 7: The apparatus of example 1, wherein the apparatus is further caused to perform: selecting a second frame, following the first frame in a coding order, to be a delayed RAP frame; setting an open bitstream unit RAP flag equal to the first value for OBUs in a temporal unit of the second frame; setting the frame type as the key frame in a frame header of the second frame; setting a show frame flag in the frame header of the second frame to a second value; and coding the second frame as an intra frame.

[0010] Example 8: The apparatus of example 7, wherein the apparatus is further caused to perform: selecting a third frame, following the second frame in the coding order, to be a non-RAP; setting the open bitstream unit RAP flag equal to a second value for OBUs in a temporal unit of the third frame; setting the frame type as inter frame in a frame header of the third frame; and coding the third frame as the inter frame.

[0011] Example 9: The apparatus of example 8, wherein, for coding the third frame, the apparatus is further caused to perform: setting at least one reference frame to the first frame.

[0012] Example 10: The apparatus of example 6, wherein the apparatus is further caused to perform: setting an open bitstream unit RAP type flag equal to the first value for open bitstream units in the temporal unit of the first frame.

[0013] Example 11: The apparatus of example 10, wherein the apparatus is further caused to perform: setting an open bitstream unit extension flag equal to the first value for open bitstream units in the temporal unit of the first frame.

[0014] Example 12: The apparatus of example 8, wherein the apparatus is further caused to perform: setting an open bitstream unit RAP type flag equal to the first value for open bitstream units in the temporal unit of the second frame.

[0015] Example 13: The apparatus of example 12, wherein the apparatus is further caused to perform: setting an open bitstream unit extension flag equal to the first value for open bitstream units in the temporal unit of the first frame.

[0016] Example 14: The apparatus of example 7, wherein the apparatus is further caused to perform: selecting a third frame, following the second frame in the coding order, to be key frame dependent recovery point; setting the open bitstream unit RAP flag equal to a second value for open bitstream units in the temporal unit of the third frame; setting an open bitstream unit RAP type flag equal to the first value for open bitstream units in the temporal unit of the third frame; setting the frame type as inter frame in the frame header of the third frame; and coding the third frame as an inter frame.

[0017] Example 15: The apparatus of example 14, wherein the apparatus is further caused to perform: setting the open bitstream unit extension flag equal to the first value for open bitstream units in the temporal unit of the second frame.

[0018] Example 16: The apparatus of example 15, wherein the apparatus is further caused to perform: setting an open bitstream unit extension flag equal to the first value for the open bitstream units in the temporal unit of the second frame; and setting the open bitstream unit extension flag equal to the first value for the open bitstream units in the temporal unit of third frame.

[0019] Example 17: The apparatus of any of the examples 14 to 16, wherein for coding the third frame, the apparatus is further caused to perform: setting at least one reference frame to the second frame or another frame that follows the second frame in coding order, wherein the third frame is different from the another frame.

[0020] Example 18: The apparatus of example 1, wherein the first frame comprises a key frame RAP, and wherein the apparatus is further caused to perform: setting an open bitstream unit extension flag equal to the first value for open bitstream units in the temporal unit of the first frame; setting an open bitstream unit RAP type syntax element equal to the first value for all open bitstream units in the temporal unit of first frame; setting a frame type equal to a key frame in frame header of the first frame; setting show frame in the frame header of the first frame equal to the first value; selecting a second frame, following the first frame in the coding order, to be a non-RAP frame; setting open bitstream unit RAP type syntax element equal to second value and / or open bitstream unit extension flag equal to the second value for all open bitstream units in the temporal unit of the second frame; setting frame type equal to an inter frame in the frame header of the second frame; coding second frame as an inter frame.

[0021] Example 19: The apparatus of example 18, wherein for coding the second frame, the apparatus is further caused to perform: setting at least one reference frame to the first frame or another frame that follows the first frame in coding order, wherein the another frame is different from the second frame.

[0022] Example 20: The apparatus of example 1, wherein the apparatus is further caused to perform: selecting a second frame, following the first frame in coding order, to be a delayed RAP; setting an open bitstream unit extension flag equal to the first value for open bitstream units in a temporal unit of the second frame; setting an open bitstream unit RAP type syntax element equal to a third value for open bitstream units in the temporal unit of the second frame; setting the frame type equal to the key frame in a frame header of the second frame; setting a show frame flag equal to the second value in theframe header of the second frame; coding the second frame as an intra frame; selecting a third frame, following the second frame in the coding order, to be a non-RAP frame; setting the open bitstream unit RAP type syntax element and / or the open bitstream unit extension flag equal to a second value for open bitstream units in the temporal unit of the third frame; setting frame type equal to an inter frame in the frame header of the third frame; and coding the third frame as an inter frame.

[0023] Example 21 : The apparatus of example 20, wherein for coding the third frame, the apparatus is further caused to perform: setting at least one reference frame to the first frame.

[0024] Example 22: The apparatus of example 1, wherein the apparatus is further caused to perform: selecting a second frame, following the first frame in coding order, to be a delayed RAP; setting an open bitstream unit extension flag equal to the first value for open bitstream units in a temporal unit of the second frame; setting an open bitstream unit RAP type syntax element equal to a third value for open bitstream units in the temporal unit of the second frame; setting the frame type equal to the key frame in a frame header of the second frame; setting a show frame flag equal to the second value in the frame header of the second frame; coding the second frame as an intra frame; selecting a third frame, following the second frame in the coding order, to be a key frame dependent recovery point; setting open bitstream unit extension flag equal to the first value for open bitstream units in the temporal unit of the third frame; setting the open bitstream unit RAP type syntax element equal to a fourth value for open bitstream units in the temporal unit of the third frame; setting frame type equal to inter frame in the frame header of the third frame; and coding the third frame as an inter frame.

[0025] Example 23: The apparatus of example 22, wherein for coding the third frame, the apparatus is further caused to perform: setting at least one reference frame to the second frame or another frame that follows the second frame in the coding order, wherein the another frame is different from the third frame.

[0026] Example 24: The apparatus of example 1, wherein the first frame comprises a key frame RAP, and wherein the apparatus is further caused to perform: setting a two bit delimiter RAP type syntax element to a fifth value in a temporal delimiter open bitstream unit of the first frame; setting a show frame flag equal to the first value in the frame header of the first frame; selecting a second frame to be a non-RAP; setting a two bit delimiter RAP type syntax element to a second value in the temporal delimiter open bitstream unit of the second frame; setting the frame type equal to inter frame in the frame header of the second frame; and coding the second frame as an inter frame.

[0027] Example 25: The apparatus of example 24, wherein for coding the second frame, the apparatus is further caused to perform: setting at least one reference frame to the first frame or anotherframe that follows the first frame in coding order, wherein the another frame is different from the second frame.

[0028] Example 26: The apparatus of example 1, wherein the apparatus is further caused to perform: selecting a second frame, following the first frame in coding order, to be a delayed RAP; setting a two bit delimiter RAP type syntax element to a third value in a temporal delimiter open bitstream unit of the second frame; setting the frame type equal to the key frame in a frame header of the second frame; setting show frame flag equal to a second value in the frame header of the second frame; coding the second frame as an intra frame; selecting, following the second frame in the coding order, a third frame to be non-RAP; setting a two bit delimiter RAP type syntax element to a second value in the temporal delimiter open bitstream unit of the third frame; setting the frame type equal to inter frame in the frame header of the third frame; and coding the third frame as an inter frame.

[0029] Example 27 : The apparatus of example 26, wherein, for coding the third frame as an inter frame, the apparatus is further caused to perform: setting at least one reference frame to the first frame.

[0030] Example 28: The apparatus of example 1, wherein the apparatus is further caused to perform: selecting a second frame, following the first frame in coding order, to be a delayed RAP; setting a two bit delimiter RAP type syntax element to a third value in a temporal delimiter open bitstream unit of the second frame; setting the frame type equal to the key frame in a frame header of the second frame; setting a show frame flag to a second value in the frame header of the second frame; coding the second frame as an intra frame; selecting a third frame, following the second frame in the coding order, to be key frame dependent recovery point; setting two bit delimiter RAP type syntax element to a fourth value in the temporal delimiter open bitstream unit of the third frame; setting frame type equal to inter frame in the frame header of the third frame; and coding the third frame as an inter frame.

[0031] Example 29: The apparatus of example 28, wherein, for coding the third frame as an inter frame, the apparatus is further caused to perform: setting at least one reference frame to the second frame or another frame that follows the second frame in the coding order, wherein the another is different from the second frame.

[0032] Example 30: An apparatus comprising: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to: receiving a streaming or a broadcast video; reading open bitstream unit (OBU) headers in coding order; discarding open bitstream units until a criterion is met for a first frame; and decoding the first frame.

[0033] Example 31: The apparatus of example 30, wherein the criterion comprises identifying the first frame with an open bitstream unit (OBU) random access point (RAP) flag equal to a first value.

[0034] Example 32: The apparatus of any of the examples 30 or 31, wherein the apparatus is further caused to perform: decoding subsequent frames.

[0035] Example 33: The apparatus of any of the examples 30 or 31, wherein the apparatus is further caused to perform: examining a frame header of the first frame; and displaying the first frame, when a show existing frame is equal to the first value.

[0036] Example 34: The apparatus of example 33, wherein when the show existing frame is not equal to the first value, the apparatus is further caused to perform: continuing decoding frames and examining frame headers until a second frame with the show existing frame equal to the first value is found; decoding the second frame and frames following the second frame in the bitstream; and displaying the second frame and the frames following the second frame in the bitstream.

[0037] Example 35: The apparatus of any of the examples 30 or 31, wherein, when the open bitstream unit RAP type flag in the first frame is equal to the first value, the apparatus is further caused to perform: determining that the first frame comprises a key frame RAP; and decoding frames following the first frame in coding order.

[0038] Example 36: The apparatus of example 35, wherein the apparatus is further caused to perform: displaying the first frame; and / or displaying frames following the first frame in coding order.

[0039] Example 37: The apparatus of any of the examples 30, wherein, when an open bitstream unit RAP type flag in the first frame is not equal to the first value, the apparatus is further caused to perform: determining that the first frame comprises a delayed RAP; reading open bitstream unit headers until a second frame with the open bitstream unit RAP flag equal to the second value and open bitstream unit RAP type flag equal to the first value is found; decoding the second frame; and decoding frames following the second frame in coding order.

[0040] Example 38: The apparatus of example 37, wherein the apparatus is further caused to perform: displaying the second frame; and / or displaying frames following the second frame in coding order.

[0041] Example 39: The apparatus of example 30, wherein the criterion comprises identifying the first frame with an open bitstream unit random access point (RAP) type syntax element equal to a first value or a third value.

[0042] Example 40: The apparatus of example 39, wherein when an open bitstream unit RAP type syntax element in the first frame is equal to the first value, the apparatus is further caused to perform:determining that the first frame comprises a key frame RAP; and decoding frames following the first frame in coding order.

[0043] Example 41: The apparatus of example 40, wherein the apparatus is further caused to perform: displaying the first frame; and / or displaying frames following the first frame in the coding order.

[0044] Example 42: The apparatus of example 39, wherein, when the open bitstream unit RAP type syntax element in the first frame is equal to the third value, the apparatus is further caused to perform: determining that the first frame comprises a delayed RAP; reading open bitstream unit headers until a second frame with the open bitstream unit RAP type syntax element equal to a fourth value is found; decoding the second frame; and decoding frames following the second frame in the coding order.

[0045] Example 43: The apparatus of example 42, wherein the apparatus is further caused to perform: displaying the second frame; and / or displaying frames following the second frame in the coding order.

[0046] Example 44: The apparatus of example 30, wherein the criterion comprises identifying the first frame comprising a temporal delimiter open bitstream unit with a two bit delimiter RAP type syntax element equal to a fifth value or a third value.

[0047] Example 45 : The apparatus of example 44, wherein when the two bit delimiter RAP type syntax element is equal to the fifth value, the apparatus is further caused to perform: determining that the first frame comprises a key frame RAP; and decoding frames following the first frame in the coding order.

[0048] Example 46: The apparatus of example 45, the apparatus is further caused to perform: displaying the first frame; and / or displaying the frames following the first frame in the coding order.

[0049] Example 47 : The apparatus of example 44, wherein, when the two bit delimiter RAP type syntax element is equal to the third value, the apparatus is further caused to perform: determining that the first frame comprises a delayed RAP; reading open bitstream unit headers until a second frame with the temporal delimiter open bitstream unit with the two bit delimiter RAP type syntax element is equal to a fourth value is found; decoding the second frame; and decoding frames following the second frame in the coding order.

[0050] Example 48: The apparatus of example 47, wherein the apparatus is further caused to perform: displaying the second frame; and / or displaying frames following the second frame in thecoding order.

[0051] Example 49: A method comprising: selecting a first frame to be a random access point (RAP) frame; setting an open bitstream unit (OBU) RAP flag equal to a first value for OBUs in a temporal unit of the first frame; setting a frame type as a key frame in a frame header of the first frame; and coding the first frame as an intra frame.

[0052] Example 50: The method of example 49 further comprising: selecting a second frame, following the first frame in a coding order, to be a non-RAP frame; setting the open bitstream unit RAP flag equal to a second value for OBUs in a temporal unit of the second frame, wherein the second value is different from the first value; setting the frame type as an inter frame in a frame header of the second frame; and coding the second frame as an inter frame.

[0053] Example 51 : The method of example 50, wherein, for coding the second frame, the method further comprises: setting at least one reference frame to the first frame or another frame that follows the first frame in the coding order, wherein the another frame is different from the second frame.

[0054] Example 52: The method of any of the examples 49 to 51, wherein the temporal unit comprises one or more of a temporal delimiter OBU, a sequence header OBU, an optional frame header OBU, one or more tile groups OBUs, or an optional metadata OBU.

[0055] Example 53: The method of any of the examples 49 to 52, wherein the RAP frame comprises a key frame.

[0056] Example 54: The method of example 53 further comprises: setting a show frame flag in the frame header of the first frame to the first value.

[0057] Example 55: The method of example 49 further comprising: selecting a second frame, following the first frame in a coding order, to be a delayed RAP frame; setting an open bitstream unit RAP flag equal to the first value for OBUs in a temporal unit of the second frame; setting the frame type as the key frame in a frame header of the second frame; setting a show frame flag in the frame header of the second frame to a second value; and coding the second frame as an intra frame.

[0058] Example 56: The method of example 55 further comprising: selecting a third frame, following the second frame in the coding order, to be a non-RAP; setting the open bitstream unit RAP flag equal to a second value for OBUs in a temporal unit of the third frame; setting the frame type as inter frame in a frame header of the third frame; and coding the third frame as an inter frame.

[0059] Example 57: The method of example 56, wherein, for coding the third frame, the methodfurther comprises: setting at least one reference frame to the first frame.

[0060] Example 58: The method of example 54 further comprising: setting an open bitstream unit RAP type flag equal to the first value for open bitstream units in the temporal unit of the first frame.

[0061] Example 59: The method of example 58 further comprising: setting an open bitstream unit extension flag equal to the first value for open bitstream units in the temporal unit of the first frame.

[0062] Example 60: The method of example 56 further comprising: setting an open bitstream unit RAP type flag equal to the first value for open bitstream units in the temporal unit of the second frame.

[0063] Example 61: The method of example 60 further comprising: setting an open bitstream unit extension flag equal to the first value for open bitstream units in the temporal unit of the first frame.

[0064] Example 62: The method of example 55 further comprising: selecting a third frame, following the second frame in the coding order, to be key frame dependent recovery point; setting the open bitstream unit RAP flag equal to a second value for open bitstream units in the temporal unit of the third frame; setting an open bitstream unit RAP type flag equal to the first value for open bitstream units in the temporal unit of the third frame; setting the frame type as inter frame in the frame header of the third frame; and coding the third frame as an inter frame.

[0065] Example 63: The method of example 62 further comprising: setting the open bitstream unit extension flag equal to the first value for open bitstream units in the temporal unit of the second frame.

[0066] Example 64: The method of example 63 further comprising: setting an open bitstream unit extension flag equal to the first value for the open bitstream units in the temporal unit of the second frame; and setting the open bitstream unit extension flag equal to the first value for the open bitstream units in the temporal unit of third frame.

[0067] Example 65 : The method of any of the examples 62 to 64, wherein for coding the third frame, the method further comprises: setting at least one reference frame to the second frame or another frame that follows the second frame in coding order, wherein the third frame is different from the another frame.

[0068] Example 66: The method of example 49, wherein the first frame comprises a key frame RAP, and wherein the method further comprises: setting an open bitstream unit extension flag equal to the first value for open bitstream units in the temporal unit of the first frame; setting an open bitstream unit RAP type syntax element equal to the first value for all open bitstream units in the temporal unit of first frame; setting a frame type equal to a key frame in frame header of the first frame; setting show framein the frame header of the first frame equal to the first value; selecting a second frame, following the first frame in the coding order, to be a non-RAP frame; setting open bitstream unit RAP type syntax element equal to second value and / or open bitstream unit extension flag equal to the second value for all open bitstream units in the temporal unit of the second frame; setting frame type equal to an inter frame in the frame header of the second frame; coding second frame as an inter frame.

[0069] Example 67 : The method of example 66, wherein for coding the second frame, the method further comprises: setting at least one reference frame to the first frame or another frame that follows the first frame in coding order, wherein the another frame is different from the second frame.

[0070] Example 68: The method of example 49 further comprising: selecting a second frame, following the first frame in coding order, to be a delayed RAP; setting an open bitstream unit extension flag equal to the first value for open bitstream units in a temporal unit of the second frame; setting an open bitstream unit RAP type syntax element equal to a third value for open bitstream units in the temporal unit of the second frame; setting the frame type equal to the key frame in a frame header of the second frame; setting a show frame flag equal to the second value in the frame header of the second frame; coding the second frame as an intra frame; selecting a third frame, following the second frame in the coding order, to be a non-RAP frame; setting the open bitstream unit RAP type syntax element and / or the open bitstream unit extension flag equal to a second value for open bitstream units in the temporal unit of the third frame; setting frame type equal to an inter frame in the frame header of the third frame; and coding the third frame as an inter frame.

[0071] Example 69: The method of example 68, wherein for coding the third frame, the method further comprises: setting at least one reference frame to the first frame.

[0072] Example 70: The method of example 49 further comprising: selecting a second frame, following the first frame in coding order, to be a delayed RAP; setting an open bitstream unit extension flag equal to the first value for open bitstream units in a temporal unit of the second frame; setting an open bitstream unit RAP type syntax element equal to a third value for open bitstream units in the temporal unit of the second frame; setting the frame type equal to the key frame in a frame header of the second frame; setting a show frame flag equal to the second value in the frame header of the second frame; coding the second frame as an intra frame; selecting a third frame, following the second frame in the coding order, to be a key frame dependent recovery point; setting open bitstream unit extension flag equal to the first value for open bitstream units in the temporal unit of the third frame; setting the open bitstream unit RAP type syntax element equal to a fourth value for open bitstream units in the temporal unit of the third frame; setting frame type equal to inter frame in the frame header of the third frame; and coding the third frame as an inter frame.

[0073] Example 71: The method of example 70, wherein for coding the third frame, the method further comprises: setting at least one reference frame to the second frame or another frame that follows the second frame in the coding order, wherein the another frame is different from the third frame.

[0074] Example 72: The method of example 49, wherein the first frame comprises a key frame RAP, and wherein the method further comprises: setting a two bit delimiter RAP type syntax element to a fifth value in a temporal delimiter open bitstream unit of the first frame; setting a show frame flag equal to the first value in the frame header of the first frame; selecting a second frame to be a non-RAP; setting a two bit delimiter RAP type syntax element to a second value in the temporal delimiter open bitstream unit of the second frame; setting the frame type equal to inter frame in the frame header of the second frame; and coding the second frame as an inter frame.

[0075] Example 73: The method of example 72, wherein for coding the second frame, the method further comprises: setting at least one reference frame to the first frame or another frame that follows the first frame in coding order, wherein the another frame is different from the second frame.

[0076] Example 74: The method of example 49 further comprising: selecting a second frame, following the first frame in coding order, to be a delayed RAP; setting a two bit delimiter RAP type syntax element to a third value in a temporal delimiter open bitstream unit of the second frame; setting the frame type equal to the key frame in a frame header of the second frame; setting show frame flag equal to a second value in the frame header of the second frame; coding the second frame as an intra frame; selecting, following the second frame in the coding order, a third frame to be non-RAP; setting a two bit delimiter RAP type syntax element to a second value in the temporal delimiter open bitstream unit of the third frame; setting the frame type equal to inter frame in the frame header of the third frame; and coding the third frame as an inter frame.

[0077] Example 75: The method of example 75, wherein, for coding the third frame as an inter frame, the method further comprises: setting at least one reference frame to the first frame.

[0078] Example 76: The method of example 49 further comprising: selecting a second frame, following the first frame in coding order, to be a delayed RAP; setting a two bit delimiter RAP type syntax element to a third value in a temporal delimiter open bitstream unit of the second frame; setting the frame type equal to the key frame in a frame header of the second frame; setting a show frame flag to a second value in the frame header of the second frame; coding the second frame as an intra frame; selecting a third frame, following the second frame in the coding order, to be key frame dependent recovery point; setting two bit delimiter RAP type syntax element to a fourth value in the temporal delimiter open bitstream unit of the third frame; setting frame type equal to inter frame in the frameheader of the third frame; and coding the third frame as an inter frame.

[0079] Example 77: The method of example 76, wherein, for coding the third frame as an inter frame, the method further comprises: setting at least one reference frame to the second frame or another frame that follows the second frame in the coding order, wherein the another is different from the second frame.

[0080] Example 78: A method comprising: receiving a streaming or a broadcast video; reading open bitstream unit (OBU) headers in coding order; discarding open bitstream units until a criterion is met for a first frame; and decoding the first frame.

[0081] Example 79: The method of example 78, wherein the criterion comprises identifying the first frame with an open bitstream unit (OBU) random access point (RAP) flag equal to a first value.

[0082] Example 80: The method of any of the examples 78 or 79 further comprising: decoding subsequent frames.

[0083] Example 81: The method of any of the examples 78 or 79 further comprising: examining a frame header of the first frame; and displaying the first frame, when a show existing frame is equal to the first value.

[0084] Example 82: The method of example 81, wherein when the show existing frame is not equal to the first value, the method further comprises: continuing decoding frames and examining frame headers until a second frame with the show existing frame equal to the first value is found; decoding the second frame and frames following the second frame in the bitstream; and displaying the second frame and the frames following the second frame in the bitstream.

[0085] Example 83: The method of any of the examples 78 or 79, wherein, when the open bitstream unit RAP type flag in the first frame is equal to the first value, the method further comprises: determining that the first frame comprises a key frame RAP; and decoding frames following the first frame in coding order.

[0086] Example 84: The method of example 83 further comprising: displaying the first frame; and / or displaying frames following the first frame in coding order.

[0087] Example 85: The method of any of the examples 78, wherein, when an open bitstream unit RAP type flag in the first frame is not equal to the first value, the method further comprises: determining that the first frame comprises a delayed RAP; reading open bitstream unit headers until a second frame with the open bitstream unit RAP flag equal to the second value and open bitstream unit RAP type flagequal to the first value is found; decoding the second frame; and decoding frames following the second frame in coding order.

[0088] Example 86: The method of example 85 further comprising: displaying the second frame; and / or displaying frames following the second frame in coding order.

[0089] Example 87: The method of example 78, wherein the criterion comprises identifying the first frame with an open bitstream unit random access point (RAP) type syntax element equal to a first value or a third value.

[0090] Example 88: The method of example 87, wherein when an open bitstream unit RAP type syntax element in the first frame is equal to the first value, the method further comprises: determining that the first frame comprises a key frame RAP; and decoding frames following the first frame in coding order.

[0091] Example 89: The method of example 88 further comprising: displaying the first frame; and / or displaying frames following the first frame in the coding order.

[0092] Example 90: The method of example 87, wherein, when the open bitstream unit RAP type syntax element in the first frame is equal to the third value, the method further comprises: determining that the first frame comprises a delayed RAP; reading open bitstream unit headers until a second frame with the open bitstream unit RAP type syntax element equal to a fourth value is found; decoding the second frame; and decoding frames following the second frame in the coding order.

[0093] Example 91: The method of example 90 further comprising: displaying the second frame; and / or displaying frames following the second frame in the coding order.

[0094] Example 92: The method of example 78, wherein the criterion comprises identifying the first frame comprising a temporal delimiter open bitstream unit with a two bit delimiter RAP type syntax element equal to a fifth value or a third value.

[0095] Example 93: The method of example 92, wherein when the two bit delimiter RAP type syntax element is equal to the fifth value, the method further comprises: determining that the first frame comprises a key frame RAP; and decoding frames following the first frame in the coding order.

[0096] Example 94: The method of example 93 further comprising: displaying the first frame; and / or displaying the frames following the first frame in the coding order.

[0097] Example 95: The method of example 92, wherein, when the two bit delimiter RAP typesyntax element is equal to the third value, the method further comprises: determining that the first frame comprises a delayed RAP; reading open bitstream unit headers until a second frame with the temporal delimiter open bitstream unit with the two bit delimiter RAP type syntax element is equal to a fourth value is found; decoding the second frame; and decoding frames following the second frame in the coding order.

[0098] Example 96: The method of example 95 further comprising: displaying the second frame; and / or displaying frames following the second frame in the coding order.

[0099] Example 97: An apparatus comprising: means for selecting a first frame to be a random access point (RAP) frame; means for setting an open bitstream unit (OBU) RAP flag equal to a first value for OBUs in a temporal unit of the first frame; means for setting a frame type as a key frame in a frame header of the first frame; and means for coding the first frame as an intra frame.

[0100] Example 98: The apparatus of example 97, wherein the apparatus further comprises means for performing methods as described in any of the examples 50 to 77.

[0101] Example 99: An apparatus comprising: means for receiving a streaming or a broadcast video; means for reading open bitstream unit (OBU) headers in coding order; means for discarding open bitstream units until a criterion is met for a first frame; and means for decoding the first frame.

[0102] Example 100: The apparatus of example 99, wherein the apparatus further comprises means for performing methods as described in any of the examples 79 to 96.

[0103] Example 101: A computer readable medium comprising program instructions that, which when executed by an apparatus, cause the apparatus to perform: selecting a first frame to be a random access point (RAP) frame; setting an open bitstream unit (OBU) RAP flag equal to a first value for OBUs in a temporal unit of the first frame; setting a frame type as a key frame in a frame header of the first frame; and coding the first frame as an intra frame.

[0104] Example 102: The computer readable medium of example 101, wherein the computer readable medium comprises a non-transitory computer readable medium.

[0105] Example 103: The computer readable medium of any of examples 101 or 102, wherein the computer readable medium further comprises instructions for performing methods as described in any of the examples 50 to 77.

[0106] Example 104: A computer readable medium comprising program instructions that, which when executed by an apparatus, cause the apparatus to perform: receiving a streaming or a broadcastvideo; reading open bitstream unit (OBU) headers in coding order; discarding open bitstream units until a criterion is met for a first frame; and decoding the first frame.

[0107] Example 105: The computer readable medium of example 104, wherein the computer readable medium comprises a non-transitory computer readable medium.

[0108] Example 106: The computer readable medium of any of examples 104 or 105, wherein the computer readable medium further comprises instructions for performing methods as described in any of the examples 79 to 96.

[0109] The first, second, third, fourth, and fifth values, as described in previous examples, represent a natural number . In one or more embodiments, the first value, the second value, the third value and the fourth are not equal to each other. In at least one embodiment, the first value is equal to 1, the second value is equal to 0, the third value is equal to 2, and the fourth value is equal to 3. In at least one embodiment, the fifth value is equal to 1. It should however be understood that the first, second, third, fourth, and fifth values may be equal to a different natural number than what is mentioned above.BRIEF DESCRIPTION OF THE DRAWINGS

[0110] The foregoing embodiments and other features are explained in the following description, taken in connection with the accompanying drawings, wherein:

[0111] FIG. 1 shows schematically an apparatus employing embodiments of the examples described herein.

[0112] FIG. 2 shows schematically a user equipment suitable for employing embodiments of the examples described herein.

[0113] FIG. 3 further shows schematically electronic devices employing embodiments of the examples described herein connected using wireless and wired network connections.

[0114] FIG. 4 is a block diagram illustrating a system in accordance with an example.

[0115] FIG. 5 is an example apparatus, which may be implemented in hardware, and is caused to, implement examples described herein.

[0116] FIG. 6 shows a representation of an example of non-volatile memory media used to store instructions that implement the examples described herein.

[0117] FIG. 7 is an example method performed with an encoder, based on the examples describedherein.

[0118] FIG. 8 is another example method performed with an decoder, based on the examples described herein.DETAILED DESCRIPTION OF EXAMPLE EMBODIMENTS

[0119] The following acronyms and abbreviations that may be found in the specification and / or the drawing figures are defined as follows (the abbreviations may be appended with each other or with other characters using e.g. a hyphen or dash (-), and may be case insensitive):4CC four character code5G fifth generation cellular network technology5GC 5G core network a.k.a. also known asAVC advanced video codingCU coding unitDSP digital signal processorDU distributed unit eNB (or eNodeB) evolved Node B (for example, an LTE base station)EN-DC E-UTRA-NR dual connectivity en-gNB or En-gNB node providing NR user plane and control plane protocol terminations towards the UE, and acting as secondary node in EN-DCE-UTRA evolved universal terrestrial radio access, for example, the LTE radio access technologyFl or Fl-C interface between CU and DU control interface gNB (or gNodeB) base station for 5G / NR, for example, a node providing NR user plane and control plane protocol terminations towards the UE, and connected via the NG interface to the 5GCIEC International Electrotechnical Commission loT internet of thingsISO International Organization for StandardizationISOBMFF ISO base media file formatJPEG joint photographic experts groupLTE long-term evolution mdat MediaDataBoxMIME Multipurpose Internet Mail ExtensionMME mobility management entity moov MovieBoxMP4 fde format for MPEG-4 Part 14 fdesMPEG moving picture experts groupMPEG-2 H.222 / H.262 as defined by the ITUMPEG-4 audio and video coding standard for ISO / IEC14496 ng or NG new generation ng-eNB or NG-eNB new generation eNB NR new radio (5G radio)N / W or NW networkPDCP packet data convergence protocolPHY physical layerPNG portable network graphicsRAN radio access networkRFC request for commentsR C radio link controlRRC radio resource controlRRH remote radio headRU radio unitRx receiverSDAP service data adaptation protocolSGW serving gatewaySMF session management functionSPS sequence parameter setSVC scalable video codingSI interface between eNodeBs and the EPC trak TrackBoxTx transmitterUE user equipmentUICC Universal Integrated Circuit CardUPF user plane functionURL uniform resource locatorX2 interconnecting interface between two eNodeBs in LTE networkXn interface between two NG-RAN nodes

[0120] Some embodiments will now be described more fully hereinafter with reference to the accompanying drawings, in which some, but not all, embodiments may be shown. Indeed, various embodiments of the invention may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will satisfy applicable legal requirements. Like reference numerals refer to like elements throughout. As used herein, the terms ‘data,’ ‘content,’ ‘information,’ and similar terms may be used interchangeably to refer to data capable of being transmitted, received and / or stored in accordance with embodiments of the present invention. Thus, use of any such terms should not be taken to limit the spirit and scope of embodiments.

[0121] Described herein is a method and apparatus for implementing high level syntax headers in video coding.

[0122] The following describes in detail a suitable apparatus and possible method for implementing high level syntax headers in video coding according to embodiments. In this regard reference is first made to FIG. 1 and FIG. 2, where FIG. 1 shows an example block diagram of an electronic device or apparatus 100. The apparatus 100 may be an Internet of Things (loT) apparatus configured to perform various functions, such as for example, gathering information by one or more sensors, receiving or transmitting information, analyzing information gathered or received by the apparatus, or the like. The apparatus may comprise a video coding system, which may incorporate a codec. FIG. 2 shows a layout of an apparatus according to an example embodiment. The elements of FIG. 1 and FIG. 2 are explained next.

[0123] The apparatus 100 may for example be a mobile terminal or user equipment of a wireless communication system, a sensor device, a tag, or other lower power device. However, it would be appreciated that embodiments of the examples described herein may be implemented within any electronic device or apparatus which may process data by neural networks.

[0124] The apparatus 100 may comprise a housing 101 for incorporating and protecting the device. The apparatus 100 further may comprise a display 102 in the form of a liquid crystal display. In other embodiments of the examples described herein the display may be any suitable display technologysuitable to display an image or video. The apparatus 100 may further comprise a keypad 104. In other embodiments of the examples described herein any suitable data or user interface mechanism may be employed. For example the user interface may be implemented as a virtual keyboard or data entry system as part of a touch-sensitive display.

[0125] The apparatus may comprise a microphone 106 or any suitable audio input which may be a digital or analog signal input. The apparatus 100 may further comprise an audio output device which in embodiments of the examples described herein may be any one of: an earpiece 108, speaker, or an analog audio or digital audio output connection. The apparatus 100 may also comprise a battery (or in other embodiments of the examples described herein the device may be powered by any suitable mobile energy device such as solar cell, fuel cell or clockwork generator). The apparatus 100 may further comprise a camera 109 capable of recording or capturing images and / or video. The apparatus 100 may further comprise an infrared port for short range line of sight communication to other devices. In other embodiments the apparatus 100 may further comprise any suitable short range communication solution such as for example a Bluetooth wireless connection or a USB / firewire wired connection.

[0126] The apparatus 100 may comprise a controller 110, processor or processor circuitry for controlling the apparatus 100. The controller 110 may be connected to memory 112 which in embodiments of the examples described herein may store both data in the form of image and audio data and / or may also store instructions for implementation on the controller 110. The controller 110 may further be connected to codec circuitry 114 suitable for carrying out coding and / or decoding of audio and / or video data or assisting in coding and / or decoding carried out by the controller.

[0127] The apparatus 100 may further comprise a card reader 118 and a smart card 116, for example a UICC and UICC reader for providing user information and being suitable for providing authentication information for authentication and authorization of the user at a network.

[0128] The apparatus 100 may comprise radio interface circuitry 120 connected to the controller and suitable for generating wireless communication signals for example for communication with a cellular communications network, a wireless communications system or a wireless local area network. The apparatus 100 may further comprise an antenna 122 connected to the radio interface circuitry 120 for transmitting radio frequency signals generated at the radio interface circuitry 120 to other apparatus(es) and / or for receiving radio frequency signals from other apparatus(es).

[0129] The apparatus 100 may comprise a camera capable of recording or detecting individual frames which are then passed to the codec circuitry 114 or the controller for processing. The apparatus may receive the video image data for processing from another device prior to transmission and / orstorage. The apparatus 100 may also receive either wirelessly or by a wired connection the image for coding / decoding. The structural elements of apparatus 100 described above represent examples of means for performing a corresponding function.

[0130] With respect to FIG. 3, an example of a system within which embodiments of the examples described herein can be utilized is shown. The system 300 comprises multiple communication devices which can communicate through one or more networks. The system 300 may comprise any combination of wired or wireless networks including, but not limited to a wireless cellular telephone network (such as a GSM, UMTS, CDMA, LTE, 4G, 5G network, etc.), a wireless local area network (WLAN) such as defined by any of the IEEE 802.x standards, a Bluetooth personal area network, an Ethernet local area network, a token ring local area network, a wide area network, and the Internet.

[0131] The system 300 may include both wired and wireless communication devices and / or apparatus 100 suitable for implementing embodiments of the examples described herein.

[0132] For example, the system shown in FIG. 3 shows a mobile telephone network 301 and a representation of the internet 302. Connectivity to the internet 302 may include, but is not limited to, long range wireless connections, short range wireless connections, and various wired connections including, but not limited to, telephone lines, cable lines, power lines, and similar communication pathways.

[0133] The example communication devices shown in the system 300 may include, but are not limited to, an electronic device or apparatus 100, a combination of a personal digital assistant (PDA) and a mobile telephone 304, a PDA 306, an integrated messaging device (IMD) 308, a desktop computer 310, a notebook computer 312, or a head-mounted apparatus. The head-mounted apparatus may be a head-mounted display (HMD), or glasses having a device such as a camera configured to encode and / or decode images and / or video. The apparatus 100 may be stationary or mobile when carried by an individual who is moving. The apparatus 100 may also be located in a mode of transport including, but not limited to, a car, a truck, a taxi, a bus, a train, a boat, an airplane, a bicycle, a motorcycle or any similar suitable mode of transport.

[0134] The embodiments may also be implemented in a set-top box; e.g., a digital TV receiver, which may / may not have a display or wireless capabilities, in tablets or (laptop) personal computers (PC), which have hardware and / or software to process neural network data, in various operating systems, and in chipsets, processors, DSPs and / or embedded systems offering hardware / software based coding.

[0135] Some or further apparatus may send and receive calls and messages and communicatewith service providers through a wireless connection 314 to a base station 316. The base station 316 may be connected to a network server 318 that allows communication between the mobile telephone network 301 and the internet 302. The system may include additional communication devices and communication devices of various types.

[0136] The communication devices may communicate using various transmission technologies including, but not limited to, code division multiple access (CDMA), global systems for mobile communications (GSM), universal mobile telecommunications system (UMTS), time divisional multiple access (TDMA), frequency division multiple access (FDMA), transmission control protocolinternet protocol (TCP-IP), short messaging service (SMS), multimedia messaging service (MMS), email, instant messaging service (IMS), Bluetooth, IEEE 802.11, 3GPP Narrowband loT and any similar wireless communication technology. A communications device involved in implementing various embodiments of the examples described herein may communicate using various media including, but not limited to, radio, infrared, laser, cable connections, and any suitable connection.

[0137] In telecommunications and data networks, a channel may refer either to a physical channel or to a logical channel. A physical channel may refer to a physical transmission medium such as a wire, whereas a logical channel may refer to a logical connection over a multiplexed medium, capable of conveying several logical channels. A channel may be used for conveying an information signal, for example a bitstream, from one or several senders (or transmitters) to one or several receivers.

[0138] The embodiments may also be implemented in so-called loT devices. The Internet of Things (loT) may be defined, for example, as an interconnection of uniquely identifiable embedded computing devices within the existing Internet infrastructure. The convergence of various technologies has and may enable many fields of embedded systems, such as wireless sensor networks, control systems, home / building automation, etc. to be included in the Internet of Things (loT). In order to utilize the Internet loT devices are provided with an IP address as a unique identifier. loT devices may be provided with a radio transmitter, such as a WLAN or Bluetooth transmitter or a RFID tag. Alternatively, loT devices may have access to an IP -based network via a wired network, such as an Ethernet-based network or a power-line connection (PLC).

[0139] FIG. 4 is a block diagram illustrating a system or apparatus 400 in accordance with several examples. In an example, the encoder 402 is used to encode an image or video from the scene 404, and the encoder 402 is implemented in a transmitting apparatus 406. The encoder 402 produces a bitstream 408 comprising signaling that is received by the receiving apparatus 410, which implements a decoder 412. The encoder 402 sends the bitstream 408 that comprises the herein described signaling. The decoder 412 forms the image or video for the scene 404-1, and the receiving apparatus 410 wouldpresent this to the user, e.g., via a smartphone, television, or projector among many other options.

[0140] In some examples, the transmitting apparatus 406 and the receiving apparatus 410 are at least partially within a common apparatus, and for example, are located within a common housing 414. In other examples the transmitting apparatus 406 and the receiving apparatus 410 are at least partially not within a common apparatus and have at least partially different housings. Therefore in some examples, the encoder 402 and the decoder 412 are at least partially within a common apparatus, and for example are located within a common housing 414. For example, the common apparatus comprising the encoder 402 and decoder 412 implements a codec. In other examples, the encoder 402 and the decoder 412 are at least partially not within a common apparatus and have at least partially different housings, but when together still implement a codec.

[0141] In some examples, 3D media from the capture (e.g., volumetric capture) at a viewpoint 416 of the scene 404, which includes a person 418) is converted via projection to a series of 2D representations with occupancy, geometry, attributes and / or displacements. Additional atlas information is also included in the bitstream to enable inverse reconstruction. For decoding, the received bitstream 408 is separated into its components with atlas information; occupancy, geometry, displacement, and attribute 2D representations. A 3D reconstruction is performed to reconstruct the scene 404-1 created looking at the viewpoint 416-1 with a “reconstructed” person 418-1. The “-1” are used to indicate that these are reconstructions of the original. As indicated at 420, the decoder 412 performs an operation(s) or action(s) based on the received signaling.

[0142] Encoding 422 performs encoding of high level syntax headers based on the examples described herein. Decoding 424 performs decoding of high level syntax headers, based on the examples described herein.

[0143] Version 1 of the High Efficiency Video Coding (H.265 / HEVC a.k.a. HEVC) standard was developed by the Joint Collaborative Team - Video Coding (JCT-VC) of VCEG and MPEG. The standard was published by both parent standardization organizations, and it is referred to as ITU-T Recommendation H.265 and ISO / IEC International Standard 23008-2, also known as MPEG-H Part 2 High Efficiency Video Coding (HEVC). Later versions of H.265 / HEVC included scalable, multiview, fidelity range, three-dimensional, and screen content coding extensions which may be abbreviated SHVC, MV-HEVC, REXT, 3D-HEVC, and SCC, respectively.

[0144] Versatile Video Coding (VVC) (MPEG-I Part 3), a.k.a. ITU-T H.266, is a video compression standard developed by the Joint Video Experts Team (JVET) of the Moving Picture Experts Group (MPEG), (formally ISO / IEC JTC1 SC29 WG11) and Video Coding Experts Group(VCEG) of the International Telecommunication Union (ITU) to be the successor to HEVC / H.265.

[0145] A specification of the AV 1 bitstream format and decoding process were developed by the Alliance for Open Media (AOM). The AVI specification was published in 2018. AOM is reportedly working on the AV2 specification.

[0146] Some key definitions, bitstream and coding structures, and concepts of some video coding standards and specifications are described in this section for providing background for a video encoder, decoder, encoding method, decoding method, and a bitstream structure, wherein the embodiments may be implemented. It is to be understood that embodiments are not limited to the referenced video coding standards or specifications.

[0147] Video coding standards may specify the bitstream syntax and semantics as well as the decoding process for error-free bitstreams, whereas the encoding process might not be specified, but encoders may just be required to generate conforming bitstreams. Bitstream and decoder conformance can be verified with the Hypothetical Reference Decoder (HRD). The standards may contain coding tools that help in coping with transmission errors and losses, but the use of the tools in encoding may be optional and decoding process for erroneous bitstreams might not have been specified.

[0148] An elementary unit for the input to an encoder and the output of a decoder, respectively, in many cases is a picture. A picture given as an input to an encoder may also be referred to as a source picture, and a picture decoded by a decoded may be referred to as a decoded picture or a reconstructed picture.

[0149] The source and decoded pictures are each comprised of one or more sample arrays. The sample arrays of a picture may be referred to as luma (or L or Y) and chroma, where the two chroma arrays may be referred to as Cb and Cr; regardless of the actual color representation method in use. The actual color representation method in use can be indicated e.g., in a coded bitstream e.g., using the Video Usability Information (VUI) syntax of HEVC or alike. A component may be defined as an array or single sample from one of the three sample arrays (luma and two chroma) or the array or a single sample of the array that compose a picture in monochrome format.

[0150] A picture may be defined to be either a frame or a field. A frame comprises a matrix of luma samples and possibly the corresponding chroma samples. A field is a set of alternate sample rows of a frame and may be used as encoder input, when the source signal is interlaced. Chroma sample arrays may be absent (and hence monochrome sampling may be in use) or chroma sample arrays may be subsampled when compared to luma sample arrays.

[0151] A bitstream may be defined as a sequence of bits or a sequence of syntax structures. A bitstream format may constrain the order of syntax structures in the bitstream.

[0152] A syntax element may be defined as an element of data represented in a bitstream. A syntax structure may be defined as zero or more syntax elements present together in a bitstream in a specified order.

[0153] Syntax structures may be specified, for example, using arithmetic, logical, relational, bitwise, and assignment operators similar to those available in many programming languages. For example, & may indicate a bit-wise ‘AND’ operation. Furthermore, syntax structures may be specified with reference to mathematical functions.

[0154] Syntax structures and semantics may use the values of variables derived from the values of syntax elements. Naming conventions may be defined for variables. For example, variables may be named by a mixture of lower case and upper case letter and without any underscore characters. Variables starting with an upper case letter may be derived for the decoding of the current syntax structure and all depending syntax structures. Variables starting with an upper case letter may, in some cases, be used in the decoding process for later syntax structures without mentioning the originating syntax structure of the variable. Variables starting with a lower case letter may only be used in relation to the syntax structure or function they have been defined for.

[0155] Video coding specifications may define an elementary unit that for the output an of an encoder and / or for the input to a decoder. For example, such an elementary unit may be an open bitstream unit (OBU), as specified e.g. in AVI, or aNetwork Abstraction Layer (NAL) unit, as specified e g. in HEVC or VVC.

[0156] In some video codecs, an elementary unit for the output of an encoder and the input of a decoder, respectively, may be a Network Abstraction Layer (NAL) unit. For transport over packet- oriented networks or storage into structured files, NAL units may be encapsulated into packets or similar structures. A bytestream format has been specified in some video coding standards for transmission or storage environments that do not provide framing structures. The bytestream format separates NAL units from each other by attaching a start code in front of each NAL unit. To avoid false detection of NAL unit boundaries, encoders run a byte-oriented start code emulation prevention algorithm, which adds an emulation prevention byte to the NAL unit payload if a start code would have occurred otherwise. In order to enable straightforward gateway operation between packet- and stream-oriented systems, start code emulation prevention may always be performed regardless of whether the bytestream format is in use or not. A NAL unit may be defined as a syntax structure containing anindication of the type of data to follow and bytes containing that data in the form of an RBSP interspersed as necessary with emulation prevention bytes. A raw byte sequence payload (RBSP) may be defined as a syntax structure containing an integer number of bytes that is encapsulated in a NAL unit. An RBSP is either empty or has the form of a string of data bits containing syntax elements followed by an RBSP stop bit and followed by zero or more subsequent bits equal to 0.

[0157] A bitstream may be defined to logically include a syntax structure, such as a NAL unit, when the syntax structure is transmitted along the bitstream but may be included in the bitstream according to the bitstream format. A bitstream may be defined to natively comprise a syntax structure, when the bitstream includes the syntax structure.

[0158] In some coding formats or standards, a bitstream may be in the form of a network abstraction layer (NAL) unit stream or a byte stream, that forms the representation of coded pictures and associated data forming one or more coded video sequences.

[0159] In some coding formats, such as AVI, a bitstream may comprise a sequence of open bitstream units (OBUs). An OBU comprises a header and a payload, wherein the header identifies a type of the OBU. Furthermore, the header may comprise a size of the payload in bytes.

[0160] In some coding standards, NAL units include a header and payload. In some coding standards, the NAL unit header indicates the type of the NAL unit. In some coding standards, the NAL unit header indicates a scalability layer identifier (e.g., called nuh_layer_id), which may be used, e.g., for indicating spatial or quality layers, views of a multiview video, or auxiliary layers (such as depth maps or alpha planes). In some coding standards, the NAL unit header includes a temporal sublayer identifier, which may be used for indicating temporal subsets of the bitstream, such as a 30-frames-per- second subset of a 60-frames-per-second bitstream.

[0161] Bitstreams or coded video sequences may be encoded to be temporally scalable as follows. Each picture may be assigned to a particular temporal sub-layer. A temporal sub-layer may be equivalently called a sub-layer, temporal sublayer, sublayer, or temporal level. Temporal sub-layers may be enumerated, e.g., from 0 upwards. The lowest temporal sub-layer, sub-layer 0, may be decoded independently. Pictures at temporal sub-layer 1 may be predicted from reconstructed pictures at temporal sub-layers 0 and 1. Pictures at temporal sub-layer 2 may be predicted from reconstructed pictures at temporal sub-layers 0, 1, and 2, and so on. In other words, a picture at temporal sub-layer N does not use any picture at temporal sub-layer greater than N as a reference for inter prediction. The bitstream created by excluding all pictures greater than or equal to a selected sub-layer value and including pictures remains conforming.

[0162] Each picture of a temporally scalable bitstream may be assigned with a temporal identifier (also known as TID, temporal layer identifier, sub-layer identifier, sublayer identifier, temporal sublayer identifier, temporal sublayer identifier, or temporal layer ID), which may be, for example, assigned to a variable Temporalld. The temporal identifier may, for example, be indicated in a NAL unit header or in an OBU extension header. Temporalld equal to 0 corresponds to the lowest temporal level. The bitstream created by excluding all coded pictures having a Temporalld greater than or equal to a selected value and including all other coded pictures remains conforming. Consequently, a picture having Temporalld equal to tid value does not use any picture having a Temporalld greater than tid value as a prediction reference. In some video coding standards, a sub-layer or a temporal sub-layer may be defined to be a temporal scalable layer (or a temporal layer, TL) of a temporal scalable bitstream, consisting of VCL NAL units with a particular value of the Temporalld variable and the associated non- VCL NAL units.

[0163] NAL units can be categorized into Video Coding Layer (VCL) NAL units and non-VCL NAL units. VCL NAL units are typically coded slice NAL units.

[0164] A non-VCL NAL unit may be for example one of the following types: a video parameter set (VPS), a sequence parameter set (SPS), a picture parameter set (PPS), an adaptation parameter set (APS), a supplemental enhancement information (SEI) NAL unit, an access unit delimiter, an end of sequence (EOS) NAL unit, an end of bitstream (EOB) NAL unit, or a filler data NAL unit. Parameter sets may be needed for the reconstruction of decoded pictures, whereas many of the other non-VCL NAL units may not be necessary for the reconstruction of decoded sample values.

[0165] Some coding formats specify parameter sets that may carry parameter values needed for the decoding or reconstruction of decoded pictures. A parameter may be defined as a syntax element of a parameter set. A parameter set may be defined as a syntax structure that contains parameters and that can be referred to from or activated by another syntax structure, for example, using an identifier.

[0166] Instead of or in addition to parameter sets at different hierarchy levels (e.g., sequence and picture), video coding formats may include header syntax structures, such as a sequence header or a picture header.

[0167] A sequence header may precede any other data of the coded video sequence in the bitstream order. It may be allowed to repeat a sequence header in the bitstream, e.g., to provide a sequence header at a random access point.

[0168] A picture header may precede any coded video data for the picture in the bitstream order. A picture header may be interchangeably referred to as a frame header. Some video codingspecifications may enable carriage of a picture header in a dedicated picture header NAL unit or a frame header OBU or alike. Some video coding specifications may enable carriage of a picture header in a NAL unit, OBU, or alike syntax structure that also contains coded picture data.

[0169] A coded picture may be defined as a coded representation of a picture.

[0170] A random access point may be defined as a location within a bitstream where decoding can be started.

[0171] A random access picture or a Random Access Point (RAP) picture may be defined as a picture that serves as a random access point, i.e., as a picture where decoding can be started. In some contexts, the term random -access picture may be used interchangeably with the term RAP picture.

[0172] An intra random access point (IRAP) picture, when contained in a single-layer bitstream or an independent layer, may comprise only intra-coded image segments. Furthermore, an IRAP picture may constrain subsequence pictures in output order to be such that they can be correctly decoded without performing the decoding process of any pictures that precede the IRAP picture in decoding order.

[0173] In some contexts, an IRAP picture may be defined as one category of random -access pictures, characterized in that they contain only intra-coded image segments, whereas there may also be other category or categories of random-access pictures, such as a gradual decoding refresh (GDR) picture.

[0174] Gradual Decoding Refresh (GDR) often refers to the ability to start decoding at a non- IRAP picture and to recover decoded pictures that are correct in content after decoding a certain number of pictures. Said otherwise, GDR can be used to achieve random access from non-intra pictures. GDR, which is also known as Gradual random access (GRA) or Progressive Intra Refresh (PIR), alleviates the delay issue with intra coded pictures. Instead of coding an intra picture at a random access point, GDR progressively refreshes pictures by spreading intra coded regions (groups of intra coded blocks) over several pictures.

[0175] A GDR picture may be defined as a RAP picture that, when used to start the decoding process, enables recovery of exactly or approximately correct decoded pictures starting from a specific picture, known as the recovery point picture. It is possible to start decoding from a GDR picture.

[0176] In some video coding formats, such as VVC, all Video Coding Layer (VCL) Network Abstraction Layer (NAL) units of a GDR picture may have a particular NAL unit type value thatindicates a GDR NAL unit.

[0177] In some video coding formats, an SEI message, a metadata OBU or alike with a particular type, such as a recovery point SEI message of HEVC, may be used to indicate a GDR picture and / or a recovery point picture.

[0178] Some codecs use a concept of picture order count (POC). A value of POC is derived for each picture and is non-decreasing with increasing picture position in output order. POC therefore indicates the output order of pictures. The derived POC value may be assigned to the variable PicOrderCntVal. POC may be used in the decoding process for example for implicit scaling of motion vectors and for reference picture list initialization. Furthermore, POC may be used in the verification of output order conformance.

[0179] A recovery point may be indicated within a GDR picture, e.g., as a picture order count (POC) difference compared to the POC of the GDR picture. When the decoding started from the GDR picture, the decoded recovery point picture and all subsequent decoded pictures in output order are correct in content.

[0180] Pictures between the GDR picture (exclusive) and the recovery point picture (exclusive), in decoding order, may be referred to as recovering pictures. Recovering pictures may be partially correct in content, when the decoding started from the GDR picture.

[0181] It may be allowed that the recovery point picture is the same picture as the GDR picture (and consequently there are no recovering pictures). In this case, there may be pictures that follow the GDR picture in decoding order and precede the GDR picture in output order that are not correctly decodable when the decoding is started from the GDR picture.

[0182] Some coding standards or specifications, such as H.265 / HEVC, may use the NAL unit type of VCL NAL unit(s) of a picture to indicate a picture type. In H.266 / VVC, the NAL unit type indicates a picture type when mixed VCL NAL unit types within a coded picture are disabled (pps_mixed_nalu_types_in_pic_flag is equal to 0 in the referenced PPS), while otherwise it indicates a subpicture type.

[0183] Types and abbreviations for VCL NAL unit types may include one or more of the following: trailing (TRAIL), Temporal Sub-layer Access (TSA), Step-wise Temporal Sub-layer Access (STSA), Random Access Decodable Leading (RADL), Random Access Skipped Leading (RASL), Instantaneous Decoding Refresh (IDR), Clean Random Access (CRA), Gradual Decoding Refresh (GDR). When all VCL NAL units of a picture have the same NAL unit type, the types and abbreviationsmay be used as picture types, trailing picture (a.k.a. TRAIL picture).

[0184] Some VCL NAL unit types may be more fine-grained as indicated in the paragraph above. Lor example, two types of IDR pictures may be specified, IDR without leading pictures, IDR with random access decodable leading pictures (i.e., without RASL pictures).

[0185] In VVC, an IRAP picture may be a CRA picture or an IDR picture.

[0186] In HEVC and VVC, provided the necessary parameter sets are available when they are activated or referenced, an IRAP picture at an independent layer and all subsequent non-RASL pictures at the independent layer in decoding order can be correctly decoded without performing the decoding process of any pictures that precede the IRAP picture in decoding order.

[0187] In HEVC and VVC, a CRA picture may be the first picture in the bitstream in decoding order, or may appear later in the bitstream. CRA pictures allow so-called leading pictures that follow the CRA picture in decoding order but precede it in output order. Some of the leading pictures, so-called RASL pictures, may use pictures decoded before the CRA picture (in decoding order) as a reference. Pictures that follow a CRA picture in both decoding and output order are decodable if random access is performed at the CRA picture, and hence clean random access is achieved similarly to the clean random access functionality of an IDR picture.

[0188] A CRA picture may have associated RADL or RASL pictures. When a CRA picture is the first picture in the bitstream in decoding order, the CRA picture is the first picture of a coded video sequence in decoding order, and any associated RASL pictures are not output by the decoder and may not be decodable, as they may contain references to pictures that are not present in the bitstream.

[0189] A leading picture is a picture that precedes the associated RAP picture in output order and follows the associated RAP picture in decoding order. The associated RAP picture is the previous RAP picture in decoding order (if present). In some coding specifications, such as HEVC and VVC, a leading picture is either a RADL picture or a RASL picture.

[0190] All RASL pictures are leading pictures of an associated IRAP picture (e.g., CRA picture). When the associated RAP picture is the first coded picture in the coded video sequence or in the bitstream, the RASL picture is not output and may not be correctly decodable, as the RASL picture may contain references to pictures that are not present in the bitstream. However, a RASL picture can be correctly decoded if the decoding had started from a RAP picture before the associated RAP picture of the RASL picture. RASL pictures are not used as reference pictures for the decoding process of non- RASL pictures. When present, all RASL pictures precede, in decoding order, all trailing pictures of thesame associated RAP picture.

[0191] All RADL pictures are leading pictures. RADL pictures are not used as reference pictures for the decoding process of trailing pictures of the same associated RAP picture. When present, all RADL pictures precede, in decoding order, all trailing pictures of the same associated RAP picture. RADL pictures do not refer to any picture preceding the associated RAP picture in decoding order and can therefore be correctly decoded when the decoding starts from the associated RAP picture.

[0192] In HEVC and VVC, an IDR picture in an independent layer does not use inter prediction in its decoding process. An IDR picture could be the first picture in the bitstream in decoding order, or could appear later in the bitstream. Each IDR picture is the first picture of a coded video sequence in decoding order. An IDR picture does not have associated RASL pictures.

[0193] Two IDR picture types may be defined and indicated: IDR pictures without leading pictures, which may use a defined NAL unit type denoted as IDR N LP, and IDR pictures that may have associated decodable leading pictures (i.e., RADL pictures), which may use a defined NAL unit type denoted as IDR W RADL.

[0194] A trailing picture may be defined as a picture that follows the associated RAP picture in output order (and also in decoding order). Additionally, a trailing picture may be required not to be classified as any other picture type, such as STSA picture.

[0195] A sub-layer access picture may be defined as a picture from which the decoding of a sublayer can be started correctly, i.e., starting from which all pictures of the sub-layer can be correctly decoded. In HEVC there are two picture types, the temporal sub-layer access (TSA) and step-wise temporal sub-layer access (STSA) picture types, that can be used to indicate temporal sub-layer switching points. If temporal sub-layers with Temporalld up to N had been decoded until the TSA or STSA picture (exclusive) and the TSA or STSA picture has Temporalld equal to N+l, the TSA or STSA picture enables decoding of all subsequent pictures (in decoding order) having Temporalld equal to N+L The TSA picture type may impose restrictions on the TSA picture itself and all pictures in the same sub-layer that follow the TSA picture in decoding order. None of these pictures is allowed to use inter prediction from any picture in the same sub-layer that precedes the TSA picture in decoding order. The TSA definition may further impose restrictions on the pictures in higher sub-layers that follow the TSA picture in decoding order. None of these pictures is allowed to refer a picture that precedes the TSA picture in decoding order if that picture belongs to the same or higher sub-layer as the TSA picture. TSA pictures have Temporalld greater than 0. The STSA is similar to the TSA picture but does not impose restrictions on the pictures in higher sub-layers that follow the STSA picture in decoding orderand hence enable up-switching only onto the sub-layer where the STS A picture resides.

[0196] Some coding standards or specifications may indicate a picture type in a picture header or a frame header or alike.

[0197] In some coding formats, picture unit (PU) may be defined as a set of data units, such as NAL units, that are associated with each other, are consecutive in decoding order, and contain exactly one coded picture. For example, certain non-video-coding data units, such as non-VCL NAL units, may be next to coded video data units in decoding order and the respective picture unit may comprise both these non-video-coding data units and the video coding data units of a coded picture.

[0198] In some coding formats, an access unit (AU) may be defined as a set of NAL units that are associated with each other according to a specified classification rule, are consecutive in decoding order, and include at most one coded picture at any scalability layer (e.g., with any specific value of nuh layer id in some coding formats, such as HEVC or VVC). In some coding formats, an access unit comprises one or more complete picture units. In some coding formats, in addition to including the VCL NAL units of a coded picture, an access unit may also include non-VCL NAL units associated with the coded picture. Said specified classification rule may, for example, associate pictures with the same output time or picture order count value into the same access unit.

[0199] In some coding formats, a coded video sequence (CVS) may be defined as a sequence of coded pictures in decoding order that is independently decodable and is followed by another coded video sequence or the end of the bitstream.

[0200] In some coding formats, such as AVI, a coded video sequence comprises one or more temporal units. A temporal unit consists of a series of OBUs starting from a temporal delimiter, optional sequence headers, optional metadata OBUs, a sequence of one or more frame headers, each followed by zero or more tile group OBUs as well as optional padding OBUs. A temporal unit may be defined to comprise all the OBUs that are associated with a specific, distinct time instant. A temporal unit may comprise a temporal delimiter OBU, and all the OBUs that follow, up to but not including the next temporal delimiter. A temporal delimiter OBU may be defined as an indication that the following OBUs will have a different presentation / decoding time stamp from the one of the last frame prior to the temporal delimiter.

[0201] A coded layer video sequence (CLVS) may be defined as a sequence of pictures and associated other data within the same scalable layer (e.g., with the same value of nuh_layer_id) that is decodable independently of other pictures in the same layer.

[0202] One or more syntax structures for (decoded) reference picture marking may exist in a video coding system. An encoder generates an instance of a syntax structure e.g., in each coded picture, and a decoder decodes an instance of the syntax structure e.g., from each coded picture. For example, the decoding of the syntax structure may cause pictures to be adaptively marked as "used for reference" or "unused for reference".

[0203] A reference picture set (RPS) syntax structure of HEVC is an example of a syntax structure for reference picture marking. A reference picture set valid or active for a picture includes all the reference pictures that may be used as reference for the picture and all the reference pictures that are kept marked as "used for reference" for any subsequent pictures in decoding order. The reference pictures that are kept marked as "used for reference" for any subsequent pictures in decoding order but that are not used as reference picture for the current picture or image segment may be considered inactive. For example, they might not be included in the initial reference picture list(s).

[0204] A media aware network element (MANE) may be defined as an intermediate device that may adapt a scalable video according to network, terminal and / or user capabilities.

[0205] Many multipoint audio-visual conferences operate utilizing a centralized unit called Multipoint Control Unit (MCU), which may be regarded as one example of a MANE. An MCU may implement the functionality of a Real-time transport protocol (RTP) translator or an RTP mixer. An RTP translator may be a media translator that may modify the media inside the RTP stream. A media translator may for example decode and re-encode the media content (i.e. transcode the media content). An RTP mixer aggregates multiple RTP streams that are part of a session by generating one or more new RTP streams. An RTP mixer may manipulate the media data. One common application for a mixer is to allow a participant to receive a session with a reduced amount of resources compared to receiving individual RTP streams from all endpoints. A mixer can be viewed as a device terminating the RTP streams received from other endpoints in the same RTP session. Using the media data carried in the received RTP streams, a mixer generates derived RTP streams that are sent to the receiving endpoints.

[0206] In another example, a MANE is a selective forwarding unit (SFU) that selectively forwards incoming data (e.g., RTP packets) from one or more senders to one or more receivers.

[0207] Having thus introduced a suitable but non-limiting technical context for the practice of the example embodiments of the present disclosure, example embodiments will now be described in detail.

[0208] Video coding standards provide a capability to encode a bitstream containing random access points, which enable decoders to begin decoding a bitstream at random points, so that decodersare not required to decode the entire bitstream. This is particularly useful for broadcast applications, where a viewer may tune in to a particular channel at a random time.

[0209] MPEG-2 Program Elementary Stream (PES) start codes consist of a 24-bit prefix (0x000001) and an 8-bit stream_id specified in Table 2-22 of ITU-T Rec. H.222.0 | ISO / IEC 13818-1. stream_id values are in the range of 10111100b to 11111111b, i.e. 188 to 255, as can be observed from the following excerpt of Table ‘Stream_id assignments’ of ITU-T Rec. H.222.0 | ISO / IEC 13818-1:

[0210] Because of flexibility of reference frame selection in inter coded frames, the presence of an intra coded frame is not sufficient to guarantee that frames following in decoding order are decodable, because those frames might reference a frame that preceded the intra frame.

[0211] Intra coded frames are referred to as key frames in the AVI specification.

[0212] The AVI specification in section 7.6.2 defines the following random access point (RAP) and recovery point (RP) terms.

[0213] A key frame random access point is defined as being a frame:- with frame type equal to KEY FRAME- with show frame equal to 1- that is contained in a temporal unit that also contains a sequence header OBU

[0214] In an example, a temporal unit consists of all the OBUs that are associated with a specific, distinct time instant. In another example, the temporal unit consists of a temporal delimiter OBU, and all the OBUs that follow, up to, but not including the next temporal delimiter.

[0215] A delayed random access point is defined as being a frame:- with frame type equal to KEY FRAME- with show frame equal to 0- that is contained in a temporal unit that also contains a sequence header OBU

[0216] A sequence header OBU contains information for the entire sequence, including the sequence profile and activation of specific coding tools.

[0217] A key frame dependent recovery point is defined as being a frame:- with show_existing_frame equal to 1- with frame to show map idx specifying a frame to output that was a delayed random access point

[0218] The AVI specification describes OBU as follows: all structures are packetized in “Open Bitstream Units” or OBUs. Each OBU has a header, which provides identifying information for the contained data (payload).

[0219] The OBU syntax structure in the AVI specification is shown below.

[0220] The following obu type values are specified in AV 1 : :I

[0221] The AVI frame header contains a syntax element called show_frame, with the below semantics:

[0222] show_frame equal to 1 specifies that this frame should be immediately output once decoded, show frame equal to 0 specifies that this frame should not be immediately output. (It may be output later if a later uncompressed header uses show_existing_frame equal to 1).

[0223] The AVI OBU is similar to the Network Abstraction Layer (NAL) unit in AVC, HEVC, and VVC.

[0224] The HEVC NAL unit header is shown below.

[0225] The HEVC standard defines several types of random access pictures, which are identified via the nal_unit_type syntax element in the NAL unit header. The defined types include the following:- IDR: An IDR picture does not refer to any pictures other than itself for inter prediction in its decoding process, and can be the first picture in the bitstream in order or can appear later in the bitstream. Each IDR picture is the first picture of a CVS in decoding order. When an IDR picture for which each VCL NAL unit has nal unit type equal to IDR W RADL, it can have associated RADL pictures. When an IDR picture for which each VCL NAL unit has nal unit type equal to IDR N LP, it does not have any associated leading pictures. An IDR picture does not have associated RASL pictures.- CRA: A CRA picture does not refer to any pictures other than itself for inter prediction in its decoding process, and can be the first picture in the bitstream in decoding order, or can appear later in the bitstream. A CRA picture can have associated RADL or RASL pictures. A CRA picture can contain syntax elements that specify a non-empty RPS. A decoder may provide means to set the value of the NoRaslOutputFlag variable outside of the decoder. Lor example, a player application may set the value of NoRaslOutputFlag equal to 1, when decoding and playback starts from a CRA picture. When a CRA picture has NoRaslOutputLlag equal to 1, the associated RASL pictures are not output by the decoder, because they can be non- decodable, as they can contain references to pictures that are not present in the bitstream.

[0226] An HEVC decoder can identify the presence of an IDR or CRA picture from information in the NAL unit header.

[0227] The HEVC standard includes a recovery point Supplemental Enhancement Information (SEI) message that assists a decoder in determining when the decoding process will produce acceptablepictures for display after the decoder initiates random access. The syntax for the recovery point SEI message is provided below.

[0228] The semantics for the HEVC recovery point SEI message include the following: all decoded pictures in output order are indicated to be correct or approximately correct in content starting at the output order position of the recovery point picture.

[0229] The VVC standard includes a ph_recovery_poc_cnt syntax element in the picture header. The syntax element ph rcco vciyjioc cnt is present in the picture header of a GDR picture and specifies the recovery point of decoded pictures in output order. The variable recoveryPointPocVal is derived to be equal to PicOrderCntVal + ph_recovery_poc_cnt, where PicOrderCntVal is the POC value of the current picture. The value of recoveryPointPocVal specifies the POC value of the recovery point picture . If the current picture is a GDR picture and ph_recovery_poc_cnt is equal to 0, the current picture itself is also referred to as the recovery point picture. Otherwise, if the current picture is a GDR picture, and there is a picture picA that follows the current GDR picture in decoding order in the CLVS that has PicOrderCntVal equal to recoveryPointPocVal, the picture picA is referred to as the recovery point picture, otherwise, the first picture in output order that has PicOrderCntVal greater than recoveryPointPocVal in the CLVS is referred to as the recovery point picture.

[0230] When sps_gdr_cnablcd_flag is equal to 1 and PicOrderCntVal of the current picture is greater than or equal to recoveryPointPocVal of the associated GDR picture, the current and subsequent decoded pictures in output order are exact match to the corresponding pictures produced by starting the decoding process from the previous IRAP picture, when present, preceding the associated GDR picture in decoding order.

[0231] The syntax of the VVC NAL unit header is as follows:

[0232] VVC includes a forbidden_zero_bit syntax element in the NAL unit header, described as follows:

[0233] forbidden_zero_bit causes the value of the first byte of the NAL unit header to be in the range of 0 to 127, inclusive. Similarly, since obu_forbidden_bit is required to be 0, the first byte of the OBU header is in the range of 0 to 127, inclusive.

[0234] Hence, the combination of a start code prefix 0x000001 and the first byte of the NAL unit header or the OBU header does not cause an emulation of MPEG-2 PES start code.

[0235] The semantics of nuh_reserved_zero_bit are specified in VVC as follows: nuh_reserved_zero_bit shall be equal to 0. The value 1 of nuh_reserved_zero_bit could be specified in the future by ITU T | ISO / IEC. Although the value of nuh reserved zero bit is required to be equal to 0 in this version of this specification, decoders conforming to this version of this specification shall also allow the value of nuh_reserved_zero_bit equal to 1 to appear in the syntax and shall ignore (i.e., remove from the bitstream and discard) NAL units with nuh_reserved_zero_bit equal to 1.

[0236] In AVI, it is not possible to identify the location of a random access point (RAP) or recovery point (RP) simply from the information available in the OBU header, which is easily accessible to a decoder or MANE. Instead, to determine location of a RAP or recovery point, it is required to parse and process lower level syntax structures, including the frame header. This places a burden on a decoder when determining when to begin to decode bitstreams at random times and when to display decoded pictures, because of the need for additional processing.

[0237] Various embodiments propose to enable identification of random access points and recovery points using information available in a modified OBU header, which may be defined in a backwards-compatible extension to AVI or a future AV2 or AVx standard.

[0238] Backwards compatibility is beneficial because bitstreams created using the proposed modifications could still be decoded by existing AV 1 decoders which use the methods to determine the presence of random access points and recovery points.

[0239] The embodiments may also be used, for example, in a future H.267 standard, to identify the recovery point using the NAL unit header, without requiring access to the picture header or an SEI message.

[0240] In following embodiments, several options are proposed to modify the AV 1 OBU header to enable of random access points and recovery points using information available in a modified OBU header, and in some options a modified OBU extension header. Encoder and decoder operations are described for each option. Decoder / MANE operation describe how the information in the OBU header to identify the three types of random access and recovery points defined in AVI - key frame RAP, delayed RAP, and key frame dependent RP.

[0241] Example option 1 : Single bit in obu headerf )

[0242] In this example option, the existing reserved bit in the AV 1 OBU header is repurposed to carry a random access point flag to indicate when the OBU includes an RAP. A single bit is used in this case, the flag indicates that the OBU includes either a key frame RAP or a delayed RAP.

[0243] Because an existing reserved bit is used, a bitstream created using the proposed embodiment, still be properly decoded by existing decoders or MANEs, which could use existing methods to identify a key frame RAP or delayed RAP. But decoders or MANEs using the invention could more easily identify the location of a RAP for bitstreams created using the proposed embodiments.

[0244] With this option, when a decoder or MANE wishes to guarantee to avoid displaying some incorrectly decoded frames while minimizing the delay in displaying correct frames, the decoder requires additional processing to distinguish between a key frame RAP and a delayed RAP.

[0245] A modified syntax table is provided below.

[0246] In another syntax option, the following modified syntax table is used:

[0247] In an embodiment for this syntax option, obu rap flag shall be equal to 0, when obu type is greater than or equal to 7. MPEG-2 PES start code emulation can be avoided when values 188 to 255, inclusive, are not used in the first byte of the OBU header. When obu rap flag is allowed to be 1 when obu type is less than 7, the first byte of the OBU header does not exceed 183 and no MPEG-2 PES start code emulation takes place.

[0248] obu_rap_flag equal to 1 indicates that the frame is a key frame random access point or a delayed random access point, obu rap flag equal to 0 indicates that the frame is not a key frame random access point or a delayed random access point.

[0249] Encoder and decoder operation

[0250] An encoder according to this embodiment performs the following to indicate a RAP (either key frame RAP or delayed RAP) in the bitstream:- Selecting a first frame to be a IRAP (either key frame RAP or delayed RAP); - Setting obu rap flag equal to 1 for all OBUs in the temporal unit of the first frame, including:- temporal delimiter OBU;- sequence header OBU;- Optional frame header OBU;- One or more tile groups OBUs (all of at least one tile group OBUs); and / or- Optional metadata OBU- Setting frame_type = KEY_FRAME in the frame_header of the first frame; and- Coding selected frame as intra.

[0251] An encoder may additionally perform the following to indicate a non-RAP in the same bitstream:- Selecting a second frame to be non-IRAP, following the first frame in coding order;- Setting obu rap flag equal to 0 for all OBUs in the temporal unit of the second frame;- Setting frame type = INTER FRAME in the in the frame header of the second frame; and- Coding the second frame as inter, while setting the reference pictures (including alt ref frame, golden frame, last_frame, last2_frame) to the first frame or another frame that follows the first frame in coding order.

[0252] A decoder receiving a bitstream created by such an encoder can perform random access by performing the following:- Receiving streaming or broadcast video;- Reading all OBU headers in coding order and discard all OBUs until a first frame is identified with obu rap flag equal to 1 ;- Decoding the first frame; and- Decoding subsequent frames.

[0253] A MANE may also perform detection of random access points to identify locations for splicing.

[0254] In another embodiment, an encoder may perform the following steps to indicate a key frame RAP and a non-RAP frame in a bitstream:- Selecting a first frame to be a key frame RAP;- Setting obu rap flag equal to 1 for all OBUs in the temporal unit of first frame;- Setting frame type = KEY FRAME in frame header of first frame;- Setting show_frame = 1 in frame_header of first frame;- Coding frame as intra;- Selecting a second frame to be non-RAP;- Setting obu rap flag equal to 0 for all OBUs in the temporal unit of second frame;- Setting frame type = INTER FRAME in frame header of the second frame; and- Coding the second frame as inter, while setting the reference pictures, including any of alt ref frame, golden frame, last_frame, last2_frame, to the first frame or another frame that follows the first frame in coding order.

[0255] In this embodiment, an encoder may perform the following to indicate a delayed frame RAP and a non-RAP frame in a bitstream also including an additional frame:- Coding a first frame;- Selecting a second frame to be delayed RAP, following the first frame in coding order;- Setting obu rap flag equal to 1 for all OBUs in the temporal unit of second frame;- Setting frame type = KEY FRAME in frame header of second frame;- Setting show_frame = 0 in frame header of second frame;- Coding frame as intra;- Selecting a third frame to be non-RAP, following the second frame in coding order;- Setting obu rap flag equal to 0 for all OBUs in the temporal unit of third frame;- Setting frame type = INTER FRAME in frame header of third frame; and- Coding the third frame as inter, while setting at least one reference picture (which may be any of alt ref frame, golden frame, last_frame, last2_frame) to the first frame.

[0256] A decoder receiving a bitstream created by such an encoder can perform random access by performing the following:- Receiving streaming or broadcast video- Reading all OBU headers in coding order and discard all OBUs until a first frame is identified with obu rap flag equal to 1- Decoding the first frame and examine its frame header.- when show_existing_frame = 1, displaying the frame, otherwise not displaying the frame and continuing decoding frames and examine the frame header until find a second frame with show_existing_frame = 1- Decoding and displaying the second frame and following frames in the bitstream.

[0257] Example option 2: Two bits in OBU header

[0258] Two syntax options are presented, which share the same semantics, encoder, and decoder operation.

[0259] Example Option 2a: one bit in obu_header( ) and another one in obu_extension_header( )

[0260] In this option, the existing reserved bit in the AV 1 OBU header is repurposed to carry a random access point flag to indicate when the OBU includes a RAP, and a second bit in the OBU extension header is repurposed from being a reserved bit to be used to carry a RAP type flag, which is used in combination with the random access point flag to distinguish between a key frame RAP and a delayed frame RAP or to identify a key frame dependent recovery point. To utilize the OBU extension header, the extension flag in the OBU header is also set equal to 1.

[0261] A modified syntax table is provided below:

[0262] Example option 2b: two bits in obu headerf )

[0263] In this option, the existing forbidden and reserved bits in the AVI OBU header are repurposed to carry a random access point flag to indicate when the OBU includes a RAP and a RAP type flag, which is used in combination with the random access point flag to distinguish between a key frame RAP and a delayed frame RAP or to identify a key frame dependent recovery point.

[0264] A modified syntax table is provided below:

[0265] In an embodiment using this syntax option, an obu rap flag shall be equal to 0, when obu type is greater than or equal to 7. MPEG-2 PES start code emulation can be avoided when values 188 to 255, inclusive, are not used in the first byte of the OBU header. obu_rap_flag is allowed to be 1 when obu type is less than 7. When obu rap flag is allowed to be 1, the first byte of the OBU header does not exceed 183 and no MPEG-2 PES start code emulation takes place.

[0266] Semantics

[0267] obu_rap_flag equal to 1 indicates that the frame is a key frame random access point or a delayed random access point, obu rap flag equal to 0 indicates that the frame is not a key frame random access point or a delayed random access point.

[0268] obu rap type flag equal to 1, when obu rap flag is equal to 1, indicates that the frame is a key frame random access point, obu rap type flag equal to 0, when obu rap flag is equal to 1, indicates that the frame is a delayed random access point, obu rap type flag equal to 1, when obu_rap_flag equal is to 0, indicates that the frame is a key frame dependent recovery point, obu rap type flag equal to 0, when obu rap flag is equal to 0, indicates that the random access and recovery point characteristics of the frame are unspecified.

[0269] The example option 2 differs from the example option 1 in that, for example, a decoder / MANE is not required to examine the frame header to distinguish between a key frame RAP and a delayed RAP, and to identify a recovery point. This information may be used to determine when to start displaying frames to ensure to avoid displaying possibly incorrectly decoded frames. The use of two bits, one in the OBU header and one in the OBU extension header is enough to identify 4 different example frame characteristics: key frame RAP, delayed RAP, key frame dependent recovery point, and a non-RAP frame that is also not a recovery point.

[0270] The fourth example characteristic, identified by obu_rap_type_flag equal to 0 and obu rap flag equal to 0 may indicate that the RAP type is unspecified, or alternatively may specify that the frame is not a key frame RAP, delayed RAP, or key frame dependent recovery point frame. Similarly, when the OBU extension flag is not set, and hence obu_rap_type_flag is not present, indicates that the RAP type may either be unspecified, or may alternatively be specified to not be a key frame RAP, delayed RAP, or key frame dependent recovery point frame.

[0271] Encoder and decoder operation

[0272] In an example embodiment, an encoder may perform the following to indicate a key frame RAP in a bitstream that also includes an additional non-RAP frame:- Selecting a first frame to be key frame RAP;- Setting obu rap flag equal to 1 for all OBUs in the temporal unit of first frame;- Setting obu extension flag to 1 for all OBUs in the temporal unit of first frame (for example option 2a only);- Setting obu rap type flag to 1 for all OBUs in the temporal unit of first frame;- Setting frame type = KEY FRAME in frame header of first frame;- Setting show_frame = 1 in frame_header of first frame;- Coding frame as intra;- Selecting a second frame to be non-RAP;- Setting obu rap flag equal to 0 for all OBUs in the temporal unit of second frame;- Setting frame type = INTER FRAME in frame header of second frame; and- Coding the second frame as inter, while setting the reference pictures (including any of alt ref frame, golden frame, last_frame, last2_frame) to the first frame or another frame that follows the first frame in coding order.

[0273] In an example embodiment, the encoder may perform the following to indicate a delayed frame RAP and a non-RAP frame in a bitstream also including an additional frame:- Coding a first frame;- Selecting a second frame to be delayed RAP, following the first frame in coding order;- Setting obu rap flag equal to 1 for all OBUs in the temporal unit of second frame;- Setting obu extension flag to 1 for all OBUs in the temporal unit of second frame (for example option 2a only);- Setting obu rap type flag to 1 for all OBUs in the temporal unit of second frame;- Setting frame type = KEY FRAME in frame header of second frame;- Setting show_frame = 0 in frame header of second frame;- Coding second frame as intra;- Selecting a third frame to be non-RAP, following the second frame in coding order;- Setting obu rap flag equal to 0 for all OBUs in the temporal unit of third frame;- Setting frame type = INTER FRAME in frame header of third frame; and- Coding the third frame as inter, while setting at least one reference picture (which can be any of alt ref frame, golden frame, last_frame, last2_frame) to the first frame.

[0274] In an example embodiment, the encoder may perform the following to indicate a delayed frame RAP and a key frame dependent recovery point in a bitstream also including an additional frame:- Coding a first frame;- Selecting a second frame to be delayed RAP, following the first frame in coding order;- Setting obu rap flag equal to 1 for all OBUs in the temporal unit of second frame;- Setting obu extension flag to 1 for all OBUs in the temporal unit of second frame (for example option 2a only);- Setting obu rap type flag to 1 for all OBUs in the temporal unit of second frame;- Setting frame type = KEY FRAME in frame header of second frame;- Setting show_frame = 0 in frame header of second frame;- Coding second frame as intra;- Selecting a third frame to be key frame dependent recovery point, following the second frame in coding order;- Setting obu rap flag equal to 0 for all OBUs in the temporal unit of third frame;- Setting obu extension flag to 1 for all OBUs in the temporal unit of third frame (for example option 2a only);- Setting obu rap type flag to 1 for all OBUs in the temporal unit of third frame;- Setting frame type = INTER FRAME in frame header of third frame; and- Coding the third frame as inter, while setting the reference pictures (including any of alt ref frame, golden frame, last_frame, last2_frame) to the second frame or another frame that follows the second frame in coding order.

[0275] An example decoder performs the following to detect a RAP in the bitstream:- Receiving streaming or broadcast video- Reading all OBU headers in coding order and discard all OBUs until a first frame is identified with obu rap flag equal to 1- When obu rap type flag equal to 1 ;- Determining that the first frame is a key frame RAP;- Decoding and display the first frame; and- Decoding and displaying frames following first frame in coding order;- Otherwise, (obu rap type flag equal to 0);- Determining that the first frame is a delayed RAP, and decode the first frame and do not display the first frame;- Reading all OBU headers until find a second frame with obu rap flag equal to 0 and obu rap type flag equal to 1 ;- Decoding the second frame and display the second frame; and- Decoding and display frames following second frame in coding order.

[0276] Example option 3: Two bits in obu_extension_header( )

[0277] In this option, two bits in the OBU extension header are used to indicate the same four frame characteristics provided in the example option 2, but with both bits present in the extension header. The four characteristics are key frame RAP, delayed RAP, key frame dependent recovery point, and a non-RAP frame that is also not a recovery point.

[0278] In the example option 3, the OBU extension flag bit is required to be set to 1 to identify a key frame RAP, delayed RAP, or key frame dependent recovery point.

[0279] The OBU extension header is modified to repurpose two reserved bits to a RAP type syntax element as following:

[0280] When the obu_rap_type syntax element is present, the following frame characteristics can be defined. When obu_rap_type equal to 1, the frame is a key frame random access point. When obu rap type equal to 2, the frame is a delayed frame random access point. When obu rap type equal to 3, the frame is a key frame dependent random access point.

[0281] When obu_rap_type equal to 0 or obu_rap_type syntax element is not present indicates that the random access and recovery point characteristics of the frame are unspecified. Alternatively, this condition could be specified to not be a key frame RAP, delayed RAP, or key frame dependent recovery point frame.

[0282] Alternative mappings may be used for values of the obu_rap_type syntax element to the four frame characteristics.

[0283] The two bit syntax element may alternatively be described as being two separate one bit flags.

[0284] An example encoder may perform the following steps to indicate a key frame RAP in a bitstream that also includes an additional non-RAP frame:- Selecting a first frame to be key frame RAP;- Setting obu extension flag to 1 for all OBUs in the temporal unit of first frame;- Setting obu rap type to 1 for all OBUs in the temporal unit of first frame;- Setting frame type = KEY FRAME in frame header of first frame;- Setting show_frame = 1 in frame_header of first frame;- Coding frame as intra;- Selecting a second frame to be non-RAP;- Setting obu_rap_type equal to 0 and / or obu_extension_flag equal to 0 for all OBUs in the temporal unit of second frame;- Setting frame type = INTER FRAME in frame header of second frame; and- Coding second frame as inter, while setting the reference pictures (including any of alt ref frame, golden frame, last_frame, last2_frame) to the first frame or another frame that follows the first frame in coding order.

[0285] An example encoder may perform the following steps to indicate a delayed frame RAP and a non-RAP frame in a bitstream also including an additional frame:- Coding a first frame;- Selecting a second frame to be delayed RAP, following the first frame in coding order;- Setting obu extension flag to 1 for all OBUs in the temporal unit of second frame;- Setting obu rap type to 2 for all OBUs in the temporal unit of second frame;- Setting frame type = KEY FRAME in frame header of second frame;- Setting show_frame = 0 in frame header of second frame;- Coding second frame as intra;- Selecting a third frame to be non-RAP, following the second frame in coding order;- Setting obu rap type equal to 0 and / or obu_extension_flag equal to 0 for all OBUs in the temporal unit of third frame;- Setting frame type = INTER FRAME in frame header of third frame; and- Coding third frame as inter, while setting at least one reference picture (which can be any of alt ref frame, golden frame, last_frame, last2_frame) to the first frame.

[0286] An example encoder may perform the following to indicate a delayed frame RAP and a key frame dependent recovery point in a bitstream also including an additional frame:- Coding a first frame;- Selecting a second frame to be delayed RAP, following the first frame in coding order;- Setting obu extension flag to 1 for all OBUs in the temporal unit of second frame;- Setting obu rap type to 2 for all OBUs in the temporal unit of second frame;- Setting frame type = KEY FRAME in frame header of second frame- Setting show_frame = 0 in frame header of second frame;- Coding second frame as intra;- Selecting a third frame to be key frame dependent recovery point, following the second frame in coding order;- Setting obu extension flag to 1 for all OBUs in the temporal unit of third frame;- Setting obu rap type flag to 3 for all OBUs in the temporal unit of third frame;- Setting frame type = INTER FRAME in frame header of third frame; and- Coding third frame as inter, while setting the reference pictures (including any of alt ref frame, golden frame, last_frame, last2_frame) to the second frame or another frame that follows the second frame in coding order.

[0287] An example decoder using this option performs the following to detect a RAP in the bitstream:- Receiving streaming or broadcast video;- Reading all OBU headers in coding order and discard all OBUs until a first frame is identified with obu rap type equal to 1 or 2;- When obu rap type equal to 1,- Determining that the first frame is a key frame RAP;- Decoding and display the first frame;- Decoding and display frames following first frame in coding order;- Otherwise, (when obu rap type equal to 2),- Determining that the first frame is a delayed RAP, and decode the first frame and do not display the first frame;- Reading all OBU headers until find a second frame with obu rap type equal to 3;- Decoding the second frame and displaying the second frame; and- Decoding and displaying frames following second frame in coding order.

[0288] Example option 4 two bits in temporal_delimiter_obu( )

[0289] In this option, two bits in the temporal delimiter are used to indicate a RAP type syntax element. This is similar to the obu rap type syntax element proposed in option 3, but instead with a tdo rap type syntax element in the temporal delimiter.

[0290] The temporal delimiter in the existing AVI specification includes no payload bits.

[0291] In this option, it is proposed to add a two bit syntax element, tdo rap type, and a 6-bit reserved syntax element to maintain byte alignment. It is to be understood that embodiments of example option 1, using 1 bit, could likewise be realized with a one-bit syntax elements in a temporal delimiter OBU.

[0292] An example encoder may perform the following to indicate a key frame RAP in a bitstream that also contains an additional non-RAP frame:- Selecting a first frame to be key frame RAP;- Setting tdo rap type to 1 in the temporal delimiter OBU of first frame;- Setting frame type = KEY FRAME in frame header of first frame;- Setting show_frame = 1 in frame_header of first frame;- Coding frame as intra;- Selecting a second frame to be non-RAP;- Setting tdo rap type equal to 0 in the temporal delimiter OBU of second frame; and- Setting frame type = INTER FRAME in frame header of second frame.- Coding second frame as inter, while setting the reference pictures (including any of alt ref frame, golden frame, last_frame, last2_frame) to the first frame or another frame that follows the first frame in coding order.

[0293] An example encoder may perform the following to indicate a delayed frame RAP and a non-RAP frame in a bitstream also including an additional frame:- Coding a first frame;- Selecting a second frame to be delayed RAP, following the first frame in coding order;- Setting tdo rap type to 2 in the temporal delimiter OBU of second frame;- Setting frame type = KEY FRAME in frame header of second frame;- Setting show_frame = 0 in frame header of second frame;- Coding second frame as intra;- Selecting a third frame to be non-RAP, following the second frame in coding order;- Setting tdo rap type to 0 in the temporal delimiter OBU of third frame;- Setting frame type = INTER FRAME in frame header of third frame; and- Coding third frame as inter, while setting at least one reference picture (which can be any of alt ref frame, golden frame, last_frame, last2_frame) to the first frame.

[0294] An encoder may perform the following to indicate a delayed frame RAP and a key frame dependent recovery point in a bitstream also including an additional frame:- Coding a first frame;- Selecting a second frame to be delayed RAP, following the first frame in coding order;- Setting tdo rap type to 2 in the temporal delimiter OBU of second frame;- Setting frame type = KEY FRAME in frame header of second frame;- Setting show_frame = 0 in frame header of second frame;- Coding second frame as intra;- Selecting a third frame to be key frame dependent recovery point, following the second frame in coding order;- Setting tdo rap type to 3 in the temporal delimiter OBU of third frame;- Setting frame type = INTER FRAME in frame header of third frame; and- Coding third frame as inter, while setting the reference pictures (including any of alt ref frame, golden frame, last_frame, last2_frame) to the second frame or another frame that follows the second frame in coding order.

[0295] An example decoder using this option performs the following to detect a RAP in the bitstream:- Receiving streaming or broadcast video;- Reading all OBU headers in coding order and discard all OBUs until a first frame is identified including an temporal delimiter OBU with tdo rap type equal to 1 or 2;- When tdo rap type equal to 1,- Determining that the first frame is a key frame RAP;- Decoding and display the first frame;- Decoding and display frames following first frame in coding order;- Otherwise when, (tdo rap type equal to 2),- Determining that the first frame is a delayed RAP, and decode the first frame and do not display the first frame;- Reading all OBU headers until find a second frame with a temporal delimiter OBU with tdo rap type equal to 3;- Decoding the second frame and displaying the second frame;- Decoding and displaying frames following second frame in coding order; and- Recovering point signaling in NAU unit based syntax.

[0296] According to an embodiment, in a NAU-unit based bitstream syntax, such as in an extension of H.266 / V VC or a future H.267 standard, a flag is proposed to be included in a picture header or NAU unit header or other high level syntax structure to indicate that the current picture is a recovery point picture.

[0297] In an embodiment, a recovery point is indicated by an encoder in, and / or detected by a decoder from, a NAU unit header that is otherwise like the VVC NAU unit header but nuh_reserved_zero_bit is replaced by nuh_recovery_point_flag:

[0298] In an embodiment, nuh_recovery_point flag is conditionally present only for certain nal unit type values, such as GDR picture and trailing picture, and is reserved for future use or used for other purposes for other nal unit type values.

[0299] In an embodiment, a recovery point is indicated by an encoder in, and / or detected by a decoder from, a flag in a picture header.

[0300] nuh_recovery_point_flag or ph_recovery_point_flag equal to 1 specifies that the current picture is a recovery point picture.

[0301] The associated GDR picture for the recovery point picture is derived as follows:

[0302] When the current picture is a GDR picture, the associated GDR picture is the current picture.

[0303] Otherwise (the current picture is not a GDR picture), the associated GDR picture is the previous GDR picture in decoding order.

[0304] nuh_recovery_point_flag or ph_recovery_point_flag equal to 0 specifies that the current picture is not a recovery point picture.

[0305] A recovery point picture may be defined to have the following characteristics:

[0306] When the associated GDR picture starts a CLVS, the recovery point picture and all subsequent decoded pictures in output order are correct in content.

[0307] The recovery point picture and all subsequent pictures in output order shall not contain any references to pictures preceding the associated GDR picture in decoding order.

[0308] According to an embodiment, in a NAL-unit based bitstream syntax, such as in an extension of H.266 / VVC or a future H.267 standard, a flag is proposed to be included in a NAL unit header to indicate that the current picture is an IRAP picture. The flag may be referred to as nuh_irap_pic_flag without loss of generality. In an embodiment, nuh_irap_pic_flag is conditionally present for certain nal unit type values, such as nal unit type values indicating IDR or CRA VCL NAL units, and is reserved for future use or used for other purposes for other nal_unit_type values. Consequently, an IRAP picture may be detected even when mixed VCL NAL unit types within a coded picture is allowed.

[0309] It is to be understood that nuh_recovery_point_flag and nuh_irap_pic_flag may use the same bit position in the NAL unit header, when it is conditioned by the nal_unit_type value.

[0310] FIG. 5 is an example apparatus 500, which may be implemented in hardware, configured to implement the examples described herein. The apparatus 500 comprises at least one processor 502 (e.g., an FPGA and / or CPU), at least one memory 504 including computer program code 505, the computer program code 505 having instructions to carry out the methods described herein, wherein the at least one memory 504 and the computer program code 505 are configured to, with the at least one processor 502, cause the apparatus 500 to implement circuitry, a process, component, module, or function (implemented with control module 506) to implement the examples described herein, implementing high level syntax headers in video coding. Optionally included encoder 508 of the control module 506 implements encoding based on the examples described herein, and optionally included decoder 510 implements decoding based on the examples described herein. The at least one memory 504 may be a non-transitory memory, a transitory memory, a volatile memory (e.g. RAM), or a nonvolatile memory (e.g., ROM).

[0311] The apparatus 500 includes a display and / or I / O interface 512, which includes user interface (UI) circuitry and elements, that may be used to display features or a status of the methods described herein (e.g., as one of the methods is being performed or at a subsequent time), or to receive input from a user such as with using a keypad, camera, touchscreen, touch area, microphone, biometric recognition, one or more sensors, etc. The apparatus 500 includes one or more communication e.g. network (N / W) interfaces (I / F(s)) 514. The communication I / F(s) 514 may be wired and / or wireless and communicate over the Intemet / other network(s) via any communication technique including via one or more links 516. The communication I / F(s) 514 may comprise one or more transmitters or one or more receivers.

[0312] The transceiver 518 comprises one or more transmitters 520 and one or more receivers 522. The transceiver 518 and / or communication I / F(s) 514 may comprise standard well-knowncomponents such as an amplifier, filter, frequency-converter, (de)modulator, and encoder / decoder circuitries and one or more antennas, such as antennas 524 used for communication over wireless link 526.

[0313] The control module 506 of the apparatus 500 comprises one of or both parts 506-1 and / or 506-2, which may be implemented in a number of ways. The control module 506 may be implemented in hardware as control module 506-1, such as being implemented as part of the at least one processor 502. The control module 506-1 may be implemented also as an integrated circuit or through other hardware such as a programmable gate array. In another example, the control module 506 may be implemented as control module 506-2, which is implemented as computer program code (having corresponding instructions) 505 and is executed by the at least one processor 502. For instance, the at least one memory 504 store instructions that, when executed by the at least one processor 502, cause the apparatus 500 to perform one or more of the operations as described herein. Furthermore, the at least one processor 502, the at least one memory 504, and example algorithms (e.g., as flowcharts and / or signaling diagrams), encoded as instructions, programs, or code, are means for causing performance of the operations described herein.

[0314] The apparatus 500 to implement the functionality of control module 506 may correspond to any of the apparatuses depicted herein. Alternatively, apparatus 500 and its elements may not correspond to any of the other apparatuses depicted herein, as apparatus 500 may be part of a self- organizing / optimizing network (SON) node or other node, such as a node in a cloud.

[0315] The apparatus 500 may also be distributed throughout the network including within and between apparatus 500 and any network element (such as a base station and / or terminal device and / or user equipment).

[0316] Interface 528 enables data communication and signaling between the various items of apparatus 500, as shown in FIG. 5. For example, the interface 528 may be one or more buses such as address, data, or control buses, and may include any interconnection mechanism, such as a series of lines on a motherboard or integrated circuit, fiber optics or other optical communication equipment, and the like. Computer program code (e.g. instructions) 505, including control module 506 may comprise object-oriented software configured to pass data or messages between objects within computer program code 505. The apparatus 500 need not comprise each of the features mentioned, or may comprise other features as well. The various components of apparatus 500 may at least partially reside in a housing 530, or a subset of the various components of apparatus 500 may at least partially be located in different housings, which different housings may include housing 530.

[0317] FIG. 6 shows a schematic representation of non-volatile memory media 600a (e.g. computer / compact disc (CD) or digital versatile disc (DVD)) and 600b (e.g. universal serial bus (USB) memory stick) and 600c (e.g. cloud storage for downloading instructions and / or parameters 602 or receiving emailed instructions and / or parameters 602) storing instructions and / or parameters 602 which when executed by a processor allows the processor to perform one or more of the operations of the methods described herein. Instructions and / or parameters 602 may represent or correspond to a non- transitory computer readable medium.

[0318] FIG. 7 is an example method 700 performed with an encoder, based on the examples described herein. At 702, the method 700 includes selecting a first frame to be a random access point (RAP) frame. At 704, the method 700 includes setting an open bitstream unit (OBU) RAP flag equal to a first value for OBUs in a temporal unit of the first frame. At 706, the method 700 includes setting a frame type as a key frame in a frame header of the first frame. At 708, the method 700 includes coding the first frame as an intra frame.

[0319] The method 700 may be performed with an encoding apparatus, such as the apparatus 100, 500, apparatuses depicted in FIG. 3 and FIG. 4, for example, the transmitting apparatus 406 with the encoder 402, or the apparatus 400 with the encoder 402.

[0320] FIG. 8 is an example method 800 performed with a decoder, based on the example embodiments described herein. At 802, the method 800 includes receiving a streaming or a broadcast video. At 804, the method 800 includes reading open bitstream unit (OBU) headers in coding order. At 806, the method 800 includes discarding open bitstream units until a criterion is met for a first frame. At 808, the method 800 includes decoding the first frame.

[0321] The method 800 may be performed with a decoding apparatus, such as the apparatus 100, 500, apparatuses depicted in FIG. 3 and FIG. 4, for example, the receiving apparatus 410 with the decoder 412, or the apparatus 400 with the decoder 412.

[0322] As described above, FIGs. 7 and 8 include flowcharts of an apparatus (e.g., 100, 400, 500, or any other apparatuses described herein), method, and computer program product according to certain example embodiments. It will be understood that each block of the flowcharts, and combinations of blocks in the flowcharts, may be implemented by various means, such as hardware, firmware, processor, circuitry, and / or other devices associated with execution of software including one or more computer program instructions. For example, one or more of the procedures described above may be embodied by computer program instructions. In this regard, the computer program instructions which embody the procedures described above may be stored by a memory (e.g. 112 or 504) of an apparatus employingan embodiment of the present invention and executed by processing circuitry (e.g., 110 or 502) of the apparatus. As will be appreciated, any such computer program instructions may be loaded onto a computer or other programmable apparatus (e.g., hardware) to produce a machine, such that the resulting computer or other programmable apparatus implements the functions specified in the flowchart blocks. These computer program instructions may also be stored in a computer-readable memory that may direct a computer or other programmable apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture, the execution of which implements the function specified in the flowchart blocks. The computer program instructions may also be loaded onto a computer or other programmable apparatus to cause a series of operations to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide operations for implementing the functions specified in the flowchart blocks.

[0323] A computer program product is therefore defined in those instances in which the computer program instructions, such as computer-readable program code portions, are stored by at least one non-transitory computer-readable storage medium with the computer program instructions, such as the computer-readable program code portions, being configured, upon execution, to perform the functions described above, such as in conjunction with the flowchart(s) of FIGs. 7 and 8. In other embodiments, the computer program instructions, such as the computer-readable program code portions, need not be stored or otherwise embodied by a non-transitory computer-readable storage medium, but may, instead, be embodied by a transitory medium with the computer program instructions, such as the computer-readable program code portions, still being configured, upon execution, to perform the functions described above.

[0324] Accordingly, blocks of the flowcharts support combinations of means for performing the specified functions and combinations of operations for performing the specified functions for performing the specified functions. It will also be understood that one or more blocks of the flowcharts, and combinations of blocks in the flowcharts, may be implemented by special purpose hardware-based computer systems which perform the specified functions, or combinations of special purpose hardware and computer instructions.

[0325] In some embodiments, certain ones of the operations above may be modified or further amplified. Furthermore, in some embodiments, additional optional operations may be included. Modifications, additions, or amplifications to the operations above may be performed in any order and in any combination.

[0326] Some embodiments have been described in relation to streaming or broadcast video. It is to be understood that embodiments may be realized with any type of video transmission or video bitstreams, such as video conferencing.

[0327] In the above, some example embodiments have been described with the help of syntax of the bitstream. It needs to be understood, however, that the corresponding structure and / or computer program may reside at the encoder for generating the bitstream and / or at the decoder for decoding the bitstream.

[0328] In the above, some example embodiments have been described with certain assignments of syntax element values to semantics. It needs to be understood, however, that embodiments may be realized similarly with any assignment of syntax element values to semantics. For example, when an embodiment is described in relation to a flag and the semantics for values 0 and 1 of the flag, another embodiment may be realized by switching the semantics of flag values.

[0329] In the above, where example embodiments have been described with reference to an encoder, it needs to be understood that the resulting bitstream and the decoder have corresponding elements in them. Likewise, where example embodiments have been described with reference to a decoder, it needs to be understood that the encoder has structure and / or computer program for generating the bitstream to be decoded by the decoder.

[0330] Many modifications and other embodiments of the inventions set forth herein will come to mind to one skilled in the art to which these inventions pertain having the benefit of the teachings presented in the foregoing descriptions and the associated drawings. Therefore, it is to be understood that the inventions are not to be limited to the specific embodiments disclosed and that modifications and other embodiments are intended to be included within the scope of the appended claims. Moreover, although the foregoing descriptions and the associated drawings describe example embodiments in the context of certain example combinations of elements and / or functions, it should be appreciated that different combinations of elements and / or functions may be provided by alternative embodiments without departing from the scope of the appended claims. In this regard, for example, different combinations of elements and / or functions than those explicitly described above are also contemplated as may be set forth in some of the appended claims. Accordingly, the description is intended to embrace all such alternatives, modifications and variances which fall within the scope of the appended claims. Although specific terms are employed herein, they are used in a generic and descriptive sense only and not for purposes of limitation.

[0331] It should be understood that the foregoing description is only illustrative. Variousalternatives and modifications may be devised by those skilled in the art. For example, features recited in the various dependent claims could be combined with each other in any suitable combination(s). In addition, features from different embodiments described above could be selectively combined into a new embodiment. Accordingly, the description is intended to embrace all such alternatives, modifications and variances which fall within the scope of the appended claims.

[0332] References to a ‘computer’, ‘processor’, etc. should be understood to encompass not only computers having different architectures such as single / multi-processor architectures and sequential (Von Neumann) / parallel architectures but also specialized circuits such as field-programmable gate arrays (FPGA), application specific circuits (ASIC), signal processing devices and other processing circuitry. References to computer program, instructions, code etc. should be understood to encompass software for a programmable processor or firmware such as, for example, the programmable content of a hardware device such as instructions for a processor, or configuration settings for a fixed-function device, gate array or programmable logic device, and the like.

[0333] As used herein, the term ‘circuitry’ may refer to any of the following: (a) hardware circuit implementations, such as implementations in analog and / or digital circuitry, and (b) combinations of circuits and software (and / or firmware), such as (as applicable): (i) a combination of processor(s) or (ii) portions of processor(s) / software including digital signal processor(s), software, and memory(ies) that work together to cause an apparatus to perform various functions, and (c) circuits, such as a microprocessor(s) or a portion of a microprocessor(s), that require software or firmware for operation, even when the software or firmware is not physically present. This description of ‘circuitry’ applies to uses of this term in this application. As a further example, as used herein, the term ‘circuitry’ would also cover an implementation of merely a processor (or multiple processors) or a portion of a processor and its (or their) accompanying software and / or firmware. The term ‘circuitry’ would also cover, for example and when applicable to the particular element, a baseband integrated circuit or applications processor integrated circuit for a mobile phone or a similar integrated circuit in a server, a cellular network device, or another network device.

[0334] Circuitry or Circuit: As used in this application, the term ‘circuitry’ or ‘circuit’ may refer to one or more or all of the following:(a) hardware-only circuit implementations (such as implementations in only analog and / or digital circuitry); and(b) combinations of hardware circuits and software, such as (as applicable):(i) a combination of analog and / or digital hardware circuit(s) with software / firmware; and(ii) any portions of hardware processor(s) with software (including digital signal processor(s)), software, and memory(ies) that work together to cause an apparatus, such as a mobile phone or server, to perform various functions); and(c) hardware circuit(s) and or processor(s), such as a microprocessor(s) or a portion of a microprocessor(s), that requires software (e.g., firmware) for operation, but the software may not be present when it is not needed for operation.

[0335] This definition of circuitry applies to all uses of this term in this application, including in any claims. As a further example, as used in this application, the term circuitry also covers an implementation of merely a hardware circuit or processor (or multiple processors) or portion of a hardware circuit or processor and its (or their) accompanying software and / or firmware. The term circuitry also covers, for example, and when applicable to the particular claim element, a baseband integrated circuit or processor integrated circuit for a mobile device or a similar integrated circuit in server, a cellular network device, or other computing or network device.

Claims

1. CLAIMSWhat is claimed is1. An apparatus comprising: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to: selecting a first frame to be a random access point (RAP) frame; setting an open bitstream unit (OBU) RAP flag equal to a first value for OBUs in a temporal unit of the first frame; setting a frame type as a key frame in a frame header of the first frame; and coding the first frame as an intra frame.

2. The apparatus of claim 1, wherein the apparatus is further caused to perform: selecting a second frame, following the first frame in a coding order, to be a non-RAP frame; setting the open bitstream unit RAP flag equal to a second value for OBUs in a temporal unit of the second frame, wherein the second value is different from the first value; setting the frame type as an inter frame in a frame header of the second frame; and coding the second frame as the inter frame.

3. The apparatus of claim 2, wherein, for coding the second frame, the apparatus is further caused to perform: setting at least one reference frame to the first frame or another frame that follows the first frame in the coding order, wherein the another frame is different from the second frame.

4. The apparatus of any of the previous claims, wherein the temporal unit comprises one or more of a temporal delimiter OBU, a sequence header OBU, an optional frame header OBU, one or more tile groups OBUs, or an optional metadata OBU.

655. The apparatus of any of the previous claims, wherein the RAP frame comprises the key frame.

6. The apparatus of claim 5, wherein the apparatus is further caused to perform: setting a show frame flag in the frame header of the first frame to the first value.

7. The apparatus of claim 1, wherein the apparatus is further caused to perform: selecting a second frame, following the first frame in a coding order, to be a delayed RAP frame; setting an open bitstream unit RAP flag equal to the first value for OBUs in a temporal unit of the second frame; setting the frame type as the key frame in a frame header of the second frame; setting a show frame flag in the frame header of the second frame to a second value; and coding the second frame as the intra frame.

8. The apparatus of claim 7, wherein the apparatus is further caused to perform: selecting a third frame, following the second frame in the coding order, to be a non-RAP; setting the open bitstream unit RAP flag equal to the second value for OBUs in a temporal unit of the third frame; setting the frame type as an inter frame in a frame header of the third frame; and coding the third frame as the inter frame.

9. The apparatus of claim 8, wherein, for coding the third frame, the apparatus is further caused to perform: setting at least one reference frame to the first frame.

10. The apparatus of claim 6, wherein the apparatus is further caused to perform: setting an open bitstream unit RAP type flag equal to the first value for open bitstream units in the temporal unit of the first frame.

11. The apparatus of claim 10, wherein the apparatus is further caused to perform: setting an open bitstream unit extension flag equal to the first value for the open bitstream units in the temporal unit of the first frame.

12. The apparatus of claim 8, wherein the apparatus is further caused to perform: setting an open bitstream unit RAP type flag equal to the first value for open bitstream units in the temporal unit of the second frame.

13. The apparatus of claim 12, wherein the apparatus is further caused to perform: setting an open bitstream unit extension flag equal to the first value for the open bitstream units in the temporal unit of the first frame.

14. The apparatus of claim 7, wherein the apparatus is further caused to perform: selecting a third frame, following the second frame in the coding order, to be key frame dependent recovery point; setting the open bitstream unit RAP flag equal to the second value for open bitstream units in the temporal unit of the third frame; setting an open bitstream unit RAP type flag equal to the first value for the open bitstream units in the temporal unit of the third frame; setting the frame type as an inter frame in the frame header of the third frame; and coding the third frame as the inter frame.

15. The apparatus of claim 14, wherein the apparatus is further caused to perform: setting an open bitstream unit extension flag equal to the first value for the open bitstream units in the temporal unit of the second frame.

16. The apparatus of claim 15, wherein the apparatus is further caused to perform: setting the open bitstream unit extension flag equal to the first value for the open bitstream units in the temporal unit of the second frame; and setting the open bitstream unit extension flag equal to the first value for the open bitstream units in the temporal unit of third frame.

17. The apparatus of any of the claims 14 to 16, wherein for coding the third frame, the apparatus is further caused to perform: setting at least one reference frame to the second frame or another frame that follows the second frame in the coding order, wherein the third frame is different from the another frame.

18. The apparatus of claim 1, wherein the first frame comprises a key frame RAP, and wherein the apparatus is further caused to perform: setting an open bitstream unit extension flag equal to the first value for open bitstream units in the temporal unit of the first frame; setting an open bitstream unit RAP type syntax element equal to the first value for the open bitstream units in the temporal unit of first frame; setting the frame type equal to the key frame in the frame header of the first frame; setting show frame in the frame header of the first frame equal to the first value; selecting a second frame, following the first frame in coding order, to be a non-RAP frame; setting open bitstream unit RAP type syntax element equal to a second value and / or the open bitstream unit extension flag equal to the second value for the open bitstream units in the temporal unit of the second frame; setting frame type equal to an inter frame in the frame header of the second frame; and coding second frame as the inter frame.

19. The apparatus of claim 18, wherein for coding the second frame, the apparatus is further caused to perform: setting at least one reference frame to the first frame or another frame that follows the first frame in the coding order, wherein the another frame is different from the second frame.

20. The apparatus of claim 1, wherein the apparatus is further caused to perform: selecting a second frame, following the first frame in coding order, to be a delayed RAP; setting an open bitstream unit extension flag equal to the first value for open bitstream units in a temporal unit of the second frame; setting an open bitstream unit RAP type syntax element equal to a third value for the open bitstream units in the temporal unit of the second frame; setting the frame type equal to the key frame in a frame header of the second frame;setting a show frame flag equal to a second value in the frame header of the second frame; coding the second frame as the intra frame; selecting a third frame, following the second frame in the coding order, to be a non-RAP frame; setting the open bitstream unit RAP type syntax element and / or the open bitstream unit extension flag equal to the second value for the open bitstream units in the temporal unit of the third frame; setting frame type equal to an inter frame in the frame header of the third frame; and coding the third frame as the inter frame.

21. The apparatus of claim 20, wherein for coding the third frame, the apparatus is further caused to perform: setting at least one reference frame to the first frame.

22. The apparatus of claim 1, wherein the apparatus is further caused to perform: selecting a second frame, following the first frame in coding order, to be a delayed RAP; setting an open bitstream unit extension flag equal to the first value for open bitstream units in a temporal unit of the second frame; setting an open bitstream unit RAP type syntax element equal to a third value for the open bitstream units in the temporal unit of the second frame; setting the frame type equal to the key frame in a frame header of the second frame; setting a show frame flag equal to a second value in the frame header of the second frame; coding the second frame as the intra frame; selecting a third frame, following the second frame in the coding order, to be a key frame dependent recovery point; setting open bitstream unit extension flag equal to the first value for the open bitstream units in the temporal unit of the third frame; setting the open bitstream unit RAP type syntax element equal to a fourth value for the69open bitstream units in the temporal unit of the third frame; setting frame type equal to an inter frame in the frame header of the third frame; and coding the third frame as the inter frame.

23. The apparatus of claim 22, wherein for coding the third frame, the apparatus is further caused to perform: setting at least one reference frame to the second frame or another frame that follows the second frame in the coding order, wherein the another frame is different from the third frame.

24. The apparatus of claim 1, wherein the first frame comprises a key frame RAP, and wherein the apparatus is further caused to perform: setting a two bit delimiter RAP type syntax element to a fifth value in a temporal delimiter open bitstream unit of the first frame; setting a show frame flag equal to the first value in the frame header of the first frame; selecting a second frame to be a non-RAP; setting the two bit delimiter RAP type syntax element to a second value in the temporal delimiter open bitstream unit of the second frame; setting the frame type equal to an inter frame in the frame header of the second frame; and coding the second frame as the inter frame.

25. The apparatus of claim 24, wherein for coding the second frame, the apparatus is further caused to perform: setting at least one reference frame to the first frame or another frame that follows the first frame in coding order, wherein the another frame is different from the second frame.

26. The apparatus of claim 1, wherein the apparatus is further caused to perform: selecting a second frame, following the first frame in coding order, to be a delayed RAP; setting a two bit delimiter RAP type syntax element to a third value in a temporal delimiter open bitstream unit of the second frame;70setting the frame type equal to the key frame in a frame header of the second frame; setting show frame flag equal to a second value in the frame header of the second frame; coding the second frame as the intra frame; selecting, following the second frame in the coding order, a third frame to be non-RAP; setting the two bit delimiter RAP type syntax element to the second value in the temporal delimiter open bitstream unit of the third frame; setting the frame type equal to an inter frame in the frame header of the third frame; and coding the third frame as the inter frame.

27. The apparatus of claim 26, wherein, for coding the third frame as the inter frame, the apparatus is further caused to perform: setting at least one reference frame to the first frame.

28. The apparatus of claim 1, wherein the apparatus is further caused to perform: selecting a second frame, following the first frame in coding order, to be a delayed RAP; setting a two bit delimiter RAP type syntax element to a third value in a temporal delimiter open bitstream unit of the second frame; setting the frame type equal to the key frame in a frame header of the second frame; setting a show frame flag to a second value in the frame header of the second frame; coding the second frame as the intra frame; selecting a third frame, following the second frame in the coding order, to be key frame dependent recovery point; setting two bit delimiter RAP type syntax element to a fourth value in the temporal delimiter open bitstream unit of the third frame; setting frame type equal to an inter frame in the frame header of the third frame; and coding the third frame as the inter frame.

29. The apparatus of claim 28, wherein, for coding the third frame as the inter frame, the apparatus is further caused to perform: setting at least one reference frame to the second frame or another frame that follows the second frame in the coding order, wherein the another is different from the second frame.

30. An apparatus comprising: at least one processor; and at least one memory storing instructions that, when executed by the at least one processor, cause the apparatus at least to: receiving a streaming or a broadcast video; reading open bitstream unit (OBU) headers in coding order; discarding open bitstream units until a criterion is met for a first frame; and decoding the first frame.

31. The apparatus of claim 30, wherein the criterion comprises identifying the first frame with an open bitstream unit (OBU) random access point (RAP) flag equal to a first value.

32. The apparatus of any of the claims 30 or 31, wherein the apparatus is further caused to perform: decoding subsequent frames.

33. The apparatus of any of the claims 30 or 31, wherein the apparatus is further caused to perform: examining a frame header of the first frame; and displaying the first frame, when a show existing frame is equal to the first value.

34. The apparatus of claim 33, wherein when the show existing frame is not equal to the first value, the apparatus is further caused to perform: continuing decoding frames and examining frame headers until a second frame with the show existing frame equal to the first value is found; decoding the second frame and frames following the second frame in a bitstream; anddisplaying the second frame and the frames following the second frame in the bitstream.

35. The apparatus of any of the claims 30 or 31, wherein, when an open bitstream unit RAP type flag in the first frame is equal to the first value, the apparatus is further caused to perform: determining that the first frame comprises a key frame RAP; and decoding frames following the first frame in the coding order.

36. The apparatus of claim 35, wherein the apparatus is further caused to perform: displaying the first frame; and / or displaying the frames following the first frame in the coding order.

37. The apparatus of claim 30, wherein, when an open bitstream unit RAP type flag in the first frame is not equal to the first value, the apparatus is further caused to perform: determining that the first frame comprises a delayed RAP; reading open bitstream unit headers until a second frame with the open bitstream unit RAP flag equal to a second value and the open bitstream unit RAP type flag equal to the first value is found; decoding the second frame; and decoding frames following the second frame in the coding order.

38. The apparatus of claim 37, wherein the apparatus is further caused to perform: displaying the second frame; and / or displaying the frames following the second frame in the coding order.

39. The apparatus of claim 30, wherein the criterion comprises identifying the first frame with an open bitstream unit random access point (RAP) type syntax element equal to a first value or a third value.

40. The apparatus of claim 39, wherein when an open bitstream unit RAP type syntax element in the first frame is equal to the first value, the apparatus is further caused to perform:73determining that the first frame comprises a key frame RAP; and decoding frames following the first frame in the coding order.

41. The apparatus of claim 40, wherein the apparatus is further caused to perform: displaying the first frame; and / or displaying the frames following the first frame in the coding order.

42. The apparatus of claim 39, wherein, when the open bitstream unit RAP type syntax element in the first frame is equal to the third value, the apparatus is further caused to perform: determining that the first frame comprises a delayed RAP; reading open bitstream unit headers until a second frame with the open bitstream unit RAP type syntax element equal to a fourth value is found; decoding the second frame; and decoding frames following the second frame in the coding order.

43. The apparatus of claim 42, wherein the apparatus is further caused to perform: displaying the second frame; and / or displaying the frames following the second frame in the coding order.

44. The apparatus of claim 30, wherein the criterion comprises identifying the first frame comprising a temporal delimiter open bitstream unit with a two bit delimiter RAP type syntax element equal to a fifth value or a third value.

45. The apparatus of claim 44, wherein when the two bit delimiter RAP type syntax element is equal to the fifth value, the apparatus is further caused to perform: determining that the first frame comprises a key frame RAP; and decoding frames following the first frame in the coding order.

46. The apparatus of claim 45, the apparatus is further caused to perform: displaying the first frame; and / or74displaying the frames following the first frame in the coding order.

47. The apparatus of claim 44, wherein, when the two bit delimiter RAP type syntax element is equal to the third value, the apparatus is further caused to perform: determining that the first frame comprises a delayed RAP; reading open bitstream unit headers until a second frame with the temporal delimiter open bitstream unit with the two bit delimiter RAP type syntax element is equal to a fourth value is found; decoding the second frame; and decoding frames following the second frame in the coding order.

48. The apparatus of claim 47, wherein the apparatus is further caused to perform: displaying the second frame; and / or displaying the frames following the second frame in the coding order.

49. A method comprising: selecting a first frame to be a random access point (RAP) frame; setting an open bitstream unit (OBU) RAP flag equal to a first value for OBUs in a temporal unit of the first frame; setting a frame type as a key frame in a frame header of the first frame; and coding the first frame as an intra frame.

50. The method of claim 49 further comprising: selecting a second frame, following the first frame in a coding order, to be a non-RAP frame; setting the open bitstream unit RAP flag equal to a second value for OBUs in a temporal unit of the second frame, wherein the second value is different from the first value; setting the frame type as an inter frame in a frame header of the second frame; and75coding the second frame as the inter frame.

51. The method of claim 50, wherein, for coding the second frame, the method further comprises: setting at least one reference frame to the first frame or another frame that follows the first frame in the coding order, wherein the another frame is different from the second frame.

52. The method of any of the claims 49 to 51, wherein the temporal unit comprises one or more of a temporal delimiter OBU, a sequence header OBU, an optional frame header OBU, one or more tile groups OBUs, or an optional metadata OBU.

53. The method of any of the claims 49 to 52, wherein the RAP frame comprises the key frame.

54. The method of claim 53 further comprises: setting a show frame flag in the frame header of the first frame to the first value.

55. The method of claim 49 further comprising: selecting a second frame, following the first frame in a coding order, to be a delayed RAP frame; setting an open bitstream unit RAP flag equal to the first value for OBUs in a temporal unit of the second frame; setting the frame type as the key frame in a frame header of the second frame; setting a show frame flag in the frame header of the second frame to a second value; and coding the second frame as the intra frame.

56. The method of claim 55 further comprising: selecting a third frame, following the second frame in the coding order, to be a non-RAP; setting the open bitstream unit RAP flag equal to the second value for OBUs in a temporal unit of the third frame; setting the frame type as an inter frame in a frame header of the third frame; and coding the third frame as the inter frame.7657. The method of claim 56, wherein, for coding the third frame, the method further comprises: setting at least one reference frame to the first frame.

58. The method of claim 54 further comprising: setting an open bitstream unit RAP type flag equal to the first value for open bitstream units in the temporal unit of the first frame.

59. The method of claim 58 further comprising: setting an open bitstream unit extension flag equal to the first value for the open bitstream units in the temporal unit of the first frame.

60. The method of claim 56 further comprising: setting an open bitstream unit RAP type flag equal to the first value for open bitstream units in the temporal unit of the second frame.

61. The method of claim 60 further comprising: setting an open bitstream unit extension flag equal to the first value for the open bitstream units in the temporal unit of the first frame.

62. The method of claim 55 further comprising: selecting a third frame, following the second frame in the coding order, to be key frame dependent recovery point; setting the open bitstream unit RAP flag equal to the second value for open bitstream units in the temporal unit of the third frame; setting an open bitstream unit RAP type flag equal to the first value for the open bitstream units in the temporal unit of the third frame; setting the frame type as an inter frame in the frame header of the third frame; and coding the third frame as an the inter frame.

63. The method of claim 62 further comprising: setting an open bitstream unit extension flag equal to the first value for the open bitstream units in the temporal unit of the second frame.

64. The method of claim 63 further comprising: setting the open bitstream unit extension flag equal to the first value for the open bitstream units in the temporal unit of the second frame; and setting the open bitstream unit extension flag equal to the first value for the open bitstream units in the temporal unit of third frame.7765. The method of any of the claims 62 to 64, wherein for coding the third frame, the method further comprises: setting at least one reference frame to the second frame or another frame that follows the second frame in the coding order, wherein the third frame is different from the another frame.

66. The method of claim 49, wherein the first frame comprises a key frame RAP, and wherein the method further comprises: setting an open bitstream unit extension flag equal to the first value for open bitstream units in the temporal unit of the first frame; setting an open bitstream unit RAP type syntax element equal to the first value for the open bitstream units in the temporal unit of first frame; setting the frame type equal to the key frame in the frame header of the first frame; setting show frame in the frame header of the first frame equal to the first value; selecting a second frame, following the first frame in coding order, to be a non-RAP frame; setting open bitstream unit RAP type syntax element equal to second value and / or the open bitstream unit extension flag equal to the second value for the open bitstream units in the temporal unit of the second frame; setting frame type equal to an inter frame in the frame header of the second frame; and coding second frame as the inter frame.

67. The method of claim 66, wherein for coding the second frame, the method further comprises: setting at least one reference frame to the first frame or another frame that follows the first frame in the coding order, wherein the another frame is different from the second frame.

68. The method of claim 49 further comprising: selecting a second frame, following the first frame in coding order, to be a delayed RAP; setting an open bitstream unit extension flag equal to the first value for open bitstream units in a temporal unit of the second frame;setting an open bitstream unit RAP type syntax element equal to a third value for the open bitstream units in the temporal unit of the second frame; setting the frame type equal to the key frame in a frame header of the second frame; setting a show frame flag equal to a second value in the frame header of the second frame; coding the second frame as the intra frame; selecting a third frame, following the second frame in the coding order, to be a non-RAP frame; setting the open bitstream unit RAP type syntax element and / or the open bitstream unit extension flag equal to the second value for the open bitstream units in the temporal unit of the third frame; setting frame type equal to an inter frame in the frame header of the third frame; and coding the third frame as the inter frame.

69. The method of claim 68, wherein for coding the third frame, the method further comprises: setting at least one reference frame to the first frame.

70. The method of claim 49 further comprising: selecting a second frame, following the first frame in coding order, to be a delayed RAP; setting an open bitstream unit extension flag equal to the first value for open bitstream units in a temporal unit of the second frame; setting an open bitstream unit RAP type syntax element equal to a third value for the open bitstream units in the temporal unit of the second frame; setting the frame type equal to the key frame in a frame header of the second frame; setting a show frame flag equal to a second value in the frame header of the second frame; coding the second frame as the intra frame; selecting a third frame, following the second frame in the coding order, to be a key frame dependent recovery point;setting open bitstream unit extension flag equal to the first value for the open bitstream units in the temporal unit of the third frame; setting the open bitstream unit RAP type syntax element equal to a fourth value for the open bitstream units in the temporal unit of the third frame; setting frame type equal to an inter frame in the frame header of the third frame; and coding the third frame as the inter frame.

71. The method of claim 70, wherein for coding the third frame, the method further comprises: setting at least one reference frame to the second frame or another frame that follows the second frame in the coding order, wherein the another frame is different from the third frame.

72. The method of claim 49, wherein the first frame comprises a key frame RAP, and wherein the method further comprises: setting a two bit delimiter RAP type syntax element to a fifth value in a temporal delimiter open bitstream unit of the first frame; setting a show frame flag equal to the first value in the frame header of the first frame; selecting a second frame to be a non-RAP; setting the two bit delimiter RAP type syntax element to a second value in the temporal delimiter open bitstream unit of the second frame; setting the frame type equal to an inter frame in the frame header of the second frame; and coding the second frame as the inter frame.

73. The method of claim 72, wherein for coding the second frame, the method further comprises: setting at least one reference frame to the first frame or another frame that follows the first frame in coding order, wherein the another frame is different from the second frame.

74. The method of claim 49 further comprising: selecting a second frame, following the first frame in coding order, to be a delayed RAP;setting a two bit delimiter RAP type syntax element to a third value in a temporal delimiter open bitstream unit of the second frame; setting the frame type equal to the key frame in a frame header of the second frame; setting show frame flag equal to a second value in the frame header of the second frame; coding the second frame as the intra frame; selecting, following the second frame in the coding order, a third frame to be non-RAP; setting the two bit delimiter RAP type syntax element to the second value in the temporal delimiter open bitstream unit of the third frame; setting the frame type equal to an inter frame in the frame header of the third frame; and coding the third frame as the inter frame.

75. The method of claim 74, wherein, for coding the third frame as the inter frame, the method further comprises: setting at least one reference frame to the first frame.

76. The method of claim 49 further comprising: selecting a second frame, following the first frame in coding order, to be a delayed RAP; setting a two bit delimiter RAP type syntax element to a third value in a temporal delimiter open bitstream unit of the second frame; setting the frame type equal to the key frame in a frame header of the second frame; setting a show frame flag to a second value in the frame header of the second frame; coding the second frame as the intra frame; selecting a third frame, following the second frame in the coding order, to be key frame dependent recovery point; setting two bit delimiter RAP type syntax element to a fourth value in the temporal delimiter open bitstream unit of the third frame; setting frame type equal to an inter frame in the frame header of the third frame; andcoding the third frame as an the inter frame.

77. The method of claim 76, wherein, for coding the third frame as the inter frame, the method further comprises: setting at least one reference frame to the second frame or another frame that follows the second frame in the coding order, wherein the another is different from the second frame.

78. A method comprising: receiving a streaming or a broadcast video; reading open bitstream unit (OBU) headers in coding order; discarding open bitstream units until a criterion is met for a first frame; and decoding the first frame.

79. The method of claim 78, wherein the criterion comprises identifying the first frame with an open bitstream unit (OBU) random access point (RAP) flag equal to a first value.

80. The method of any of the claims 78 or 79 further comprising: decoding subsequent frames.

81. The method of any of the claims 78 or 79 further comprising: examining a frame header of the first frame; and displaying the first frame, when a show existing frame is equal to a first value.

82. The method of claim 81, wherein when the show existing frame is not equal to the first value, the method further comprises: continuing decoding frames and examining frame headers until a second frame with the show existing frame equal to the first value is found; decoding the second frame and frames following the second frame in a bitstream; and displaying the second frame and the frames following the second frame in the bitstream.

83. The method of any of the claims 78 or 79, wherein, when an open bitstream unit RAP type flag in the first frame is equal to the first value, the method further comprises:82determining that the first frame comprises a key frame RAP; and decoding frames following the first frame in the coding order.

84. The method of claim 83 further comprising: displaying the first frame; and / or displaying the frames following the first frame in the coding order.

85. The method of claim 78, wherein, when an open bitstream unit RAP type flag in the first frame is not equal to the first value, the method further comprises: determining that the first frame comprises a delayed RAP; reading open bitstream unit headers until a second frame with the open bitstream unit RAP flag equal to a second value and the open bitstream unit RAP type flag equal to the first value is found; decoding the second frame; and decoding frames following the second frame in the coding order.

86. The method of claim 85 further comprising: displaying the second frame; and / or displaying the frames following the second frame in the coding order.

87. The method of claim 78, wherein the criterion comprises identifying the first frame with an open bitstream unit random access point (RAP) type syntax element equal to a first value or a third value.

88. The method of claim 87, wherein when an open bitstream unit RAP type syntax element in the first frame is equal to the first value, the method further comprises: determining that the first frame comprises a key frame RAP; and decoding frames following the first frame in the coding order.8389. The method of claim 88 further comprising: displaying the first frame; and / or displaying the frames following the first frame in the coding order.

90. The method of claim 87, wherein, when the open bitstream unit RAP type syntax element in the first frame is equal to the third value, the method further comprises: determining that the first frame comprises a delayed RAP; reading open bitstream unit headers until a second frame with the open bitstream unit RAP type syntax element equal to a fourth value is found; decoding the second frame; and decoding frames following the second frame in the coding order.

91. The method of claim 90 further comprising: displaying the second frame; and / or displaying the frames following the second frame in the coding order.

92. The method of claim 78, wherein the criterion comprises identifying the first frame comprising a temporal delimiter open bitstream unit with a two bit delimiter RAP type syntax element equal to a fifth value or a third value.

93. The method of claim 92, wherein when the two bit delimiter RAP type syntax element is equal to the fifth value, the method further comprises: determining that the first frame comprises a key frame RAP; and decoding frames following the first frame in the coding order.

94. The method of claim 93 further comprising: displaying the first frame; and / or displaying the frames following the first frame in the coding order.8495. The method of claim 92, wherein, when the two bit delimiter RAP type syntax element is equal to the third value, the method further comprises: determining that the first frame comprises a delayed RAP; reading open bitstream unit headers until a second frame with the temporal delimiter open bitstream unit with the two bit delimiter RAP type syntax element is equal to a fourth value is found; decoding the second frame; and decoding frames following the second frame in the coding order.

96. The method of claim 95 further comprising: displaying the second frame; and / or displaying the frames following the second frame in the coding order.

97. An apparatus comprising: means for selecting a first frame to be a random access point (RAP) frame; means for setting an open bitstream unit (OBU) RAP flag equal to a first value for OBUs in a temporal unit of the first frame; means for setting a frame type as a key frame in a frame header of the first frame; and means for coding the first frame as an intra frame.

98. The apparatus of claim 97, wherein the apparatus further comprises means for performing methods as claimed in any of the claims 50 to 77.

99. An apparatus comprising: means for receiving a streaming or a broadcast video; means for reading open bitstream unit (OBU) headers in coding order; means for discarding open bitstream units until a criterion is met for a first frame; and means for decoding the first frame.

100. The apparatus of claim 99, wherein the apparatus further comprises means for performing methods as claimed in any of the claims 79 to 96.

101. A computer readable medium comprising program instructions that, which when executed by an apparatus, cause the apparatus to perform: selecting a first frame to be a random access point (RAP) frame; setting an open bitstream unit (OBU) RAP flag equal to a first value for OBUs in a temporal unit of the first frame; setting a frame type as a key frame in a frame header of the first frame; and coding the first frame as an intra frame.

102. The computer readable medium of claim 101, wherein the computer readable medium comprises a non-transitory computer readable medium.

103. The computer readable medium of any of claims 101 or 102, wherein the computer readable medium further comprises instructions for performing methods as claimed in any of the claims 50 to 77.

104. A computer readable medium comprising program instructions that, which when executed by an apparatus, cause the apparatus to perform: receiving a streaming or a broadcast video; reading open bitstream unit (OBU) headers in coding order; discarding open bitstream units until a criterion is met for a first frame; and decoding the first frame.

105. The computer readable medium of claim 104, wherein the computer readable medium comprises a non-transitory computer readable medium.

106. The computer readable medium of any of claims 104 or 105, wherein the computer readable medium further comprises instructions for performing methods as claimed in any of the claims 79 to 96.

Images