hypothetical reference decoder

Abstract

A video data stream in which video is encoded is described. It includes first timing information regarding a first coded image buffer (CPB) removal time to be applied at the decoder side when buffering a full version of the video data stream, and second timing information regarding a second CPB removal time to be applied at the decoder side to buffer a reduced version of the video data stream, the reduced version of the video data stream being different from the full version by removing a portion of the video data stream. Also described are a corresponding video encoder, a video decoder, a network node, means for managing the coded image buffer (CPB) of the video decoder, and means for splicing the video data streams together.

Description

Technical Field

[0001] This application is a divisional application of application number 202080032895.0, entitled "Hypothetical Reference Decoder". The hypothetical reference decoder (HRD) and its use for checking the consistency between the bitstream and the decoder are typically specified in video coding standards, such as AVC (Advanced Video Codec) and HEVC (High-Efficiency Video Coding).

[0002] According to embodiments of the present invention, there are provisions concerning video streams, video encoders, video decoders, network nodes, means for managing the encoded image buffer (CPB) of the video decoder, and means for splicing video data streams together. Background Technology

[0003] The following section will provide an introduction to the background technology.

[0004] To this end, an HRD buffer model is defined, which consists of a hypothetical stream scheduler (HSS), an encoded image buffer (CPB), a decoding process (which is considered instantaneous), a decoded image buffer (DPB), and an output cropping process, as follows: Figure 2 As shown in the image.

[0005] The model defines the timing and bit rate for feeding the bitstream to the encoded image buffer, the time when its decoding unit (access unit AU or VCL NAL (VCL: Video Coding Layer, NAL: Network Abstraction Layer) is removed from the CPB and instantaneously decoded, and the output time of the image from the DPB.

[0006] By doing so, the CPB size can also be limited to the size required by the decoder to avoid buffer overflow (more data sent to the decoder than can be stored in the CPB), or to avoid less data being sent to the decoder (at a lower bit rate than required), and to avoid underruns where the required decoding unit is not at the decoder at the correct time for decoding.

[0007] Currently, advanced technology decoding standards specify different parameters to describe bitstreams, HRD requirements, and buffer models.

[0008] For example, in HEVC, the hrd_parameter defined for each sublayer consists of one or more pairs of Bitrate(i) and CPBsize(i), which indicate that if the stream scheduler feeds to a CPB with size CPBsize(i) at a bit rate of Bitrate(i), no overflow or underflow will occur.

[0009] Combined with the hrd_parameter syntax element, additional timing information exists in the bitstream specifying the removal time of the image removed from the CPB, which indicates when to send the VCLNAL unit belonging to that image for decoding.

[0010] For ease of understanding, the description of sub-image processing is omitted.

[0011] The relevant information is present in the buffer periodic SEI (auxiliary enhancement information) messages with InitialCPBRemovalDelay(i), InitialCPBRemovalOffset(i), and AuCPBRemovalDelay, as well as in the image temporal SEI messages with AuCPBRemovalDelay.

[0012] For the most basic operations, only InitialCPBRemovalDelay(i) and AuCPBRemovalDelay are used.

[0013] In this scenario, the first access unit being decoded is the random access point (RAP) with its corresponding buffer period SEI message, and time 0 is defined as the time when the first RAP enters the CPB. Then, at time InitialCPBRemovalDelay(i), the image corresponding to the RAP and other non-RAP images are removed from the CPB. The removal of the CPB occurs at InitialCPBRevovalDelay(i) + AuCPBRemovalDelay (HEVC defines some parameters to convert delay into time, i.e., ClockTick, but this is omitted here for simplicity).

[0014] When the next RAP arrives, the removal time is calculated as before for non-RAP images, i.e., InitialCPBRevovalDelay(i) + AuCPBRemovalDelay, and this new value is used as the anchor for further increments until another RAP, i.e.

[0015] anchorTime=InitialCPBRemovalDelay(i)+AuCPBRemovalDelay

[0016] Subsequently, image removal becomes anchorTime + AuCPBRemovalDelay, and anchorTime is updated at the next RAP with the buffered SEI message.

[0017] anchorTime = anchorTime + AuCPBRemovalDelay, etc.

[0018] The operating mode described is the simplest one. Other situations need to be considered.

[0019] Under the current circumstances, the present invention overcomes several disadvantages. Details are as follows. Summary of the Invention

[0020] An embodiment of the invention relates to a video data stream in which video is encoded, and includes first timing information regarding a first coded image buffer (CPB) removal time to be applied at the decoder side when a full version of the video data stream is buffered, and second timing information regarding a second CPB removal time to be applied at the decoder side to buffer a reduced version of the video data stream, the reduced version being different from the full version of the video data stream by removing a portion of the video data stream.

[0021] A further embodiment of the invention relates to a video data stream in which video is encoded, including first HRD timing information data associated with a minimum CPB feed bit rate and second HRD timing information data associated with a maximum CPB feed bit rate, wherein the first HRD timing information data and the second HRD timing information data allow a third HRD timing information for the actual CPB feed bit rate to be determined by linear interpolation between the first and second HRD timing information data.

[0022] A further embodiment of the invention relates to a video data stream comprising: a sequence of image portions, each image portion containing an image encoded with video, the image portion including a first type image portion and a second type image portion serving as a CPB removal time reference; first timing information in each image portion relating to a first CPB removal time of the corresponding image portion, the first CPB removal time measuring the elapsed time since the removal of a previous first type image portion; and second timing information in each of the predetermined first type image portions relating to a second CPB removal time of the corresponding predetermined first type image portion in the case of reconstructing the decoded video data stream backward from the corresponding predetermined first type image portion, the second CPB removal time measuring the elapsed time since the arrival of the first CPB of the corresponding predetermined first type image portion; and a cut-off point indication indicating an image portion whose final CPB arrival time differs from its CPB removal time by a predetermined threshold, such that the image portion can be used as the last image portion before a cut-off point with another video data stream.

[0023] A further embodiment of the invention relates to a video encoder that encodes video into a video data stream, and provides first timing information and second timing information to the video data stream, the first timing information relating to a first CPB removal time to be applied at the decoder side when the full version of the video data stream is buffered, and the second timing information relating to a second CPB removal time to be applied at the decoder side to buffer a reduced version of the video data stream, the reduced version being different from the video data stream by removing a portion of the video data stream from the full version of the video data stream.

[0024] A further embodiment of the present invention relates to a video encoder that encodes video into a video data stream and provides the video data stream with first HRD timing information related to a minimum CPB feed bit rate and second HRD timing information related to a maximum CPB feed bit rate, wherein the first HRD timing information and the second HRD timing information allow a third HRD timing information for the actual CPB feed bit rate to be determined by linear interpolation between the first and second HRD timing information.

[0025] According to a further embodiment of the invention, reference is made to a video encoder for encoding video into a video data stream such that the video data stream includes a sequence of image portions, each image portion containing an image of video, wherein the video encoder classifies the image portions into a first type of image portion and a second type of image portion as a reference for CPB removal time; provides the video data stream with first timing information for each image portion regarding a first CPB removal time of the corresponding image portion, the first CPB removal time being a measurement of the elapsed time since the removal of a previous first type of image portion, and second timing information for each of the predetermined first type of image portions regarding the time elapsed since the removal of the corresponding first type of image portion. In the case of forward recovery decoding of the video data stream for a first type of image portion, a second CPB removal time is defined for the corresponding predetermined first type of image portion. The second CPB removal time measures the elapsed time from the arrival of the first CPB of the corresponding predetermined first type of image portion. For each of the predetermined image portions, it is checked whether the time difference between its final CPB arrival and its CPB removal exceeds a predetermined threshold, so that the predetermined image portion can be used as the last image portion before the splicing point with another video data stream. A splicing point indication is provided to the video data stream to indicate those predetermined image portions whose time difference between their final CPB arrival and their CPB removal exceeds the predetermined threshold.

[0026] A further embodiment of the present invention relates to a network node for forwarding video data streams, which is used to remove portions of the video data stream from the video data stream.

[0027] A further embodiment of the present invention relates to an apparatus for managing a video decoder's CPB, the video decoder decoding a video data stream via a CPB buffer, the apparatus managing the CPB according to first timing information when the video data stream includes the portion therein, and managing the CPB according to second timing information when the portion has been removed.

[0028] A further embodiment of the present invention relates to an apparatus for managing a video decoder's CPB, the video decoder decoding a video data stream in which video is encoded, the apparatus deriving first HRD timing information related to a minimum CPB feed bit rate from the video data stream, deriving second HRD timing information related to a maximum CPB feed bit rate, determining third HRD timing information for an actual CPB feed bit rate by linear interpolation between the first and second HRD timing information, and managing the CPB using the third HRD timing information.

[0029] A further embodiment of the present invention relates to an apparatus for managing a video decoder's CPB (Concurrent Processing Block), the video decoder decoding a video data stream via a CPB buffer. The apparatus checks whether a concatenation flag in a first predetermined first-type image portion indicates that a cut-in has occurred at the first predetermined first-type image portion, and determines the time for removing the first predetermined first-type image portion from the CPB. Wherein, if the concatenation flag in the first predetermined first-type image portion indicates that no cut-in has occurred at the first predetermined first-type image portion, the determination is based on first timing information of the first predetermined first-type image portion; and if the concatenation flag in the first predetermined first-type image portion indicates that a cut-in has occurred at the first predetermined first-type image portion, the determination is based on second and third timing information of the first predetermined first-type image portion.

[0030] A further embodiment of the present invention relates to an apparatus for splicing a first video data stream with a second video data stream, each of the data streams comprising: a sequence of image portions, each image portion containing an image encoded with video, the image portion including a first type image portion and a second type image portion serving as a CPB removal time reference; first timing information in each image portion relating to a first CPB removal time of the corresponding image portion, the first CPB removal time measuring the elapsed time since the removal of a previous first type image portion; and second timing information in each of the predetermined first type image portions relating to a corresponding predetermined first type image portion in the case of resuming the decoded video data stream backward from the corresponding predetermined first type image portion. The second CPB removal time for the type image portion, the second CPB removal time is measured from the arrival of the first CPB of the corresponding predetermined first type image portion, wherein the second video data stream includes a concatenation flag in the first predetermined first type image portion and third timing information, the concatenation flag is set to indicate a second state to the second video data stream that the first predetermined first type image portion continues the second video data stream, and the third timing information indicates a third CPB removal time for the first predetermined first type image portion, used to determine the CPB removal time when the second video data stream is cut and accessed at the first predetermined first type image portion, the third CPB removal time is measured from the most recent CPB removal of the non-discardable image portion. The apparatus of the present invention checks whether the splice point indication in the first video data stream indicates that the time difference between the arrival and removal of the CPB of a predetermined image portion exceeds a predetermined threshold, such that the predetermined image portion can be used as the last image portion before the first video data stream to be spliced ​​with the second video data stream. If so, the concatenation flag in the first predetermined first type image portion of the second video data stream is set to a first state, the first state indicating that the second video data stream has been spliced ​​at the first predetermined first type image portion, and the first data stream and the second data stream are concatenated at the predetermined image portion and the first predetermined first type image portion, respectively, to obtain a spliced ​​video data stream.

[0031] The concepts described above can be implemented by methods according to embodiments of the present invention. These methods are based on the same considerations as those described above regarding the decoder, encoder, apparatus, and data stream. However, it should be noted that the methods can be supplemented by any of the features, functionalities, and details also described herein with respect to the decoder, encoder, apparatus, and data stream. Furthermore, the methods can be supplemented individually and in combination by the features, functionalities, and details of the decoder, encoder, apparatus, and data stream.

[0032] Finally, the concept can also be used to generate encoded data streams according to embodiments of the present invention. The data stream can also be supplemented individually and in combination by the features, functionality, and details of the decoder, encoder, apparatus, and method. Attached Figure Description

[0033] Embodiments of the present invention will then be described with reference to the accompanying drawings, in which:

[0034] Figure 1 This illustrates an encoded data stream according to an embodiment of the present application.

[0035] Figure 2 A flowchart for a hypothetical reference decoder according to an embodiment of this application is shown.

[0036] Figures 3 to 7 as well as Figure 9 This illustrates the state of an encoded image buffer, including time removal, according to an embodiment of this application.

[0037] Figure 8 , Figure 10 , Figure 11 , Figures 20 to 2 4 and Figures 26 to 34 Exemplary signal notification syntax according to embodiments of this application is shown.

[0038] Figures 12 to 14 , Figure 16 as well as Figure 18 The following diagram illustrates data stream structures for different frame rates according to embodiments of this application.

[0039] Figure 15 , Figure 17 as well as Figure 19 Exemplary removal time values ​​are shown according to embodiments of this application when different frame rates are provided under the same bitstream conditions.

[0040] Figure 25 This illustrates a spliced ​​encoded data stream according to one embodiment of this application.

[0041] Figure 35 , Figure 36 as well as Figure 37 A table showing the calculated values ​​of the CPB size according to embodiments of this application, and

[0042] Figure 38 A graph showing an exemplary CPB fill level according to an embodiment of this application.

[0043] In the accompanying drawings, similar reference numerals indicate similar elements and features. Detailed Implementation

[0044] In the following sections, some considerations upon which the invention is based will be discussed and several solutions will be described. Specifically, several details will be disclosed, which may optionally be incorporated into any of the embodiments disclosed herein. Specifically, the following description begins with a simple presentation of the problems encountered when performing splicing, stream extraction, stripping scalable bitstreams, and bitstream feeding under various transmission conditions. Then, some modifications to the problems are presented subsequently using corresponding modified embodiments.

[0045] Check and resolve the following issues.

[0046] Splicing The removal time of the previous RAP with buffered SEI is used as the anchor for adding AuCPBRemovalDelay, so that the removal time of the CPB of the image is used.

[0047] However, when splicing occurs, the previous value of anchorTime cannot be easily obtained at the splice point. Therefore, in this case, the CPB removal time of the spliced ​​RAP with buffered SEI is derived differently.

[0048] To address this issue, HEVC includes two additional parameters: `concatenation_flag`, which indicates whether splicing has occurred at the RAP containing the buffered periodic SEI message; and `AuCPBRemovalDelta`, which, instead of referring to the previous RAP with the buffered periodic SEI message, is the time increment up to the previously non-discardable image. This prevents the need to calculate any value and rewrite the buffered periodic SEI message at the splice point. The CPB removal time from the RAP where splicing occurred will then be...

[0049] Remove(previousNonDiscardable)+AuCPBRemovalDelta.

[0050] This assumes a seamless splicing scenario. However, it cannot be guaranteed that the CPB state is the same at the splice point of the original bitstream (when the RAP splicing result with a buffered periodic SEI message has not been spliced, but is within its original bitstream) and at the time of splicing. More specifically, Figure 3 The problem described is illustrated, where the bitstream at the top of frame #3 is spliced ​​with the bitstream at the bottom from frame #2 forward.

[0051] Therefore, instead of having a removal time as discussed above (which was originally earlier than when frame t had arrived), af 3 o'clock)

[0052] Removal(previousNonDiscardable)+AuCPBRemovalDelta,

[0053] Instead, use removal time.

[0054] Removal(previousNonDiscardable)+InitialCPBRevovalDelay(i)+t af 2-t rm 2.

[0055] That is, the removal time of the spliced ​​RAP is:

[0056] Removal(previousNonDiscardable)+SplicingDelta

[0057] SplicingDelta is set to AuCPBRemovalDeta and InitialCPBRevovalDelay(i)+t. af 2-t rm The maximum value of 2.

[0058] When the second value is exactly greater than the first value, for example... Figure 3 As shown, a non-seamless splicing will occur.

[0059] This splicing operation only takes effect when the first stream ends at the corresponding NonDiscardable image. This is because the AuFinalArrivalTime and the removal times of subsequent images after the NonDiscardable image are unknown, and therefore, if both values ​​t... af X and t rm If the difference in X is different from that in the NonDiscardable image, the derived removal time may be incorrect (InitialCPBRemovalDelay(i)+t). af Xt rm X).

[0060] CPB fullness and latency Another problem with the HRD model is that in order to utilize the CPB buffer, a time delay is required for the removal of the first access unit.

[0061] When the first RAP enters the CPB after splicing, the removal of the AU is delayed until after the InitialCPBRemovalDelay(i), as discussed. A given CPB fullness, such as CPBA, is achieved by feeding the CPB with Bitrate(i) during InitialCPBRemovalDelay(i). Afterwards, it is impossible to achieve a fullness greater than CPB at other RAPs with buffered periodic SEI messages. A Large CPB B The reason can be seen mathematically: check the earliest time when AU can enter CPB.

[0062] initArrivalTime[n]=Max(AuFinalArrivalTime[n-1], initArrivalEarliestTime[n]), where initArrivalEarliestTime[n]=RemovalTime[n]-InitCpbRemovalDelay(i)

[0063] This means that if an AU with a buffered SEI message cannot enter the CPB before its removal time InitialCPBRemovalDelay(i), it cannot achieve a time greater than CPB. A CPB B This is because feeding CPB with Bitrate(i) during InitialCPBRemovalDelay(i) only implements CPB. A CPB fullness.

[0064] To address this issue, the idea is to hypothetically schedule the first RAP with buffered SEI messages by delaying the scheduling of the transmitter (or the HSS in the attached diagram) by a given time offset InitialCPBRemovalOffset(i), as... Figure 4 As shown in the image.

[0065] It should be mentioned that this only applies to VBR and not CBR, because in CBR,

[0066] initArrivalTime[n]=AuFinalArrivalTime[n-1].

[0067] In this case, the scheduling changes as follows:

[0068] initArrivalEarliestTime[n]=RemovalTime[n]-InitCpbRemovalDelay(i)-

[0069] InitialCPBRemovalOffset(i)

[0070] This means CPB B The size of the CPB can correspond to the size achieved by feeding the CPB with Bitrate(i) to InitCpbRemovalDelay(i) + InitialCPBRemovalOffset(i).

[0071] AU discard The HRD model in HEVC also supports the discarding of some NAL units. More specifically, "discardable" images associated with RAP images can be discarded. "Discardable" images refer to RASL (Random Access Skip Before) images, which are images in an open GOP (Group of Pictures) structure that precede the RAP image in presentation order but follow the RAP image in decoding order, and also reference images that precede the RAP image in decoding order. Therefore, when performing random access at the corresponding RAP image, the RASL image cannot be decoded. Thus, sending the RASL image is wasteful when splicing this bitstream with another bitstream or when starting a session at the RAP location, and in some cases, these images are discarded before transmission.

[0072] Clearly, the timing information, namely the initial arrival time and removal time of the image, changes when the bitstream is modified.

[0073] Therefore, the buffered periodic SEI message contains an alternative timing for the case where the RASL image associated with the RAP that has the buffered periodic SEI message is discarded:

[0074] o AltInitCpbRemovalDelay(i)

[0075] o AltInitialCPBRemovalOffset(i)

[0076] Time scalabilityWhen a sub-layer is discarded, the timing information also changes. For example, if the original bitstream is 60Hz, the distance in the nominal removal time between two consecutive images in the decoding order is expected to be 1 / 60 of a second. And when discarding every other image, the distance will be 1 / 30 of a second. This means that the image timing SEI message will need to be replaced. In addition, InitCpbRemovalDelay(i) and InitialCPBRemovalOffset(i) will also need to be changed. Therefore, the buffer periodic SEI message will need to be replaced. For this operation, additional image timing SEI messages and buffer periodic SEI messages are typically included in so-called nested SEI messages. When intermediate software performs sub-layer bitstream extraction (e.g., discarding every other frame to obtain a 30Hz bitstream from a 60Hz bitstream), the original image timing SEI message and buffer periodic SEI message are replaced by the corresponding image timing SEI message and buffer periodic SEI message from the nested SEI message.

[0077] Bit rate variant HRD parameters are typically provided for several values, such as initial_removal_delay and initial_removal_delay_offset, as well as Bitrate and CPB_siz. The idea is that the bit rate fed into the CPB from the positioning stream can vary, and thus provide several values ​​that produce an effective HRD model.

[0078] However, there are some aspects that have not yet been properly considered:

[0079] o may only exist at the bit rate that generates CBR.

[0080] If the appropriate bitrate is not known prior to the bitstream, the bitstream cannot provide a valid HRD model (its parameters).

[0081] As outlined above, splicing at nondiscardable points has been a difficult task to achieve until now, and therefore, the embodiments described below aim to accurately realize the possibility of splicing even at nondiscardable points. A first simple correction is to adjust the formula so that instead of always using the nondiscardable image as the anchor, it uses the last image received after the splice point. That is, from the following formula:

[0082] Removal(previousNonDiscardable)+SplicingDelta

[0083] SplicingDelta is set to AuCPBRemovalDelta and InitialCPBRevovalDelay(i)+t. af 2-t rm The maximum value of 2

[0084] Change to the maximum value among the following:

[0085] -Removal(previousNonDiscardable)+AuCPBRemovalDelta

[0086] -Removal(lastPictureBeforeSplicing)+InitialCPBRevovalDelay(i)+t af 2-t rm 2, where Removal(lastPictureBeforeSplicing) is actually t rm 2. (In the example above)

[0087] Similarly, as can be seen in the formula, if the splicer decides to treat the previous image as the last image before splicing, for example t rm 1. In this case, the maximum value will always be Removal(previousNonDiscardable) + AuCPBRemovalDelta. This is unless the splicer changes the value of AuCPBRemovalDelta accordingly and determines (t... af 2-t rm 2) The difference makes seamless switching feasible; otherwise, this situation would prevent seamless splicing at any AU.

[0088] Changing the value of AuCPBRemovalDelta during the buffer period SEI would be feasible. However, the splicer maintains tracking (t af X and t rm The value of X) is not simple, because keeping track (t) af X and t rm X) is not that simple.

[0089] Since seamless splicing at different points is desirable, it would be appropriate to provide the splicer with some signaling notification indicating whether seamless splicing of a given image is permitted. This can be achieved by checking (t) on the encoder side. af X and t rm X) value, and ensure the difference (t) rm Xt afX) will not become less than a given value to achieve this. This signal notification can, for example, appear in the image time-series SEI associated with the last image that can be used as a cutoff point. An example can be found in... Figure 26 I saw it in the middle.

[0090] Alternatively, even at the non-discardable image level, it can be guaranteed that subsequent discardable images will meet the requirements. An example can be found... Figure 27 I saw it in the middle.

[0091] Due to (t) rm Xt af The minimum requirement that X) must meet is related to the initial_removal_delay of the spliced ​​bitstream. There should be some indication at the buffer period SEI regarding what values ​​can be used for seamless splicing of the image. An example can be found... Figure 28 I saw it in the middle.

[0092] Therefore, as a result of the ideas for possible modifications that have just been outlined, the following embodiments were evaluated.

[0093] exist Figure 25 and Figure 26 , Figure 27 and Figure 28 In the example of signal notification, it can be seen that the video data stream may include a sequence 23a of image portions, such as an access unit, and an image 10a encoded with video 12a in each image portion. Figure 25 Two data streams 14a and 14b are depicted, and it is shown that streams 14a and 14b are spliced ​​at splice point 92 such that the end of the portion of stream 14a before splice point 92 (i.e., to the left) is adjacent to the beginning of the portion of stream 14b after splice point 92 (i.e., to the right). The result of the splicing performed by a splicing device such as a server, video conferencing system, or master-slave streaming system is shown at 14c, i.e., the spliced ​​data stream.

[0094] The image portion may include a first type of image portion, which is shown as crosshairs or simply as shading in some figures, and is referred to above as a marked image. A corresponding example is an AU having or including a buffer period SEI. These first type of image portions serve as a CPB removal time reference, that is, their CPB removal is used as a reference anchorTime for other timings given a time offset such as AuCPBRemovalDelay.

[0095] The image portion may further include a second type of image portion, which is shown in some figures without shadow lines, such as an image that does not include any buffered period SEI.

[0096] Data streams 14a and 14b additionally include first timing information 95, which compares the AuCPBRemovalDelay or au_cpb_removal_delay_minus1 with respect to the first CPB removal time of the corresponding image segment in each image segment. The first CPB removal time is measured as the elapsed time since the removal of the previous first type of image segment, i.e., in Figure 25 The image portion shown is located earlier in the encoding or bitstream order, especially the closest preceding image portion.

[0097] Data streams 14a and 14b further include second timing information, comparing an initial_removal_delay in each of the predetermined first-type image portions. This initial_removal_delay may exist in the image timing syntax of the buffered period SEI, for example, before the splice point indication discussed below. These predetermined first-type image portions in Figure 25 Crosshairs are used to indicate the first type of image portion, distinguishing it from other first type image portions that are simply shown as shading and are specific first type image portions, such as, for example, an AU with a buffer period SEI indicated as RAP, or in other words, an AU including a buffer period SEI and indicated as RAP. Second timing information, when resuming the decoded video data stream backward from the corresponding predetermined first type image portion, informs the second CPB removal time for the corresponding predetermined first type image portion. The second CPB removal time measures the elapsed time from the arrival of the first CPB of the corresponding predetermined first type image portion, i.e., the time to the CPB of the decoder.

[0098] At least one of the data streams (in) Figure 25 14a, which indicates that the stream should be split or spliced ​​to obtain an additional stream attached to its splice point 92 (here, 14b), may further include splice point indicators 94, 94', such as splicable_flag or following_pic_spliceable_flag, which indicate the image portion (e.g., without RAP), the time difference between the arrival and removal of the final CPB of the image portion (such as t indicated above). rm #-t af #) Exceeding a predetermined threshold, making it usable as the last image portion before the splice point 92 of another video data stream 14b, i.e., suitable for allowing splicing.

[0099] It should be noted that the threshold can be calculated, for example, as follows:

[0100] Assume the removal time is equal to 1 / frame rate. Then, the newly spliced ​​AU (such as...) Figure 25The expected removal time for AU 23b' in the code should be t. rm #+1 / frame rate, where t rm # represents the removal time of the AU or image portion immediately preceding 23'a.

[0101] t rm #+1 / Frame rate should be equal to or greater than t af #+initial_removal_delay, where t af # represents the final arrival time of the CPB for the corresponding AU23b'. Therefore, the "predetermined threshold" is initial_removal_delay - 1 / frame rate.

[0102] Furthermore, in the variant of max_val_initial_removal_delay_for_seamless_splicing, i.e., when this threshold is explicitly indicated in the last data stream (such as data stream 14a to which other stream 14b is to be attached), t rm #+1 / Frame rate equal to or greater than t af #+max_val_initial_removal_delay_for__seamless_splicing. Then, the "predetermined threshold" is max_val_initial_removal_delay_for_seamless_splicing - 1 / frame rate.

[0103] That is, video data stream 14a may also include an indication 99 of a maximum second CPB removal time value, which indicates that if the second CPB removal time of the initial first type image portion 23b' of the other video data stream 14b cascaded with video data stream 14a at the splice point 92 is lower than the maximum second CPB removal time value, then the other video data stream 14b will be cascaded to the time difference t between its final CPB arrival and its CPB removal. rm #-t af #Any image portion exceeding a predetermined threshold, as indicated by the splice point indicator, will result in seamless splicing.

[0104] It should be noted that the SEI buffer cycle example given above includes indicator 99, cascading flag 95, and third CPB removal time 98. This indicates an example where both streams 14a and 14b carry the same kind of information data. In other examples, this data may be different.

[0105] The splice point indicators 94, 94' may further include a flag 94 present at each of the image portions or at each of the set of image portions (e.g., without RAP), indicating the time difference t between the final CPB arrival and its CPB removal for the corresponding image portion. rm #-t af #Whether it exceeds the predetermined threshold.

[0106] Alternatively or additionally, the splice point indications 94, 94' may further include a flag 94' present at each of the image portions or at each of the set of image portions (e.g., in these image portions without RAP), the flag indicating whether, for each image portion moving forward from the corresponding image portion to the next first type image portion, the time difference trm#-taf# between its final CPB arrival and its CPB removal exceeds a predetermined threshold.

[0107] The sequence of image portions may further include images of video 12a encoded in some way using temporal inter-prediction, such as motion-compensated prediction 90, such that the image portions are included in some accompanying figures (such as...). Figure 25 The non-discardable image portions are shown without underlined text in the figure, while the discardable image portions are shown with underlined text in the figure.

[0108] These essential image portions are decodeable independently, while the discardable image portions are not necessary for decoding the essential image portions, but may be required to make them decodeable themselves. It should be noted that without decoding the discardable image portions, the video can be decoded without any problems in the future.

[0109] That is, discardable image portions are not used, for example, as reference images for intertemporal prediction, while non-discardable images can be used as reference images for intertemporal prediction. An example of representing these image portions using a hierarchical temporal scaling encoded data stream is that the image portions for the lowest (base) temporal layer can be non-discardable, while other portions may be discardable. Another example is an open GOP encoded data stream, where RASL images can be discardable.

[0110] Video data stream 14c is spliced ​​at splice point 92, such that splice point indicator 94' indicates the time difference t between its final CPB arrival and its CPB removal. rm #-t af #The first predetermined first type image portion 23b' following the predetermined image portion 23a' that exceeds the predetermined threshold and is adjacent to the video data stream 14b originating from the clip access is the first predetermined first type image portion 23b'.

[0111] The video data stream 14c includes a concatenation flag 96 and third timing information 98 in the first predetermined first type image portion, comparing AuCPBRemovalDelta or au_cp_removal_delay_delta_minus1. The concatenation flag 96 and the third timing information 98 may have already existed in the spliced ​​data stream 14b before splicing, but the splicing device may have reset the flag 96 from an unspliced ​​state to an spliced ​​state, and / or may have set the third timing information 98. That is, the concatenation flag 96 is set to a first state indicating that the video data stream has been spliced ​​at the first predetermined first type image portion, and the third timing information indicates a third CPB removal time for the first predetermined first type image portion 23b', which is used to determine the CPB removal time in the case of splicing at the first predetermined first type image portion 23b'. The third CPB removal time measures the elapsed time since the most recent CPB removal of the non-discardable image portion. For example, starting with removing the most recently discardable image portion of the video data stream that has been encoded in the spliced ​​data stream 14c.

[0112] Optionally, each of the predetermined first type image portions shown in crosshairs in some of the accompanying drawings may include a concatenation flag 96 and third timing information 98, the concatenation flag being set to a first state and a second state, the second state indicating that the video data stream has not yet been spliced ​​at the corresponding predetermined first type image portion.

[0113] The video encoder according to this embodiment can encode video into a video data stream in the following manner. For example, imagine that this encoder will encode data stream 14a. It can also be configured to encode data stream 14b, as discussed above, which can be considered the same. The encoder will perform encoding such that video data stream 14a includes a sequence 23a of image portions, each image portion encoding an image 10a of video 12a. The video encoder can then classify the image portions into a first type of image portion used as a CPB removal time reference, and a second type of image portion, as discussed above. The encoder can provide a video data stream with first timing information and second timing information, and for each of the predetermined image portions (which may include, for example, discardable image portions), check whether the time difference between its final CPB arrival and its CPB removal exceeds a predetermined threshold. If so, the corresponding predetermined image portion can be used as the last image portion before the splice point 92 with another video data stream 14b.

[0114] The video encoder can further provide a video data stream 14a with splice point indications 94, 94'.

[0115] For each predetermined image segment, the video encoder may further use a value determined based on the second CPB removal time of the most recent first type predetermined image preceding the corresponding predetermined image segment as a predetermined threshold, such as the image before 23a' simply shown in the shading in some of the accompanying figures.

[0116] For each predetermined image portion, the video encoder may also use a value determined based on the maximum second CPB removal time value as a predetermined threshold, and write the indication 99 of the maximum second CPB removal time value into the most recent first type predetermined image before the corresponding predetermined image portion, such as the image before 23a' simply shown in the figure with shading.

[0117] The following describes how the spliced ​​data stream, as discussed above, can be processed within the decoder, such as by the HRD 46 within the decoder, or in other words by the device 46 for managing the CPB 48 of the video decoder 44. The video decoder then decodes the video data stream 14c via the CPB buffer, i.e., by receiving the image portion via the CPB at the removal time, where the image portion fully arrives at the aforementioned final arrival time. This device can check whether the cascading flag 96 in the first predetermined first type image portion 23b' indicates that splicing has occurred at the first predetermined first type image portion. The device can then determine the time when the first predetermined first type image portion 23b' is removed from the CPB.

[0118] If the cascading flag in the first predetermined first type image portion indicates that no clipping has occurred at the first predetermined first type image portion, the removal time can be determined based on the first timing information 95 of the first predetermined first type image portion 23b'.

[0119] Furthermore, if the cascading flag 96 in the first predetermined first type image portion indicates that a cut access has occurred at the first predetermined first type image portion, the removal time can be determined based on the second timing information of the first predetermined first type image portion 23b', such as initial_removal_delay and the third timing information 98 of the first predetermined first type image portion.

[0120] Optionally, the apparatus can make this determination by determining the maximum value between the following two: the most recent CPB removal of the non-discardable image portion plus the third CPB removal time of the first predetermined first type image portion, followed by the CPB removal of the predetermined image portion 23a' adjacent to the first predetermined first type image portion plus the second CPB time minus the time difference t between the final CPB arrival and CPB removal of the predetermined image portion 23a'. rm #-t af#. This time is then used to remove the first pre-defined first type image portion 23b' from the CPB.

[0121] Furthermore, according to this embodiment, a device is described, namely... Figure 25 The apparatus depicted is used to splice two video data streams (here 14a and 14b) together to obtain 14c. Each data stream is as described above, such that each data stream includes a sequence of image portions 23a, 23b, each image portion containing images 12a, 12b encoded with video 12a, 12b.

[0122] The image portion, as described above, includes first and second type image portions and first and second timing information.

[0123] The second video data stream will then include a concatenation flag 96 and third timing information 98 in the first predetermined first type image portion. The concatenation flag 96 is configured to indicate to the second video data stream that the first predetermined first type image portion continues in a second state of the second video data stream, and the third timing information indicates a third CPB removal time for the first predetermined first type image portion. This third CPB removal time is used to determine the CPB removal time when the second video data stream is cut and inserted at the first predetermined first type image portion. The third CPB removal time is measured as the elapsed time since the most recent CPB removal of the non-discardable image portion.

[0124] The device can then check, for a predetermined image portion (e.g., without RAP), whether the splice point indication 94;94' in the first video data stream indicates that the time difference between its CPB arrival and its CPB removal exceeds a predetermined threshold, so that the predetermined image portion can be used as the last image portion of the first video data stream before it is spliced ​​with the second video data stream.

[0125] For example, in one variant, the predetermined threshold is determined based on the initial_removal_delay of the buffer SEI of the first stream 14a or based on max_val_initial_removal_delay_for_seamless_splicing.

[0126] If the result of this check is positive, i.e. "yes", the device can set the concatenation flag 96 in the first predetermined first type image portion of the second video data stream to a first state indicating that the second video data stream has been spliced ​​at the first predetermined first type image portion, and concatenate the first and second data streams at the predetermined image portion 26a' and the first predetermined first type image portion 26b' respectively, so as to obtain the spliced ​​video data stream.

[0127] Furthermore, if so, the device can rewrite the third timing information 98 to measure the elapsed time since the most recent removal of the non-discardable image portion from the first video stream within the spliced ​​video stream, rather than from the second video stream.

[0128] Furthermore, if so, the device may further check whether the elapsed time since the most recent removal of the non-discardable image portion preceding the first predetermined first type image portion has changed when considering the spliced ​​video data stream compared to the second video data stream, and if so, rewrite the third timing information 98 to measure the elapsed time since the most recent removal of the non-discardable image portion of the first video data stream within the spliced ​​video data stream.

[0129] Typically, the device can also check the following checks, and if both checks show yes, then perform setup and cascading.

[0130] First, whether the second CPB removal time equal to or greater than the second CPB removal time of the most recent first type predetermined image portion of the first video data stream 14a preceding the corresponding predetermined image portion, or the upper limit determined from the second CPB removal time, is greater than the second CPB removal time of the first predetermined first type image portion 23b' of the second video data stream.

[0131] The most recent first type of predetermined image portion of the first stream 14a is simply shown in the shading in some of the accompanying drawings.

[0132] The second CPB removal time for the most recently selected first-type pre-defined image portion can also have a CPB feed delay time for the addition of the most recently selected first-type pre-defined image portion. This measures the elapsed time for delaying the arrival of the first CPB of the most recently selected first-type pre-defined image portion.

[0133] The corresponding predetermined image portion is shown simply as a shading line in some of the accompanying drawings, preceding 23a'.

[0134] The second check is whether the maximum second CPB removal time value 99 indicated in the most recent first type of predetermined image portion is greater than the second CPB removal time of the first predetermined first type of image portion 23b' of the second video data stream.

[0135] For example, in one variant, the predetermined threshold might simply be the initial_removal_delay of the first-order buffer SEI. Alternatively, the threshold could be initial_removal_delay + initial_removal_delay_offset, i.e., the CPB feed delay. Further alternatively, as described in the second option, the maximum value max_val_initial_removal_delay_for_seamless_splicing could be sent.

[0136] For example, AuCPBRemovalDelta only needs to be rewritten if the original value of the distance to the non-discardable image in the original second bitstream 14b is different from the distance at the spliced ​​bitstream 14c.

[0137] For example, it can be assumed that both bit streams have the same frame rate, and that the following bit streams, shown in decoding order, are concatenated, i.e. spliced:

[0138] The second stream: RAP1, B0, B1 (not discardable), B2, RAP2 (splicing_point)

[0139] First stream: rap1, b0 (cannot be discarded), b1, b2, b3, b4

[0140] spliced ​​together:

[0141] rap1, b0 (cannot be discarded), b1, b2, b3, RAP2 (splicing_point)

[0142] Originally, the buffer period SEI for RAP2 will include a concatenation_flag set to 0 and an AuCPBRemovalDelta equal to 2 / frame rate.

[0143] In the spliced ​​bitstream, concatenation_flag will be set to 1 and AuCPBRemovalDelta will be equal to 4 / frame rate.

[0144] However, if the spliced ​​bitstream is rap1, b0 (non-discardable), b1, RAP2 (splicing_point), then concatenation_flag is set to 1 and AuCPBRemovalDelta is equal to 2 / frame rate.

[0145] Therefore, the first case requires rewriting AuCPBRemovalDelta, but the second case does not.

[0146] For spliced ​​streams with an initial removal delay less than max_val_initial_removal_delay_for_seamless_splicing, seamless splicing can be achieved if the flag at the image temporal SEI indicates this. If the initial removal delay of the spliced ​​stream is high, it becomes impossible to know whether the stream can be spliced.

[0147] The preceding embodiments addressed the problem of implementing splicing at non-discardable points, while the following description addresses how to implement available removal times that can also be used for removable image portions such as DRAP. Specifically, subsequent embodiments address providing timing information (removal time) in the case of performing the dropping of some AUs, such as in RASL images with open GOP structures during random access or splicing operations. In other words, does removal time also apply to DRAP images that depend on random access points?

[0148] The idea here is to modify the initial_removal time of the RAP's buffer period SEI. Different scenarios are discussed below, along with examples of AU removal times.

[0149] Figure 1 This illustrates the inventive concept according to this application. It depicts a video data stream encoded with video 12, and, in the case of a complete version 15 including a buffered video data stream 14, a first CPB removal time t to be applied at the decoder side. rm (1) The first timing information 16, and the second CPB removal time t to be applied at the decoder side to buffer the reduced version 20 of the video data stream 14. rm (2) The second timing information 18, the reduced version is different from the video data stream by removing 22 portions 24 of the video data stream from the full version of the video data stream.

[0150] Optionally, the second timing information 18 is derived from information about how to modify the first CPB removal time t. rm (1) To obtain the second CPB removal time t rm (2) The timing modification information limits the second CPB removal time t rm (2) An example of this sequence modification information is init_removal_delay_correction_offset.

[0151] Furthermore, the first timing information 16 can signal the first CPB removal time t image by image and by means of the increment of the preceding marked image relative to the decoding order 26.rm (1) .

[0152] The increment can be signaled, for example, by AuCPBRemovalDelay or au_cpb_removal_delay_minus1. The labeled image can be, for example, at the clean random access point CRA and is an image containing the buffered period SEI. It is typically a RAP image but can be other basic images, such as a time layer 0 image.

[0153] In other words, each image portion 23 of the video data stream includes a measurement of the increment of its removal from the CPB 48 relative to the delay of the RAP image portion 23, which precedes the image portion along the encoding order 26. The image portion 23 is also referred to as an access unit AU, such as in the case of HEVC.

[0154] Alternatively, the labeled image can be labeled using buffered periodic messages within the image portion associated with the labeled image in the video data stream.

[0155] Furthermore, the means 46 for managing the CPB 48 of the video decoder 44 for managing the decoded video data stream can manage the CPB according to the first timing information 16 when the video data stream includes the portion therein, and can manage the CPB according to the second timing information 18 when the portion 24 has been removed.

[0156] Figure 5 The complete bitstream is shown. The initial removal time for the Clean Random Access Point (CRA) is 8.

[0157] exist Figure 7 In this process, the RASL image is removed. The initial removal time for CRA is 7. Compared to the previous case, the removal time for subsequent images has an incremental shift of 4.

[0158] Figure 6 The image shows the removed RASL image and the TRAIL (normal post-processed) image up to (but excluding) TRAIL4 (which is DRAP). The initial removal of CRA is now 11, making the CPB level the same as at the start of the bitstream.

[0159] One option would be to add alternative timing sequences for different possibilities of dropping frames. That is, one timing sequence for removing RASL images, one timing sequence for removing up to the first DRAP, one timing sequence for removing up to the second DRAP, and so on.

[0160] However, this would increase the size of the buffer period SEI and would require instructing the receiver on the conditions under which it applies, such as the removal of RASL, all AUs up to the first DRAP, all AUs up to the second DRAP, etc. This can be done through SEI.

[0161] However, it will also require encoding of all DRAPs so that the buffered SEI can be written properly, which will result in additional latency at the encoder / transmitter side.

[0162] Alternatively, a single value is indicated at the buffered SEI message, and this value can be modified by subsequent SEIs, such as the timing SEI message of the next unremoved AU. Then, when AU removal is performed, only the flag aus_since_rap_removed_flag needs to be set. A corresponding example can be found in... Figure 8 I saw it in the middle.

[0163] As can be seen from the attached diagram mentioned above, the second timing information 18 can limit the second CPB removal time t by means of the first timing modification information 19 or by means of the first timing replacement information. rm (2) The first timing modification information relates to how to modify the first CPB removal time t indicated by the first timing information relative to the first image portion 28' in the video data stream, which is related to the first predetermined image CRA and precedes portion 24' in the video data stream. rm (1) The second CPB removal time t is obtained relative to the first image portion 28'. rm (2) The first timing substitution information is used to replace the first CPB removal time relative to the first image portion to obtain the second CPB removal time.

[0164] The first timing modification information 19 can be notified, for example, by a signal from init_removal_delay_correction_offset.

[0165] Furthermore, the first timing modification information 19 or the first timing replacement information is signaled in the video data stream, after portion 24' of the video data stream and within the second image portion 30' of the video data stream related to the second predetermined image TRAIL1.

[0166] It should be noted that SEI messages can also be buffered in the CPB, and therefore, these messages can be considered when determining the first and second timing information. The first and second timing information can be calculated and transmitted once for SEI messages that include the first and second timing information, and once for discarded SEI messages.

[0167] Alternatively, the second image portion may include a signal notification 32 indicating whether the portion has been removed.

[0168] Furthermore, the first timing modification information 19 or the first timing replacement information can be signaled in the video data stream, regardless of whether the signal notification 32 indicates that the part has been removed or the signal notification 32 indicates that the part has not been removed.

[0169] The first predetermined image can be a labeled image, such as an Intra-Frame Random Access Point (IRAP) or Dependent Random Access Point (DRAP) image, i.e. an image used as a timing reference, as detailed above.

[0170] Furthermore, the second predetermined image can be a TRAIL or DRAP image.

[0171] Furthermore, the second timing information 18 can be further limited by the second timing modification information 21 to the second CPB removal time t. rm (2) The second timing modification information pertains to how the first CPB removal time t, indicated by the first timing information, is modified relative to a set of one or more third image portions 31' of the video data stream. rm (1) The second CPB removal time t is obtained relative to a set of one or more third image portions 30'. rm (2) The third image portion is associated with a third predetermined image, follows portion 24' in the video data stream and includes the second image portion 28', and the second timing modification information 21 can be signaled in the second image portion 30' in the video data stream.

[0172] The second timing modification information 21 can be notified, for example, by a signal from cpb_removal_delay_offset.

[0173] The second image portion may also include a signal notification 32 indicating whether the portion has been removed, and a second timing modification information 21 may be signaled in the video data stream, regardless of whether the signal notification 32 indicates that the portion has been removed or indicates that the portion has not been removed.

[0174] One option is to extend a set of one or more third image portions 31' of the video data stream up to a fourth image portion associated with the labeled image. The labeled image can be used as the starting point for a timing reference.

[0175] Finally, the labeled image can be labeled using buffered periodic messages within the image portion associated with the labeled image in the video data stream.

[0176] It should be noted that, in the case shown for DRAP, the initial removal of CRA is 11 instead of 8 because the full bit stream is taken into account and therefore the CPB padding level is higher than in the original case. This could introduce potential problems and buffer overflows.

[0177] Another option would be to allow (in the case of DRAP) removal times from the CPB to be unequal for all frames. More specifically, for RAP and DRAP, the distances could be unequal, while for all other AUs, the distances would be equal.

[0178] In this case, the buffered periodic SEI message for RAP can have two timings. One timing is used for the entire bitstream or when RASL is removed, and one timing is used for the DRAP case. Since no RAP is output in the second case, the RAP removal time can make the final arrival time of RAP equal to its removal time. Then, the DRAP AU will have a second removal time, which will build the buffer to the desired level. In this case, the final arrival time of RAP is the earliest arrival time of DRAP in both the CBR and VBR cases. The described solution is shown in... Figure 9 middle.

[0179] An exemplary signal notification syntax is shown in Figure 10 as well as Figure 11 middle.

[0180] In one embodiment, the second timing information 18 may define the second CPB removal time t by means of third 33 and / or fourth 35 timing modification information regarding how to modify it, or by means of third 33 or fourth 35 timing replacement information. rm (2) .

[0181] The third timing modification / replacement information 33 pertains to how to modify or replace the first CPB removal time t indicated by the first timing information relative to the first predetermined image CRA and in the video data stream prior to portion 24”. rm (1) The second CPB removal time t is obtained relative to the first image portion 28'. rm (2) .

[0182] The third timing modification / replacement information 33 can be signaled, for example, by `drap_operation_initial_removal_delay`. It should also be noted that `cpb_drap_operation_removal_offset` relates to the arrival time, i.e., the time when a portion of the image, marked by the buffered SEI message, enters or is fed into the CPB. Feeding may fall under the responsibility of a means other than the means for managing the CPB described in this application. Adherence to the arrival time may depend on the intermediate network nodes involved in the transmission.

[0183] The fourth timing modification / replacement information 35 pertains to how the first CPB removal time t, as indicated by the first timing information, is modified or replaced relative to the set of one or more third image portions 31” of the video data stream after portion 24”, in relation to the third predetermined image. rm (1) The second CPB removal time t is obtained relative to a set of one or more third image portions 30". rm (2) .

[0184] The fourth timing modification / replacement information 35 can be notified by a signal, for example, cpb_drap_operation_removal_delay_offset.

[0185] Typically, the device 46 used to manage the CPB 48 video decoder can also perform the modifications or replacements detailed above.

[0186] Optionally, the third timing modification / replacement information 33 may be signaled in the video data stream within the first image portion 28' of the video data stream, and the fourth timing modification / replacement information 35 may be signaled in the video data stream within the second image portion 30' of the video data stream, the second image portion being preceding the encoding order 26 in a set of one or more third image portions.

[0187] Furthermore, the first image portion 28' may include a signal notification 36 indicating whether the distance between the second CPB removal times of the first and second image portions is allowed to deviate from the distance between the second CPB removal times of the consecutive second and third image portions, wherein conditionally when the signal notification 36 indicates that the distance between the second CPB removal times of the first and second image portions is allowed to deviate from the distance between the second CPB removal times of the consecutive second and third image portions, a third timing modification information 33 or a third timing replacement information 33 is signaled in the first image portion 28' of the video data stream.

[0188] The video encoder can check whether the second predetermined image is a DRAP of the reference first predetermined image (which is a RAP), set a signal notification 36 to indicate whether the second predetermined image is a DRAP of the reference first predetermined image (which is a RAP), and if the second predetermined image is a DRAP of the reference first predetermined image (which is a RAP), then encode a third timing modification / replacement information 33 in the first image portion 28' of the video data stream.

[0189] It should be noted that au_cpb_removal_delay_minus1 in the pic_timing SEI message typically increases progressively between consecutive image segments, i.e., at equal intervals. However, in cases where images encoded up to the first image segment 28' are discarded or not rendered or output, it is acceptable if the time interval of the rule is interrupted up to DRAP, and then becomes valid from there, because these images are the actual output images. In other words, in the example described, au_cpb_removal_delay_minus1 in the pic_timing SEI message increases progressively at equal intervals. This is the usual case, but not prohibited. The example shows the initial full-bitstream situation when nothing is removed. However, this is no longer the case when everything between RAP and DRAP is removed, but it is still acceptable because, in the case of using DRAP functionality, RAP is not output or displayed at all.

[0190] In addition, the second image portion 30” may include a signal notification 38 indicating whether the portion 24' has been removed.

[0191] The device 46 for managing the CPB 48 of the video decoder 44 according to the present invention can postpone the removal of the first image portion of the video data stream after the inspection signal notification 32; 38.

[0192] Alternatively, the network node 42 for forwarding video data streams according to the invention may be configured with signal notifications 32; 36 to indicate that the portion has been removed from the video data stream.

[0193] Network node 42 can also be removed from the video data stream, that is, part 24 of the video data stream is discarded.

[0194] like Figure 5As can be seen, optionally, the second timing information can limit the second CPB removal time, such that when the buffer removal of the second image portion is performed according to the first CPB removal time, after the first image portion of the video data stream preceding the portion in the buffered video data stream, the first CPB fill level 39a'; 39a" at the decoder side is equal to the second CPB fill level 39b'; 39b" at the decoder side. The first portion and the second image portion of the video data stream are after the portion in the video data stream, and the second CPB fill level is displayed at the decoder side after the first and second image portions of the buffered video data stream when the buffer removal of the second image portion is performed according to the second CPB buffer time without buffering the portion.

[0195] The video encoder can set first and second timing information to make the video data stream conform to the above content.

[0196] More generally, the second timing information can define the second CPB removal time for portions associated with one or more RASL image sequences, or for portions associated with an image sequence preceding a certain DRAP.

[0197] Furthermore, the video data stream may include signal notifications 32, 35 indicating whether the portion has been removed.

[0198] The DRAP offsets used for random access will be used to calculate the removal time of subsequent AUs. Clearly, instead of indicating the offsets in the image temporal SEI message, it is possible to specify that the DRAP includes buffered periodic SEI messages and indicate these offsets in the buffered periodic SEI message.

[0199] The embodiments described subsequently address the issue of temporal scaling and how to indicate information about image buffer timing and buffer periodic SEI messages in this coding environment. Therefore, embodiments are described that utilize alternative information in the temporal scaling reference image for both image temporal SEI messages and buffer periodic SEI messages. First, embodiments utilizing alternative information in the temporal scaling reference image for both image temporal SEI messages and buffer periodic SEI messages are presented, specifically embodiments using temporal offsets in the image temporal SEI messages.

[0200] Figures 12 to 14 Examples of removal time values ​​(i.e., time scaling) are shown when different frame rates are provided under the same bitstream conditions.

[0201] Figure 17The table shows that the deltaTime for "blue" frames is 0, 1 / 60, 3 / 120, 2 / 60, and 6 / 120. The "red" frames are 3 / 120 and 6 / 120. Blue frames are indicated by these rows in the table, including values ​​of 30, 60, and 120 fps. Red frames are indicated by these rows in the table, which only include values ​​of 60 and 120 fps, excluding the 30 fps value.

[0202] for Figure 16 The table shows the open GOPs with the highest frame rates. The following content applies, and also... Figure 15 As shown in [the image]. Figure 15 The table shows that the deltaTime for "blue" frames is 0, 1 / 60, and 3 / 120. The deltaTime for "red" frames is 1 / 120 and 2 / 120. Similarly, the blue frames are indicated by these rows in the table, including values ​​of 30, 60, and 120 fps. The red frames are referenced by these rows, which only include values ​​of 60 and 120 fps, excluding the 30 fps value.

[0203] Overall, due to the GOP structure of the encoded bitstream, there are some repeating patterns in deltaTime at different frame rates for the removal time.

[0204] Another aspect to consider is scene segmentation, which occurs because the described pattern is interrupted. Different locations of scene segmentation will produce different values. For simplicity, Figure 18 Only one location is shown.

[0205] As from Figure 19 As can be seen from the table, it is clear that the GOPs affected by scene segmentation do not follow the described pattern.

[0206] Figure 20 An exemplary signal notification structure (i.e., target TemporalId) is shown, which references different values ​​of image timing SEI messages with different frame rates.

[0207] exist Figure 14 In the embodiment shown, the video data stream is encoded in a time-scalable manner using inter-temporal prediction in units of image groups 50, such that the video's images are divided into 12 groups of different hierarchical levels 50. i The images are provided, where for each layer level, all images of that layer level depend only on the images of that layer or the layers below it. In the accompanying figures, only the beginning portion of one GOP is shown.

[0208] Furthermore, part 24 consists of images belonging to the highest level 503, or images belonging to the highest level 503 and one or more levels 502 other than the lowest level 501 that are immediately below the highest level 503 in the hierarchical order 52.

[0209] Next, the first timing information 16 signals the first CPB removal time t image by image and by means of the increment of the preceding marked image 53 relative to the current image group in the decoding order. rm (1) This order is indicated by the sequential order of the score values ​​used to label the image.

[0210] Similarly, increments can be signaled, for example, by AuCPBRemovalDelay or au_cpb_removal_delay_minus1. The current GOP is the image relative to the preceding image of the GOP to which each image belongs.

[0211] In addition, labeled images can be labeled using buffered periodic messages within the image portion associated with the labeled image in the video data stream.

[0212] In one embodiment, the second timing information 18 may define the second CPB removal time t by means of the fifth timing modification information 19 or the fifth timing replacement information 19 used for replacement. rm (2) The information is communicated in each of the fifth image portions 23 of the video data stream using signals, the fifth image portion 23 being associated with the fifth image 54, which is associated with one or more hierarchical levels other than the highest hierarchical level.

[0213] The fifth timing modification information 19 can be signaled, for example, by au_cpb_removal_pattern_offset, and the fifth timing replacement information 19 can be signaled, for example, by au_cpb_removal_delay_minus1[i], where i is not equal to the current layer level to which the image sending pic_timing belongs.

[0214] Fifth timing modification / replacement information 19 concerns how to modify or replace the first CPB removal time t indicated by the first timing information relative to the corresponding fifth image portion 23. rm (1) The second CPB removal time t is obtained relative to the fifth image portion 23. rm (2) .

[0215] The first timing information can be signaled, for example, by au_cpb_removal_delay_minus1[i], where i is equal to the current layer level to which the image sending pic_timing belongs, or au_cpb_removal_delay_minus1.

[0216] In section 24, where the highest layer level extends downwards to the corresponding layer level, the fifth timing modification / replacement information 19 is signaled for each layer level above the corresponding layer level between the highest and lowest layer levels (i.e., 501 and 502 in the illustrated example), and for each layer level above the corresponding layer level between the highest and lowest layer levels (i.e., 503 for 502 and 503 and 502 for 501 in the illustrated example). As an example, the number is indicated by num_sub_layer_cpb_removal_delays_minus1 when buffering_period_pattern_flag is one, and by num_sub_layer_cpb_removal_delays_minus1+1 when buffering_period_pattern_flag is zero.

[0217] Figure 21 as well as Figure 22 An exemplary signal notification structure is shown, wherein alternatively, patterns may be defined within each buffer cycle SEI and these patterns may be indicated in the image timing SEI message. When, as described above, such as for scene slicing, patterns may not be applied, and the value may be explicitly notified by a signal (or in the form of some offset).

[0218] According to this embodiment, the second timing information 18 limits the second CPB removal time t by means of the fifth timing modification information 19, such as au_cpb_removal_pattern_offset. rm (2) The fifth timing modification information 19 indicates how the first CPB removal time should be modified using offset values, and the video data stream includes data 60, which indicates a list of representative offset values ​​taken by the offset values ​​indicated by the fifth modification information for modifying the first CPB removal time in one or more picture groups. Then, the fifth timing modification information 19 signals the offset values ​​by means of pointers pointing to the list of representative offset values.

[0219] More generally, in the above embodiment, the video data stream is reduced by portion 24, resulting in reduced version 20, and the video data stream includes first and second timing information 18. In other words, in the event that certain hierarchical levels are dropped at an intermediate network device, the second timing information is not nested so as not to be nested and replace the corresponding first timing information. In practice, the first and second timing information remain unchanged and, on the decoder side, the correct timing information is selected for CPB processing based on signals transmitted in the video data stream, and is configured to explain to intermediate network devices which portion has been removed, or more precisely, which previous highest hierarchical level has been removed / dropped.

[0220] Another example of the embodiment for alternative information in the temporally scaled reference image in the image temporal SEI message and the buffered periodic SEI message is the increment at the buffered periodic SEI message.

[0221] like Figure 23 As can be seen, the different values ​​of vcl_initial_cpb_removal_delay[i] and vcl_initial_cpb_removal_offset[i] should also be explicitly indicated in the buffer period SEI message, because a single value may not be sufficient for all possible frame rates.

[0222] Another embodiment involves multiple bit rate considerations. As discussed above, the current approach to supporting different transmission rates or CPB feed rates is to indicate several values ​​for the bit rate, CPB size, and the corresponding initial removal time and offset.

[0223] The problem with this current solution is that all potential CPB feed bit rates need to be known in advance, and the rate control at the encoder needs to take all bit rates into account to ensure that the information provided is valid.

[0224] While this allows for a degree of flexibility, where the mentioned parameters can be varied in many ways, in some practical situations it may be desirable to consider many potential CPB feed rates from the transmitter side, but this would be difficult to implement at the encoder side. In these cases, a “very” restrictive rate control that involves some parameterization of the HRD model may be useful, which means considering two extreme values—the maximum bit rate (maxBitrate) and the minimum bit rate (minBitrate)—and linear interpolation of anything in between. This solution will be more restrictive in terms of HRD modeling than the flexible solution, but will allow for any rate between the maximum and minimum bit rates at the transmitter side, and will still ensure that there is no underflow or overflow of CPBsize or initial removal time that can be calculated based on the maximum and minimum signal notification values.

[0225] hrd_parameters is divided into 3 parts.

[0226] ·cbr_parameters_present_flag

[0227] ·vbr_min_parameter_present_flag

[0228] ·vbr_max_parameter_present_flag

[0229] Any other bit rate between vbr_min_parameter_present_flag and vbr_max_parameter_present_flag can be calculated. An example signal notification is shown in... Figure 24a , Figure 29 as well as Figure 30 middle.

[0230] Similarly, if CBR, minimum, and maximum values ​​also exist in hrd_parameters, then the buffered periodic SEI message will include those values, such as... Figure 24b as well as Figure 31 As shown in the image.

[0231] Based on this, HRD time-series information data, such as first HRD time-series information data, second HRD time-series information data, or any one of multiple HRD time-series information data, and each HRD time-series information data includes one or more of the following:

[0232] -Indication 104 of the CPB feed bit rate related to the corresponding HRD timing information.

[0233] - An indication 106 of the CPB size required for the corresponding CPB feed bit rate related to the HRD timing information.

[0234] - An indication 108 of the CPB removal time for each of the marked image portions of the video data stream, which measures the elapsed time from the arrival of the CPB at the corresponding CPB feed bit rate associated with the corresponding HRD timing information from the first bit of the corresponding marked image portion—here the marked image portions are image portions with buffer periods SEI such as RAP, which are shown in crosshairs in other related figures, but there are also other images, such as image portions simply shown in shading in other related figures.

[0235] - An indication 110 of the CPB feed delay time for each of the marked image portions of the video data stream, the measurement of which is the elapsed time of the delay of the arrival of the first CPB of the corresponding marked image portion relative to the respective CPB feed bit rate associated with the respective HRD timing information.

[0236] If a minimum and a maximum exist, then any value in between will result in linear interpolation of, for example, the minimum and the maximum.

[0237] Furthermore, in some cases where greater flexibility is required, it may be desirable not to limit the HRD model to a linear interpolation model (e.g., non-linearly reducing initial_removal_delay and increasing the bit rate). Therefore, depending on the scenario, the encoder can choose which mode to use: a mode with linear interpolation or a mode with multiple values. Exemplary signal notifications are shown in... Figure 32 , Figure 33 as well as Figure 34 middle.

[0238] As shown in Figure 24, Figure 29 , Figure 31 , Figure 32 as well as Figure 34 As shown, the video stream may include first HRD timing information data 70a, 70b related to the minimum CPB feed bit rate, and second HRD timing information data 72a, 72b related to the maximum CPB feed bit rate. The first HRD timing information data 70a, 70b and the second HRD timing information data 72a, 72b then allow a third HRD timing information to be determined for the actual CPB feed bit rate by linear interpolation between the first and second HRD timing information data. The actual CPB feed bit rate is, for example, the bit rate at which the video data stream is actually fed into the CPB.

[0239] The video data stream may further include, for example, a flag 74 of hrd_interpolation-mode_flag, which indicates the presence of first HRD timing information data 70a, 70b and second HRD timing information data 72a, 72b, or multiple HRD timing information data 76 associated with each of the set of desired CPB feed bit rates.

[0240] The video encoder may then decide to provide the video data stream with first HRD timing information 70 and second HRD timing information 72, or to provide the video data stream with multiple HRD timing information data 76 associated with each of the set of desired CPB feed bit rates.

[0241] For all CPB feed bit rates between the minimum and maximum CPB feed bit rates, the video encoder may optionally perform the decision based on a feasibility check of whether linear interpolation between the first and second HRD timing information results in no CPB overflow and underflow.

[0242] The device 46 for managing the CPB 48 of the video decoder 44 can then use the third HRD timing information to manage the CPB.

[0243] Similarly, the device can examine flags 74 in the video data stream, such as hrd_interpolation-mode_flag, and depending on the flags, perform the extraction of first HRD timing information data 70a, 70b and second HRD timing information data 72a, 72b from the video data stream, or extract multiple HRD timing information data 76 associated with each of the set of desired CPB feed bit rates from the video data stream.

[0244] Figure 38 An example of three operation points is shown.

[0245] For interpolation, the following linear interpolation formula is used:

[0246] -BR interpolated =BR min +(BR max -BR min )*alpha

[0247] -IND interpolated =IND min +(IND max -IND min )*alpha(where IND = InitialRemovalDelay)

[0248] It should be noted that IND min (For minimum bit rate) greater than IND max (For the highest bit rate)

[0249] Figure 37 Involving IND min The following abbreviations are used:

[0250] RmD = Remove Delay

[0251] Afinal = Final Arrival Time

[0252] ArrivalEar = the first digit of the image, possibly the earliest arrival time due to the frame rate.

[0253] ArrivalInit = Actual arrival time of the first image element.

[0254] It can be seen that the maximum CPB size is 15.

[0255] Figure 36 Involving IND max Furthermore, the required CPB size for this second case is 12,1125.

[0256] Given the calculated initial removal delay and a bit rate linearly having α of 0.3, Figure 35 The required CPB size for this case is 13,87912. If linear interpolation is performed, the calculated CPB size will correspond to 12,97875. It can be seen that the values ​​are different.

[0257] Therefore, if the encoder wishes to comply with the constraint of linearly deriving the HRD parameters involved with full flexibility within the maximum and minimum values, the rate control at the encoder should take this constraint into account when determining the image size, so as to satisfy the CPB size limit during interpolation.

[0258] Since this may not always be the case, one option would be to signal only if the encoder notices that interpolation is possible, and if not, to signal any actual values ​​that have been used at discrete numbers of operating points.

[0259] As previously stated, the concepts described above can be employed by video streams, video encoders, video decoders, network nodes, means for managing the encoded image buffer (CPB) of the video decoder, and means for splicing video data streams together.

[0260] Similarly, the aforementioned concepts can be implemented by corresponding methods according to embodiments of the present invention. These methods are based on considerations identical to those described above regarding video streams, video encoders, video decoders, network nodes, means for managing the encoded image buffer (CPB) of the video decoder, and / or means for splicing video data streams together. However, it should be noted that the methods can be supplemented by any of the features, functionality, and details described herein that also relate to video streams, video encoders, video decoders, network nodes, means for managing the encoded image buffer (CPB) of the video decoder, and / or means for splicing video data streams together. Furthermore, the methods can be supplemented individually and in combination by the features, functionality, and details of video streams, video encoders, video decoders, network nodes, means for managing the encoded image buffer (CPB) of the video decoder, and / or means for splicing video data streams together.

[0261] Finally, this concept can also be used to generate encoded data streams according to embodiments of the present invention. The data streams can also be supplemented individually and in combination by the features, functionality, and details of the method, video encoder, video decoder, network node, means for managing the encoded image buffer (CPB) of the video decoder, and / or means for splicing video data streams together.

[0262] Finally, the embodiments described herein may optionally be supplemented by any of the key points or aspects described herein. However, it should be noted that the key points and aspects described herein may be used individually or in combination, and may be incorporated individually and in combination into any of the embodiments described herein.

[0263] Although some aspects have been described in the context of the apparatus, it is apparent that these aspects also represent a description of the corresponding method, wherein the apparatus or a portion thereof corresponds to a method step or a feature of the method step. Similarly, aspects described in the context of the method steps also represent a description of a corresponding apparatus or a portion thereof, or an object or feature of the corresponding apparatus. Some or all of the method steps may be performed by (or using) hardware devices such as (e.g.) a microprocessor, a programmable computer, or electronic circuitry. In some embodiments, one or more of the most important method steps may be performed by this apparatus.

[0264] Depending on certain implementation requirements, embodiments of the present invention can be implemented in hardware or software. Implementation can be performed using a digital storage medium, such as a floppy disk, DVD, Blu-ray, CD, ROM, PROM, EPROM, EEPROM, or flash memory, on which electronically readable control signals are stored, which cooperate (or are capable of cooperating with) a programmable computer system to cause various methods to be executed. Therefore, the digital storage medium can be computer-readable.

[0265] Some embodiments of the invention include a data carrier having electronically readable control signals, which is capable of cooperating with a programmable computer system to perform one of the methods described herein.

[0266] Typically, embodiments of the present invention can be implemented as a computer program product having program code that, when run on a computer, is operatively used to perform one of the methods. The program code may, for example, be stored on a machine-readable medium.

[0267] Other embodiments include a computer program stored on a machine-readable medium for performing one of the methods described herein.

[0268] In other words, therefore, embodiments of the inventive method are computer programs having program code for executing one of the methods described herein when a computer program is running on a computer.

[0269] Therefore, a further embodiment of the method of the present invention includes a data carrier (or digital storage medium, or computer-readable medium) on which a computer program for performing one of the methods described herein is recorded. The data carrier, digital storage medium, or recording medium is typically tangible and / or non-transient.

[0270] Therefore, a further embodiment of the method of the present invention represents a data stream or signal sequence for performing one of the methods described herein. The data stream or signal sequence may, for example, be configured to be transmitted via a data communication connection (e.g., via the Internet).

[0271] Further embodiments include processing components, such as a computer or programmable logic device configured or adapted to perform one of the methods described herein.

[0272] Further embodiments include a computer on which a computer program for performing one of the methods described herein is installed.

[0273] Further embodiments of the invention include an apparatus or system configured to transmit (e.g., electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver may be, for example, a computer, a mobile device, a memory device, etc. The apparatus or system may include, for example, a file server for transmitting the computer program to the receiver.

[0274] In some embodiments, a programmable logic device (e.g., a field-programmable gate array) may be used to perform some or all of the functionalities of the methods described herein. In some embodiments, the field-programmable gate array may cooperate with a microprocessor to perform one of the methods described herein. Generally, the methods are preferably performed by any hardware device.

[0275] The apparatus described herein can be implemented using hardware devices, or using a computer, or a combination of hardware devices and a computer.

[0276] The apparatus described herein, or any element thereof, may be implemented, at least in part, in hardware and / or software.

[0277] The methods described in this article can be performed using hardware devices, computers, or a combination of hardware devices and computers.

[0278] The methods described herein, or any part thereof, may be implemented at least in part by hardware and / or software.

[0279] The embodiments described above merely illustrate the principles of the invention. It should be understood that modifications and variations to the configurations and details described herein will be apparent to those skilled in the art. Therefore, it is intended to be limited only by the scope of the following claims, and not by the specific details presented by way of the description and explanation of the embodiments herein.

Claims

1. An apparatus for managing an encoded image buffer (CPB) of a video decoder, the video decoder decoding a video data stream buffered by the CPB, the apparatus comprising: The inspection unit is configured to determine that the cascading flag in the buffer period BP auxiliary enhancement information SEI message of the image portion of the video data stream is set; as well as The determining unit is configured to, in response to the setting of the cascading flag, determine a removal time for removing the image portion from the CPB, the removal time being determined based on the maximum of a first removal time and a second removal time. The first removal time is the sum of the removal time of the previously undiscardable image portion and the time elapsed since the removal of the previously undiscardable image portion, and The second removal time is the removal time of the last image portion before concatenation, the initial CPB removal delay of the image portion, and the sum of the time difference representing the final arrival time of the last image portion before concatenation minus the removal time of the last image portion.

2. The apparatus of claim 1, wherein the image portion, the previously non-discardable image portion, and the last image portion are access units.

3. The apparatus of claim 1, wherein the BP SEI message indicates the time elapsed since the removal of the previously non-discardable image portion.

4. A method for managing an encoded image buffer (CPB) of a video decoder used to decode a video data stream, the method comprising: The cascading flag in the BP-assisted enhancement information SEI message, which determines the buffering period of the image portion of the video data stream, is set. as well as In response to the cascading flag being set, a removal time for the image portion to be removed from the CPB is determined, the removal time being based on the maximum of a first removal time and a second removal time. The first removal time is the sum of the removal time of the previously undiscardable image portion and the time elapsed since the removal of the previously undiscardable image portion, and The second removal time is the removal time of the last image portion before concatenation, the initial CPB removal delay of the image portion, and the sum of the time difference representing the final arrival time of the last image portion before concatenation minus the removal time of the last image portion.

5. The method of claim 4, wherein the image portion, the previously non-discardable image portion, and the last image portion are access units.

6. The method of claim 4, wherein the BP SEI message indicates the time elapsed since the removal of the previously non-discardable image portion.

7. A non-transitory computer-readable medium comprising instructions that, when executed by at least one processor, cause the at least one processor to manage an encoded image buffer (CPB) of a video decoder by performing the following steps: The buffering period of the image portion of the video data stream is determined by the cascading flag in the SEI message, which is set; and In response to determining that the cascading flag is set, a removal time for removing the image portion from the CPB is determined, the removal time for the image portion being determined based on the maximum of a first removal time and a second removal time. The first removal time is the sum of the removal time of the previously undiscardable image portion and the time elapsed since the removal of the previously undiscardable image portion, and The second removal time is the removal time of the last image portion before concatenation, the initial CPB removal delay of the image portion, and the sum of the time difference representing the final arrival time of the last image portion before concatenation minus the removal time of the last image portion.

8. The non-transitory computer-readable medium of claim 7, wherein the image portion, the previously non-discardable image portion, and the last image portion are access units.

9. The non-transitory computer-readable medium of claim 7, wherein the BP SEI message indicates the elapsed time since the removal of the previously non-discardable image portion.

10. A video encoder for encoding video into a video data stream, such that the video data stream includes a sequence of image portions, each image portion containing an image encoded as video, wherein the video encoder comprises: The first indication unit is configured to indicate the maximum value of the initial removal delay in the first buffer period BP auxiliary enhancement information SEI message; as well as The second indication unit is configured to indicate a first flag in an image timing SEI message corresponding to the first image portion of the video data stream, wherein the first flag equal to 1 indicates that when the first image portion is followed by a second image portion having a second BP SEI message with its concatenated flag equal to 1 and the initial removal delay is less than the maximum value of the initial removal delay, a nominal encoded image buffer CPB removal time based on the time elapsed since the removal of the previously non-discardable image portion is applied.

11. The video encoder of claim 10, wherein the second BP SEI message indicates the time elapsed since the removal of the previously non-discardable image portion.

12. The video encoder of claim 10, wherein the first image portion, the second image portion, and the previously non-discardable image portion are access units.

13. A method for encoding video into a video data stream, the method causing the video data stream to include a sequence of image portions, each image portion containing an image encoded as video, the method comprising: The maximum value of the initial removal delay is indicated in the BP auxiliary enhancement information SEI message during the first buffer cycle; as well as A first flag is indicated in the image timing SEI message corresponding to the first image portion of the video data stream, wherein the first flag being equal to 1 indicates that when the first image portion is followed by a second image portion having a second BP SEI message with its cascading flag equal to 1 and the initial removal delay is less than the maximum value of the initial removal delay, then the nominal encoded image buffer CPB removal time based on the time elapsed since the removal of the previously non-discardable image portion is applied.

14. The method of claim 13, wherein the second BP SEI message indicates the time elapsed since the removal of the previously non-discardable image portion.

15. The method of claim 13, wherein the first image portion, the second image portion, and the previously non-discardable image portion are access units.

16. A non-transitory computer-readable medium comprising instructions that, when executed by at least one processor, cause the at least one processor to encode video into a video data stream by performing the following steps, such that the video data stream comprises a sequence of image portions, each image portion containing an image encoded with video: The maximum value of the initial removal delay is indicated in the BP auxiliary enhancement information SEI message of the first buffer cycle; and A first flag is indicated in the image timing SEI message corresponding to the first image portion of the video data stream, wherein the first flag being equal to 1 indicates that when the first image portion is followed by a second image portion having a second BP SEI message with its cascading flag equal to 1 and the initial removal delay is less than the maximum value of the initial removal delay, then the nominal encoded image buffer CPB removal time based on the time elapsed since the removal of the previously non-discardable image portion is applied.

17. The non-transitory computer-readable medium of claim 16, wherein the second BP SEI message indicates the time elapsed since the removal of the previously non-discardable image portion.

18. The non-transitory computer-readable medium of claim 16, wherein the first image portion, the second image portion, and the previously non-discardable image portion are access units.

Images