Network device and error handling method

Abstract

To provide a network device with reduced packets, and a method for error handling.SOLUTION: In order to ease transmission of a data stream 20, an encoder 10 encodes a video 12 into the data stream 20. Slices 26a, 26b, which are portions of the data stream 20, are restricted to either completely lying within one tile 22, i.e., to not cross any tile boundary, or to be composed of two or more tiles in tile order completely. In encoding the video 12 in units of the slices 26a and 26b, the encoder 10 uses entropy coding and in particular context-adaptive entropy coding with continuous adaptation of the contexts' entropy probabilities so as to adapt the probabilities used for entropy encoding to the actual symbol statistics and picture content, respectively. The contexts' probabilities are reset at the beginning of each slice.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a network device and a method of error handling associated with a transport stream, a series of packets in which a video data stream is transmitted.

[0002] In some applications, transport packet-based video data stream transmissions suffer from packet loss, which can result, for example, from transmission errors exceeding the error correction capabilities of any used forward error correction of the transport stream, the lack of any uplink connection for sending acknowledgements of received signals, or a combination of both. Regardless of the validity of the receipt uplink acknowledgment, it is desirable to have as little affected part of the video data stream as possible that cannot be decoded due to non-reception of lost packets. [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] B. Bross, W.-J. Han, J.-R. Ohm, GJ Sullivan, T. Wiegand (Eds.), “High Efficiency Video Coding (HEVC) text specification draft 10“, JCTVC-L1003, Geneva, CH, Jan. 2013 [Non-patent document 2] G. Tech, K. Wegner, Y. Chen, M. Hannuksela, J.Boyce (Eds.), “MV-HEVC Draft Text 3 (ISO / IEC 23008-2 PDAM2)“, JCT3V-C1004, Geneva, CH, J an. 2013 [Non-patent document 3] G. Tech, K. Wegner, Y. Chen, S. Yea (Eds.), “3D-HEVC Test Model Description, draft specification“, JCT3V-C1005, Geneva, CH, Jan. 2013 [Non-patent document 4] WILBURN, Bennett, et al. High performance imaging using large camera arrays. ACM Transactions on Graphics, 2005, 24. Jg., Nr. 3, S. 765-776. [Non-Patent Document 5] WILBURN, Bennett S., et al. Light field video camera. In: Electronic Imaging 2002. International Society for Optics and Photonics, 2001. S. 29-36. [Non-patent document 6] HORIMAI, Hideyoshi, et al. Full-color 3D display system with360 degree horizontal viewing angle. In: Proc. Int. Symposium of 3D and Contents. 2010. S. 7-10. Summary of the Invention [Problem to be solved by the invention]

[0004] However, packets of the transport stream may provide the necessary information for decoding the content carried by subsequent packets of the transport stream. In the HEVC standard (High Efficiency Video Coding), for example, a video data stream includes independent slice portions and dependent slice portions, where a dependent slice portion depends on the independent slice portion insofar as, for example, slice header data contained in the immediately preceding independent slice portion and inherited for decoding the dependent slice portion is concerned.

[0005] It would therefore be advantageous to have an immediate concept that can reduce the amount of affected undecodable portions of a video data stream in the presence of packet loss.

[0006] It is therefore an object of the present application to provide a concept of this kind for dealing with errors occurring in a transport stream of a series of packets in which a video data stream is transported, i.e., a concept that can affect, as little as possible, packets after the lost packet that are received but cannot be decoded. [Means for solving the problem]

[0007] This object is achieved by the subject matter of the attached independent claims. The finding of the present application is to provide error resilience in a series of packets, analyze the number of packets (albeit correctly received) due to packet loss, and for each run of one or more lost packets in the series of packets, identify the first packet in the series of packets after each run of one or more lost packets that carries the start of any tile of the video data stream and simultaneously carries a slice, the slice header of which is included in any packet in the series and is not lost. In particular, the side information overhead for transmitting the error resilience data is relatively low compared to the loss of packets adversely affected by packet loss.

[0008] Advantageous embodiments are the subject matter of the dependent claims and preferred embodiments of the present application relate to any of the drawings described below. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 shows a schematic diagram of an encoder by which video is encoded to generate a video data stream, and embodiments of the present application may be supported by the encoder of FIG. [Figure 2A] FIG. 2A is a schematic diagram showing a decoder, the video reconstructed thereby based on a video data stream, and the transport of the video data stream via a series of packets, and embodiments of the present application can be applied to the decoder of FIG. 2. [Figure 2B] FIG. 2B is a schematic diagram showing a decoder, the video reconstructed thereby based on the video data stream, and the transport of the video data stream via a series of packets, and embodiments of the present application can be applied to the decoder of FIG. 2. [Figure 3] FIG. 3 shows diagrammatically an image 14 partitioned into tiles and slices according to the first option. [Figure 4] FIG. 4 shows a schematic diagram of an image 14 using another segmentation option. [Figure 5] FIG. 5 shows an example of two packet streams over a lossy channel to illustrate the problem addressed by embodiments of the present application. [Figure 6] FIG. 6 shows a schematic block diagram of a network device according to an embodiment, of which the network device may be part or may be connected in front of the decoder of FIG. [Figure 7] FIG. 7 illustrates in more detail, using the structure of a schematic flow diagram, a possible mode of operation of the error handler of FIG. DETAILED DESCRIPTION OF THE INVENTION

[0010] The following description of embodiments of the present application begins with a description of a typical video codec or a typical encoder / decoder structure. Subsequently, problems resulting from packet loss are described. Subsequently, embodiments of the present application are described. These embodiments are particularly applicable with respect to the encoder / decoder structure described above.

[0011] 1 shows an encoder 10 configured to encode a video 12 consisting of a sequence of images 14 arriving at the input of the encoder 10 into a data stream at the output of the encoder 10. The encoder 10 may be configured to encode the sequence of images 14 using a coding order that may, but need not necessarily, follow the temporal order 16 of the images 14. More precisely, the encoder 10 may be a hybrid video encoder configured to select different available prediction modes for the blocks 18 into which the images 14 are divided. Such prediction modes may include, for example, spatial prediction from a previously coded portion of the same picture and temporal prediction from a previously coded portion of a previously coded picture. However, additionally or alternatively, other prediction modes may be supported by encoder 10 as well, such as inter-layer prediction modes from previously coded layers of lower quality, or inter-view prediction from previously coded views showing the same scene in time to align pictures 14 of video 12. Encoder 10 signals the selected prediction mode and prediction parameters associated with the selected prediction mode along with encoding the residual prediction in data stream 20 at its output. For example, spatial prediction can include an extrapolation direction in which already coded samples are copied to the current block 18, indicating the direction / extrapolation along the neighborhood, and temporal prediction modes can be implemented such as prediction compensation with motion vectors as motion prediction parameters. Inter-view prediction modes can also be implemented in a motion compensated manner, which results in disparity vectors as prediction parameters. In performing prediction, the "previously coded portion" of video 12 is defined by the coding order described above, which sequentially passes through pictures 14. Within each picture 14, the coding order also passes through blocks 18 in a predetermined order, which, for example, in a raster scan manner, leads from the top left corner of picture 14 towards its bottom right corner.

[0012] Video 12 Parallel encoding and decoding of image 14, and / or selective / partial decoding of images 14 of video 12 of To make this possible, the encoder 10 of FIG. 1 supports so-called tile division. If , each image 14 may be, for example, ,Ta Il 22 array It is divided into In Figure 1, one image 14 is illustratively divided into an array of 2x2 tiles 22, It is also possible to use any m×n division ( If tiling is enabled, m+n>1. ). Tile 22 Split into Block 18 Overcoming So that That is, to match the block boundary, Limited There is something The tiles are, for example, p×q blocks of 18. array in There may be, As a result, the tiles There are q equal tiles in a row and p equal tiles in a column. do.

[0013] The encoder 10 signals the tile division of the image 14 in the data stream 20 and specifically encodes each tile 22, i.e., such that each tile 22 is individually decodable from the data stream 20, e.g., as predicted and entropy decoded, such that interdependencies arising from, e.g., spatial prediction, context selection of the entropy-encoded data stream 20 are constrained at the tile boundaries so as not to intersect with the latter. The encoding described above is suitable for tile dividing. Within each image 14 adapted for tile dividing, the encoding order first passes through the image 14 within the first one of the tiles 22, then traverses the next tile in the tile order. The tile order may be a raster scan order leading from the top left tile to the bottom right tile of the image 14.

[0014] For ease of explanation, FIG. 1 shows the encoding sequence for one exemplary image 14 , designated by reference numeral 24 .

[0015] To facilitate transmission of the data stream 20, the encoder 10 The video 12 is converted into a data stream 20 in so-called slices in the manner described above. Encode the slices as follows: Portions of the data stream 20 that follow the coding order described above. Slices are It is restricted to either being entirely within one tile 22, i.e. not crossing any tile boundaries, or being entirely comprised of two or more tiles in tile order, i.e. covering two or more tiles in total and matching the contours of the tiles it covers at the slice boundaries.

[0016] FIG. 1 shows the image 14 of FIG. 1 divided into two slices 26a, 26b, with the first slice 26a in the coding order 24 consisting of the first two tiles 22 in tile order, and the second slice 26b covering the bottom half of the image 14, i.e. the third and fourth tiles 22 in tile order. When encoding video 12 in units of slices 26a and 26b, encoder 10 uses entropy coding, in particular context-adaptive entropy coding, with continuous adaptation of context entropy probabilities, to adapt the probabilities used for entropy coding to the actual symbol statistics and picture content, respectively, which are reset at the start of each slice 26a and 26b or, within each slice, reset or initialized at each tile boundary.

[0017] 1 exemplarily illustrates a slice 26 of data stream 20. The slice contains data for the first two tiles 22 of image 14. Additionally, slice 26 includes a slice header 30 that indicates some high-level information about the corresponding portion of image 14 and the coding type selected to code slice 26, i.e., whether the first two tiles 22, e.g., tile 26, relate to an intra-coded portion, a p-type coded portion, or a b-type coded portion. Without the information in slice header 30, the tiles of slice 26a would not be correctly decodable.

[0018] Another mechanism is to further subdivide the slices so that transmission of the coded data stream 20 can be further subdivided. According to this principle, each slice 26a and 26b constitutes exactly one independent slice segment, in fact, the case with slice 26a, or a series of independent slice portions followed by dependent slice portions. Slice 26a cannot be further divided. Encoder 10 can thus simply output slice 26a in its entirety. Things are different for slice 26b. Slice 26b is composed of an independent slice portion 28a that follows, in coding order, a dependent slice portion 28b, with the tile boundary of tile 22 in slice 26b coinciding with the boundary between slice portions 28a and 28b. Slice segments 28a and 28b therefore have similar properties, as do slices, i.e., they are independently decodable except for the slice header. Dependent slice portion 28b inherits the same attributed slice header 30 from the independent slice portion 28a that precedes, i.e., leads, slice 26b.

[0019] Before discussing issues arising from possible packet loss during transmission, a decoder 50 compatible with the encoder 10 of FIG. 1 will be described with reference to FIG. 2, which thus illustrates an embodiment for a network device for processing a data stream. The decoder 50 receives the data stream 20 and reconstructs the video 14 therefrom. The decoder 50 receives, for example, slice 26a followed by slice 26b. For example, decoder 50 may be of a hybrid video decoding type, i.e., a hybrid video decoder, that uses the identified prediction nodes to reconstruct portions of image 14 of video 12, corresponding to slices 26a and 26b. In decoding slice 26a, for example, decoder 50 uses slice header 30 to determine the slice type of slice 26a and reconstruct first and second tiles 22 of image 14 from slice 26a in a manner dependent on the slice type. For example, for an I slice, the availability of P and B slices is provided, whereas temporal prediction modes are not available. Therefore, analysis of payload data for slice 26a can rely on slice header 30. In particular, decoder 50 can entropy encode slice 26a in the context-adaptive manner outlined above, for example, by initializing context probabilities at the beginning of slice 26a, then using the prediction mode and prediction parameters signaled in slice 26a to predict first and second tiles 22 within slice 26a, and combining the resulting prediction signal with a prediction residual contained within the payload data of slice 26a. In decoding tiles 22, decoder 50 follows the encoding order outlined above. However, decoder 50 may perform some of the decoding tasks in parallel as far as tiles 22 are concerned. This is true for predictions, for example, so that they are configured not to cross tile boundaries, thus avoiding interdependencies between decoding of tiles of the same image 14, and as far as entropy decoding is concerned, entropy decoding can be performed in parallel.

[0020] In decoding slice portion 26b, decoder 50 is able to decode this slice 26b independently from slice 26a. In particular, the independent slice portion 28a providing data for the third tile 22 of the image 14 includes a slice header alone, allowing the decoder 50 to reconstruct this third tile without needing any other data. However, as far as dependent slice portion 28b is concerned, decoder 50 inherits the slice header data from the slice header 30 contained in the independent slice portion that immediately precedes independent slice portion 28a of the same slice 26b, and therefore decoding the fourth tile requires knowledge of the slice header of slice portion 28a in addition to the existence of slice portion 28b.

[0021] As far as the transmission of the data stream 20 is concerned, it is in the form of slice segments 26a, 28a and 28b or framed in Network Abstraction Layer (NAL) units. In the following description, slice parts and slice part NAL units are not specifically identified, because slice parts are almost the same as the NAL units that carry them. The headers of small NAL units simply contain a NAL unit type that indicates the content of the NAL unit, being a slice part.

[0022] However, it should also be noted that during transmission, slice portions may be more fragmented to fit into the payload portion of the transformed packet. Preferably, this is done in a way that the beginning or start of a new tile within a particular slice 26a is inserted into a new transport packet. As will be further described below, this same means that slice header data is available and can be decoded even if, for example, a previous packet is lost, for dependent slice portion 28b, whose beginning indicates the beginning of the encoding of a new tile. 2 illustrates the division of slice portions 26a, 28a, and 28b into the payload data section 32 of a transport packet 34, which includes, in addition to the payload portion 32, a transport packet header 36. In this regard, FIG. 2 also shows that the trailing portion of the payload portion of the transport packet, including the end of the slice portion, may be filled with padding bits 38 that are distinguished from the size portion data by showing the slice segment data within packet 34 simply as hatched and showing the heading bits 38 shaded.

[0023] A problem arises whenever a packet is lost during transmission. In particular, imagine that slice portion 28a is never received at decoder 50 because, for example, the second and third packets of slice portion 28a are lost and fragmented. However, first transport packet 34 carries slice header 30. Thus, the decoder 50 can resume decoding the picture 14 with the dependent slice segment 28b, and the decoder 50 can be sure that the slice header 30 of the independent slice segment 28a received before the lost packet is the slice header belonging to the dependent slice segment 28b. However, this cannot be guaranteed in any case for the decoder 50.

[0024] For example, turning now to Figure 3, which shows a typical image 14 subdivided / divided into six tiles, i.e., three columns of two rows of tiles 22, with one slice segment per tile. In particular, in the case of Figure 3, the first tile is incorporated into independent slice portion 28a, while the next five slice portions are dependent slice portions 28b. 4 shows the same tile being divided, but showing that the first three tiles in the first row of tiles 22 create one slice 26a consisting of a first separate slice portion 28a covering the first tile. 4. The second slice 26b then includes a sequence of independent slice portions 28a covering the fourth tile of the image 14, followed by two dependent slice portions 28b covering the fifth and sixth tiles of the image 14. Similarly, the second slice 26b includes a sequence of independent slice portions 28a covering the fourth tile of the image 14, followed by two dependent slice portions 28b covering the fifth and sixth tiles of the image 14. In the case of receiving all the data for the image 14, it is not a problem to decode the image 14, regardless of whether the encoding side chose the option of FIG. 3 or the option of FIG. However, a problem arises when, for example, the fourth slice portion is missing. In the case of Figure 3, there is in fact no problem with subsequent slice segments associated with the fifth and sixth tiles of the same image 14, which inherit the slice header data from the first slice segment. However, in the case of Figure 4, the slice portions for the fifth and sixth tiles are no longer of value, as they require the slice header data of the lost fourth packet, which in the case of Figure 4 is an independent slice portion.

[0025] In the case of Figure 3, the concept outlined below is suggested to enable the decoder 50 to resume encoding of the image 14 with respect to the fifth and sixth tiles 22. A data stream with error resilience data is provided that allows identification of packets carrying slice headers of independent slice segments from dependent slice segments.

[0026] It should be noted that in a slice decoding operation, the encoder and decoder 50 reset the successive context probabilities, i.e., the context probabilities, whenever the first syntax element of the second or subsequent tile of each slice portion occurs, while decoding the entropy contained in one partial slice, such as slice 26a of FIG. 2, to a default value. For this reason, if there is, for example, one slice portion, such as 26a in FIG. 2, if the first transport packet 34 carrying the slice header 30 is received correctly, then the second transport packet carrying slice 26a will still transport the second tile of slice portion 26a, which is decodable. The decoder 50 can perform entropy decoding using default decoding initialization values ​​for context probabilities for entropy to decode the data of slice 26a for the second tile 22, using slice header data 30 consisting of the first transport packet 34 of the six packets into which slice 26a is fragmented into tiles, i.e., opening a new packet 34 with the first syntax element for the second tile and filling the preceding packet, including the end of the data of slice 26a for the first tile, with padding data. Thus, the case described in FIG. 3 is very similar to the case where a single independent slice portion covers the entire image 14. In this entropy coding / decoding of independent slice portions, the context probabilities are newly initialized every time a tile boundary between consecutive tiles in the tile order occurs. Then, in response, the decoder can resume decoding of the fifth tile despite packet loss, for example for the fourth tile, provided that the slice header at the beginning of the independent slice segment is correctly received.

[0027] The issues outlined with respect to Figures 3 and 4 are, in other words, outlined again below: In video transmission scenarios, it is often expected that losses will occur. This type of loss may result in data that is correctly received but is not decodable due to dependencies on the lost data. For example, in RTP, and as illustrated in FIG. 2, slices 26a,b may be transported over several RTP packets 34 (commonly called fragmentation units). If one of those packets 34 of a slice is lost, many decoders, such as decoder 50, will need to discard all of the data for the corresponding slice, or decode the lost portion and discard the remaining received data for that slice. However, slices 26a,b may contain independent parts that can be decoded separately. This is the case for multiple tiles contained in a single slice for HEVC [1]. When multiple tiles 22 are included in a single slice (e.g., two or more, as illustrated in Figures 3 and 4), transport of data taking into account tile boundaries (by the first tile of the slice included in the independent slice portion and the remaining tiles within the slice included in the dependent slice portion) may be desirable in a single independent slice segment (cp. 26a in Figure 2) or one slice segment per tile (cp. 26b in Figure 26b). That is, the RTP packets 34 can be aligned to tile boundaries, or in other words, each RTP packet contains data for only one tile, rather than for several tiles, and smart fragmentation of data may be performed, e.g., an RTP packet may be aligned to a tile boundary in the case of an independent slice portion carrying several tiles. By doing this, if data for some tiles is lost or some selective decoding of partial data is performed, it is still possible to decode other tiles since they do not depend on non-received tiles for decoding.

[0028] In the described cases, where both slice portions for multiple tiles (cp. 26a) or dependent slices are used (cp. 26b in Figures 2, 3 and 4), all slices 22 of the tiles require correct reception of the slice segment header 30 of the independent slice 28a. However, there are scenarios where not all data has been received and some packets 34 are missing. Without the concepts outlined further below, it is not possible to know whether the slice segment header of the last independent slice segment is consistent with a given slice segment after a loss (and includes, for example, an independent tile). Or, the required slice segment header of an independent slice was not received due to a loss. An example is shown in FIG. 5.

[0029] In the top-level embodiment of Figure 5, the fifth packet, which illustratively corresponds to packaging each slice portion of Figure 3 into a separate packet, can be decoded if the necessary slice segment header information contains a separate portion of data from when it was received (e.g., a tile), as described above. On the other hand, in the bottom-level embodiment of Figure 5, which illustratively corresponds to packaging each slice portion of Figure 4 into a separate packet, the fifth packet cannot be decoded since the header information included in the previous packet is lost.

[0030] The concepts outlined below take advantage of the fact that there is some data that is independent of others and can be used to provide some kind of error resilience in a lossy environment. The problem is that without the concepts outlined below it is not possible to detect whether this important information contained in the previous slice segment header of an independent slice has been received or lost.

[0031] Therefore, according to the concepts outlined below, some signaling is added that allows the receiver to detect whether previously received slice header data from an independent slice segment applies to the currently received data, or whether some required data applies to data that is missing.

[0032] Examples of this kind of signaling could be, for example, some auxiliary information in NAL units specific to the RTP payload, or that PACSI for RTP payloads defined for SVC (RFC6190) formats but for its extensions including identifiers in slice segment headers required for decoding the data in HEVC or RTP packets.

[0033] This signaling may, for example, involve a flag (e.g., a T flag) indicating the presence / absence of this kind of error resilience information in the form of an identifier. The supplemental information is used to assign this identifier to a particular slice segment header or to indicate which slice segment headers of individual slice segments with a given identifier are required for certain data to be codeable. That is, if this information directly precedes the data with the slice segment header of an independent slice segment, the identifier is assigned to the slice segment header of this independent slice segment, and if not, it indicates which is the identifier of the slice segment header necessary to correctly decode the following data.

[0034] one In the examples, Former The data is all data About one time Only the specified Contains independently decodable data that requires header information. If this is done, even if some data is lost, additional supplemental information will be available that allows the identification of the important header information required. Previously received header information and If there is a match, it becomes possible to decode other independently decodable data received.

[0035] The concepts outlined above, which merely provide and analyze error resilience data for packets in which a video data stream is transmitted, are explained in more detail below in connection with the following figures.

[0036] 6 illustrates a network device 200 that may be located in front of or form part of the decoder 50. The network device comprises a receiver 202 and an error handler 204.

[0037] The transport stream received by receiver 202 is shown at 206 . It is formed of a series of packets 208, corresponding elements 34 of Figures 1 and 2, through which a video data stream 210 is transported. As mentioned above, the packets may be, for example, RTP packets, although alternative embodiments may use IP packets or the like. The video data stream 210 comprises tiles 212 corresponding to element 20 of Figures 1 and 2, corresponding to element 12 of Figures 1 and 2, corresponding to element 14 of Figures 1-4 of video 216, corresponding to element 22 of Figures 1-4 of image 214, encoded therein along some encoding order 218, corresponding to element 24 of Figures 1 and 2, and then leading, for example, through the image tiles in raster scan order, to step towards the next image 214 in the image encoding order, which may however not necessarily coincide with the order of presentation times between the images 214. In particular, tiles 212 are coded into data stream 210 using entropy coding and spatial prediction, with context-guided entropy coding and spatial prediction constrained to not cross the boundaries of tiles 212, illustrated in the figure using dotted lines. The relationship between successive portions covered by tiles 212 of a representative image 214 of data stream 210 is illustrated using the same wavy line type of image 212 on the one hand and data stream 210 on the other hand. Using constraints, tiles 212 are encodable and decodable in parallel as far as entropy coding and spatial prediction are concerned.

[0038] Video Data Stream 6, tiles 212 are coded in slices 220 according to a coding order 218. Each slice 220 may contain data for only one tile 212, as exemplarily shown for the two slices illustrated on the right side of Fig. 6, or may completely contain data for more tiles, as on the left side of Fig. 6, which contains data for the first three images according to the coding order 218. Each slice corresponds to element 30 in Figures 1 and 2. ,example For example, quantization step size, default coding mode, Regarding the entire slice, such as the slice type discussed above as an example Collects certain higher-level syntax elements that are globally valid , starting with a slice header 222. That is, the slice 220 covers multiple tiles. In the case of All tiles except the first tile in that slice require the slice's slice header data to be successfully decoded, which is located at the beginning of the slice.

[0039] Although not mentioned above, slices may be further subdivided along the coding order into so-called independent and dependent slice segments. An independent slice segment has an explicit slice header at its start and is followed by one or more dependent slice segments that inherit at least part of the slice header of the independent slice segment. Thus, at least part of the slice header of the inherited independent slice segment must be available for the dependent slice segment to be successfully decoded. The start of each tile may coincide with the start of a slice segment, whether dependent or independent.

[0040] Video Data Stream 210 are packetized into a sequence of packets 208 according to an encoding order 218, with each packet carrying data for only one tile. This is also done using four different dash types associated with the four different tiles of the example image. figure 6.

[0041] receiver 202 While receiving the transport stream 206, the error handler 204 Sequence of Of 208 lost Packets, i.e., not received It is not received in time, or it is received but with an error, or for example send Too many bit errors during Forward error correction Even if you receive it without being able to of The device is configured to identify the Additionally, the error handler 204: Packet sequence 208 packets By analyzing the error tolerance data in For each run of one or more lost packets in the sequence of packets, the sequence of one or more lost packets contains a slice header that carries the beginning of a tile and is included in a slice in which any non-lost packet in the sequence of packets contains a slice header. Identify the first packet. For example, arrow 224 Assume that the packet 208 identified in step 2 is a lost packet. Usually the same slice Two packets, namely 226 and 228, form the fragment of is discarded by the transport layer. here in, Error handler 204 detects that packet 226 The above requirements Identify packets that meet all the criteria. (1) One or more lost packet, i.e., packet 224 After each run. (2) the slice header 222 of the slice 220 to which the data included in the packet 226 belongs; Is not lost. (3) Packet 226 but Tile 212 It is carrying the starting part. (4) This packet 226 is a packet that satisfies the above (1) to (3). Our It's the first one. Lost Packet 224 and above Packet 228, which is between packets 226, does not satisfy requirement (2) above. So the error handler 204 is For packet 226, rather than discarding the content, i.e., the tile 212 of packet 226, Of course, the error handler 204 can extract the It continues to process packets in order until it encounters another run of one or more lost packets.

[0042] The just-mentioned error resilience data may be included in the packet headers of the transport packets 208 or may be included in supplemental enhancement NAL units of the data stream 210 interspersed among the payload slices 220, for example.

[0043] Error Handler 204 Operation Mode is described in further detail below in connection with FIG. In particular, the error handler must be In process 300, lost To identify the packet, Introverted packet Noshi The identification 302 may, for example, include a unintentionally and inspecting packet headers 36 (compare Figures 1 and 2) such as round-robin packet numbers. It may be accompanied by The result of the process 300 is the detection of a run of one or more lost packets in a sequence of packets, as shown at 304, where lost packets are placed above the sequence error 306 and correctly received packets are placed below the sequence error 306 to show the sequence of packets 208 in sequential order. As can be seen, one exemplary run can be seen at 308. Another process 310, executed continuously by the error handler 204, detects the loss of one or more lost packets in a sequence of packets, as shown at 304, where correctly received packets are placed below the sequence error 306 to show the sequence of packets in sequential order. It concerns the analysis error resilience data in packets. In this process 310, the error handler 204 identifies, for each run 308, the first packet in the sequence of packets after the respective run 308 that carries the beginning of one of the tiles and that carries a slice whose slice header is included in one of the non-lost packets in the sequence of packets. The process 310 is repeated with the received packets following the run 308. After Run 308 First received on Packet 308 is Using A in Figure 7 is shown. In process 310, The error handler 204 checks whether each packet A is at the start of one of the tiles. Transporting parts Check whether This check 312 In The error handler 204 may, for example, send payload data section 32 of packet A to the parsing process. Attached This Identify whether the payload data section 32 begins with the start of any tile encoding, or is at least parsable up to such encoding. For example, payload data section 32 The beginning of Slice segment NAL unit The Beginning , the error handler 204 reads the edge field from the slice segment to evaluate whether the slice segment starts encoding a new tile.

[0044] Alternatively, the error handler 204 can: for example Based on the fragmentation parameters in the transport packet header 36, Packet A Check if it is the first fragment of a NAL unit, If so, Packet A of Start of new tile in payload data 32 It is assumed that there is a part In step 312, the packet is The Beginning If it is determined that it does not match, process 310 proceeds to the next received packet, here B. however, If the check result of check 312 is positive, i.e., the tile starting point If found, process In 310, The error handler 204 checks whether the current packet A itself contains a slice header. Check. If YES There is no problem, and the decoding procedure is repeated from packet A to packet B after the run of lost packet 308, as shown in step 316. It can be resumed. However, if the check in step 314 reveals that the current packet itself does not contain a slice header, then the error handler 204 examines the error resilience data in the current packet A to identify the slice headers of the slices carried by the respective packet, i.e., the slice headers inherited by the dependent slice segment relating to the new tile, or the slice to which the new tile identified in step 312 belongs, i.e., the slice headers of the independent slice segment. Identification 318 as follows It may function. For example, packet A The transport packet header 36 may itself comprise error resilience data, This error resilience data may be a pointer to some of the preceding packets. If it belongs to the received packet 208 a check is performed at 320 , then a resumption of decoding is performed at 316 . On the other hand, the necessary slice header is lost. If it belongs to a packet, i.e. received If it does not belong to the packet, then process 310 corresponds to the concatenation of steps 312 and 314. Step 322 searches for packets with new slice headers but, Any "dependent tile" lostIf the current packet does not contain a new slice header because it requires a slice header belonging to one of the packets, return to step 312.

[0045] It should be mentioned that instead, certain NAL units may be interspersed among the actual slice segment NAL units described so far in order to carry error resilience data.

[0046] Upon resuming decoding in step 316, decoding of the video data stream is resumed after each run of one or more lost packets from the identified packet by applying a slice header included in any of the received packets to be used in decoding the tile whose start was identified in step 312, as identified by the error resilience data.

[0047] Thus, the above description has clarified the transfer of partial slice error recovery.

[0048] The error resilience data just outlined can point to the required slice header, for example, by indicating the packet number of the packet in which the required slice header is located. This can be done in an absolute or relative manner, i.e., using an offset value from the current packet containing the error resilience data to the packet containing the required slice header. Alternatively, the required packet can be indexed by the slice address of a separate slice segment containing the required slice header. As mentioned above, every slice segment includes a slice address that indicates the position in the image 14 of the first block encoded in this slice segment.

[0049] In order to maintain backward compatibility with other devices that cannot process / parse the error resilience data, an extension mechanism containing respective flags can be used to enable older decoders to ignore / skip the error resilience data and discard the same accordingly.

[0050] Of course, the above concept of using error resilient data manifests itself in a corresponding network device at the transmitting side, which may be included within or connected to the output of the encoder of Figure 1. This transmitting network device is configured to transmit the video data stream via a transport stream of a series of packets. The video data has tiles of video images into which the image is divided and coded along with entropy coding and spatial prediction using a coding order. The tiles are then coded into a video data stream with contextual derivation of the entropy coding, and spatial prediction is constrained so as not to cross tile boundaries. The video data stream has tiles with video images coded along a slice-based coding order, with each slice containing data for only one tile or entirely containing two or more tiles. Each slice begins with a slice header. The network device is configured to package the video data stream into a sequence of packets in an encoding order such that each packet carries data for only one tile, and, for each packet in the sequence that does not include a slice header for a slice that the respective packet partially carries, insert error resilience data into the packet in the sequence of packets to identify that packet as preceding the packet in the sequence that includes the slice header of the respective packet.

[0051] Although some aspects have been described in the context of an apparatus, it is clear that these aspects also refer to a description of the corresponding method, where a block or apparatus corresponds to a method step or a feature of a method step. Similarly, when described in the context of a method step, an aspect refers to a description of the corresponding block or item or feature of the corresponding apparatus. Some or all of the method steps may be implemented in (or using) a hardware device such as a microprocessor, a programmable computer, or electronic circuitry. In some embodiments, one or more of the most important method steps can be performed by such an apparatus.

[0052] Depending on particular implementation requirements, embodiments of the invention can be implemented in hardware or in software. Implementation can be performed using a digital storage medium, such as a floppy disk, DVD, Blu-ray, CD, ROM, PROM, EPROM, EEPROM or FLASH storage, having the electronically readable control signals stored thereon. The digital storage medium may be computer readable, and may cooperate (or be capable of cooperating) with a programmable computer system so that each method is performed.

[0053] Some embodiments according to the present invention include a data storage medium having electronically readable control signals, which can cooperate with a programmable computer system so that one of the methods described herein can be performed.

[0054] Typically, embodiments of the present invention can be implemented as a computer program product with program code, the program code being embodied to perform one of the methods when the computer program product runs on a computer, and the program code may for example be stored on a machine-readable carrier.

[0055] Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.

[0056] In other words, an embodiment of the inventive method is, therefore, a computer program having a program code when running on a computer for performing one of the methods described herein.

[0057] A further embodiment of the inventive method is therefore a data storage medium (or digital storage medium or computer-readable medium) having recorded thereon a computer program for performing one of the methods described herein. The data storage medium, digital storage medium or recorded medium is typically tangible and / or non-transitory.

[0058] A further embodiment of the inventive method is, therefore, a data stream or a sequence of signals representing the computer program for performing one of the methods described herein. For example, the data stream or sequence of signals can be adapted to be transferred through a data communication connection, for example by the Internet.

[0059] A further embodiment comprises a processing means, for example a computer, or a programmable logic device, adapted to or adapted to perform one of the methods described herein.

[0060] A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.

[0061] Further embodiments according to the invention include an apparatus or system configured to transfer (e.g., electronically or optically) to a receiver a computer program for performing one of the methods described herein. The receiver may be, for example, a computer, a mobile device, a memory device or the like. The apparatus or system may include, for example, a file server for transferring computer programs to the receiver.

[0062] In some embodiments, a programmable logic device (e.g., a field programmable gate array) may be used to perform some or all of the functionality of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor to perform one of the methods described herein. In general, the methods are preferably performed by any hardware apparatus.

[0063] The above described embodiments are merely illustrative for the principles of the present invention. It is understood that modifications and variations of the preparations and details described herein will be apparent to others skilled in the art, and it is therefore the intention to be limited only by the scope of the impending patent claims and not by the specific details set forth herein by way of illustration and description of the embodiments.

[0064] reference [1]B. Bross, W.-J. Han, J.-R. Ohm, GJ Sullivan, T. Wiegand (Eds.), “High Efficiency Video Coding (HEVC) text specification draft 10“, JCTVC-L1003, Geneva, CH, Jan. 2013 [2] G. Tech, K. Wegner, Y. Chen, M. Hannuksela, J.Boyce (Eds.), “MV-HEVC Draft Text 3 (ISO / IEC 23008-2 PDAM2)“, JCT3V-C1004, Geneva, CH, Jan. 2013 [3]G. Tech, K. Wegner, Y. Chen, S. Yea (Eds.), “3D-HEVC Test Model Description, draft specification“, JCT3V-C1005, Geneva, CH, Jan. 2013 [4]WILBURN, Bennett, et al. High performance imaging using large camera arrays. ACM Transactions on Graphics, 2005, 24. Jg., Nr. 3, S. 765-776. [5]WILBURN, Bennett S., et al. Light field video camera. In: Electronic Imaging 2002. International Society for Optics and Photonics, 2001. S. 29-36. [6]HORIMAI, Hideyoshi, et al. Full-color 3D display system with 360 degree horizontal viewing angle. In: Proc. Int. Symposium of 3D and Contents. 2010. S. 7-10.

Claims

1. 1. An error handler for a video decoder, the error handler configured to receive a transport stream of a sequence of packets in which a video data stream is transported, wherein: The video data stream includes tiles of a video image divided into the tiles, the tiles being coded in a coding order using entropy coding and spatial prediction; In the video data stream, the tiles of the images of the video are coded in slices according to the coding order, with each slice either containing data for only one tile or completely containing two or more tiles, each slice beginning with a slice header; the video data stream is packetized into the sequence of packets in the coding order, each packet carrying data for only one tile; The error handler further comprises: and analyzing the error resilience data in packets of the sequence of packets to identify, for each run of one or more lost packets of the sequence of packets, a first packet in the sequence of packets after the respective run of one or more lost packets, the first packet in the sequence of packets carrying a starting portion of one of the tiles and being involved in carrying a slice, at which decoding of the video data stream will be resumed; and based on the error resilience data of the first packet, identifying a packet of the sequence of packets that is not a lost packet and that includes the slice header of the slice carried by the first packet, wherein decoding is resumed using the slice header of the slice carried by the first packet. Error handler.

2. The error handler: checking whether each packet following each run of the one or more lost packets carries the beginning of any of the tiles; and For each packet that the check finds to match the beginning of one of the tiles, identifying a slice header for a slice carried by the respective packet based on the error resilience data of the respective packet; and checking whether the slice header of the slice carried by each of said packets is contained in any packet of said sequence of packets that is not a lost packet and that precedes said each run of one or more lost packets; configured to sequentially inspect each packet following each run of the one or more lost packets by The error handler of claim 1 .

3. The error handler of claim 1 , wherein the error handler is configured to read the error resilience data from a transport packet header of the packet.

4. The error handler of claim 1 , wherein the error handler is configured to obtain from the error resilience data of the respective packet a pointer or identifier to a packet containing the slice header of the slice that the respective packet carries.

5. The error handler of claim 1 , wherein the error resilience data of the first packet includes an offset value from the first packet to a packet containing the slice header.

6. 1. A method for error handling, comprising: receiving a transport stream of a sequence of packets in which a video data stream is transported, wherein The video data stream includes tiles of a video image divided into the tiles, the tiles being coded in a coding order using entropy coding and spatial prediction; In the data stream, the tiles of the images of the video are coded in slices according to the coding order, with each slice either containing data for only one tile or completely containing two or more tiles, each slice beginning with a slice header; the video data stream being packetized into the sequence of packets in the coding order, each packet carrying data for only one tile of an image; receiving the signal; identifying a lost packet in a sequence of packets; analyzing error resilience data in packets of the sequence of packets to identify, for each run of one or more lost packets of the sequence of packets, a first packet in the sequence of packets after each run of one or more lost packets that carries a start of one of the tiles and that is involved in carrying a slice, at which decoding of the video data stream will resume; and identifying, based on the error resilience data of the first packet, a packet of the sequence of packets that is not a lost packet and that includes the slice header of the slice carried by the first packet; Including, A method wherein decoding is resumed using the slice header of the slice carried by the first packet.

7. A computer program having a program code for performing the method according to claim 6, when the computer program runs on a computer.

Images