Method and apparatus for encoding or decoding video data having frame portions - Patents.com

Abstract

To provide a method for facilitating extraction and recoupling of frame portions during decoding while restricting a necessary rewriting process.SOLUTION: The method is capable of inter-prediction being used in rectangular regions. In a situation where different identifiers can be assigned to each of the plurality of rectangular regions in a frame, the method includes steps for signaling third information into a bitstream, the third information being information of the number of bits used to represent the identifier of each of the plurality of rectangular regions in the frame and information different from first information corresponding to the number of rectangular regions in the frame and signaling all of the identifiers, the first information, and second information of the locations of the rectangular regions in the frame, to one parameter set of the bitstream.SELECTED DRAWING: None

Description

[Technical field]

[0001] The present disclosure relates to a method and apparatus for encoding or decoding video data having spatial portions. [Background technology]

[0002] HEVC tiles were introduced and designed for parallel coding. However, with high size video content, there are some use cases where tiles are used differently. In particular, there is an increasing need to stream individual tiles or sets of tiles. Some applications also have an increasing need to combine different tiles from the same sequence or different sequences to compose a new video sequence.

[0003] Current mechanisms in HEVC were not designed with this kind of scenario in mind. Implementing these scenarios with current HEVC mechanisms would mean adding encoding constraints on tiles, and any configuration of tiles at decoding time would involve data rewriting operations. In particular, data rewriting involving manipulation of slice segment headers is commonly required. Summary of the Invention

[0004] The present invention has been devised to address one or more of the aforementioned problems. It relates to the definition of frame parts and the signaling of these frame parts in the bitstream. The invention aims to facilitate the extraction and recombination of these frame parts during decoding, while limiting the rewriting operations required to do so.

[0005] According to a first aspect of the present invention there is provided a method of encoding video data comprising frames into a bitstream, the frames being spatially divided into frame parts, the method comprising the steps of: encoding at least one frame portion into one or more first coding units, The method further comprises: - signaling at least one frame portion identifier to the first encoding unit, the frame portion identifier identifying an encoded frame portion; - providing frame portion configuration information including frame portion identifiers and spatial information regarding the frame portions; has.

[0006] The first aspect of the invention has the advantage of offering more flexibility and easier operation whilst allowing the possibility of improving compression compared to known tiling designs such as HEVC tiling.

[0007] In one embodiment, the frame portion configuration information is provided to one second encoding unit.

[0008] In one embodiment, at least one frame portion is encoded independently.

[0009] In one embodiment, the method further includes providing a flag indicating that the frame portions are independently encoded.

[0010] In one embodiment, one or more of the first encoding units includes a flag indicating for each frame portion that the frame portion was independently encoded.

[0011] In one embodiment, one or more of the first encoding units includes a flag indicating that at least one frame portion has been independently encoded.

[0012] In one embodiment, one or more of the first encoding units have a flag that indicates a level of encoding constraint used to encode the frame portion.

[0013] In one embodiment, the frame portion is a slice, the first coding unit is a slice unit that includes the data portion, and the flag is included in a slice segment header of the data portion of the slice unit.

[0014] In one embodiment, the frame portion is a slice, the first coding unit is a slice unit containing the data portion, and the frame portion identifier is included in a slice segment header of the data portion of the slice unit.

[0015] In one embodiment, the first encoding unit includes a header portion and a data portion including an encoded frame portion, and the frame portion identifier is included in the header portion.

[0016] In one embodiment, a frame portion identifier is signaled in every frame portion coding unit, and a predefined frame portion identifier value indicates that the frame portions are not independently coded.

[0017] In one embodiment, the second coding unit is a parameter set dedicated to information about one or more frames.

[0018] In one embodiment, the second coding unit is a parameter set dedicated to frame portion information.

[0019] In one embodiment, the first coding unit has a specific type that indicates that the frame portions are independently coded.

[0020] In one embodiment, the frame portion identifier is encoded using a fixed, predetermined number of bits.

[0021] In one embodiment, the frame portion identifier is encoded using the signaled number of bits.

[0022] In one embodiment, the spatial information includes the location of the frame portion given by a coding tree unit address.

[0023] In one embodiment, the spatial information includes the location of the frame portion given by a sample address.

[0024] In one embodiment, the spatial information includes a size of the frame portion.

[0025] In one embodiment, the position of the frame portion is given relative to the frame.

[0026] In one embodiment, several parameter data units are signaled in a bitstream that contains different frame portion configurations for the same frame portion.

[0027] In one embodiment, the second encoding unit has a flag indicating whether a given post-filtering algorithm is available for a frame portion.

[0028] In one embodiment, the same frame portion identifier may be used to identify several frame portions that define a frame portion set.

[0029] In one embodiment, the header portion includes a layer identifier, which is used to signal a frame portion identifier.

[0030] According to a second aspect of the present invention, there is provided a method of decoding video data comprising frames from at least one bitstream, the frames being spatially divided into frame parts, the method comprising: - obtaining frame portion configuration information from the bitstream, the frame portion configuration information including a frame portion identifier and spatial information regarding the frame portion; - extracting at least a frame portion from one or more first coding units in a bitstream, the frame portion including a frame portion identifier; - determining a position of a frame portion within a frame based on the spatial information; - decoding the frame portion to render the frame portion into a frame according to the determined position; has.

[0031] In one embodiment, the frame portion configuration information is provided to one second encoding unit.

[0032] In one embodiment, at least one frame portion is encoded independently.

[0033] In one embodiment, the method further includes obtaining a flag indicating that the frame portions are independently encoded.

[0034] In one embodiment, one or more of the first encoding units includes a flag indicating for each frame portion that the frame portion was independently encoded.

[0035] In one embodiment, one or more of the first encoding units includes a flag indicating that at least one frame portion has been independently encoded.

[0036] In one embodiment, one or more of the first encoding units have a flag that indicates a level of encoding constraint used to encode the frame portion.

[0037] In one embodiment, the frame portion is a slice, the first coding unit is a slice unit that includes the data portion, and the flag is included in a slice segment header of the data portion of the slice unit.

[0038] In one embodiment, the frame portion is a slice, the first coding unit is a slice unit containing the data portion, and the frame portion identifier is included in a slice segment header of the data portion of the slice unit.

[0039] In one embodiment, the first encoding unit includes a header portion and a data portion including an encoded frame portion, and said frame portion identifier is included in the header portion.

[0040] In one embodiment, a frame portion identifier is signaled in every frame portion coding unit, and a predefined frame portion identifier value indicates that the frame portions are not independently coded.

[0041] In one embodiment, the second coding unit is a parameter set dedicated to information about one or more frames.

[0042] In one embodiment, the second coding unit is a parameter set dedicated to frame portion information.

[0043] In one embodiment, the first coding unit has a specific type that indicates that the frame portions are independently coded.

[0044] In one embodiment, the frame portion identifier is encoded using a fixed, predetermined number of bits.

[0045] In one embodiment, the frame portion identifier is encoded using the signaled number of bits.

[0046] In one embodiment, the spatial information includes the location of the frame portion given by the coding tree unit address.

[0047] In one embodiment, the spatial information includes the location of the frame portion given by a sample address.

[0048] In one embodiment, the spatial information includes a size of the frame portion.

[0049] In one embodiment, the position of the frame portion is given relative to the frame.

[0050] In one embodiment, several parameter data units are obtained from a bitstream that contains different frame portion configurations for the same frame portion.

[0051] In one embodiment, the second encoding unit has a flag indicating whether a given post-filtering algorithm is available for a frame portion.

[0052] In one embodiment, the same frame portion identifier may be used to identify several frame portions that define a frame portion set.

[0053] In one embodiment, the header portion includes a layer identifier, which is used to signal a frame portion identifier.

[0054] According to a third aspect of the present invention, there is provided a method for generating a new bitstream comprising video data comprising frames, the frames being spatially divided into frame parts, the method comprising the steps of: - determining a plurality of frame portions to be extracted from a plurality of bitstreams and merged into a new bitstream, the plurality of bitstreams being coded according to any one of claims 1 to 25, determining a frame portion identifier of the frame portion to be extracted; - generating frame portion configuration information for a new bitstream; - extracting a plurality of frame portions to be extracted from a plurality of bitstreams; Embedding the multiple frame parts and the generated frame part configuration information into a new bitstream. has.

[0055] In one embodiment, the method further comprises: determining a new frame portion identifier for the extracted frame portion; replacing the frame part identifier of the extracted frame part by a new frame part identifier; has.

[0056] In one embodiment, extracting the plurality of frame portions comprises: - parsing a plurality of bitstreams; - Extracting frame portion coded data units which include one of the determined frame portion identifiers.

[0057] According to a fourth aspect of the present invention, there is provided an apparatus for encoding video data comprising frames into a bitstream, the frames being spatially divided into frame parts, the apparatus comprising: - a circuit configured to encode at least one frame portion into one or more first encoding units; The method further comprises: - signaling at least one frame portion identifier to the first encoding unit, the frame portion identifier identifying an encoded frame portion; - providing frame portion configuration information including frame portion identifiers and spatial information regarding the frame portions; has.

[0058] According to a fifth aspect of the present invention, there is provided an apparatus for decoding video data comprising frames from at least one bitstream, the frames being spatially divided into frame parts, the apparatus comprising: - obtaining frame portion configuration information from the bitstream, the frame portion configuration information including a frame portion identifier and spatial information regarding the frame portion; extracting at least a frame portion from one or more first coding units in the bitstream, the frame portion including a frame portion identifier; determining a position of a frame portion within a frame based on the spatial information; - Decoding the frame portion to render the frame portion into a frame according to the determined position. The circuit has a configuration as follows.

[0059] According to a sixth aspect of the present invention, there is provided an apparatus for generating a new bitstream comprising video data comprising frames, the frames being spatially divided into frame parts, the apparatus comprising: - determining a number of frame portions to be extracted from a number of bitstreams and merged into a new bitstream, the number of bitstreams being coded according to any one of claims 1 to 25, determining a frame portion identifier for the frame portion to be extracted; generating frame portion configuration information for the new bitstream; extracting a plurality of frame portions to be extracted from a plurality of bitstreams; Embedding the multiple frame parts and the generated frame part configuration information into a new bitstream. The circuit has

[0060] According to a seventh aspect of the present invention there is provided a computer program product for a programmable device, the computer program product comprising a series of instructions for performing the method according to the present invention when loaded and executed on a programmable device.

[0061] According to an eighth aspect of the present invention there is provided a computer readable storage medium storing computer program instructions for carrying out a method according to the present invention.

[0062] According to a ninth aspect of the present invention there is provided a computer program which, when executed, causes a method according to the present invention to be carried out.

[0063] At least some of the methods according to the invention may be computer-implemented. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, microcode, etc.), or an embodiment combining software and hardware aspects, all of which may be referred to generally herein as a "circuit," "module," or "system." Furthermore, the present invention may take the form of a computer program product embodied in any tangible medium of expression having computer usable program code embodied in the medium.

[0064] Since the present invention can be implemented in software, the present invention can be implemented as computer readable code for provision to a programmable device on any suitable carrier medium. Tangible, non-transitory carrier media can include storage media such as floppy disks, CD-ROMs, hard disk drives, magnetic tape devices or solid-state memory devices. Transient carrier media can include signals such as electrical, electronic, optical, acoustic, magnetic or electromagnetic signals, e.g. microwave or RF signals. [Brief description of the drawings]

[0065] Embodiments of the invention will now be described, by way of example only, with reference to the following drawings, in which: [Figure 1] FIG. 1 shows a system in which the present invention can be integrated. [Diagram 2] FIG. 2 shows an image coding structure of a block-based video encoder, for example, HEVC. [Diagram 3] FIG. 3 shows the division of an image into two types of partitions called slice segments and tiles in HEVC. [Figure 4] FIG. 4 shows the quadtree inference mechanism used in HEVC for CTUs that cross image boundaries. [Diagram 5] FIG. 5 illustrates the boundary extension mechanism used, for example, in HEVC. [Figure 6] FIG. 6 shows an example of an HEVC bitstream structure. [Figure 7] FIG. 7 shows an example of HEVC partitioning for region of interest (ROI) streaming. [Figure 8a] FIG. 8a shows two different example usage scenarios for region of interest combination. [Figure 8b] FIG. 8b shows two different example usage scenarios for region of interest combination. [Figure 9] FIG. 9 illustrates a typical encoding process of a video encoder in which the present invention is integrated. [Figure 10]FIG. 10 shows a typical decoding process of a video decoder in which the present invention is integrated. [Figure 11] FIG. 11 illustrates an exemplary use of the present invention. [Figure 12] FIG. 12 shows the details of the encapsulation step. [Figure 13a] FIG. 13a shows the signaling of the frame portion structure performed by the encoding process. [Figure 13b] FIG. 13b shows the signalling of the frame portion structure performed by the encoding process. [Figure 13c] FIG. 13c shows the signaling of the frame portion structure performed by the encoding process. [Figure 14] FIG. 14 shows an example of non-lattice based partitioning. [Figure 15] FIG. 15 shows an alternative embodiment for signaling a CTile identifier. [Figure 16a] Figure 16a shows an XPS that contains a list of dependencies for each CTile. [Figure 16b] Figure 16b shows a first example of a CTile dependency. [Figure 16c] Figure 16c shows a second example of a CTile dependency. [Figure 17a] FIG. 17a provides an example of an embodiment in which a CTile may change position or size between successively encoded frames. [Figure 17b] FIG. 17b provides an example of an embodiment in which a CTile may change position or size between successively encoded frames. [Figure 18] FIG. 18 is a schematic block diagram of a computing device for implementing one or more embodiments of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0066] Encoding frames of a video sequence into spatial frame portions is particularly useful, for example, in scenarios related to streaming of so-called 360-degree videos, which are in fact the result of the projection of a 360-degree panoramic or spherical video onto a classical 2D video representation.

[0067] 360-degree video (or just 360-degree video) is video that can have a very high resolution to provide a good user experience. When displayed inside a head-mounted display (or on a screen), only a spatial subpart of the 360-degree video content is presented to the user.

[0068] Therefore, it is interesting to utilize streaming protocols such as Dynamic Adaptive Streaming over HTTP (DASH) to, for example, request high quality spatial frame portions only for regions that the user is viewing, and for regions that are not viewed (i.e., regions that the user is not looking at), the spatial frame portions can simply be skipped.

[0069] The application of the present invention refers to a streaming approach that applies streaming to the user's gaze direction. In other words, it refers to viewport-dependent streaming. In such an approach, one good compromise between storage cost, computation cost, and user experience is to encode the sequence into independent spatial frame parts with different qualities. The frame parts can then be randomly accessed, extracted, and / or combined with other frame part sequences according to needs and bandwidth constraints. No additional encoding or transcoding is required. An example of such a scenario is described with reference to Fig. 8a.

[0070] An application concerns a video surveillance system in which spatial frame portions of several different videos are rearranged into a new video to match a configuration requested by a system operator. For example, the operator may only want a portion of the original video. This use is particularly shown in FIG. 8b.

[0071] Finally, in another application, a new "video" containing only a single spatial frame portion extracted from a full video sequence may involve rewriting of coding parameters if the new position of the spatial frame portion in the new video is different.

[0072] When using HEVC, the coding of spatial frame portions is based on HEVC tiles, however HEVC tiles, or more generally HEVC type tiles, are not designed to address the above mentioned applications.

[0073] FIG. 1 illustrates an embodiment of a system (eg, an interactive streaming video system) in which the present invention can be integrated.

[0074] A video bitstream is transmitted from a server or proxy server 100 to a client 102 over a network 101. The server 100 uses a video stream (or video file) generated by a video encoder 103 conforming to a block-based video codec, for example the HEVC video codec specification.

[0075] The encoder compresses a set of video sequences with different rate / distortion tradeoffs while providing spatial random access to some spatial frame portions in accordance with the invention, as described below.

[0076] A server 100 receives a request for a description of available video streams for interactive streaming via a communication network 101. The communication network 101 is based on the Internet Protocol standard. The standard protocol adopted for transmitting the media presentation over the IP network 101 is preferably MPEG DASH: Dynamic Adaptive Streaming over HTTP. However, the invention can also be used for any other streaming protocol.

[0077] Figure 2 shows the division of an image by two types of partitions: slice segments and spatial frame parts. An image 206 is divided into three slice segments. A slice segment is a part of an image or the whole image. Each slice segment contains an integer number of coding blocks (which can correspond to coding units in HEVC). A coding block is composed of samples.

[0078] There are two types of slice segments: independent slice segments 207 and dependent slice segments 208. Each slice segment is embedded in one NAL unit, which is a structure with a generic format for use in both packet-oriented and bitstream-oriented transport systems. The difference between the two types of slice segments lies in the fact that the data specified in the independent slice segment header defines all parameters necessary to decode the coding blocks of the slice segment. On the other hand, dependent slice segments have a reduced header and rely on the first preceding independent slice segment to infer parameters that are not available in the header. The destination of the first coding unit in the slice is specified in the independent slice segment header.

[0079] FIG. 3 shows another division into spatial frame parts (SPFs), allowing each frame to be divided into independently coded rectangular regions, as shown in frame 305.

[0080] Like HEVC type tiles, a spatial frame portion contains an integer number of coding blocks. Like slice boundaries, the SPF boundary 310 breaks all intra prediction mechanisms.

[0081] Like HEVC type tiles, SPFs are defined in picture parameter sets that are included in a particular NAL unit used to initialize the decoding process. The PPS NAL unit contains syntax elements that can specify the number of tile rows and the number of tile columns in a picture and their associated sizes. Other parameter set NAL units (such as Video Parameter Set or VPS, Sequence Parameter Sets or SPS) convey parameters that describe the coding settings of the bitstream. In this invention, any of these parameter sets are referred to as XPS (X is used as a wildcard character). The location of the SPF in one slice segment, e.g., its offset in bits, is identified in syntax elements available at the end of the slice segment header.

[0082] SPF and slice segments can be used together, but with some constraints: one or both of the following conditions must be verified: - All coding blocks of one slice (or slice segment) belong to the same SPF - All coding blocks of one SPF belong to the same slice (or slice segment)

[0083] A slice (or slice segment) means that it may contain several whole SPFs or may only be a sub-portion of a single tile. Secondly, an SPF may contain several whole slices (or slice segments) or may only be a sub-portion of a single slice (or slice segment).

[0084] FIG. 4 shows, for illustration purposes only, a schematic representation of the quadtree inference mechanism used in HEVC for coding units that cross an image boundary. In HEVC, images are not limited to have width and height that are multiples of the size of the coding units. Then, the rightmost coding unit of the frame crosses the right boundary 401 of the image, and the bottommost coding unit of the frame crosses the bottom boundary 402 of the image. In these cases, HEVC defines a quadtree inference mechanism for coding units that cross boundaries. This mechanism consists in recursively splitting any CUs of coding units that cross an image boundary until there are no more CUs that cross the boundary or until the maximum quadtree depth is reached for these coding units. For example, coding unit 403 is not automatically split, while coding units 404, 405 and 406 are split. There is no signaling of the inferred quadtree: the decoder must infer the same quadtree on the image boundary. However, the automatically obtained quadtree may be further refined for coding units that are inside the frame, for example, by signaling split information for that coding unit (if the maximum quadtree depth is not reached), as in 407.

[0085] 6 shows a typical video bitstream 600 transmitted from a server to a client. The bitstream may be compliant with HEVC or a block-based bitstream.

[0086] The bitstream 600 is structured as a series of Network Abstract Layer (NAL) units. There are several types of NAL units. Parameter set NAL units (e.g., VPS, SPS, and PPS for HEVC) describe the settings of the coding tool used to code the sequence. They also describe some information about the characteristics of the image (resolution, frame rate, etc.).

[0087] The first NAL unit 601 is a Video Parameter Set (VPS) that provides information about the entire bitstream, in particular the number of scalability layers in the bitstream.

[0088] The next NAL unit 602 is the sequence parameter set (SPS). It provides sequence level parameters. It is followed by a picture parameter set (PPS) NAL unit 603, which provides picture level parameters. Next, slice segments 604 can be provided. It is common to have one slice segment per frame. Slice segments 604 can be included in NAL units with various NAL unit types (CRA, IDR, BLA, RASL, RADL, STSA, TSA, or TRAIL...). NAL units that include slice segments consist of a NAL header 605 (further description of the NAL header is provided in the description of FIG. 10) and a raw byte sequence payload (RBSP) 606. The NAL header 605 contains information including the NAL unit type. The RBSP (i.e., the NAL unit data) contains information specific to the NAL unit type. In the case of a slice segment, the RBSP contains a slice segment header 607 followed by slice segment data 608. The slice segment data is a continuation of the coded data of the raster scan ordered coding tree unit 609 of the slice segment.

[0089] In one embodiment (not shown here), a parameter set, called TPS (for Tiling Parameter Set), can be inserted into the bitstream before a slice segment NAL unit. The corresponding parameters remain valid until a new TPS is found. The TPS describes the partitioning configuration of the frame.

[0090] In another embodiment, if no TPS is present in the bitstream, it is assumed that there is only one spatial frame portion in the bitstream, said spatial frame portion having the same dimensions as the video frame and located at its origin.

[0091] The bitstream may contain independent frame portions or regions of interest (ROIs). Figure 7 shows a schematic example of a region of interest, here considered as a rectangular region in a frame. ROIs are well known in HEVC and block-based codecs.

[0092] Streaming ROI or independent frame parts implies a partitioning strategy, which impacts coding efficiency since the introduction of tile boundaries breaks some HEVC prediction mechanisms.

[0093] 7, a frame 700 is divided into a 4x4 SPF grid. To access a predefined ROI 701, NAL units embedding corresponding slice segments for SPFs 6, 7, 10, and 11 are selected and sent to the client.

[0094] Preferably, in the present invention, one independent slice segment and zero or more dependent slice segments are embedded in the SPF, the advantage being that it guarantees access to the ROI independent of the rest of the frame that contains this ROI.

[0095] Indeed, it should be noted that for HEVC, and more generally for block-based codecs, HEVC tiles or similar suspend all intra-frame prediction mechanisms (except loop filtering processes) at their boundaries. Therefore, spatial prediction mechanisms are not allowed. However, some prediction mechanisms rely on temporal redundancy of data between frames of a video sequence to improve compression. For example, one block in an HEVC tile can be predicted from a predictive block that is partially or entirely outside the current HEVC tile boundary. Furthermore, HEVC provides a well-known boundary extension mechanism that extends the image boundary to allow the predictive block to be partially or entirely outside the reference image, so that the predictive block may be partially or entirely outside the frame boundary.

[0096] Finally, the predictive block may be located at a sub-pixel position, meaning that the reference block pixel values ​​are the result of a sub-pixel interpolation filter, which generates sub-pixel values ​​from a range of up to four pixels outside the block of pixels located at the corresponding whole pixel coordinate of the predictive block. As a result, temporal prediction can introduce coding dependencies between blocks within an HEVC tile and sets of pixel data located outside the HEVC tile boundary.

[0097] The second HEVC mechanism involving temporal prediction consists in the predictive coding of motion vectors using a motion vector predictor.

[0098] Finally, HEVC provides a set of loop filters that introduce dependencies between pixels of successive tiles. These loop filters are, in particular, a deblocking filter and a SAO filter that remove some artifacts introduced by the quantization of residual blocks. HEVC provides flags in the picture parameter set to indicate if these loop filters are disabled at tile and / or slice boundaries. When disabled, no coding dependencies between tiles are introduced by these compressors.

[0099] In order to guarantee decoding of the region of interest (meaning decoding the region of interest independently), a solution is to disable some or all of the aforementioned prediction mechanisms.

[0100] This leads to a higher bitrate and less efficient compression for the resulting bitstream. To optimize the bitrate of the resulting bitstream, it is possible to adapt the activation / deactivation of the prediction mechanism depending on the region of interest usage scenario.

[0101] 8a and 8b show two different application examples of the combination of regions of interest, already mentioned above.

[0102] For example, in a first example, Fig. 8a represents two frames 800 and 801 from two different video streams consisting of four regions of interest. The first video stream 800 has high quality encoding parameters and the second 801 is a lower quality and therefore lower bitrate version. The client effectively combines the high quality version of region of interest #3 with the lower quality regions of interest of regions 1, 2, and 4. This allows the quality of region of interest #3 to be emphasized while keeping the bitrate relatively low for the other less important regions.

[0103] In a second example, a set of four video streams (803, 804, 805, 806) is shown in Fig. 8b. In this usage scenario, a client forms a new mosaic video of different interest areas in each video stream. He relocates or combines the interest areas of each video stream to new positions in the resulting video stream.

[0104] According to an embodiment of the present invention, it is proposed to define a spatial frame portion, here called a constrained tile (shortcut as CTile in the following description). It refers to a spatial frame portion belonging to a sequence of frames divided into spatial frame portions that can be randomly accessed and completely decoded without decoding errors. The decoding of a CTile can be performed independently of its spatial location and / or its neighbours. In other words, a CTile is coded in such a way that a decoder can always decode it without errors or coded independently.

[0105] The coded data corresponding to the samples that form a CTile are coded independently. For example, data is coded into NAL units or coding units that form a slice (or any other part of a frame with similar functionality) in such a way that one parser can extract the samples that correspond to a CTile. As a result, two CTiles are coded into two different sets of coding units. In order to be decoded at any spatial location, a CTile cannot be part of a slice that contains another CTile. Thus, the coded data of a slice that corresponds to a CTile is independent of the coded data from other slices.

[0106] The coded data corresponding to a slice may be further divided into several slice segment coding units.

[0107] In one embodiment, a CTile is strictly independently decodable, meaning that all data necessary to analyze the coded data forming the CTile is contained in said CTile. Furthermore, the prediction mechanism uses prediction information calculated from the coded data of the same CTile. In the case of inter prediction, the reference block is taken from the same CTile in another frame.

[0108] In other embodiments, the coding restrictions may be lifted.

[0109] In a first alternative embodiment, a boundary extension mechanism is used for image boundaries. This mechanism, shown diagrammatically in Figure 5, is used for the boundaries of CTiles to allow for unconstrained and more efficient motion compensation.

[0110] Figure 5 shows a simplified boundary extension mechanism, for example used in HEVC, which allows motion compensation in inter prediction (a well-known prediction mode that allows the use of data outside the current frame) with reference to sample values ​​outside the frame.

[0111] When predicting block 501 while encoding frame 502, it is useful to allow prediction from block 503 that crosses the boundary of the reference frame 504. This makes it possible, for example, to predict moving content from the same content that was partially outside the field of view of the previous frame. Preferably, a sample padding method is defined that allows access to samples within the margin around the frame boundary of the reference picture.

[0112] In a second alternative embodiment, the derivation mechanism of the motion vector predictor (or any other predictor) may be approved for the CTile in a manner that is completely independent of neighboring tile information or the tiling configuration.

[0113] FIG. 9 illustrates an example of an encoding process implemented in a video encoder according to the present invention.

[0114] First, for each input video sequence 900 considered, the encoder determines the partitioning of the frames into frame portions (i.e., frame portion configuration) in step 901. In some embodiments, the size of the frame portions is predetermined such that one frame portion covers a single or part of the region of interest. For example, a frame portion may have a size of 512×512 pixels.

[0115] The encoder then determines which frame portion needs to be encoded as a CTile, in step 902. For example, such a frame portion may correspond to a region of interest (ROI) that the client may wish to decode alone or that the client may wish to configure with one or more other regions of interest.

[0116] The encoder then determines and assigns identifiers to all CTiles in step 903. In an alternative, the encoder may determine the identifiers and assign identifiers only to a selection of CTiles. In an alternative, identifiers for CTiles may be inferred.

[0117] If CTiles are assigned the same identifier in multiple encoded frames, it means that these CTiles belong to the same CTile sequence. A sequence of CTiles (or a CTile sequence) can be decoded independently of other frame parts. Only data from a CTile sequence is needed to decode said CTile sequence. In other words, CTiles from a CTile sequence can have a temporal dependency together.

[0118] After determining the CTile and CTile identifier, the encoder compresses and encodes the frame portions 904 according to the encoding settings. Encoding the frame portions ensures that any decoder can decode them as previously described.

[0119] The encoder generates frame portion configuration information in step 905. The frame portion configuration information consists in determining the description parameters of the frame to be partitioned into frame portions and in determining the description parameters of the CTiles by associating CTile identifiers with their position within the frame or within a sequence of frames. Different signaling alternatives are proposed, which are explained with reference to Figures 13a, 13b and 13c (first alternative) or Figure 15 (second alternative). Step 905 comprises generating signaling tiling parameters in a parameter set (XPS). In a variant, step 905 is performed before the encoding step 904 instead of after.

[0120] Step 906 involves optionally encapsulating the NAL units of the XPS and the compressed CTile frame portions into a bitstream.

[0121] For example, based on a streaming protocol, this step may further include encapsulation of the bitstream within a higher level video description format, such as the ISO BMF file format. For example, it is also possible to multiplex video data with audio data.

[0122] Steps 901, 902, and 903 may be accomplished by using one or more configuration files that provide predefined frame portion locations, provide information on whether a frame portion is a CTile, and which identifier should be used for a CTile. In alternative embodiments, frame portions and CTiles may be determined automatically from analysis of the video content, for example using a deep neural network, or a simpler segmentation algorithm.

[0123] As described in some embodiments, step 901 can be used to determine a partitioning that is constant within the entire video sequence (or at least for several consecutive frames within a video segment), meaning that the positions and sizes of the CTiles are constant within the entire CTile sequence in at least some frames that comprise a portion of the CTile sequence.

[0124] Alternatively, the determined frame portions may have variable sizes and positions between frames.

[0125] 10 illustrates an example of a decoding process implemented in a video decoder, the decoding process including the use of CTile, as defined above.

[0126] First, the video decoder extracts a NAL unit that contains a parameter set (XPS). In step 1000, frame portion structure information is obtained from the parameter set.

[0127] For each frame portion 1001 considered, the decoder determines in step 1002 from the frame portion configuration information whether the frame portion is a CTile.

[0128] If the frame part is signaled as a CTile, after test 1003, branch "yes", the decoder extracts (or infers) a CTile identifier from the frame part configuration information and determines in step 1005 the decoding position of the CTile thanks to the CTile identifier and the CTile position information associated with it. Otherwise, if the frame part is not a CTile, after test 1003, branch "no", the decoder determines in step 1005 the decoding position of the frame part from the positioning information described in the frame part encoded data and from the XPS information.

[0129] Finally, the decoder decodes the frame portion encoded data 1006, taking into account whether the frame portion is a CTile or not, and places the decoded sample values ​​into the rendered picture buffer.

[0130] Figure 11 shows an example of the merging process (see application in Figures 8a and 8b) of two bitstreams generated by the encoding process in Figure 9. The merging process means that the extracted CTiles are combined into a new video bitstream that is sent to the client.

[0131] The merging process begins in step 1100 by determining a set of CTiles to be extracted from one or more video bitstreams and merged into a new bitstream. For example, a graphical user interface may allow a user to select a set of CTiles and reposition them within a frame. In another example, the selection is performed automatically based on the content of the bitstreams. An application may select the set of CTiles that contain the content to be moved.

[0132] The process begins in step 1101 by determining the new position of the CTile when merged into the new video bitstream.

[0133] Once the CTiles to be extracted are known, in step 1102, their new identifiers are determined by obtaining the current CTile identifiers of each of the determined CTiles to be extracted. These identifiers are signaled in the frame structure information in accordance with an embodiment of the present invention. As mentioned above, in an alternative embodiment, the frame structure information may be described in a file format used to encapsulate the input bitstream. The frame structure information may be present in XPS and file formats.

[0134] In the case of an identifier collision, meaning that two or more CTiles have the same identifier, step 1101 further includes determining new CTile identifiers to resolve these collisions.

[0135] Next, the process generates frame portion configuration information for the merged video sequence in the new video bitstream (1103), which includes generating a parameter in one of the XPSs that associates new positions of the CTiles in the merged bitstream with their new CTile identifiers.

[0136] In step 1104, the coded frame part data of the set of CTiles determined in step 1100 is extracted or obtained, which involves fetching the NAL units that contain the coded frame part data of the CTile. This can be done by parsing all NAL units in the input bitstream to extract those that have the CTile identifier determined in step 1102. If the input bitstream conforms to a file format specification, all NAL units corresponding to one frame part are encapsulated in one container, e.g., a video track for ISOBMFF. Then, step 1104 involves fetching the data corresponding to the track of the selected frame part.

[0137] Finally, in optional step 1105, a new bitstream is generated by embedding the NAL units of the XPS and the NAL units containing the extracted CTile-coded frame portion data into a new bitstream, and possibly encapsulating this bitstream in a higher level description format.

[0138] For those CTiles for which new CTile identifiers were determined in step 1101 due to a CTile identifier collision, step 1105 further includes modifying headers included in the NAL units containing the original CTile identifiers. These headers are modified such that the original CTile identifiers are replaced by the CTile identifiers determined in step 1102.

[0139] In one example, the merging process in Figure 11 consists in extracting subsets of CTiles from the same bitstream. In such a case, there is no need to handle identifier collisions.

[0140] 13a, 13b and 13c show examples of frame portion structure signaling performed by the encoding process according to some embodiments of the present invention.

[0141] FIG. 13a illustrates identification of CTiles in a bitstream according to one embodiment of the present invention.

[0142] Here, a CTile identifier named ctile_unique_identifier 1301 is indicated in the frame portion encoded data. Preferably, the identifier is indicated in each data sequence (i.e., slice segment header) belonging to the frame portion encoded data. Thus: - easy identification of which parts of the bitstream belong to the CTile, and -Quick access or extraction of these parts is possible.

[0143] More precisely, in the embodiment shown in FIG. 13a, a CTile identifier 1301 is signaled in a slice segment header (slice_segment_header) 1302 of each slice segment that corresponds to a CTile having identifier 1301.

[0144] As mentioned above, the decoder parses the CTile identifier to determine the associated location of the CTile based on the frame portion configuration information. According to an embodiment, the frame portion configuration information is provided in a parameter set, e.g., TPS), as described below with reference to Figure 13b or Figure 13c.

[0145] Unless explicitly mentioned or applicable, for the sake of brevity below, no distinction is made between CTiles and CTile sequences. Furthermore, a CTile identifier can also be considered as a CTile sequence identifier.

[0146] In an embodiment, information such as ctile_flag 1303 may be used in a data sequence belonging to the frame portion coded data (e.g. in a slice segment header) to distinguish HEVC type tiles, which do not necessarily require an identifier, from CTiles. If ctile_flag is inactive (e.g. set to 'false'), HEVC type tile parameters 1304 are provided. These parameters may include tile positioning information such as first_slice_in_pic_flag and CTU destination (slice_segment_address) and references to other bitstream elements such as slice_pic_parameter_set_id. These syntax elements depend on the frame partitioning and may differ from one video sequence to another.

[0147] When ctile_flag is active, these parameters are omitted and CTile specific information constituting a CTile unique identifier 1301) is provided instead. To allow having multiple slices in a CTile, one solution is to provide an information named here ctb_addr_offset_inside_tile 1305. This information 1305 is also used to specify the position to start decoding the slice segment relative to the CTile position with the considered frame. For example, the ctb_addr_offset_inside_tile information is independent of the CTile encoding / decoding position, since this position is expressed as the number of ordered coding blocks of the raw scan (e.g., CTB, the HEVC standard coding tree block) relative to the beginning of the CTile and its width (CTB).

[0148] In another embodiment, the flag ctile_flag is not used. For example, the CTile identifier is present in all tiles, CTile and other tiles (HEVC type tiles). A predefined value, for example the value 0, may be used to identify HEVC type tiles.

[0149] In one embodiment, information is provided to identify whether a spatial frame portion is a CTile.

[0150] In another embodiment, no information is provided to identify whether a spatial frame portion is a CTile or not, assuming that the CTile is the only frame portion used.

[0151] Preferably, in a given frame, there is no more than one CTile with a given identifier. The same CTile identifier is used in all CTiles that are temporally dependent (e.g., in a CTile sequence). Thus, when CTiles with the same CTile identifier in consecutively coded pictures are extracted, they will be decoded properly.

[0152] In other words, a CTile identifier is a unique identifier that identifies a CTile in a coded video sequence. In one embodiment, a CTile identifier is inserted into the slice header of the slice segment that the CTile contains. This means that in the bitstream, the NAL unit (slice segment) that corresponds to the CTile contains the CTile identifier. Therefore, any CTile can be parsed and easily extracted from the bitstream based on its CTile identifier.

[0153] It is advantageous to signal CTile configuration information in the bitstream, for example defined by the number of CTiles, the associated CTile identifiers, and the position of the CTile within the frame.

[0154] FIG. 13b illustrates a CTile configuration according to one embodiment of the present invention.

[0155] In a first embodiment, an encoder identifies additional signaling information related to frame portion configuration information of CTiles in a picture to be decoded. The signaling information is provided in a parameter set (XPS), preferably in a tiling parameter set (TPS). Preferably, the additional signaling information includes a number of CTiles in the picture 1311, referred to herein as num_ctiles. For each CTile, a unique identifier 1312 of the CTile is associated with a CTile position 1313, referred to herein as tile_ctb_addr, signifying a decoding position within the picture. The CTile position is provided as a decoding position within the picture, which may be expressed as a CTB index number (e.g., relative to the raster scan order).

[0156] In another embodiment, a parameter named here slice_pic_parameter_set_id, also referred to in the slice segment header depicted in FIG. 13a in the portion designated by 1304, refers to a unique identifier representing a TPS. In a variant, the unique identifier represents a PPS. In this other embodiment: - Each TPS includes a tile_parameter_set_id (not shown for simplicity) parameter that identifies the TPS - meaning that the encoder may generate a new TPS, for example, every time the CTile configuration changes in a picture - so in order to avoid rewriting each slice header of a frame, it is RECOMMENDED to generate TPSs with unique identifiers for the same TPS (or PPS). - slice_pic_parameter_set_id 1314 in the slice segment header is equal to the tile_parameter_set_id of the TPS that applies to the slice. In such a case, slice_pic_parameter_set_id in the slice segment header can be renamed to slice_tile_parameter_set_id.

[0157] In one alternative, the TPS identifier is not specified in the slice data, and the decoder infers that the last TPS NAL unit preceding the slice NAL unit contains the frame portion structure of the current CTile.

[0158] 13c illustrates a CTile construction according to another embodiment of the present invention. According to this embodiment, the TPS 1320 includes a parameter value indicating the number of tiles minus one, e.g., 'num_tiles_minus1' 1321. Alternatively, the TPS includes a parameter value named instance 'num_tiles' that directly or indirectly provides the number of tiles in the frame.

[0159] In one embodiment, if the TPS indicates that there is only one frame part, this is assumed to be a CTile that has the same dimensions as the video frame and is located at its origin. Otherwise (the TPS describes several frame parts), the frame part position is described as in the previous embodiment.

[0160] In another embodiment, if there is no TPS, it is assumed that there is one CTile with the same dimensions as the video frame, which is located at its origin.

[0161] According to another embodiment, the TPS may specify a syntax similar to the HEVC lattice, e.g., 'num_tile_rows_minus1', 'num_tile_cols_minus1', and 'uniform_spacing_flag', and may describe a spatial frame sublattice with this syntax. If 'uniform_spacing_flag' is not set, the width of each column and the height of each row (except for the size of the column and the size of the last row, which can be estimated) are also specified. If 'uniform_spacing_flag' is set, the width and height of the CTile are calculated from the width and height of the picture, e.g., as in the HEVC specification. In such an embodiment, the CTile position may be expressed by the CTile number (e.g., using the raster scan order of the tiles) corresponding to the spatial frame sublattice index, since the lattice index allows to localize the corresponding CTile.

[0162] According to an alternative embodiment, 'ctile_flag' is replaced by 'ctile_level' which can take several values, each value indicating a different level of encoding constraint applied to the CTile. For example, ctile_level equal to zero indicates that the CTile is not constrained (like HEVC type tiles). ctile_level equal to '1' indicates that the CTile is constrained such that it can be decoded with the original neighborhood but may not be properly decoded when shuffled with other CTiles, or that it can be extracted alone (without the original neighborhood) and properly decoded. ctile_level equal to '2' indicates that the CTile is constrained such that it can be shuffled with any neighborhood (equivalent to ctile_flag equal to 1 in the previous embodiment) and decoded anywhere.

[0163] In another embodiment, 'ctile_level' simply provides information that the encoder made encoding decisions to satisfy the level of constraint, and thus the decoding process for CTiles with any level of constraint can be performed by the same decoding process as for HEVC type tiles (e.g., no boundary extension is performed on CTile boundaries).

[0164] In another embodiment, the encoding and decoding process is not the same for all levels of constraints. For example, a CTile with ctile_level '1' uses the same decoding process as a HEVC type tile (some restrictions that are used in the encoder and do not affect the decoder), while a CTile with ctile_level '2' needs to be decoded using a specific derivation process of the list of motion vector predictors, using boundary extension of the CTile boundaries.

[0165] According to another embodiment, even HEVC type tiles may need to have an identifier (e.g., to associate their parameters in the XPS), which is specified in the slice segment header in a similar manner as the CTile identifier. In a given frame, no HEVC type tile will have the same identifier as a CTile, nor will any other HEVC type tile have the same identifier.

[0166] According to one embodiment, an encoder indicates that a spatial frame part is a CTile by signaling a value of 'ctile_flag' for each spatial frame part in one of the parameter sets, e.g., the PPS or the TPS. For example, the encoder generates a unique identifier for each tile of the frame. When describing the frame part configuration, the encoder associates a flag (e.g., ctile_flag) with each tile-specific identifier. This flag is true when the encoding of the corresponding tile (i.e., the tile with an identifier equal to the associated unique identifier) ​​is constrained to ensure independent decoding. Conversely, the flag is false if the encoding of the tile is not sufficiently constrained to ensure independent decoding.

[0167] According to a second embodiment, the encoder generates frame portion configuration information including another flag (e.g., all_ctile_flag). If this flag is set to "1", it means that all tiles described in the frame portion configuration are CTiles. The flag indicating whether the spatial frame portion is a CTile (e.g., each ctile_flag) is omitted and inferred to be equal to true. If this flag is set to zero, the CTile is explicitly described using one of the previous embodiments. If a parameter is specific to a HEVC type tile, it is signaled in the XPS, such as in the TPS, rather than in the slice segment header. For example, slice_segment_address, which is incorporated by reference in 1304 for another embodiment, is specific to a HEVC type tile. In one embodiment, if the TPS also indicates that the spatial frame portion is not a CTile, it is indicated in the TPS. This embodiment allows simplifying the parsing and syntax of the slice segment header.

[0168] According to another embodiment, instead of using 'ctile_flag' in the slice segment header, the encoder defines a new NAL unit type for slice data corresponding to a CTile. For example, the encoder defines a CTILE_IDRNAL unit for slice NAL units from instantaneous decoding refresh (IDR) frames that are within the CTile. The encoder defines as many new NAL unit types as the encoding format specifies for regular slice data. For example, HEVC defines the following NAL unit types: CRA_NUT for slice segments of clean random access (CRA) pictures, IDR_W_RADL for slice segments of random access decodable leading (RADL) IDR pictures, IDR_N_LP for slice segments of IDR pictures for which there are no associated leading pictures in the bitstream, BLA_W_LP, BLA_W_RADL, BLA_N_LP for slice segments of broken link access (BLA) pictures, RASL_N, RADL_R for slice segments of random access skipped leading (RASL) pictures, RADL_N, RADL_R for slice segments of RADL pictures, STSA_N, STSA_R for slice segments of stepwise temporal sublayer access (STSA) pictures, TSA_N, TSA_R for slice segments of temporal sublayer access (TSA) pictures, and TRAIL_N, TRAIL_R for slice segments of non-STSA tail, non-TSA pictures. W_LP: Can have an associated RASL or RADL picture. W_RADL: No associated RASL picture. N_LP: No associated leading picture. * _N: The picture is a sub-layer non-referenced (SLNR) picture (otherwise it is a sub-layer reference picture). * _R: The picture is simply a sub-layer reference picture.

[0169] These HEVC NAL unit types are replaced by new corresponding NAL unit types CTILE_BLA_, ... * , CTILE_CRA_ * , CTILE_IDR_ * , CTILE_RASL_ * , CTILE_RADL_ * , CTILE_STSA_ * , CTILE_TSA_ * , CTILE_TRAIL_ * Using one of these new NAL unit types indicates that a NAL unit belongs to a CTile.

[0170] This alternative simplifies the decoding process because the encoder only needs to parse the first bit of each NAL unit to determine whether the slice data is within a CTile.

[0171] Preferably, the uniqueness of the CTile identifier is guaranteed by the encoding-time construction for a given sequence, which is meant in a given bitstream. However, when shuffling CTiles from different sequences, which are meant to come from different bitstreams, the uniqueness is not guaranteed. According to one embodiment, to facilitate shuffling of spatial frame parts with CTiles potentially coming from various sequences, the CTile identifier is unique in a limited number of bits. The unique value can be a random value, e.g. a hash value, or any other value that does not necessarily represent its position, thus reducing the probability of having identifier collisions when taking CTiles of different bitstreams.

[0172] In one embodiment, when performing shuffling of CTiles from multiple sequences, in case of a collision between two CTile identifiers, it is sufficient to replace the collided CTile identifier. To do this efficiently by not being forced to regenerate all slice segment headers, in a preferred embodiment, a fixed, predefined number of bits are used to encode the CTile identifier. For example, in Figures 13a and 13b, the CTile identifier is encoded in 8 bits.

[0173] In an alternative embodiment, all CTile identifiers of a sequence or picture are coded with the same number of bits. This number of bits is specified by parameters in one parameter set, e.g. SPS, PPS, RPS, 'uid_num_bits'. In slice segment headers, it is preferable to have a byte alignment mechanism after the CTile identifier (in case it takes a number of bits that is not a multiple of 8). Alternatively, the number of bits can be expressed in number of bytes (8 bits), e.g. 'uid_num_bytes'. When shuffling CTiles together from various sequences, the CTile identifiers may need to be changed when they do not all have the same number of bits. This requires modifying some slice segment headers, but is easier than updating slice segment headers, since only bytes need to be added / removed or replaced.

[0174] In yet another alternative embodiment, each CTile identifier may be encoded with a variable number of bits, the number of bits being specified in the slice segment header, or the number of bits may be determined automatically from the code used for the CTile identifier: a variable byte length code may be used, for example, Exponential-Golomb encoding (or equivalently a variable length code followed by byte alignment bits).

[0175] According to an embodiment, the CTile identifier is not signaled in the dependent slice segment header to reduce the signaling size. The CTile identifier of the dependent slice segment header is then inferred from the previous independent slice segment header. According to an alternative embodiment, the CTile identifier is signaled in the dependent slice header to facilitate parsing and extraction of the sub-bitstream containing the CTile.

[0176] Instead of signaling the CTile position with the 'tile_ctb_addr[i]' 1313 or 'slice_segment_address' 1306 coding unit address, a fine-grained CTile is introduced to provide finer positioning. The granularity can be refined down to the luma sample position, but in another embodiment, a granularity of a number of luma samples corresponding to a power of "2" (smaller than the CTU size) is sufficient. In some embodiments, the granularity may be predetermined. In alternative embodiments, the granularity is signaled, for example, in VPS, SPS, or PPS. When fine-grained CTiles are used, the dimensions of the CTile are not necessarily multiples of the CTU size.

[0177] If the size of a CTile is not a multiple of the CTU size, the coding units to the right and bottom of the CTile use an automatic splitting mechanism similar to that used for the HEVC CTUs to the right and bottom of the picture, as shown in Figure 4.

[0178] According to an alternative embodiment, even if the coding unit is incomplete, the syntax describes the complete encoding, giving some space for rate-distortion optimization of the decomposition tree (e.g., quatree or QTBT) and allowing eventual padding of suitable information to improve compression.

[0179] For HEVC type tiles, dimensions are specified using a grid. Thus, all HEVC type tiles are aligned by row and column, all HEVC type tiles in a given row have the same height, and all HEVC type tiles in a given column have the same width. The width of each column and the height of each row are specified in the XPS. For fine-grained HEVC type tiles, it may be useful to allow for a less strict arrangement, for example to allow for more efficient encoding of multiple ROIs.

[0180] According to one embodiment, the dimensions of a CTile may be specified in the slice segment header of the CTile's slice segment. To reduce the size of the resulting bitstream, the dimensions of a CTile are specified only in the first slice segment. The following slice segments reuse the same CTile dimensions. Instead, the dimensions of all CTiles are provided in the XPS, e.g., together with the position of the CTile.

[0181] As another alternative, the dimensions are provided in both the first slice segment header of the CTile and in the XPS.

[0182] As another alternative, the dimensions of the CTile are not provided, but are inferred from the ordering used to provide the tile information in the XPS (e.g., position or 'ctile_flag') and from the CTile positions, e.g., the CTile positions are declared in the XPS and the CTile positions are ordered such that each of the corresponding bottom right corners of the CTiles are ordered in ascending order (for example) in raster scan order. Figure 14 below provides an example of such an ordering.

[0183] According to an embodiment, the dependent_slice_segment_enabled_flag used in the example of Fig. 13a has the same meaning as in HEVC. It is used to indicate whether dependent slice segments are allowed or not. In HEVC, the dependent_slice_segment_enabled_flag is signaled in the PPS. According to our preferred embodiment, the dependent_slice_segment_enabled_flag is signaled for each CTile in the tiling_parameter_set (TPS) (to allow a CTile coded with dependent slice segments to be used with a CTile coded without dependent slice segments in the same bistream). To reduce the syntax of the TPS for the common use case where all CTiles are coded with or without dependent slice segments, another flag is used at the root of the TPS structure: dependent_slice_segment_enabled_flag_for_all_ctiles. If this flag is set to 1, the dependent_slice_degment_enabled_flag is not signaled for each CTile. Instead, CTile_dependent_slice_segment_enabled_flag is also signaled at the root of the TPS structure, providing the value inferred for each CTile's dependent_slice_degment_enabled_flag. For HEVC type tiles, dependent_slice_segment_enabled_flag can be signaled in the PPS, but in the preferred embodiment is signaled in the TPS.

[0184] According to an alternative embodiment, the dependent_slice_segments_enabled_flag is not signaled at all and is always inferred as true to simplify the syntax.

[0185] Figure 14 shows an example of non-grid based partitioning. Frame 1401 is partitioned into 15 CTiles numbered from #1 to #15. This numbering provides a declaration order for the tile locations in the XPS such that the bottom right corner of each tile is ordered in raster scan order. This ordering can be used to estimate the size of each CTile. For example, taking the last CTile, CTile #15, its dimensions can be estimated since it is the last tile and therefore its bottom right corner is last in raster scan order and therefore it is the bottom right corner of the frame. The dimensions of slice #15 are the dimensions of the frame minus its position: h#15=h_frame-y#15; w#15=w_frame-x#15. Tile #14 must have the last bottom right corner before CTile #15 as its own bottom right corner, therefore its bottom right corner location is bottommost (bottom of frame) and rightmost (left edge of previous tile). The dimensions of CTile#14 are h#14=h_frame-y#14; w#14=x#15-x#14. The same is repeated for tiles #13 and #12. Then for tile #11, the bottommost position is filled, so the new bottommost position is y#14. Then h#11=y#14-x#11. And so on down to CTile#1.

[0186] According to an alternative embodiment, instead of specifying the CTile position in the XPS, only the dimensions of the CTile are specified, and the CTile position is calculated from the CTile's dimensions, with the CTiles ordered according to top-left position (e.g., increasing raster scan order). The algorithm for calculating position from dimensions can be easily derived from the algorithm previously described for calculating dimensions from position.

[0187] According to one embodiment, CTile parameters described in XPS can provide CTile positions (and / or CTile dimensions) for non-existent CTiles for which no slice segments exist. This CTile description is necessary to enable proper inference in embodiments where only positions or dimensions are provided and the dimensions or position are inferred.

[0188] For video rendering, a default sample value or padding method is used to fill missing CTiles, or a padding method index or value is provided in the XPS parameters. This can be done, for example, in - Initializing the frame contents in the rendering buffer with appropriate default sample values, and / or Adding a new step after every frame part has been decoded consisting of padding all areas not covered by a tile or CTile, for example by using an inpainting method This can be implemented by adding a preliminary step at the beginning of the decoding process (eg, before step 900 of FIG. 9) consisting of:

[0189] According to one embodiment, it is possible to process multiple CTiles at the same spatial location, or overlapping CTiles. For each CTile identifier, there is an associated decoded CTile buffer (equivalent to the Decoded Picture Buffer (DPB) in HEVC, but now containing only decoded CTile data). For a given frame, each CTile is decoded using the temporal data available in the associated decoded CTile buffer. Then, according to a first alternative, the rendering order of the CTiles is the same order as the CTile order in the bistream. In a second alternative, the CTiles are associated with a rendering order that can be determined from the XPS data. For both alternatives, the samples resulting from the decoding of each CTile are put into the frame of the rendering frame buffer in the rendering order of the CTile (then possibly erasing / masking samples previously ordered by the previous CTile).

[0190] According to one embodiment, a CTile sample further includes an alpha channel indicating the level of transparency to be applied when rendering the CTile within a frame of the rendering frame buffer, or alternatively, the sample further includes a binary mask value indicating which samples of the CTile should be rendered within a frame of the rendering frame buffer.

[0191] According to one embodiment, if it is possible to handle multiple CTiles at the same position or overlapping CTiles, both the CTile position and the CTile size must be specified in the XPS, since it is not possible to infer one from the other in that context.

[0192] According to an embodiment of any given post-filtering algorithm (e.g., deblocking filter, sample adaptive offset, or adaptive loop filter), a CTile boundary post-filtering flag may be specified in XPS to indicate whether the post-filtering algorithm is usable for CTile. The CTile boundary post-filtering flag, e.g., 'usable_for_post_filtering_flag', indicates that the given post-filtering algorithm may be applied to CTile boundaries of rendered frames in the rendering frame buffer (but not to the decoded picture buffer, since temporal decoding may be modified there). Advantageously, for example, to improve visual quality. The flag may be specified for the entire frame level and / or for each of the CTiles. This flag may be useful to prevent filtering of some edges that are known to be prone to introducing artifacts when post-filtered. For example, the flag is true for CTile shuffling in the context of adaptive quality streaming, but false if the CTile boundary is between two faces of a cubic projection of 360° content and the faces are not adjacent on that edge. A postfiltered CTile boundary is one where two sides of an edge specify that postfiltering can be applied, or when an edge is between an HEVC type tile and a CTile allows postfiltering.

[0193] According to an alternative embodiment, the CTile boundaries may be post-filtered in the decoded picture buffer (DPB), in which the samples used for prediction are not post-filtered samples when inter prediction is used to ensure that the decoding is correct in any decoding configuration. Therefore, the boundary extension mechanism is applied to the last sample on the boundary before the post-filtered one using the boundary information, which means that the boundary extension is performed on the unfiltered samples.

[0194] According to an alternative embodiment, two or more CTiles may have the same CTile identifier. In this embodiment, the CTile identifier becomes a CTile set identifier. The settings of CTiles that form a CTile set must all be kept together and in the same relative position in order to be properly decoded.

[0195] In these embodiments, the location and size of a CTile collection are inferred from the XPS, which corresponds to the location and size of the bounding box of the set of CTiles that belong to the CTile collection. Thus, in the XPS, a CTile collection identifier is associated with one or more locations and sizes, one for each CTile in the CTile collection.

[0196] In these embodiments, the slice segment header 'ctb_addr_offset_inside_tile' 1305 information may be replaced with 'ctb_addr_offset_inside_tile_set' information, which allows for inference of which CTiles belong to the slice segment, and therefore the geometry to be used when decoding the slice segment.

[0197] In one such embodiment, any sample in the set of CTiles can be used for temporal motion compensation. If motion compensation uses a sample value outside the CTile set, the sample value is set to the value of the spatially closest sample in any one of the CTile sets (equivalent to applying boundary extension, but only for CTile boundary portions that are not shared by two CTiles). If any sample outside the CTile has more than one closest CTile sample, a simple rule is used to determine which sample to use, e.g., the sample with the smallest raster scan order.

[0198] Figure 15 shows an alternative embodiment to that shown in Figures 13a, 13b, and 13c for signaling CTile identifiers. In a current block-based codec, typically HEVC, a NAL unit header 1501 contains the following fields: -0: 1 bit set to (false); - 6 bits containing the NAL unit type: (type); - 6 bits containing a layer identifier: (LayerID), which is always equal to zero in HEVC, but corresponds to a scalable layer index in Scalable HEVC (SHVC), or for example to a view index in Multiview HEVC (MV-HEVC). - 3 bits indicating a temporal layer identifier: (TID), which in HEVC corresponds to the temporal layer index for temporal scalability.

[0199] In one embodiment based on the NAL unit header 1501, the encoder divides the video sequence into frame parts. The encoder uses one coding or scalability layer for each frame part. This can be seen as layered coding based on the spatial domain. The encoder can code each spatial domain layer independently of the other regions. In such a case, each spatial domain layer corresponds to one CTile. In this particular case, all slices of the spatial domain layer at the time of coding have ctile_flag set to true. The main difference is that each spatial domain layer can be further divided into tiles of HEVC type.

[0200] The encoder signals different spatial domain layers using LayerId. It sets the value of LayerId equal to the identifier of CTile. As a result, the CTile identifier is not needed in slice segment headers. Because it is of fixed bit length, the processing of the CTile identifier remains simple when shuffling frame parts of a video stream.

[0201] The encoder signals the frame portion configuration in one of the parameter sets, e.g., the VPS, which indicates the decoding position of each spatial domain layer by associating a unique identifier of the spatial domain layer with the decoding position in a syntax corresponding to the syntax described in the previous embodiment.

[0202] The encoder also describes the dependencies between different layers of the video stream, and the decoder determines which spatial domain layers are coded independently of other layers through analysis of the inter-layer dependencies described in the parameter set NAL units.

[0203] The encoder compresses a subset of spatial domain layers as CTiles, independent of other spatial domain layers. If a spatial domain layer depends on another spatial domain layer (a slice of this layer has ctile_flag set to false) in the previous frame (i.e., has the same CTile identifier), the encoder adds a reference frame from this dependent layer to the decoded picture buffer of the current layer. An upsampling or downsampling filter is applied if the sizes of the two layers are different, such that the reference frame has a size equal to the size of the current layer.

[0204] According to one embodiment, the LayerId can also be used to infer the ctile_flag: when LayerId is zero, the NAL unit belongs to a tile of HEVC type. If LayerId is non-zero, the NAL unit belongs to a CTile with a CTile identifier equal to LayerId. Alternatively, one bit of LayerId is reserved for signaling ctile_flag. The advantage of using LayerId to transport the CTile identifier is that the complexity of parsing the bitstream to extract the CTile is significantly reduced.

[0205] In another embodiment, spatial domain scalability is defined similarly to temporal scalability in HEVC, i.e., different layer identifiers identify the spatial-temporal domain from other scalability layers (e.g., SNR, resolution, multiview). Indeed, the advantage of this approach is that it is possible to use both spatial domain scalability and SNR or resolution scalability layers.

[0206] The NAL unit header 1502 is extended with a Random Access Identifier (RAID) that indicates the current frame part identifier. The LayerId semantics remains the same as in HEVC, i.e., it indicates a multiview, SNR, or resolution scalability layer.

[0207] The encoder specifies the position of each spatial domain layer by associating its RAID value with a decoding position in one of the parameter sets, e.g., the VPS. Each NAL unit that encodes the spatial domain (including the SPS, PPS and VCL NAL units) has a RAID equal to the frame part identifier (CTile(set) identifier) ​​that corresponds to the spatial domain.

[0208] As a result, the above-mentioned merging process (which consists of extracting CTiles from a set of video bitstreams and combining them into a new video bitstream) extracts the CTile identifiers of the spatial domain layers to be merged from the frame sub-structures associated with the video streams to be combined, and then extracts all NAL units with RAID values ​​equal to the set of extracted identifiers.

[0209] To limit the risk of identifier collisions when combining two video sequences, the encoder sets the RAID value with a random value, including the case where the video sequences contain a single frame portion.

[0210] According to one embodiment, RAID specifies whether the spatial region is a CTile (replacing the signaling of ctile_flag in the slice segment header): when RAID is zero, the NAL unit belongs to a tile of HEVC type. If RAID is not zero, the NAL unit belongs to a CTile with an identifier equal to RAID. Alternatively, one bit of RAID is reserved for signaling ctile_flag. In alternative embodiments, the RAID identifier is 16 or 24 bits to allow more CTiles.

[0211] According to one embodiment, a sequence of CTiles is considered as an independent bitstream. For example, in some embodiments, the sequence order of CTiles with the same identifier may be different from the sequence order of CTiles with another identifier (i.e., the GOP structure may be different between two CTiles). Thus, two CTiles in the same frame may have different NAL unit types or TIDs.

[0212] In another embodiment, the XPS contains additional information that describes any dependencies between CTiles in a way that allows the decoder to process the CTiles without error. It has been found that the independence of CTiles is considered at the level of the set of CTiles, rather than at the level of each CTile. In this configuration, some CTiles in a CTile set may have some dependencies.

[0213] For example, Figure 16a shows an XPS that includes a dependency list for each CTile. The dependency list provides CTile identifiers 1601 that depend on a given CTile with identifier 1600. When a given CTile is shown as having a dependency on another CTile, it means that the given CTile cannot be extracted without the other CTile.

[0214] Figure 16b shows a first example of CTile dependencies: CTile #1 in a current frame 1602 uses sample values ​​from CTile #2 when performing motion compensation from a previous coded frame 1603, and CTile #2 uses sample values ​​from CTile #1. In such an example, the XPS indicates that the CTile with identifier #1 has a dependency on the CTile with identifier #2, and the CTile with identifier #2 has a dependency on the CTile with identifier #1. Signaling these interdependencies is an alternative to signaling CTile sets.

[0215] FIG. 16c illustrates a second example of CTile dependency. In such an example, CTile #3 exists in frames 1605 and 1607 as a dependency on CTile #1 and #2 in frames 1604 and 1607. In such an example, CTile #3 cannot be extracted without also extracting CTile #1 and #2. However, CTile #1 and #2 have no dependency and can be extracted alone. According to an embodiment, this scenario can be applied to facilitate extraction of CTiles at various frame rates, for example, when frames 1604-1606 are temporally ordered. Alternatively, it can be used for scalable coding to facilitate extraction of CTiles of different quality layers, for example, when frame 1605 is a refinement layer of frame 1604 and frame 1607 is a refinement layer of frame 1606.

[0216] According to some embodiments, a CTile can change spatial position or size between successive frames.

[0217] According to an embodiment, a 'tile parameter set', TilePS, is also introduced. A TPS can update CTile parameters of a subset of CTiles (only moving and / or resizing), e.g. includes a 'num_updated_tiles' value, and then associates tile identifiers with a new property of the 'num_updated_tiles' modified CTile.

[0218] Classically, a motion vector gives the location of a predicted block in a reference image relative to a block co-located with the block to be coded. For a given block to be coded, the first step is to identify a co-located block in the reference image. A co-located block is defined as a block in the reference image that has the same location, meaning the same origin (top left position) and the same size as the block to be coded. The motion vector is then applied to the origin of the co-located block to determine the origin of the predicted block.

[0219] When considering the CTile, the determination of the co-located block is adapted to consider blocks that have the same position in the CTile and are no longer in the frame. When the CTile is shuffled, it means that its position in the frame is modified in the frame at the time of decoding relative to its position in the frame at the time of encoding. However, considering that the prediction is constrained in the CTile to ensure independent decoding, the correct predicted block can still be determined by applying a motion vector to a block co-located to the coded block in the CTile. This is true as long as the CTile preserves its size and position from frame to frame. Difficulties arise when the CTile changes its position in the frame and / or its size between two successive frames. In this case, the encoder and the decoder agree on a method to determine the position of the co-located block in the reference frame to which the motion vector is applied to accurately determine the predicted block.

[0220] According to an embodiment where a CTile may change position or size between consecutively coded frames, the relative position of a CTile in two consecutive frames may not be the same in two different bitstreams. Figure 17 provides an example of such an embodiment, where a first bitstream includes frames of a video surveillance 1700. In the first frame 1701, there are several sptail frame portions including a moving region of interest CTile 1702 with a given ctile_id. In another frame 1703, a CTile with a given ctile_id has moved and changed size 1704. A second bitstream includes a video 1705 generated by assembling the CTiles extracted from 1700, where the generated CTiles include a uniform color (e.g., black). In the first frame 1706, a CTile with a given ctile_id 1702 is extracted from the first bitstream and placed at the center of frame 1707. In another frame 1708, a CTile with a given ctile_id 1704 is extracted from the first bitstream and centered in frame 1709. In the first bitstream, CTile 1704 uses inter prediction with a temporal reference to CTile 1701. Thus, in the generated video 1705, CTile 1709 uses inter prediction with a temporal reference to CTile 1707. The relative spatial position between CTiles 1702 and 1704 is not the same as between 1707 and 1709. Thus, in order for whatever the decoded relative position is to be properly decoded, when an inter prediction mode is used, the coded motion vector does not take into account the change in CTile position (i.e., the relative spatial position between successive frames).

[0221] According to a first alternative, the motion vectors are calculated as if a given reference point of CTile in two successively coded frames were at the same spatial location (e.g. top left, top right, bottom left, bottom right, top center, bottom center, left center, right center or center). The coded motion vectors thus correspond to the motion vectors of the blocks in the frame referential minus the motion vectors between the reference points of CTile, and also the motion vectors in the frame referential, thus giving the motion vectors in the referential for CTile. As a result, CTile is decodable independently of subsequent spatial changes.

[0222] Figure 17b shows the case of a CTile 1710 coded in frame 1711 using a reference CTile 1712 with the same ctile_id in reference frame 1713. Block 1714 is coded using a motion vector 1715 that corresponds to the difference between the motion vector in frame 1716 and the motion vector between predefined reference points 1717 (in that example the predefined reference point is the top left corner of the CTile). Figure 17b also shows that even if the relative time decoding position of the CTile is not the same as the coding position, the coding vector 1715 is still valid when decoding block 1718 by adding it to the motion vector between predefined reference points in decoded frame 1719 to obtain the motion vector in decoded frame 1720.

[0223] According to a second alternative, the motion vector is calculated as if a given point of the CTile were at the same spatial location in two consecutively encoded frames: the given point is signaled in the CTile encoded data as an index in a list of predefined points (e.g., top-left, top-right, bottom-left, bottom-right, middle-top, middle-bottom, middle-left, middle-right, or center).

[0224] According to a third alternative, a fixed point (or alternatively a signaled point) is taken into account and a motion vector is coded in the CTile coding data. The motion vector coded in the CTile coding data provides a motion vector that is added to each of the inter motion vectors related to the temporal predictions in the CTile, taking into account that the fixed (or signaled) point of the CTile in two consecutively coded frames is at the same spatial location. It allows the encoder to reduce the coding cost of the motion vectors. For example, the encoder can select the average motion vector of the motion compensation blocks of the CTile. Seeing for example FIG. 17b, this average motion vector is subtracted to vector 1715. The result of this subtraction is the motion vector to be coded. An alternative that is equivalent in terms of results is to provide a (sub)pixel position of the reference point in the coded CTile instead of (or in addition to) a fixed point or a fixed point index and a motion vector.

[0225] According to a fourth alternative, a fixed point (or alternatively a signaled point) is taken into account. In the CTile coding data, parameters of a motion vector field are coded. The motion field allows to determine a motion vector to be added to each of the inter motion vectors related to the temporal predictions in the CTile when considering that the fixed (or signaled) point of the CTile in two consecutively coded frames is at the same spatial location. For example, the encoder can estimate the motion vectors of the blocks in the CTile and can estimate a motion vector field that minimizes their predictions in order to minimize their residuals and reduce the cost of their coding. Then, each motion compensation vector of the inter coding block is the result of subtracting the motion vector calculated from the motion vector field parameters to the motion compensation vector (e.g. 1715).

[0226] According to an embodiment, an inter prediction mode can refer to two or more previously coded reference frames, in which the previous embodiment can be extended to consider that the fixation (or signaled point) of CTile is aligned in the coding frame and all reference frames.

[0227] In embodiments where motion vectors or motion vector fields are signaled, the extension to multiple frames can be done in two alternative ways. by signaling as many motion vectors (or motion vector field parameters) as there are reference frames, or by signalling only one motion vector (or motion vector field) 'x' which is used to derive one motion vector (or motion vector field) for each reference frame according to its temporal difference with the coding frame. Either

[0228] For example, linear scaling is used. If the temporal position of the reference frame is 't-s' (where 's' is a constant temporal sampling period between frames and 't' is time) and the temporal position of the coding frame is 't', the scaling factor used is '(t) / s-(ts) / s=1', whereas if the temporal position of the reference frame is 't+2s', the scaling factor used is '(t) / s-(t+2s) / s=-2'. The scaling factors are applied to calculate the motion vectors of each reference frame. For example, looking at FIG. 17b, if the reference frame 1713 is 't+2s', the motion vector 1715 is -2. * 'x' is subtracted. The result of the subtraction, 'y', is the value of the motion vector to be coded (e.g., if the coding mode is an inter prediction mode in HEVC, 'y' is the motion vector predicted using the motion vector prediction index). In other words, at the decoder side, the motion vector 'y' is decoded for the motion compensated block, and then it is multiplied by -2. *This is added to 'x' to obtain vector 1715. Furthermore, motion vector 1717 is added to obtain a frame-level motion vector.

[0229] FIG. 12 provides details of the encoder-side encapsulation step 906 or 1105 when the bitstream is encapsulated into a higher level description format, as mentioned in the previous description.

[0230] In a preferred embodiment, the video bitstream with the CTile is encapsulated according to the ISO Base Media File Format (ISOBMFF, ISO / IEC 14496-12 and 14496-15). In the following description with reference to Figure 12, the word "sample" corresponds to a "frame", i.e., a set of NAL units from the video bitstream that corresponds to a coded picture, as defined for ISOBMFF.

[0231] The encapsulation is handled by the ISOBMFF or mp4 writer. This writer includes a parser for NAL unit headers. It is able to extract NALU types, identifiers and corresponding compression data. Typically, the extracted NALU data is placed in the encapsulated file's media data container: 'mdat' box. Metadata for the description of the NAL units are placed in a structured hierarchy of boxes under the main 'moov' box. One video bitstream encapsulated in a video track is described by a 'trak' box with its subboxes.

[0232] For split video frames, there are different possible encapsulations depending on the expected use of the video. This use can be hard-coded in the mp4 writer application or can be provided as an input parameter by some user or another program, for example in the initialization step 1200. In one embodiment, it may be convenient to encapsulate one frame portion or a given set of frame portions into one video track, thus leading to multi-track encapsulation.

[0233] Once the ISOBMFF writer initialization is done, the encoder starts parsing the video bitstream by reading NALU types, especially those corresponding to parameter sets (XPS), in step 1201. As already mentioned above, parameter sets are specific NAL units that provide high-level general information about the coding structure of the video bitstream.

[0234] From the parsing of these parameter sets, the mp4 writer can determine in test 1202 whether the video bitstream contains frame parts (e.g. whether a TPS or a specific partitioning configuration is present in one of the parameter sets). If frame parts are present, the mp4 writer determines in the same test 1202 whether these are "constrained tiles", i.e. CTiles. If the bitstream does not contain frame parts or does not contain a CTile, test 1202 is false and the video bitstream is encapsulated as one video track in step 1203.

[0235] A Tiling Parameter Set (TPS) is considered as one NALU of parameter set information and can be embedded in metadata providing decoder configuration or in metadata providing setup information such as a DecoderConfigurationRecord box, in one of the boxes dedicated to sample description such as the 'stsd' box, typically in some codec-specific sample entry.

[0236] Alternatively, according to one embodiment of the present invention, the TPS can be processed as a NAL unit for video data (VCL NALU) and stored as one sample data in the 'mdat' box. It may also be present in both sample entry and sample data. If the frame substructure changes along the video sequence, it is more convenient to store it at the sample level (sample data) than at the sample description level (sample entry).

[0237] If a change in the frame portion partitioning configuration requires a decoder reset at the receiver, the ISOBMFF writer preferably stores the TPS and CTile related information from the video bitstream in the sample entry. This decoder reset allows the device receiving or consuming the file to take the new partitioning configuration into account. The new partitioning configuration may, for example, include an indication of the encoding tools (i.e., profile) to support the amount of data to process (i.e., level). Depending on the profile and level values ​​of the partitioning configuration, or other parameters, the device may or may not support the new partitioning configuration. If not, the device may adapt to the transmission or select an alternative version of the video, if available. If the new partitioning configuration is supported, the device continues decoding and rendering the file.

[0238] If spatial access to the ROI is required, in the branch "yes" after test 1204, the ISOBMFF writer can have different encapsulation strategies depending on the use case. Spatial access means, for example, ROI or partition-based display (i.e., extraction and decoding only a set of frame parts or a frame part or a part or a part of data corresponding to the ROI), or ROI or part-based streaming (i.e., sending only a set of frame parts or a part or a part of data and metadata corresponding to the ROI). If the foreseen use case is storage for local display, which corresponds to test 1205 being true (branch "yes"), it may be advantageous to store the partitioned video bitstream in one track, but include a NALU mapping to the ROI or the set of frame parts or the frame parts for which spatial access is required. The NALU mapping is generated in step 1206. This consists, for an ISOBMFF writer, in listing for each NALU of the video bitstream the NAL units that are related to a given CTile or have the same random access identifier (RAID criterion 1502 in FIG. 15) (i.e. correspond to selectable and decodable frame parts or spatial regions). To be able to perform the listing, the NALU parser module of the ISOBMFF writer checks the value of the identifier assigned to the ROI or frame part or set of frame parts for which spatial access is required in step 903, depending on the embodiment of the bitstream generation.

[0239] If the bitstream does not contain a unique identifier in the NALU header for the set of frame parts or frame portion or ROI for which spatial access is required, the ISOBMFF writer requests the slice header parser to obtain the value of the identifier for the CTile assigned in 903 (e.g., ctile_unique_identifier referenced in 1301 in Figure 13a).

[0240] A NALUMapEntry structure 'nalm' is then created in step 1206 as a box under the 'trak' box hierarchy, containing the list of NALUs and their mapping to frame parts or sets of frame parts. For each frame part or set of frame parts, a SampleGroupDescriptionBox of type 'trif' provides a description of the frame part or set of frame parts, e.g. providing the parameters of a TileRegionGroupEntry from ISO / IEC 14496-15. The group_ID value of the frame part identifier 'trif' is set to the value of the identifier of the encapsulating CTile.

[0241] Then, in step 1203, the data for all frame parts or sets of frame parts are encapsulated as a single track. In response to test 1208 being true (branch "yes"), if the use case is streaming, it may be convenient to split each frame part or set of frame parts corresponding to a spatial access level in the video into a dedicated track, and single track encapsulation is performed.

[0242] In streaming use cases, for each frame part or set of frame parts, in step 1209, a frame part description in terms of NALU mapping is generated. The number of frame parts or sets of frame parts can be determined by parsing the TPS. Even the default sample group can be used, since the 'trif' sample group is used and there is one frame part or one set of frame parts per frame part track (track for encapsulating data related to one frame part or set of frame parts) generated in step 1210. Then, all samples are mapped to the same sample group description, which is the frame part descriptor 'trif' according to ISO / IEC 14496-15. The group_ID value of the frame part descriptor 'trif' is set to the value of the identifier of the encapsulating CTile (or RAID, if present).

[0243] Then, in step 1210, each frame part or set of frame parts is inserted into its own track, the frame part track. The frame part track contains a specific sample entry that indicates that the sample is indeed a spatial part of the video and, if there are no frame parts or frame part sets left to be encapsulated, references the frame part base track created in step 1212, test 1211. This frame part base track contains specific NAL units that correspond to a parameter set, including the tiling parameter set (TPS). The frame part base track references each frame part track in turn with a specific track reference type, allowing implicit reconstruction of any selection of frame parts or sets of frame parts. Step 1212 can be replaced by a composite track, where NAL units called extractors provide explicit reconstruction from one or more frame part tracks.

[0244] The extractor then allows for any configuration of frame parts or sets of frame parts different from the original frame parts, for each sample of the composite track, simply by having the extractor point to a given identifier of a frame part or set of frame parts, typically by referencing the identifier of the corresponding CTile (or RAID, in some cases).

[0245] If composite tracks are used in step 1212, the frame part tracks in step 1210 may actually be decodable frame part tracks, meaning that each contains a frame part description (generated in 1209) and a parameter set. The presence of a TPS in each frame part track is optional, since the extractors can be recombined differently. The sample description can then indicate the sample entry that conforms to the codec in use, for example 'hvc1' or 'hvc2' if HEVC is in use, 'avc1' or 'avc2' if AVC (Advanced Video Coding) is in use, a reserved four-letter coder to unambiguously identify the video coder in use.

[0246] 18 is a schematic block diagram of a computing device 1800 for implementing one or more embodiments of the present invention. The computing device 1800 may be a device such as a microcomputer, a workstation, or a light portable device. The computing device 1800 may include: - a central processing unit 1801 such as a microprocessor, denoted CPU; - a random access memory 1802, denoted RAM, for storing the executable code of the method of an embodiment of the invention, as well as registers adapted to record variables and parameters necessary for implementing the method according to an embodiment of the invention, the memory capacity of which can be expanded, for example, by any RAM connected to an expansion port; - a read-only memory 1803, indicated as a ROM, for storing a computer program for implementing an embodiment of the invention; - the network interface 1804 is typically connected to a communications network over which the digital data to be processed is transmitted and received. The network interface 1804 may be a single network interface or may be composed of a set of different network interfaces (e.g. wired and wireless interfaces, or different types of wired or wireless interfaces). Data packets are written to the network interface for transmission or read from the network interface for reception under the control of a software application running within the CPU 1801; - A user interface 1805 may be used to receive input from a user or display information to a user; - A hard disk 1806, denoted HD, may be provided as mass storage device; - Input / Output module 1807 can be used to send and receive data to and from external devices such as video sources or displays A communication bus is connected to the

[0247] The executable code may be stored either in the read-only memory 1803, in the hard disk 1806 or on a removable digital medium, such as a disk. According to a variant, the executable code of the program may be received by means of a communications network, via the network interface 1804, in order to be stored in one of the storage means of the communications device 1800, such as the hard disk 1806, before execution.

[0248] The central processing unit 1801 is adapted to control and direct the execution of software code portions or instructions of a program or programs according to an embodiment of the present invention, the instructions being stored in one of the aforementioned storage means. After power-on, the CPU 1801 can execute instructions from the main RAM memory 1802 relating to a software application, for example after those instructions have been loaded from a program ROM 1803 or a hard disk (HD) 1806. Such a software application, when executed by the CPU 1801, performs the steps of the flowcharts of the present invention.

[0249] Any step of an algorithm of the present invention may be implemented in software by execution of a set of instructions or programs by a programmable computing machine such as a PC ("personal computer"), a DSP ("digital signal processor") or a microcontroller, or in hardware by a machine or dedicated component such as an FPGA ("field programmable gate array") or an ASIC ("application specific integrated circuit").

[0250] Although the present invention has been described with reference to specific embodiments, it is understood that the present invention is not limited to those embodiments and modifications within the scope of the present invention will be apparent to those skilled in the art.

[0251] Many further modifications and variations will be suggested to those skilled in the art by reference to the foregoing exemplary embodiments, which are given by way of example only and are not intended to limit the scope of the invention, which is determined solely by the appended claims. In particular, different features from the various embodiments may be interchanged as appropriate.

[0252] Each of the above-described embodiments of the present invention may be implemented alone or in combination in several embodiments, and features from the various embodiments may be combined where necessary or where a combination of elements or features from the individual embodiments in a single embodiment is beneficial.

[0253] Each feature disclosed in this specification (including the accompanying claims, abstract and drawings), unless expressly stated otherwise, may be replaced by alternative features serving the same, equivalent or similar purpose. Thus, unless otherwise stated, each feature disclosed is only one example of a generic series of equivalent or similar features.

[0254] In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite articles "a" or "an" do not exclude a plurality. The mere fact that different features are recited in mutually different dependent claims does not indicate that a combination of these features cannot be used to advantage.

Claims

1. 1. A method of encoding a frame into a bitstream, the frame being spatially divided into a plurality of rectangular regions, each of the plurality of rectangular regions capable of including at least one slice, the method comprising: encoding a rectangular region in the frame; signaling an identifier for each of the rectangular regions in the frame; signaling first information corresponding to a number of the rectangular regions in the frame; signaling second information of a position of the rectangular region in the frame; signaling a flag for loop filtering at a boundary of at least one of said rectangular regions in said frame; having Inter prediction may be used in the rectangular region; In a situation where a different identifier is assigned to each of the rectangular regions in the frame, third information is signaled in the bit stream, the third information being information of a number of bits used to represent the identifier of each of the rectangular regions in the frame, the information being different from the first information corresponding to the number of the rectangular regions in the frame; The identifier, the first information, and the second information are all signaled in one parameter set of the bitstream. A method comprising:

2. 2. The method of claim 1, wherein the identifiers of each of the rectangular regions included in the frame are signaled in the bitstream with the same number of bits.

3. 2. The method of claim 1, wherein the number of bits is a variable number of bits.

4. 2. The method of claim 1, wherein each of the rectangular regions included in the frame is coded independently.

5. 3. The method of claim 2, further comprising providing a flag indicating that each of the rectangular regions included in the frame has been independently coded.

6. 2. The method of claim 1, wherein the flag indicates whether loop filtering is enabled at a boundary of the rectangular region.

7. 2. The method of claim 1, wherein in response to the flag, adaptive loop filtering at a boundary of the rectangular region is disabled.

8. 1. A method of decoding video data, including a frame, from a bitstream, the frame being spatially divided into a plurality of rectangular regions, each of the plurality of rectangular regions capable of including at least one slice, the method comprising: obtaining, from a parameter set of the bitstream, an identifier for each of the rectangular regions in the frame, first information corresponding to a number of rectangular regions in the frame, and second information of positions of the rectangular regions in the frame; determining a position of the rectangular region within the frame based on the second information; decoding the rectangular regions from the bitstream; having Inter prediction may be used in the rectangular region; In a situation where the identifier, the first information, and the second information are all signaled in the one parameter set of the bit stream, a flag regarding loop filtering of the boundary of the rectangular region is signaled in the bit stream, and each of the rectangular regions in the frame is assigned a different identifier, information on the number of bits used to represent the identifier of each of the rectangular regions in the frame, Third information, which is different from the first information corresponding to the number of the rectangular regions in the frame, is signaled in the bitstream. A method comprising:

9. 9. The method of claim 8, wherein the identifiers of each of the rectangular regions included in the frame are signaled in the bitstream with the same number of bits.

10. 9. The method of claim 8, wherein the number of bits is a variable number of bits.

11. 9. The method of claim 8, wherein each of the rectangular regions included in the frame is coded independently.

12. The method of claim 11 , further comprising obtaining a flag indicating that each of the rectangular regions included in the frame has been independently coded.

13. 9. The method of claim 8, wherein the flag indicates whether loop filtering is enabled at a boundary of the rectangular region.

14. 9. The method of claim 8, wherein in response to the flag, adaptive loop filtering at a boundary of the rectangular region is disabled.

15. 1. An encoding device for encoding a frame into a bitstream, the frame being spatially divided into a plurality of rectangular regions, each of the plurality of rectangular regions capable of including at least one slice, the encoding device comprising: means for encoding a rectangular area in the frame; means for signaling an identifier for each of the rectangular regions in the frame; means for signaling first information corresponding to the number of the rectangular regions in the frame; means for signaling second information of a position of the rectangular region in the frame; means for signaling a flag related to loop filtering at a boundary of at least one of said rectangular regions in said frame; having Inter prediction may be used in the rectangular region; In a situation where a different identifier is assigned to each of the rectangular regions in the frame, third information is signaled in the bit stream, the third information being information of a number of bits used to represent the identifier of each of the rectangular regions in the frame, the information being different from the first information corresponding to the number of the rectangular regions in the frame; The identifier, the first information, and the second information are all signaled in one parameter set of the bitstream.

13. An encoding device comprising:

16. 1. A decoding device for decoding video data including a frame from a bitstream, the frame being spatially divided into a plurality of rectangular regions, each of the plurality of rectangular regions capable of including at least one slice, the decoding device comprising: means for obtaining, from one parameter set of the bit stream, an identifier for each of the rectangular regions in the frame, first information corresponding to the number of rectangular regions in the frame, and second information of positions of the rectangular regions in the frame; means for determining a position of the rectangular region within the frame based on the second information; means for decoding the rectangular regions from the bitstream; having Inter prediction may be used in the rectangular region; In a situation where the identifier, the first information, and the second information are all signaled in the one parameter set of the bit stream, a flag regarding loop filtering of the boundary of the rectangular region is signaled in the bit stream, and each of the rectangular regions in the frame is assigned a different identifier, information on the number of bits used to represent the identifier of each of the rectangular regions in the frame, Third information, which is different from the first information corresponding to the number of the rectangular regions in the frame, is signaled in the bitstream. A decoding device comprising:

Images