Image Data Encapsulation

Abstract

To improve a file format for a still image.SOLUTION: A method according to an embodiment of the present invention is to output data corresponding to one or more items of an arbitrary type based on a file having a hierarchical structure of a box storing meta data of the items of an arbitrary type, and the method acquires a file including at least (1) one or more properties listed in the box, and (2) association information for associating identification information on each of the one or more properties and identification information on each of the one or more items of an arbitrary type with each other, and outputs data corresponding to the items of an arbitrary type based on the acquired file.SELECTED DRAWING: Figure 8

Description

[Technical field]

[0001] The present invention relates to storing image data, such as still images, bursts of still images or video data, in a media container along with descriptive metadata, such metadata generally providing easy access to the image data and portions of the image data. [Background technology]

[0002] Although some of the approaches described in this section could be pursued, they are not necessarily approaches that have been previously conceived or pursued. As such, the approaches described in this section are not necessarily prior art to the claims in this application and are not admitted to be prior art by inclusion in this section.

[0003] The HEVC standard specifies a profile for the coding of still images and describes specific tools for compressing a single still image or a burst of still images. An extension to the ISO Base Media File Format (ISOBMFF) used for such image data has been proposed for inclusion in Part 12 of the ISO / IEC 23008 standard, under the name "Image file format". The standard covers two storage formats corresponding to different use cases: - Storage of image sequences, optionally with timing used in the decoder, where images may depend on other images. -Memory of single images and collections of independently coded images.

[0004] In the first case, the encapsulation is close to the encapsulation of video tracks in the ISO base media file format (see document "Information technology - Coding of audio-visual objects - Part 12: ISO base media file format", ISO / IEC 14496-12:2014, Fifth edition, Avril 2015), and the same tools and concepts are used, such as the "trak" box and sample groupings for description. A "trak" box is a file format box that contains sub-boxes to describe a track, i.e. a timed sequence of related samples.

[0005] In the second case, a set of ISOBMFF boxes is used, namely "meta" boxes. These boxes and their hierarchy offer fewer descriptive tools than the "track" boxes and relate to "information items" or "items" instead of to related samples.

[0006] The image file format can be used to display multimedia files locally or to stream multimedia presentations. HEVC still images have many applications, but they pose many problems.

[0007] Image bursts are one application example: an image burst is a sequence of still photos captured by a camera and stored as a single representation (many photo items refer to one data block). A user may want to perform several types of operations on these photos: selecting photos as thumbnails or covers, adding effects to these photos, etc.

[0008] Therefore, descriptive metadata is needed to identify the list of photos with their corresponding bytes in the data block.

[0009] Computational photography is another application, where users can access the same photo at different resolutions (different exposures, different focus, etc.). These different resolutions must be stored as metadata so that they can be selected and the corresponding parts of the data can be located and extracted for processing (rendering, editing, sending, etc.).

[0010] Therefore, as photographic resolution increases in terms of size, there is a need to provide sufficient descriptions so that only some spatial portions of these large photographs can be easily identified and extracted.

[0011] Another type of application is accessing specific pictures from a video sequence, for example for video summarization, forensic images in video surveillance data, etc.

[0012] For such applications, in addition to the compressed video data and video track metadata, image metadata is needed to allow easy access to the primary image.

[0013] Moreover, professional cameras have reached high spatial resolution. Videos or images with 4K2K resolution are now common. Even 8K4K videos or images are now becoming common. At the same time, videos are increasingly being played on mobile devices and connected devices with video streaming capabilities. Therefore, if a user of a mobile device wants to view or focus on a subpart of the video, it becomes important to divide the video into tiles, by maintaining or even improving the quality. By using tiles, the user can therefore interactively request a spatial subpart of the video.

[0014] There is therefore a need to compactly describe these spatial subparts of a video in a file format to make them accessible without further processing beyond simply parsing the metadata boxes. When an image corresponds to a video so described, a user would also be interested in accessing the spatial subparts.

[0015] In addition, users typically transform or compose images to generate new derived images, which are obtained by applying one or more specified operations, such as rotation or clipping, to other images or sets of images.

[0016] Therefore, in order to retrieve a derived image from the original image, the operations that are to be applied to one or more input images must be described as metadata in a file format.

[0017] The ISO / IEC 23008-12 standard covers two recently discussed methods for encapsulating still images into file formats.

[0018] One method is based on the concept of "track" boxes and timed sequences of related samples with associated descriptive tools, another method is based on "meta" boxes based on information items that provide fewer descriptive tools instead of samples, specifically for subject area description and tiling support.

[0019] Therefore, there is a need to provide tiling support in new image file formats.

[0020] The use of tiles is commonly known in the art, especially during compression. For indexing in the ISO Base Media File Format, there is a tiling descriptor in the proposed amendment to Part 15 of the ISO / IEC 14496 standard "Carriage of NAL unit structured video in the ISO Base Media File Format".

[0021] However, these descriptors rely on "track" boxes and sample grouping tools and cannot be used in still image file formats when using a "meta" based approach, which would otherwise complicate the selection and extraction of tiles from encoded pictures stored in this file format.

[0022] FIG. 1 shows a description of a tile-encoded still image within the "meta" box (100) of the ISO Base Media File Format, as disclosed in MPEG contribution m32254.

[0023] In addition to the respective information items for each tiled picture (102, 103, 104 and 105), an information item for the full picture 101 is defined. These information items are stored in a box called "ItemInfoBox" (iinf). A box called "ItemReferenceBox" (106) from the ISOBMFF standard is used to indicate the existence of a "tile" relationship (107) between an information item of the full picture and the four information items corresponding to the tiled pictures (108). An identifier for each information item is used so that a box called "ItemLocationBox" (109) gives the byte range(s) in the coded data (110) that represent each information item. To associate the EXIF ​​metadata (111) with the information item for the full picture (101), another box "ItemReferenceBox" (112) is used and a corresponding data block (111) is created in the metadata box (110). Also, an additional information item (113) is created to identify the EXIF ​​metadata.

[0024] Even if the full picture and its tiles are introduced as information items, no tiling information is given here. Furthermore, when associating additional metadata with an information item (such as EXIF), no data blocks are created that are referenced using additional "ItemReferenceBoxes".

[0025] Even if we reuse the tiling information from EXIF ​​and the mechanisms specified in the proposed still image file format, it is not possible to describe non-regular grids with existing EXIF ​​tags.

[0026] Thus, there remains a need for improved file formats for still images, particularly HEVC still images, and in particular for methods for extracting regions of interest within still images stored in this file format. The present invention is within the above context. Summary of the Invention

[0027] According to a first aspect of the present invention, there is provided a method of encapsulating an encoded bitstream representing one or more images, the method comprising: - providing tile description information including spatial parameters for dividing an image area into one or more tiles; - providing tile picture item information identifying portions of a bitstream that represent tiles of a single image; - providing reference information linking said tile picture items to said tile description information; - outputting said bitstream together with said given information as an encapsulated data file; Includes.

[0028] The output may be performed according to a defined standard and is readable and decodable.

[0029] The method according to the first aspect allows for easy identification, selection and extraction of tiles from, for example, ultra-high resolution images (4K2K, 8K4K...) without complex calculations by parsing syntax elements.

[0030] The description tools in the metadata box of the ISO Base Media File Format can be extended, in particular to allow tile descriptions to be associated with information items.

[0031] Part of the "meta" box hierarchy can be extended to provide additional description tools, in particular to support tile-based access within still images.

[0032] The method according to the first aspect allows for easy extraction of interest regions based on HEVC tiles from an encoded HEVC still picture.

[0033] Embodiments of the present invention provide tile description support and tile access for still images encoded according to the HEVC standard.

[0034] This allows preserving the region of interest features available for the video track for still images. In general, the part of the still picture that corresponds to the user-defined region of interest can be identified and easily extracted for rendering or transmission to a media player.

[0035] For example, the encapsulated encoded bitstream also includes information identifying a timed portion of the data stream that corresponds to a video sequence.

[0036] Therefore, dual indexing can be provided on a single data portion giving the same accessibility to the video as in the several still images that are part of this video. For example, the description information includes a set of spatial parameters for each tile picture item. For example, the tile description information includes spatial parameters that are common to two or more tile picture items. For example, the tile description information is embedded within the bitstream. For example, the tile description information is provided as metadata. For example, the reference information includes a reference type and additional descriptive metadata including the tile description information described above. For example, the reference information includes a reference type and reference parameters associated with the tile description information.

[0037] The method may further comprise providing a metadata item for referencing said tile description information in the bitstream. For example, tile picture items are grouped and reference information is provided to link the group of tile picture items to the tile description information. For example, all references linking a metadata item to another item are contained within a single reference box in the encapsulated data file. For example, all relationships from one item, of any type, are stored within a single item information descriptor. For example, the outputting is performed by a server module for adaptive streaming. For example, the outputting is performed for storage in a memory. For example, the outputting is performed by a display module for display. For example, the outputting is performed by a communication module for transmission. For example, the encapsulated data file corresponds to a standardized file format. For example, the encapsulated data file is decodable and playable.

[0038] According to a second aspect of the present invention there is provided a method of processing an encapsulated data file containing an encoded bitstream corresponding to one or more images and information including tile description information including spatial parameters for dividing an image area into one or more tiles, the method comprising the steps of: - selecting an image region of interest; - identifying a tile from said tile description information that corresponds to a selected area of ​​interest; - selecting one or more tile picture items to be linked to the identified tiles, each tile picture item identifying a portion of a bitstream representing a tile of a single image; - extracting a portion of the bitstream identified by the selected tile picture item(s); - outputting said extracted bitstream portion; and Includes.

[0039] For example, the outputting is performed by a server module for adaptive streaming. For example, the outputting is performed for storage in a memory. For example, the outputting is performed by a display module for display. For example, the outputting is performed by a communication module for transmission. For example, the encapsulated data file corresponds to a standardized file format. For example, the encapsulated data file is decodable and playable.

[0040] According to a third aspect of the present invention there is provided a method of processing image data representing at least one image for encapsulation in an encapsulation file, the method comprising the steps of: - obtaining a spatial subdivision of said at least one image into a plurality of image portions; - determining at least one portion identification data identifying a data portion within said image data representing an image portion of said plurality of image portions; The image data is at least subdivision description data representative of said subdivision of said at least one image, The partial identification data; reference data linking said subdivision description data and said part identification data; encapsulating both in said encapsulation file; Includes.

[0041] For example, the image data represents a plurality of images of a video sequence, the method further comprising identifying at least one time identification data identifying a data portion within the image data that represents a time portion of the video sequence, the image data being encapsulated with the time identification data. For example, a plurality of portion identification data are identified, each representing the same image portion of images of the time portion of the video sequence. For example, at least said subdivision description data is encapsulated as metadata into the image data. For example, the spatial subdivision is embedded within a bitstream that includes the image data. For example, for each image portion, respective portion identification data is identified. For example, common portion identification data for multiple image portions is identified.

[0042] The method may further include outputting the encapsulated file into a bitstream for adaptive streaming by a server device.

[0043] The method can further include outputting the encapsulated file in a bitstream for transmission to a display device for displaying the image data.

[0044] The method can further include outputting the encapsulated file into a bitstream for transmission to a client device.

[0045] The method may further include storing the encapsulated file in a storage device.

[0046] For example, the reference data includes a reference type and additional descriptive metadata including the subdivision description data. For example, the reference data includes a reference type and a reference parameter associated with said subdivision description data. For example, said subdivision description data is referenced in a metadata item. For example, the partial identification data are grouped and the reference data links the groups of partial identification data to said partial identification data. For example, the encapsulated file includes a single reference box that contains all the reference data for the image data. For example, the encapsulated file includes a description that includes a representation of relationships between the subdivision description data, part identification data, and reference data.

[0047] According to a fourth aspect of the present invention there is provided a method of processing an encapsulated file, the encapsulated file comprising: image data representing at least one image; - subdivision description data describing a spatial subdivision of said at least one image data into a plurality of image portions; - at least one portion identification data identifying a data portion within said image data representing an image portion of said plurality of image portions; - reference data linking said subdivision description data with said partial information; Including, The method is as follows: - identifying a region of interest in said at least one image; - identifying at least one image portion that belongs to said region of interest based on said subdivision description data; - accessing at least one portion identification data identifying a data portion within said image data representing said at least one image portion belonging to said region of interest based on said reference data; - extracting said data portion within said image data; Includes.

[0048] For example, the image data includes a plurality of images of a video sequence, the encapsulation file further includes at least one time identification data identifying a data portion within the image data representing a time portion of the video sequence, a target region for an image of the time portion of the video sequence is identified, and a data portion corresponding to the target region within the plurality of images of the time portion of the video sequence is extracted. For example, the plurality of portion identification data each represent the same image portion of images of the time portion of the video sequence. For example, at least said subdivision description data is encapsulated as metadata into the image data. For example, for each image portion, respective portion identification data is identified. For example, common portion identification data for multiple image portions is identified.

[0049] The method can further include receiving the encapsulated file as a bitstream that is adaptively streamed by a server device. The method may further include displaying the region of interest. For example, the reference data includes a reference type and additional descriptive metadata including the subdivision description data. For example, the reference data includes a reference type and a reference parameter associated with the subdivision description data. For example, said subdivision description data is referenced in a metadata item. For example, the partial identification data are grouped and the reference data links the groups of partial identification data to said partial identification data. For example, the encapsulated file includes a single reference box that contains all the reference data for the image data. For example, the encapsulated file includes a description that includes a representation of relationships between the subdivision description data, part identification data, and reference data.

[0050] According to a fifth aspect of the invention there is provided a device configured to carry out a method according to the first aspect.

[0051] The device is - a processing unit configured to provide tile description information comprising spatial parameters for dividing an image area into one or more tiles, to provide picture item information identifying portions of a bitstream representing tiles of a single image, and to provide reference information linking said tile picture items to said tile description information; a communication unit configured to output said bitstream together with said given information as an encapsulated data file; It can be provided with:

[0052] According to a sixth aspect of the invention there is provided a device configured to carry out the method according to the second aspect.

[0053] The device may be configured to process an encapsulated data file that includes an encoded bitstream corresponding to one or more images and information including tile description information that includes spatial parameters for dividing an image area into one or more tiles. The device may also include: - a processing unit configured to select an image area of ​​interest, to identify from said tile description information a tile corresponding to the selected area of ​​interest, to select one or more picture items that identify portions of a bitstream that are linked to said identified tiles, each picture item representing a tile of a single image, and to extract the portions of the bitstream identified by the selected tile picture item(s); a communication unit configured to output said extracted bitstream portions; It can be provided with:

[0054] According to a seventh aspect of the invention there is provided a device configured to carry out a method according to the third aspect.

[0055] The device may be configured to process image data representing at least one image for encapsulation in an encapsulation file, the device obtaining a spatial subdivision of the at least one image into a plurality of image portions, identifying at least one portion identification data identifying a data portion within the image data representing an image portion of the plurality of image portions, and encapsulating the image data into at least one of: - subdivision description data representative of said subdivision of said at least one image; said partial identification data; - reference data linking said subdivision description data with said part identification data; Both may include a processing unit configured to encapsulate within said encapsulation file.

[0056] According to an eighth aspect of the present invention there is provided a device configured to carry out a method according to the fourth aspect.

[0057] The device is image data representing at least one image; - subdivision description data describing a spatial subdivision of said at least one image data into a plurality of image portions; - at least one portion identification data identifying a data portion within said image data representing an image portion of said plurality of image portions; - reference data linking said subdivision description data with said partial information; The file can be configured to process encapsulated files containing:

[0058] The device may also comprise a processing unit configured to identify an area of ​​interest in said at least one image, identify at least one image portion belonging to said area of ​​interest based on said subdivision description data, access at least one portion identification data identifying a data portion in said image data representing said at least one image portion belonging to said area of ​​interest based on said reference data, and extract said data portion in said image data.

[0059] According to a ninth aspect of the present invention, a first device according to the fifth aspect or the seventh aspect, a second device according to the sixth or eighth aspect for processing files from said first device; A system is provided comprising:

[0060] According to a tenth aspect of the present invention there is provided a computer program and a computer program product comprising instructions for carrying out the methods according to the first, second, third and / or fourth aspect of the present invention when loaded and executed on a computing means of a programmable device.

[0061] According to an eleventh aspect of the present invention, A method is provided for encapsulating an encoded bitstream representing one or more images, the encapsulated bitstream including a data part and a metadata part, the method comprising: providing image item information identifying sub-images or portions of a data part representing an image of a single image; providing image description information including parameters including display parameters and / or transformation operators associated with one or more images; outputting said bitstream together with said provided information as an encapsulated data file; Including, Image description information is stored in the metadata part.

[0062] In one embodiment, each parameter included in the image description information is: -Type information, and / or -An identifier used to link the image item information to the above parameters, The information is associated with additional data including:

[0063] In one embodiment, the metadata part is contained within the "meta" data box of the ISOBMFF. In one embodiment, the additional data is a header. In one embodiment, the additional data is a virtual item. In another embodiment, each transformation operator included in the image description information is associated with additional data including an identifier used to link the transformed item to said transformation operator. In one embodiment, a box stored in the metadata part contains at least one transformation operator.

[0064] The data part of the encapsulated bitstream contains the transformed items associated with one or more transformation operators, and the metadata part contains - information for identifying the original image to which the transformation operator is applied; and -information for locating the transformed item in the data part; and Further includes.

[0065] In one embodiment, the transformed item includes at least one transformation index, the index enabling identification of one of the transformation operators in the metadata part.

[0066] According to a twelfth embodiment of the present invention, there is provided a method for processing an encapsulated data file comprising a data part and a metadata part, the data part including an encoded bitstream corresponding to one or more images in the data part and information in the metadata part including image or sub-image description information including display parameters and / or transformation operators relating to one or more images or sub-images, the method comprising: - selecting an image or sub-image of interest; - identifying from said referenced image or sub-image description information relevant display parameters and / or transformation operators from the metadata part; - if a transformation operator is identified, applying a transformation to the image or sub-image and displaying the final transformed image or sub-image according to said display parameters; Includes.

[0067] In one embodiment, the method further comprises, prior to the identifying step, retrieving additional data contained within said parameters, said additional data comprising: -Type information, and / or - an identifier used to link the image or sub-image item information to the above parameters, Includes. In one embodiment, the metadata part is contained within the "meta" data box of the ISOBMFF. In one embodiment, the additional data is a header. In one embodiment, the additional data is a virtual item. In another embodiment, each transformation operator included in the image description information is associated with additional data including an identifier used to link the transformed item to said transformation operator. In one embodiment, a box stored in the metadata part contains at least one transformation operator. In one embodiment, the data part of the encapsulated bitstream includes transformed items associated with one or more transformation operators, and the metadata part includes: - information for identifying the original image to which the transformation operator is applied; and -information for locating the transformed item in the data part; and Further includes. In one embodiment, the transformed item includes at least one transformation index, the index enabling identification of one of the transformation operators in the metadata part.

[0068] According to a thirteenth aspect of the present invention there is provided a server device for encapsulating an encoded bitstream representing one or more images, the server device being configured to implement the encapsulation method according to the eleventh aspect of the present invention.

[0069] According to a fourteenth aspect of the present invention there is provided a client device encapsulating an encoded bitstream representing one or more images, the client device being configured to implement a processing method according to the twelfth aspect of the present invention.

[0070] According to a fifteenth aspect of the present invention there is provided a computer program and a computer program product comprising instructions for carrying out the methods according to the eleventh and twelfth aspects of the present invention when loaded and executed on a computing means of a programmable device.

[0071] According to a sixteenth aspect of the present invention there is provided a method of encapsulating an encoded bitstream representing one or more images, said encapsulated bitstream comprising a data part and a metadata part, the method comprising the steps of: providing image item information identifying portions of said data parts representing a single image and / or a sub-image or image of a set of a single image; providing image description information including parameters including display parameters and / or transformation operators associated with one or more images; outputting said bitstream together with said provided information as an encapsulated data file; A method is provided, comprising:

[0072] The image item information includes one or more characteristics including at least a portion of the image description information dedicated to the sub-image or single image or set of single images under consideration, the image description information being defined in one or more boxes.

[0073] This aspect of the invention makes it possible to provide a clear separation of data and metadata for an efficient referencing mechanism.

[0074] In one embodiment, the image item information is a box, each image item information property is a box, and the property boxes are organized to form a table of boxes.

[0075] In one embodiment, each property is applied to the sub-images, or the set of images and / or the single image by following the order of appearance or the corresponding box in the table of boxes.

[0076] In one embodiment, the sub-image or the single image and / or the set of single images are associated with a video sequence and one of the image item information properties comprises one or more initialization parameters that reference initialization information for the video sequence.

[0077] In one embodiment, the parts of the image description information shared among several sub-images or a single image and / or a set of single images are defined in one dedicated shared box, each image description information being retrievable via a structure for linking the considered image item information to at least one image description information, said link structure comprising: a first identifier for each image item information under consideration, said first identifier being defined as a property of said image item information and referencing a second identifier having the same value in said dedicated share box; one or several second identifiers contained in said dedicated shared box, each second identifier referring to one image description information; Includes.

[0078] In one embodiment, portions of image description information shared among several sub-images or a single image and / or set of single images are defined within two dedicated shared boxes, one shared box related to said display parameters and the other shared box related to said transformation operator, each image description information being searchable via a structure for linking said image item information to at least one image description information.

[0079] In one embodiment, the link structure includes two reference type parameters linking the image item information and at least one image description information, each reference type parameter being specific to one of the dedicated shared boxes.

[0080] In one embodiment, the link structure comprises: a first identifier and a second identifier for each image item information under consideration, the first identifier being defined as a property of the image item information and referencing a third identifier in the dedicated share box related to the display parameter, and the second identifier being defined as a property of the image item information and referencing a fourth identifier in the dedicated share box related to the transformation operator; one or several third and fourth identifiers respectively included in the dedicated shared boxes related to the display parameters and the transformation operator, each third and fourth identifier referring to a display parameter and a transformation operator, respectively; Equipped with.

[0081] In one embodiment, one of the display parameters is a grid for defining a set of a single image corresponding to a portion of a single image.

[0082] In one embodiment, the images from the single image set relate to the same single image.

[0083] According to a seventeenth aspect of the present invention there is provided a method of obtaining an encapsulated bitstream representing one or more images, said encapsulated bitstream including an encoded data part and a metadata part, the method comprising the steps of: - obtaining image item information identifying portions of said data part representing a single image and / or a sub-image or image of a set of a single image; obtaining image description information including parameters including display parameters and / or transformation operators associated with one or more images; extracting the bitstream along with the determined information as an encapsulated data file; A method is provided, comprising:

[0084] The image item information includes one or more characteristics including at least a portion of image description information dedicated to the considered sub-image or single image or set of single images, the image description information being defined in one or more boxes.

[0085] In one embodiment, the image item information is a box, each image item information property is a box, and the property boxes are organized to form a table of boxes.

[0086] In one embodiment, each property is applied to the sub-images, or the set of images and / or the single image by order of appearance or by following the corresponding box in the table of boxes.

[0087] In one embodiment, the sub-image or the single image and / or the set of single images is associated with a video sequence and one of the image item information properties comprises one or more initialization parameters that reference initialization information of the video sequence.

[0088] In one embodiment, the parts of the image description information shared among several sub-images or a single image and / or a set of single images are defined in one dedicated shared box, each image description information being retrievable via a structure for linking the considered image item information to at least one image description information, said link structure comprising: a first identifier for each image item information under consideration, said first identifier being defined as a property of said image item information and referencing a second identifier having the same value in said dedicated share box; one or several second identifiers contained in said dedicated shared box, each second identifier referring to one image description information; Includes.

[0089] In one embodiment, portions of image description information shared among several sub-images or a single image and / or set of single images are defined within two dedicated shared boxes, one shared box related to said display parameters and the other shared box related to said transformation operator, each image description information being searchable via a structure for linking said image item information to at least one image description information.

[0090] In one embodiment, the link structure includes two reference type parameters linking the image item information and at least one image description information, each reference type parameter being specific to one of the dedicated shared boxes.

[0091] In one embodiment, the link structure comprises: a first identifier and a second identifier for each image item information under consideration, the first identifier being defined as a property of the image item information and referencing a third identifier in the dedicated share box related to the display parameter, and the second identifier being defined as a property of the image item information and referencing a fourth identifier in the dedicated share box related to the transformation operator; one or several third and fourth identifiers respectively included in the dedicated shared boxes related to the display parameters and the transformation operator, each third and fourth identifier referring to a display parameter and a transformation operator, respectively; Equipped with. In one embodiment, one of the display parameters is a grid for defining a set of a single image corresponding to a portion of a single image. In one embodiment, the images from the single image set relate to the same single image.

[0092] According to an eighteenth aspect of the present invention there is provided a device for encapsulating an encoded bitstream representing one or more images, the device being configured to perform a method according to the sixteenth aspect of the present invention.

[0093] According to a nineteenth aspect of the present invention there is provided a device for processing an encapsulated bitstream representing one or more images, the device being configured to perform a method according to the seventeenth aspect of the present invention.

[0094] According to a twentieth aspect of the present invention, there is provided a first device according to the eighteenth aspect of the present invention; a second device according to the nineteenth aspect of the invention for processing files from the first device; A system is provided, comprising:

[0095] According to a twenty-first aspect of the present invention there is provided a computer program product comprising instructions for performing a method according to the sixteenth or seventeenth aspect of the present invention when the program is loaded and executed by a programmable device.

[0096] According to a twenty-second aspect of the present invention there is provided a non-transitory information storage means readable by a computer or microprocessor storing instructions for a computer program for performing a method according to the sixteenth or seventeenth aspect of the present invention, when said program is loaded and executed by a computer or microprocessor.

[0097] According to a twenty-third aspect of the present invention there is provided a method of encapsulating an encoded bitstream representing one or more images, said encapsulated bitstream comprising a data part and a metadata part, the method comprising the steps of: providing image item information identifying portions of said data parts representing a single image and / or a sub-image or image of a set of a single image; providing image description information including parameters including display parameters and / or transformation operators associated with one or more images; outputting said bitstream together with said provided information as an encapsulated data file; A method is provided, comprising:

[0098] The image description information is defined within one or two dedicated boxes, each image description information being retrievable via a structure for linking the image item information to at least one image description information.

[0099] In one embodiment, the image description information is defined in one dedicated box, and the link structure includes a reference type parameter linking the image item information and at least one image description information.

[0100] In one embodiment, the image description information is defined within one or two dedicated boxes, and the link structure includes one or two sets of indexes for linking the image item information with at least one image description information, each set being associated with one of the dedicated boxes.

[0101] In one embodiment, the image description information is defined in two dedicated boxes, one box relating to the display parameters and the other box relating to the transformation operators.

[0102] In one embodiment, the image description information is defined within two dedicated boxes, and the link structure includes two reference type parameters associated with respective ones of the two dedicated boxes, each reference type parameter linking the image item information with at least one image description information in the associated dedicated box.

[0103] According to a twenty-fourth aspect of the present invention there is provided a method of obtaining an encapsulated bitstream representing one or more images, said encapsulated bitstream including an encoded data part and a metadata part, the method comprising the steps of: obtaining image item information identifying a portion of said data part representing a single image and / or a sub-image or image of a set of a single image; obtaining image description information including parameters including display parameters and / or transformation operators associated with one or more images; extracting the bitstream along with the determined information as an encapsulated data file; Including, A method is provided in which the image description information is defined within one or two dedicated boxes, and each image description information is searchable via a structure for linking the image item information to at least one image description information.

[0104] In one embodiment, the image description information is defined in one dedicated box, and the link structure includes a reference type parameter linking the image item information and at least one image description information.

[0105] In one embodiment, the image description information is defined within one or two dedicated boxes, and the link structure includes one or two sets of indexes for linking the image item information with at least one image description information, each set being associated with one of the dedicated boxes.

[0106] In one embodiment, the image description information is defined in two dedicated boxes, one box relating to the display parameters and the other box relating to the transformation operators.

[0107] In one embodiment, the image description information is defined within two dedicated boxes, and the link structure includes two reference type parameters associated with respective ones of the two dedicated boxes, each reference type parameter linking the image item information with at least one image description information in the associated dedicated box.

[0108] According to a twenty-fifth aspect of the present invention there is provided a device for encapsulating an encoded bitstream representing one or more images, the device being configured to perform a method according to the twenty-third aspect of the present invention.

[0109] According to a twenty-sixth aspect of the present invention there is provided a device for processing an encapsulated bitstream representing one or more images, the device being configured to perform a method according to the twenty-fourth aspect of the present invention.

[0110] According to a 27th aspect of the present invention, there is provided a first device according to the 25th aspect of the present invention; a second device according to the 26th aspect of the invention for processing files from the first device; A system is provided, comprising:

[0111] According to a 28th aspect of the present invention there is provided a computer program product comprising instructions for performing a method according to the 23rd or 24th aspect of the present invention when the program is loaded and executed by a programmable device.

[0112] According to a twenty-ninth aspect of the present invention there is provided a non-transitory information storage means readable by a computer or microprocessor storing instructions for a computer program for performing a method according to the twenty-third or twenty-fourth aspect of the present invention, when said program is loaded and executed by a computer or microprocessor.

[0113] Other features and advantages of the invention will become apparent from the following description of non-limiting exemplary embodiments, which refers to the attached drawings in addition to FIG. [Brief description of the drawings]

[0114] [Figure 1] FIG. 2 illustrates a description of a tile-encoded still image within the "Meta" box of the ISO Base Media File Format. [Diagram 2] FIG. 2 illustrates an example of tiled video. [Diagram 3] A diagram showing various tile / slice configurations in HEVC. [Figure 4]FIG. 1 illustrates tile encapsulation according to the ISO Base Media File Format with "Track" boxes. [Diagram 5] FIG. 1 illustrates standard metadata for describing information items in the ISOBMFF “meta” box. [Figure 6] FIG. 1 illustrates an exemplary extension to an information item description. [Figure 7] FIG. 2 illustrates a reference mechanism between information items. [Figure 8] FIG. 1 illustrates the context of implementation of an embodiment of the present invention. [Figure 9] 1 is a schematic block diagram of a computing device for implementing one or more embodiments of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0115] In the following, embodiments of the present invention are described. To better understand the technical situation, video tiling is explained with reference to Fig. 2, which shows a video 200 with consecutive time frames. Each frame 201 is divided into 8 parts (here rectangular parts) called "tiles" T1 to T8. The number and shape of the tiles can be different. In the following, for any index of a video frame, the tiling is considered to be the same.

[0116] The result of this tiling are eight independent sub-videos 202, which represent a section of the entire global video. Each independent sub-video can be coded as an independent bitstream, for example according to the AVC or HEVC standard. The sub-videos can also be part of the bitstream of a single video, for example as tiles in the HEVC standard or slices in the AVC standard.

[0117] The HEVC standard defines different spatial subdivisions of a picture: tiles, slices and slice segments. These different subdivisions (or partitions) have been introduced for different purposes: slices are related to streaming problems, while tiles and slice segments are defined for parallel processing.

[0118] A tile defines a rectangular region of a picture that contains an integer number of Coding Tree Units (CTUs). Figure 3 shows a tiling of an image 300 defined by row and column boundaries 301, 302. This makes tiles good candidates for region of interest description in terms of location and size. However, the HEVC standard bitstream organization in terms of syntax and its encapsulation into Network Abstraction Layer (NAL) units is rather based on slices (similar to the AVC standard).

[0119] According to the HEVC standard, a slice is a set of slice segments, where at least the first slice segment is an independent slice segment, and the other slice segments, if present, are dependent slice segments. A slice segment contains an integer number of consecutive CTUs (in raster scan order). A slice segment does not necessarily have a rectangular shape (and is therefore less suitable than a tile for representing an area of ​​interest). A slice segment is coded in the HEVC bitstream as a header called "slice_segment_header" followed by data called "slice_segment_data". Independent and dependent slice segments differ by their headers: dependent slice segments have shorter headers since they reuse information from the independent slice segment's header. Both independent and dependent slice segments contain a list of entry points in the bitstream to tiles or to an entropy decoding synchronization point.

[0120] 3 shows different configurations of slices, slice segments and tiles in images 310 and 320. These configurations are different from that of image 300, where one tile has one slice (containing only one independent slice segment). Image 310 is partitioned into two vertical tiles (311, 312) and one slice (having five slice segments). Image 320 is divided into two tiles (321, 322), with the left tile 321 having two slices (each having two slices) and the right tile 322 having one slice (having two slice segments). The HEVC standard specifies organization rules between tiles and slice segments, which can be summarized as follows (either one or both conditions must be satisfied): -All CTUs in a slice segment belong to the same tile. -All CTUs in a tile belong to the same slice segment.

[0121] To have consistent region of interest support and transport, the configuration 300 where one tile contains one slice with one independent segment is preferred, however, the encapsulation solution will also work with the other configurations 310 or 320.

[0122] While tiles are suitable supports for regions of interest, slice segments are the entities that will actually be put into NAL units for transport over the network and aggregated to form access units (coded pictures or samples at the file format level). According to the HEVC standard, the type of NAL unit is specified in the NAL unit header. For a NAL unit of type "coded slice segment", slice_segment_header indicates the address of the first coding tree block in the slice segment via the "slice_segment_address" syntax element. The tiling information is given in the PPS (Picture Parameter Set) NAL unit. The relationship between slice segments and tiles can then be inferred from these parameters.

[0123] By definition, at tile boundaries, spatial prediction is reset. However, nothing prevents a tile from using a temporal predictor from a different tile in the reference frame(s). To construct independent tiles, at encoding time, the motion vectors for the prediction units in a tile are constrained to stay in the tile located at the same location in the reference frame(s). Furthermore, at tile boundaries, the in-loop filters (deblocking and SAO) must be turned off so that no error drift is introduced when decoding a single tile. This control of the in-loop filters is already available in the HEVC standard and is set in the slice segment header with a flag called "loop_filter_across_tiles_enabled_flag". By explicitly setting this flag to 0, pixels at the tile boundaries do not depend on pixels located at the boundaries of adjacent tiles. A tile is said to be "independently decodable" or "independent" when two conditions regarding motion vectors and in-loop filters are met.

[0124] When a video sequence is coded as a set of independent tiles, it can be decoded frame-by-frame using tile-based decoding without risking losing reference data or propagating reconstruction errors. This configuration makes it possible, for example, to reconstruct only the spatial part of the original video that corresponds to the region of interest.

[0125] In the following, independent tiles are considered. With reference to Figure 4, the encapsulation of tiles into the ISOBMFF file format is described. For example, each tile is encapsulated in a dedicated track. Common setup and initialization information for all tiles is encapsulated in a specific track, for example called "tile base track". In this way, the full video is encapsulated as a composition of all these tracks, i.e., a set of tile base tracks and tile tracks.

[0126] Figure 4 shows an example encapsulation. One way to encapsulate tiled video according to the ISOBMFF standard is to split each tile into a dedicated track, encapsulate the setup and initialization information common to all tiles in a specific track, e.g., called "tile base track", and encapsulate the full video as the composition of all these tracks, i.e., the set of tile base track + tile track. Therefore, the encapsulation is called "multi-track tile encapsulation". An example of multi-track tile encapsulation is given in Figure 4.

[0127] Box 401 represents the main ISOBMFF box "moov" and contains the full list of tracks together with their identifiers. For example, boxes 411-414 represent tile tracks (four tiles in this example) and box 420 represents a tile-based track. Additional tracks such as audio or text tracks may be used and may be encapsulated within the same file. However, for the sake of brevity, such additional tracks are not discussed here.

[0128] As represented in FIG. 4, the tile data is divided into independent and addressable tracks so that any combination of tile track(s) can be easily reconstructed from the tile base tracks that reference the tile tracks for decoding and display. The tile base tracks are also sometimes called "composite tracks" or "reference tracks" because they are designed to be able to combine any tile, i.e. one tile, many tiles, or all tiles. The tile base track 420 contains information common to all tile tracks in the "mdat" box, as well as a list of samples 450 (only the first sample is represented in FIG. 4). Each sample 450 of the tile base track 420 is constructed by referencing each tile track through the use of extractors (451-454, each representing one extractor to each tile). Each tile track 411-414 represents a spatial portion of the entire video, i.e. a full frame. The tile description (position, size, bandwidth, etc.) is stored in the track header box (not represented) of each tile track 411-414. The tile base track and each tile track are cross-referenced (405) using a box "TrackReferenceBox" in each track. Each tile track 411-414 refers to the base track 420 as a "tbas" track ("tbas" is a specific code indicating the coding dependency from each tile track to the tile base track, in particular where to find the parameter "HEVCDecoderConfigurationRecord" that allows to set up the video decoder that will process the elementary stream resulting from the file format parsing). Conversely, to allow full video reconstruction, the tile base track 420 indicates a dependency of type "scal" to each tile track (405). This is an indication of the coding dependency and reflects the definition of the tile base track samples 450 as extractors to the tile track data. These extractors are specific extractors that can support the absence of data when parsing.In Fig. 4, to provide a streamable version of the file, each track is decomposed into media segments (431-434 for tile tracks and 460 for tile-based tracks). Each media segment contains one or more movie fragments, indicated by the "moof" box + data. For tile tracks, the data part corresponds to a spatial subpart of the video, while for tile-based tracks it contains a set of parameters, a SEI message when present, and a list of extractors. The "moov" box 401 will fit in the initialization segment for streaming applications. Fig. 4 shows only one segment, but a track can be decomposed into any number of segments, the constraint being that the segments for tile tracks and the segments for tile-based tracks follow the same temporal decomposition (i.e. they are aligned in time), which allows switching from the full video to a tile or set of tiles. The granularity of this temporal decomposition is not described here for the sake of brevity.

[0129] The file format has descriptive metadata (e.g., "VisualSampleGroupEntries," or track reference types in the "tref" box) that describes the relationships between tracks so that data corresponding to a single tile, a combination of tiles, or all tiles can be easily identified by parsing the descriptive metadata.

[0130] In the following, still images are described at the same level. Thus, any tile, combination of tiles or all tiles of a picture can be easily identified and extracted when the user selects them. If a picture is mixed with video data, the description comes in parallel with the description metadata for the video. Thus, for the same data set, an additional indexing layer for the picture is provided (in addition to the indexing layers for video and audio).

[0131] In still image file formats using the "meta" box, pictures are described as information items together with related information. As shown in Figure 5, the information items are listed in a dedicated sub-box "ItemInfoBox" 500 of the "meta" box. This sub-box gives the number of information items present in the file. For each item, the sub-box also gives descriptive metadata represented as "ItemInfoEntry" 501. With the evolution of the ISOBMFF standard, there are several versions 502 of this box (0, 1, 2, 3).

[0132] "Meta" items may not be stored contiguously within a file. Also, there are no specific constraints on the interleaving of item data. Thus, two items within the same file may share one or several data blocks. This is especially useful in the case of HEVC tiles (tiles can be stored contiguously or non-contiguously) since it can be simple to have one item per independently decodable tile. This item indicates the data offset within the main HEVC picture and the length of the slice(s) used for the tile through an ItemLocationBox.

[0133] According to an embodiment, a new item type for describing tiled pictures may be added or reused from ISO / IEC 14496-15: "hvt1", for example called "hvct" or "tile". Each item representing a tiled picture (whatever 4-letter code is chosen) may have a reference of type "tbas" to an "hvc1" item from which it is derived. Each item has an identifier "item_ID" 503 and is further described in the box "ItemLocationBox" in terms of its byte location and size in the Media Data Box, which contains the compressed data for the picture.

[0134] Such a syntax allows a file format reader (or “parser”) to determine, through a list of information items, how many information items are available, along with information about their type 504, e.g., “tile” indicating that the information item is a tiled picture of a full picture.

[0135] It is therefore possible to select a subset, combination or the full set of information items in a file to download only one tile of an image and associated decoder configuration while skipping other tiles.

[0136] If an HEVC tile depends on another HEVC tile for decoding, that dependency should be indicated by an item reference of type "dpnd" (or any specific 4-letter code indicating encoding dependency) as described in document w14123, WD "Enhanced carriage of HEVC and support of MVC with depth information" of ISO / IEC 14496-15:2013 AMD 1, MPEG 107 San Jose January 2014.

[0137] This specification specifies tools for associating HEVC tile NALUs with sample group descriptions (using the "TileRegionGroupEntry" descriptor) that indicate the spatial location of the tiles. However, there is no direct equivalent of sample grouping for metadata information items that would allow these descriptors to be reused.

[0138] Therefore, according to an embodiment, a tile description item is defined for each tile and the tile is linked to its description using a modified version of the "ItemReferenceBox" box, as described below.

[0139] According to another embodiment, preferably in a general way, only one tiling description is given, so the item list does not become too long.

[0140] This design can be done as follows: -Allows some items to describe a set of metadata that is similar to a sample group, but that is specific to each item type. - For any item, add the ability to specify one parameter for a given type of item reference. The parameter will then be interpreted depending on the type of the referenced item (similar to a grouping type).

[0141] As will be explained below with reference to FIG. 6, an upgrade of the descriptive metadata for an information item may be required.

[0142] According to the ISOBMFF standard, the sample grouping mechanism is based on two main boxes with the "grouping_type" parameter as follows: - The box "SampleGroupDescriptionBox" has a parameter "sgpd" that specifies a list of properties (a list of "SampleGroupEntry"). - The box "SampleToGroupBox" has a parameter "sbgp" that specifies a list of sample groups with their mapping to properties.

[0143] The "grouping_type" parameter links a list of sample groups to a list of properties, and the mapping of a sample group to one property in the list is specified in the box "SampleToGroupBox".

[0144] To provide the same functionality for information items, a list of information item groups and a list of characteristics must be described, and it should be possible to map each group of information items to one characteristic.

[0145] In the following, a method is described that allows such descriptive metadata to be embedded in a still image file format, in other words a method is described that links the descriptor to an image item. Even though the use case is described for the HEVC still image file format, the following features can be used in other standards such as ISO / IEC 14496-12 to associate any kind of information item with additional descriptive metadata.

[0146] According to an embodiment, as shown in Fig. 6, the existing "ItemInformationEntry" box 601 with the parameter "infe" is extended with new version numbers (602 and 603) to link each item to one characteristic through a new parameter called "iref_type" 604. This allows to avoid the creation of new boxes and to improve the description while keeping it short.

[0147] The original specification of an ItemInformationEntry box is given by:

number

[0148] A new version creation that links a tile picture to its description can be as follows:

number

[0149] According to another embodiment, which is closer to the box "SampleToGroupBox", the definition of the box "ItemInformationBox" with the four-letter code "iinf" is modified, for example, by introducing a new version of this box as follows: Current version:

number

number

[0150] Alternatively, to tell if a group is in use or not, the current version is:

number

[0151] The "group_entry_count" parameter specifies the number of information item groups in the media file. For each group of information items, the number of information items is indicated, starting with item_ID=0. In contrast to samples, the encapsulation module can assign information item identifiers in any order, since the information items have no time constraints and relationships. By assigning increasing identifier numbers after each item group, the list of information groups can be represented more efficiently using the parameter "item_run", which identifies the run of consecutive information item identifiers within a group.

[0152] Related information items have an index, for example called "property_index". This "property_index" parameter, associated with the "grouping_type" parameter, allows a file format parser (or "reader") to identify a reference to the descriptive metadata, or the descriptive metadata itself. Figure 7 shows two exemplary embodiments.

[0153] The group feature in the box "SingleItemTypeReferenceBox" 701 can be used with the group identification "group_ID" instead of the information item identification (item_ID) commonly used for the value of the from_item_ID parameter. By design, the box "SingleItemTypeReferenceBox" makes it easier to find all references of a particular type or a particular item. Its use with "group_ID" instead of "item_ID" allows to find a group of items to easily identify all references of a particular type. Advantageously, there is at most one box "ItemInformationBox" per encapsulated file, so there is no need to specify a group identification. The encapsulation module (during encoding) and the parsing module (during decoding) can perform a respective counter on the list of information item groups (as the "g" variable in the box "ItemInformationBox") as they are created or retrieved. Alternatively, the parser may be informed by the flag "group_used_flag" whether to maintain a group identification counter or not.

[0154] Returning to the example where a group of information items corresponds to a tile picture, the group may contain four entries and reference 700 “SingleItemTypeReference” may point to a list 704 of information items on which the four tile picture information items depend and for a particular reference type 703.

[0155] According to another exemplary embodiment, information items are used in a new kind of box, "ItemReferenceBox", as described later, which allows listing multiple reference types 723 from one item 722 to various other information items 724.

[0156] In the latter case, a particular box "ItemReferenceBox" 721 might be realized as follows:

number

[0157] As with the standard box "ItemInformationBox", a list of item entries is written, but now with a different order depending on the grouping. In the tile example, this might result in a first group of four information items corresponding to the tile pictures collected in one group with a parameter that might be called "tile", followed by ungrouped information items for composition information, full picture information items, and optionally EXIF ​​metadata.

[0158] Thus, one box is modified and one box is created, which is a particular kind of ItemReferenceBox. Below, this new kind of ItemReferenceBox is described.

[0159] The box 'ItemReferenceBox' can also be extended by distinguishing between different kinds of ItemReferenceBox through the use of flags parameters in a box 'FullBox' that is part of the ItemReferenceBox, as follows:

number

[0160] Using the box "MultipleItemTypeReferenceBox" 721, one picture with four tiles can be described as follows:

number

[0161] This design makes it extremely easy to find all references of any kind from a particular item.

[0162] The description support 711 for a list of items 712 that refer to the same item 714 with a given type 713 may be as follows:

number

[0163] For an example picture with 4 tiles, it can be as follows:

number

[0164] The design of box 'SharedItemTypeReferenceBox' makes it easier to find all references of a particular type that point to a particular item. This is in contrast to box 'SingleItemTypeReferenceBox'. However, since most of the 'reference_types' defined for track references are not bidirectional, box 'SingleItemTypeReferenceBox' may not be used with some unidirectional reference types to inform all nodes that have this reference type to other items. Alternatively, a flag may be provided in 'SingleItemTypeReference' to indicate whether it is a direct or reverse reference, thereby alleviating the need for a new SharedItemTypeReferenceBox.

[0165] In view of the above, information items can be associated with tiling information, a description of which must now be provided.

[0166] For example, each tile can be described using a tile descriptor, such as the "iref_parameter" 605 of the extended "ItemInfoEntry" 601. The specific descriptor can be as follows:

number

[0167] According to an embodiment, a descriptor for a grid of tiles may be used to apply to one or more pictures that are to be stored.

[0168] Such a descriptor may be as follows:

number

[0169] This descriptor, "TileInfoDataItem", allows describing a tiling grid (regular or irregular). The grid is described row by row starting from the top left.

[0170] The descriptor should be stored as an item of type "tile". When another item references this item, it should use a reference of type "tile" to this description and have the parameter "iref_parameter" specified, whose value is the 0-based index of the cell in the grid specified by the descriptor, where 0 is the top-left item, 1 is the cell immediately to the right of cell 0, and so on.

[0171] In this descriptor: - "version" indicates the version of the syntax for the TileInfoDataItem. Only the value 0 is defined. - "regular_spacing" indicates whether all tiles in the grid have the same width and height. - "reference_width, reference_height" indicate the units in which the grid is written. These units may or may not match the pixel resolution of the image that references this item. If the grid is regular, then "reference_width" (or "reference_height") should be a multiple of "nb_cell_horiz" (or "nb_cell_vert"). - "cell_width" gives the horizontal division of the grid within the irregular tile, starting from the left. - "cell_height" gives the vertical division of the grid within the irregular tile, starting from the top. The above approach allows sharing of tiling information for all tiles.

[0172] Furthermore, if there are multiple pictures that share the same tiling, then even more descriptions may be shared by simply referencing cells in the tile's grid.

[0173] The tiling configuration can be placed in a media data box or in a dedicated box that is shared (by reference) between tile information items.

[0174] The above descriptors are pure spatial descriptors in the sense that they give only the spatial location and size for the sub-image(s) within a larger image. In some use cases, e.g. image acquisition or image composition, the spatial location is not sufficient to describe the image, as they typically overlap. This is one limitation of the above TileInfoDataBlock descriptor. To be able to compose any image, i.e. tiles or independent / complete images, it may be useful to specify a descriptor that contains, on the one hand, the image location and size (spatial relations) and, on the other hand, the display information for that picture (color, cropping...). For example, color information can be given to transform the sub-image from one color space to another for display. This kind of information can be carried in the ColorInformationBox "colr" of the ISOBMFF. For compactness, it may be useful to have the same data prepared for different kinds of display, by giving only the transformation parameters to be applied, instead of carrying two different such transformed pictures. Furthermore, in this descriptor, a pixel aspect ratio can be placed, such as the PixelAspectRatio box "pasp" defined in ISOBMFF part 12, to redefine the width and height of each picture, which may be different from the coded width and height. This will indicate the scale ratio to be applied by the display after decoding of the image, with the coded size stored in the video sample entry (e.g. the "stsd" box) and the display size inferred from the "pasp" box. Another possible information for the display can be the clean aperture information box "clap", also defined in ISOBMFF. According to the standard SMPTE 274M, the clean aperture defines an area where the picture information is essentially not contaminated by any edge transient distortions (possible ringing effects at the borders of the image after analog / digital conversion).This list of parameters useful for display is not limiting and any other descriptive metadata box can be placed as an optional component in the sub-image descriptor. These parameters can be mentioned explicitly since they are already part of the standard and provide a generic tool to indicate image cropping, sample aspect ratio changes and color adjustments. Unfortunately, their use was only possible in the case of media tracks and not possible in the case of image file formats that rely on "meta" boxes. Therefore, we propose a new descriptor, called for example "SimpleImageMetaData", to support the spatial description of image items, together with other characteristics such as clean aperture or sample aspect ratio. This applies to any sub-image (tile or independent image) that is intended to be composed within a larger image or, conversely, extracted from a larger image:

number

number

[0175] Here, ImageSpatialRelationBox is an extension of TileInfoDataBlock as explained below. Another useful parameter to consider is the possibility of structuring images as layers. Therefore, we propose to insert a parameter indicating the level associated with the image in such a layered configuration. This is usually useful when images overlap. It can be called for example "layer" with a layer information indication. An exemplary syntax for such a descriptor is given. Definition: Box type: "isre" Container: Simple Image Metadata Item ("simd") Required:Non-required Quantity: 0 or 1 per item syntax:

number

[0176] This new "isre" box type provides the ability to describe the relative position of one image to other images in an image collection. It provides a subset of the functionality of the transformation matrices commonly found in track header boxes of movies or media files. Coordinates within an ImageSpatialRelationBox are expressed on a square grid that gives the intended display size of the collection's author. These units may or may not match the encoded size of the images. The intended display size is defined by: - horizontal: the maximum of (horizontal_display_offset+display_width) for all 'isre' boxes. -Vertical: The maximum of (vertical_display_offset+display_height) for all 'isre' boxes.

[0177] When some images do not have any "isre" associated with them while other images in the file have an "isre" associated with them, the default image without any "isre" will be treated as if its horizontal and vertical offsets are 0, its display size is the intended display size, and its layer is 0.

[0178] ImageSpatialRelationBox indicates the relative spatial position of the image after any cropping or sample aspect ratio has been applied to it. This means that when "isre" is combined with "pasp" etc. in SimpleImageMetaData, the image is decoded, "pasp", "clap", "colr" are applied if present, and then the image is moved and scaled by the offsets and sizes declared in the "isre" box.

[0179] This new Descriptor can be used as a description of an image (tile or single image) by defining an association between Item Information representing an image and Item Information representing a Descriptor (we give the type "simd" for SimpleImageMetadata Definition, allowing any inverted 4-letter code to be acceptable for an mp4 parser to easily identify the type of metadata it is currently processing). This association is done using the ItemReferenceBox and a new reference type; "simr" indicating "spatial image relationship". The following exemplary illustration shows the case of a composition of four images, which itself has no associated items. Each image item is associated to a SimpleImageMetaData item through an item reference of type "simr", and shares the DecoderConfigurationRecord information in a dedicated "hvcC" item.

number

[0180] The above organization of data is given as an example. For example, an image and metadata can be combined in a media data box to have an image + its metadata addressable as a single byte range. When receiving this description, the parser is informed by parsing the information in the "simd" item whether the sub-image is cropped from a full picture or vice versa whether the full picture is constructed from a sub-image. In case of a crop, the full picture item and the cropped image will share the same data range and the same decoder configuration information as in the following example. The sub-image will then be associated with a "simd" item that only has "clap" information and no positioning and therefore no "isre".

[0181] Compositional case: In such a case, the full picture item will be associated with a "simd" item that contains only the "isre" information, and the sub-picture will be associated with a "simd" item that reflects its position in the full picture.

[0182] The following example shows four images composed into one larger image. All images, including the composed image, are listed as playable items using the proposed descriptors.

number

[0183] This other example shows the case where a full picture is actually a tiled HEVC picture (4 tiles).

number

[0184] Depending on the use case, you could have several image items that share the same metadata, for example when the same cropping is to be applied to all images. You could also have an image item have multiple "simr" references to different SimpleImageMetaData, for example when cropping is shared between images but not spatial information.

[0185] An alternative embodiment for the new version of ItemInfoEntry (shown in FIG. 6) is to specify more than one parameter 605 per information item entry and reference. In the embodiment of FIG. 6, the iref_parameter is a 4 byte code, which is useful in the case of a tile index to reference a cell in the tiling grid. However, to have a richer description, and to be able to embed linked descriptions inside the item info entry itself, rather than with the data (in the mdat box), the following extensions may be useful:

number

[0186] In the above expansion, -item_iref_parameter_count gives the number of reference types for which parameters are given. This is unchanged for item 605 in Figure 6. -iref_type gives the reference type, as shown in the "iref" box, whose parameters apply for this item. This is unchanged for item 605 in Figure 6. The -parameter is different from the iref_parameter (item 605 in Figure 6) because we now provide an extension mechanism via a new box, ItemReferenceParameterEntry. By specializing this new box (as was done above with TileIndexItemReferenceParameterEntry for tile indexes in a tiling configuration), any kind of additional metadata can be associated with the information item entry, provided that the encapsulation and parsing module knows the structure of this specialized box. This can be done by the standard type of ItemReferenceParameterEntry, or by giving the structure of the parameter entry, either by configuration or in a negotiation step. The meaning of the parameter is given by the meaning of the item with type iref_type.

[0187] In the following, exemplary descriptive metadata for an information item describing a picture having four tiles and the EXIF ​​metadata of the full picture are given.

[0188] In the prior art, tile pictures were listed as information items without any corresponding description being given, as shown below in this specification. Furthermore, the setup information, represented by the "hvcC" type, was not described as an item. This makes it possible to factorize common data related to HEVC parameter sets and SEI messages that apply to all tile pictures and full pictures.

number

[0189] According to an embodiment using an extension with version 4 of the ItemInfoEntry box 601 (see Figure 6, 602, 603), the tile picture information is listed with associated references to parts of the tiling configuration, also described as information items (ID = 8).

number

[0190] Figure 8 shows the situation of implementation of an embodiment of the invention. First different media are recorded, i.e. for example audio in step 800a, video in step 800b and one or more pictures in step 800c. Each media is compressed during respective steps 801a, 801b and 801c. During these compression steps elementary streams 802a, 802b and 802c are generated. Then, at the application level (user selection from a graphical user interface, configuration of a multimedia production system, etc.), an encapsulation mode is selected to determine whether all these elementary streams should be fused or not. If the "fused" mode is activated (test 803, "yes"), during step 806c, the data for audio, video and still images are encapsulated in the same file, as explained herein above. If the "Merge" mode is not activated (test 803, "no"), two encapsulated files are generated during steps 806a and 806b, either sequentially or simultaneously, resulting in one file for synchronized time media data during step 807a and an additional file with only still images 807b, respectively. To provide the tile description and the region of interest features, the audio and video elementary streams are encapsulated according to the ISOBMFF standard during step 806a, and the still pictures are encapsulated during step 806b, as described herein above. Finally, the media presentation 807 is obtained, either as a whole or after some parts (such as tiles) have been extracted by parsing the description metadata, and can be given to a DASH generator to prepare it for streaming (step 820a), stored in memory (step 820b), rendered on a display unit (step 820c) or sent to a remote entity (step 820d).

[0191] It should be noted that according to the preceding description of the embodiment, descriptive metadata, such as for example SimpleImageMetadata ("simd") boxes (also called ISOBMFFMetaData in the final version of the still image file format specification) are described as items with all attributes. Additional descriptive or normative metadata are also specified by the still image file format specification, as described in document w14878, committee draft paper "Information technology - MPEG systems technologies - Part 12: Image File Format" of ISO / IEC23008-12:2013 1st Edition, MPEG 110 Strasbourg October 2014. Examples of descriptive or normative metadata are CleanApertureBox ("clap"), ImageRotation ("irot"), ExifDataBlock ("exif") or ImageOverlay ("iovl"). More generally, descriptive metadata is metadata that provides additional information or a description for an item such as an image or sub-image (e.g., Exif metadata), and normative metadata is an operation or transformation that should be applied to the item (e.g., a rotation, crop, or a combination of several items that form a transformation operator).

[0192] However, having to store such descriptive or normative metadata as items with all their attributes in the specification can seem quite cumbersome. These are just pseudo-items, and the descriptive or normative metadata needs to be stored together with the encoded data in the mdat box 110, and entries in itemLocationBox (iloc) 109, itemInfoBox (iinf) and itemProtectionBox (ipro) need to be defined. Requiring these entries in iloc, iinf and ipro for this is a significant overhead. For example, an entry in itemInfoBox requires the use of a full box with at least a 12-byte header, and in addition, an item_protection_index (16 bits) + an empty item_name (8 bits) must be defined for a total additional consumption of 15 bytes per entry in itemInfoBox (iinf). An entry in itemLocationBox (iloc) also requires at least 9 bytes in the better case (base_offset_size=offset_size=length_size=1,1extent). In practice, itemLocationBox entries are used with base_offset_size=offset_size=length_size=2 or 4, which means an additional consumption of 12 or 18 bytes. Furthermore, this metadata is generally small, allowing to read other items efficiently. Storing them as dedicated items may complicate file parsing, especially partial fetching of files (e.g. multiplying HTTP requests).

[0193] In an alternative embodiment, all descriptive and normative metadata can be defined as embedded items, which can be stored as part of other boxes in the meta box 100 rather than in the mdat box 110, thus avoiding the additional expenditure of defining itemInfoBox and itemLocationBox entries.

[0194] To store the descriptive and normative metadata in the meta box, a virtual item box called "VirtualItemBox" is defined. According to this embodiment, all descriptive and normative metadata boxes are inherited from this virtual item class.

[0195] A virtual item has a set of boxes, as well as an item_ID and an item_type assigned to it. A virtual item is additional data that is typically used to describe metadata that is to be associated with other items. For example, a virtual item allows to associate an entry in an itemInfoBox that identifies an item (image or subimage) with an operation or transformation that is to be applied to this item. Typically, this association can be described by specifying an entry of type "simr" in an itemReferenceBox from the item_ID of the image to the item_ID of the metadata operation or transformation description box. Virtual items may only be referenced in item reference boxes and primary item boxes and should not be declared or referenced in any other boxes (e.g. itemLocationBox (iloc), itemInfoBox (iinf), itemProtectionBox (ipro)). The "VirtualItemBox" is defined as follows:

number

[0196] Optionally, in one variant, the "VirtualItemBox" can also contain an additional parameter called "descriptor_family". The descriptor family indicates whether the metadata box is descriptive or normative metadata. In one variant, the descriptor family indicates the type of the metadata box from a predefined list of values. For example: transform_operator, composed_image, descriptive_metadata...

[0197] By inheriting from this virtual Item Box, all descriptive and normative metadata boxes can be stored in the MetaBox without the need to specify associated entries in itemInfoBox (iinf) and itemLocationBox (iloc), yet retain the advantage of being addressable by the Item Reference Box.

[0198] According to this embodiment, ImageOverlay (iovl), SubSampleItemData (subs), AuxiliaryConfiguration (auxC), ExifDataBlock (exif), SimpleImageMetadata (simd) and derived image items inherit from the VirtualItem class.

[0199] Nevertheless, according to this embodiment, a single generic item type called "dimg" is introduced, with item references of type "simr" to items of type "simd". This approach allows for reuse of properties when appropriate, and reduces the number of items and item references. An ImageRotationBox is added inside SimpleImageMetadata (simd). The "simr" reference type defines a link from an image item to a "simd" item to provide direct access to the image description metadata.

[0200] Additionally, the ImageOverlay (iovl) metadata box is redesigned as follows so that it is no longer dependent on reference order:

number

[0201] An explicit item_id is added to each entry in the loop to explicitly identify the item being constructed.

[0202] In an alternative embodiment, all boxes contained in the SimpleImageMetadata (simd) are defined as independent metadata boxes inherited from the Virtual Item box.

[0203] In an alternative embodiment, a simple image rotation can be declared by incorporating the rotation operation directly within the image metadata descriptor SimpleImageMetadata ("simd") box (also called ISOBMFFMetaData in the final version of the Still Image File Format specification) as follows:

number

[0204] Although a rotated box is slightly larger than an 'irot' item (12 bytes), the advantage of using this technique is apparent when combining transformations, such as rotation and clean aperture, since only one 'simd' is required rather than a series of derived items.

[0205] In such cases, the generic derived item "dimg" (above) can be used to refer to both the image item and the metadata description. Such items can then be listed as primary items in a PrimaryItemBox ("pitm").

[0206] Another advantage of this approach is that authors can explicitly indicate that they want only rotated items to be displayed.

[0207] The following paragraphs propose an alternative to the above embodiment, which is advantageously simpler in terms of how transformations (or "effects") can be applied to images in ISO still image file format. In particular, the following problems are solved by this alternative embodiment: -A large number of item references. -The number of items increases when chaining effects together. -The effect for a given set of items, meaning a set of images or parts of images such as target areas, cannot be reciprocal.

[0208] Existing solutions have been proposed to make the effect as different extents (meaning byte offsets within the data part) of an item reciprocal. More precisely, the extents mean that a derived image is described as a list of extents in the itemLocationBox ("iloc"), where each extent identifies a fragment of the data part ("mdat"), and each fragment corresponds to one or more descriptive or normative or transformation metadata.

[0209] However, this solution has several inherent drawbacks. - Creating encapsulated image files becomes significantly more complex: touching an effect in a derived image item means inspecting all derived images to see if they share the same extents, and potentially rewriting parts of them. Parsing is also not that easy, since the image file reader needs to figure out (without directly signaling) whether a set of transforms / effects is the same on different items in the file or not. -A new extent in itemLocationBox("iloc") will be required for each transform / effect whenever a new transform / effect is not continually stored with the transform / effect in the set of transforms / effects to be applied. Furthermore, combinations or chains of effects can be costly when not stored on contiguous extents within a data part.

[0210] Furthermore, in order to understand the type (previously the type of the effect was given by item_type), these solutions require a storage implementation, which involves creating a box to store the effect. By specifying a new box format for effects, a simpler solution would be to specify effects separately from items and have a direct mapping between items and effects without incurring any additional expenditures.

[0211] An alternative embodiment proposes to simplify the handling of effects by having a clear separation in the file format. - A regular item (an image or part of an image (e.g. hvc1, ...) that is linked with descriptive metadata (through the "init" or "simr" reference types as suggested above, or through any reference type that describes the descriptive metadata). - A "Derived Image" which is a collection of effects (or transformations) applied to one or more source items (images or parts of images) identified through "dimg" item references from the "Derived Image" item to the source items. -A transformation / effect construct that contains a collection of several different effects.

[0212] The advantages of this alternative embodiment are: -Effect reusability: declared once, potentially referenced multiple times. - A more compact description by specifying a collection of effects (more on that below). -Overall readability, including no new extents required for itemLocationBox. -The number of item references remains low.

[0213] According to this alternative embodiment, a new single derived item is defined with item type "dimg." This single derived item is specifically represented by:

number

[0214] where nb_effects represents the number of effects to be applied to the source image to construct the derived image, and effect_id is a unique identifier within the encapsulated file of the effect to be applied. The effects are applied in reverse order of their appearance in the list of effects.

[0215] A derived image or transformed item, called a "DerivedImage", defines an image as a set of effects that will be applied to a source image before presentation to a user or display screen, for example. The source image is identified by an item reference of type "dimg" (or any inverse reference type) from the derived item to the source image. The source image itself can be any image item defined in the ISO Still Image File Format specification (image or part of an image, image overlay, derived image). There should not be more than one "dimg" item reference from the same item (but there may be multiple "dimg" item references to the same item if this item is reused multiple times for different configurations). The derived image is stored in the data part of the file.

[0216] When editing an encapsulated file, for example removing an effect from an image file, all references to this effect should be removed from the derived image.

[0217] Effects can be applied to an image, part of an image, a composed image or a derived image through a DerivedImage item. Each effect is described by a box that derives from the BaseEffectBox structure shown below.

number

[0218] However, it has the following meaning: effect_type is the box type for effects derived from this class, a unique four-letter code that identifies this kind of box. effect_id is a unique identifier for a given effect or transformation. This identifier should be unique within a "meta" box. nb_bits_effect is derived from the version value and indicates the number of bits used to represent the effect_id.

[0219] Effects can be declared in an optional EffectDeclarationBox, which is contained in a "meta" box. Box Type: "effd" Container:meta Required:Non-required Quantity: 0 or 1

number

[0220] For example, the following effects may be specified (this is a non-exhaustive list): -Rotate Effect: The Rotate Effect translates the source image in a counterclockwise direction in units of 90 degrees. Box type: "erot" Container: effd Required:Non-required Quantity: 0 or more

number

number

[0221] Effect Collection: The Effect Collection box allows to specify a set of several effects as a single effect in order to reuse effects for several images, thus reducing the description consumption in terms of bytes. Box type: "ecol" Container: effd Required:Non-required Quantity: 0 or more

number

[0222] Effects in an effects collection are applied in the same order as the effects in the DerivedImaged item. For example, each effect should be applied to the input in the reverse order of its appearance in the list of effects.

[0223] The OverlayEffectBox declares a composition of images as an overlay. For this particular effect, the resulting image does not have references to any source images, since this effect declares the list of source images that are part of its structure.

number

[0224] According to another aspect of the invention, the storage of all descriptive and normative metadata can be further optimized compared to the above embodiment, depending on whether the descriptive and / or normative metadata is specific to a particular image item or is shared among several image items. Such sharing is possible without using byte range sharing or without defining a large list of item references as required by the above embodiment. According to this alternative embodiment, all descriptive and normative metadata is still stored only inside the box hierarchy in the "meta" box 100, allowing an ISOBMFF reader to parse all system information without the need to fetch "idat" or "mdat" boxes. Thus, the number of image items (in the "iinf" box) and item references (in the "iref" box) is limited only to address media data or to express relationships between several image items. Such a design makes the file easier to parse and facilitates a high level understanding of the file format.

[0225] An important aspect of this embodiment is that all system level item information is contained in a dedicated box (using the ISOBMFF full box) that is accessible to the parser without fetching any "mdat" or "idat" boxes and is either directly included in or directly referenced by the item information entries.

[0226] This embodiment introduces the following changes: - A new dedicated box called SharedItemPropertiesBox ("sitp") is defined to contain descriptive and normative metadata about box structures shared between items. - Modify the Item Info Entry ("infe") to associate box structure descriptive and normative metadata with the item. The metadata can be stored directly in the "infe" box if it is only relevant to this item, or in a "sitp" box and referenced from the "infe" box if it is shared between several items. -New box (SampleDescriptionEntryReference "sder" representing initialization parameters) to allow sharing the same initialization data between image items and samples in a track.

[0227] A new box called SharedItemPropertiesBox("sitp") is defined as follows: Box type: "sitp" Container: MetaBox ("meta") Required:Non-required Quantity: 0 or 1

[0228] The SharedItemProperties box (a dedicated shared box) contains a list of boxes that specify descriptive metadata (display parameters) and normative metadata (transformation operators) (also called properties) that may be applicable to some items declared in a parent "meta" box. These boxes are referenced by zero-based indexes from the ItemInfoEntry boxes. This box has the following syntax:

number

[0229] Regarding the changes in the Item Info Entry, a new version (4) is defined with the following semantics: The ItemInfoEntry box provides the possibility to include or reference additional boxes in the Item Info Entry that give the properties for this item. In the set of included and referenced properties there should be at most one property of a given type. Properties may be order-dependent, in which case the order given in the ItemInfoEntry box should be used, i.e. the first included property is applied first, then all other included properties, then all referenced properties.

[0230] The additional syntax is specified below.

number

[0231] The associated meanings are: included_prop_count: The number of properties (descriptive metadata or normative metadata) included in the array item_properties. item_properties: An array of boxes or a table of boxes that give additional information (item information properties) for this item. The allowed boxes are the same as for SharedItemPropertiesBox. indexed_prop_count: The number of references to the property in the SharedItemProperties box. box_prop_idx: A 0-based index into the list of boxes stored within the SharedItemProperties box of the "meta" box.

[0232] According to this embodiment, all descriptive and normative metadata are ISOBMFF full boxes that will be stored in the SharedItemProperties box or in the item_properties array within the ItemInfoEntry box.

[0233] For example, the canonical metadata for image rotation is specified as follows: Box type: "irot" Container:SharedItemProperties Required:Non-required Quantity: 0 or more

[0234] The image rotation box gives the rotation angle in a counter-clockwise direction in units of 90 degrees. Only one such box should be assigned as a property of an image item. The syntax of this box is specified as follows:

number

[0235] Normative metadata for image overlays is specified as follows: Box type: "iovl" Container:SharedItemProperties Required:Non-required Quantity: 0 or more

[0236] An image overlay box identifies one or more input images in a given layering order within a larger canvas. The input images are listed in the order in which they layer, i.e., bottom-most input image first and top-most input image last, in a SingleItemTypeReferenceBox of type "dimg" for the derived image item that contains or references this box as a property. Only one such box should be assigned as a property of an image item.

[0237] The syntax of this box is specified as follows:

number

[0238] Normative metadata for image grids is specified as follows: Box type: "grid" Container:SharedItemProperties Required:Non-required Quantity: 0 or more

[0239] An image grid box forms an output image from one or more input images in a given grid order within a larger canvas. Only one such box should be allocated as a property of an image item. The input images are inserted in row-major order, top row first, left to right, in the order of a SingleItemTypeReferenceBox of type "dimg" for derived image items that use this box within an ItemReferenceBox. The rows from this item to the input image * There should be a columns item reference. All input images should have the exact same width and height, called tile_width and tile_height. The tiled input images should completely "cover" the output image grid canvas, except for the tile_width * columns is equal to or greater than output_width and tile_height *The output image is formed by tiling the input image, without gaps or overlaps, into a grid with column widths equal to tile_width (potentially excluding the right-most column) and row heights equal to tile_height (potentially excluding the bottom-most row), then cropping on the right and bottom to the indicated output_width and output_height.

[0240] The syntax of this box is specified as follows:

number

[0241] As well as all other descriptive and normative metadata such as the auxiliary configuration box ("auxC"), the image space extent box ("ispe"), the pixel information box ("pixi"), the relative location box ("rloc"), the clean aperture box ("clap") (a non-exhaustive list) are all inherited from the ISOBMFF full box.

[0242] According to this embodiment, an item is a derived image when it contains a "dimg" item reference to one or more other image items that are inputs to the derivation. A derived image is obtained by performing a specified operation, such as rotation, on a specified input image. The operation performed to obtain the derived image is identified by the item_type of the item. The image items used as inputs to a derived image can be coded images or can be other derived image items.

[0243] For example, a clean aperture derived image item is identified by an item_type value "clap". It stores no data and should not have an associated entry in the "iloc" table. It should contain or reference an item property of type CleanApertureBox as specified in ISO / IEC 14496-12. It should have an item reference of type "dimg" to an image item. As another example, an image rotation derived image item is identified by an item_type value "irot". It stores no data and should not have an associated entry in the "iloc" table. It should contain or reference an item property of type ImageRotationBox as specified above. It should have an item reference of type "dimg" to an image item.

[0244] Similarly, an image overlay derived image item is identified by item_type "iovl". It has no data and should not have an associated entry in the "iloc" table. It should contain or reference an item property of type ImageOverlayBox as defined above. It should have an item reference of type "dimg" to a set of image items. An image grid derived image item is identified by item_type value "grid". It has no data and should not have an associated entry in the "iloc" table. It should contain or reference an item property of type ImageGridBox as defined above. It should have an item reference of type "dimg" to a set of image items.

[0245] Below are some examples illustrating the use of the SharedItemProperties box and the extended ItemInfoEntry box to assign descriptive and normative metadata (or properties) to an image.

[0246] In the example below, two property boxes ("hvcC" and "ispe") are assigned directly to the image item in its associated itemInfoEntry in the array item_properties.

number

[0247] In the following example, further to the previous example, an image rotation operator ("irot") is assigned to the image item in the same manner.

number

[0248] In the following example, multiple images with different HEVC configurations share the same dimensions, which are described in a common image space extent box ("ispe") stored in a SharedItemProperty box ("sitp"). Each image itemInfoEntry box contains its own HEVC configuration box ("hvcC") and uses an index into the SharedItemProperty box (item property index) to reference the common image space extent box ("ispe").

number

[0249] The entries of the table item properties indices form a set of identifiers. Another set of identifiers is formed by the rank (here: 0) of the image description information (here: "ispe") in a dedicated shared box (SharedItemPropertiesBox).

[0250] In another embodiment, the other identifier can be formed by another ID assigned to the image description information in the dedicated shared box. For example, this other ID assigned to the image description information can be defined by inheriting from "VirtualItemBox" (mentioned above) instead of ISOBMFF "FullBox". This embodiment advantageously allows the image description information in the dedicated shared box to be rearranged without affecting the set identifier.

[0251] Any set of identifiers forms a structure for linking image item information (represented by an entry in the ItemInfoBox) to at least one image description.

[0252] The following example describes a derived image composed of multiple images in a rotated grid. All images that compose the grid share the same HEVC configuration and the same image dimensions via the "hvcC" and "ispe" boxes located in the SharedItemProperty box and referenced by box property indexes. The derived image representing the grid is described via an itemInfoEntry that contains an image grid box. The rotation to be applied is described in an image rotation box associated with the derived image. The input images that compose the derived image are referenced via item reference entries in the item reference box ("iref") box.

number

[0253] The following example describes a HEVC tiling image. In this example, all items (full image (itemID=1) and tiles (itemID=2, 3, 4, 5)) share the same HEVC composition box, and all tiles share the same image space extent box that specifies the tile size (Wt, Ht) via a SharedItemPropertiesBox. Additionally, every tile item contains its own relative location box ("rloc") that gives the xy coordinates of each tile.

number

[0254] Furthermore, some image formats require initialization data to decode the image item data. The initialization data is codec specific and can be the same as or similar to the decoder configuration record defined for the video track. In such cases, it is useful to share the initialization data rather than repeating it in the file format. If such initialization data is required, it is given in the item information by a specific type of descriptive metadata (characteristics). Several image items can share such the same characteristic. To allow sharing of the same initialization data between an image item and several samples of a track, a new descriptive metadata box called SampleDescriptionEntryReference ("sder") is defined as follows: Box type: "sder" Container:SharedItemProperties Required:Non-required Quantity: 0 or more

[0255] The SampleDescriptionEntryReferenceBox allows one to indicate that an image item reuses the same initialization data as some samples of a track. It identifies the track and the sample description entries of those samples of that track. This box has the following syntax:

number

[0256] The following example illustrates sharing HEVC configuration between tracks and image items via a SampleDescriptionEntryReference box ("sder") associated with an image item InfoEntry.

number

[0257] When image item data represents HEVC tiles, each HEVC tile item should contain or reference a property of type HEVCConfigurationBox with all parameter sets required to decode tiles present in the HEVC tile item. Several HEVC tile items may share the same HEVCConfigurationBox property. An HEVC tile item should also contain or reference a RelativeLocationBox property ("rloc") indicating the location of the HEVC tile item within the respective HEVC image item. Several HEVC tile items corresponding to tiles belonging to different HEVC images may share the same RelativeLocationBox. An ImageSpatialExtentsBox property ("ispe") should be used for each HEVC tile item. The display_width and display_height of the ImageSpatialExtentsBox should be set to the width and height of the HEVC tile item.

[0258] In a variation on the above alternative embodiment, rather than grouping all shared descriptive and normative metadata in a single container SharedItemPropertiesBox, two different container boxes can be defined, one dedicated to descriptive metadata and the other dedicated to normative metadata. In such a case, the extended ItemInfoEntry contains two different property index arrays (box_prop_idx and box_ope_idx), or a type of metadata (descriptive or normative) is associated with each entry of the property index array (box_prop_idx) to search for the associated container.

[0259] The entries in box_prop_idx and box_ope_idx form a set of identifiers. Another set of identifiers is formed by the rank of the image description information in the two private shared boxes.

[0260] In another embodiment, another set of identifiers may be formed by other IDs assigned to the image description information in each dedicated share box. This embodiment advantageously allows the image description information in the dedicated share box to be rearranged without affecting the set of identifiers.

[0261] Any set of identifiers forms a structure for linking image item information (represented by an entry in the ItemInfoBox) to at least one image description. Further examples of this last aspect of the invention are described in the Appendix.

[0262] In another aspect of the invention, all descriptive and normative metadata can still be grouped into one or two boxes similar to the SharedItemPropertiesBox, but rather than modifying the itemInfoEntry box, an item reference box can be used to associate an image item with its descriptive and normative metadata. In this alternative embodiment, two different container boxes are defined, one for the descriptive properties (e.g., SharedItemProperties) and one for the normative properties (e.g., SharedItemOperators).

number

[0263] Instead of modifying the "infe" box, the itemReferenceBox "iref" box is used to associate images and derived image items with their descriptive and normative metadata (also called operators).

[0264] Two new reference types are defined, eg, "sipr" for descriptive metadata and "sior" for normative metadata.

[0265] Depending on the association type ("sipr" or "sior"), the "to_item_ID" parameter in the Item Reference Box is interpreted as an index into a SharedItemPropertiesBox or a SharedItemOperatorsBox, respectively. The reference type associated with the "to_item_ID" (here "sipr" or "sior") forms a structure for linking image item information (represented by an entry in the ItemInfoBox) to image description information (descriptive metadata and normative metadata).

[0266] For any other existing reference type, the attribute "to_item_ID" is still interpreted as pointing to the itemID in the ItemInfoBox.

[0267] Below is an example using the "sipr" and "sior" association types to describe multiple images in a rotated grid.

number

[0268] As a variant, each image description in the shared box is associated with an appropriate ID. This embodiment advantageously allows the image description in the dedicated shared box to be rearranged without affecting the identifiers.

[0269] In one variant, each existing reference type is implicitly associated with either an itemInfoBox, a SharedItemProperties box, or a SharedItemOperators box. For example, descriptive metadata reference types like "ispe", "rloc", "clap", "hvcC" are associated with a SharedItemProperties box, and normative metadata reference types like "irot", "iovl", "grid" are associated with a SharedItemOperators box.

[0270] 9 is a schematic block diagram of a computing device 900 for implementation of one or more embodiments of the present invention. The computing device 900 may be a device such as a microcomputer, a workstation or a lightweight portable device. The computing device 900 includes a communication bus connected to the following: A central processing unit 901, such as a microprocessor, represented as a CPU. a random access memory 902, represented by a RAM, for storing the executable code of the methods of the embodiments of the invention, as well as registers configured to record variables and parameters required to implement the methods for reading and writing manifests and / or for encoding videos and / or for reading or generating data under a given file format, the storage capacity of which can be expanded, for example, by an optional RAM connected to an expansion port. A read-only memory 903, represented as a ROM, for storing a computer program for implementing an embodiment of the invention. - The network interface 904 is typically connected to a communication network through which the digital data to be processed is transmitted or received. The network interface 904 can be a single network interface or can consist of a set of different network interfaces (e.g. wired and wireless interfaces, or different types of wired or wireless interfaces). Data is written to the network interface for transmission or read from the network interface for reception under the control of software applications running in the CPU 901. A user interface 905 for receiving input from a user or displaying information to a user. -Hard disk 906, represented by HD. - An I / O module 907 for receiving / sending data from / to external devices such as video sources or displays.

[0271] The executable code can be stored in the read-only memory 903 or on the hard disk 906 or on a removable digital medium, such as a disk. According to one variant, the executable code of the program can be received by means of a communications network, via the communications interface 904, for storage in one of the storage means of the communications device 900, such as the hard disk 906, before being executed.

[0272] The central processing unit 901 is configured to control and direct the execution of instructions of one or more programs or parts of software code according to an embodiment of the present invention, the instructions being stored in one of the storage means mentioned above. Upon power-up, the CPU 901 can execute instructions from the main RAM memory 902 associated with software applications, after those instructions have been loaded, for example, from a program ROM 903 or a hard disk (HD) 906. Such software applications, when executed by the CPU 901, perform the steps of the methods according to the embodiments.

[0273] Alternatively, the present invention can be implemented in hardware (eg, in the form of an application specific integrated circuit or ASIC).

[0274] The present invention can be embedded in devices such as cameras, smartphones, or tablets that act as remote controllers for TVs, for example to zoom in on specific areas of interest. It can also be used to have a personalized browsing experience of TV programs by selecting specific areas of interest from the same device. Another use from these devices by users is to share some selected subparts of their favorite videos with other connected devices. It can also be used in smartphones or tablets to monitor what happens in a specific area of ​​a building under surveillance, if the surveillance camera supports the production part of the present invention.

[0275] While the present invention has been explained and described in detail in the drawings and the above description, such explanations and descriptions are to be considered as explanatory or exemplary and not restrictive. The present invention is not limited to the disclosed embodiments. Those skilled in the art can understand and make other variations to the disclosed embodiments, from a study of the drawings, the disclosure, and the appended claims, when practicing the claimed invention.

[0276] In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. A single processor or other unit may fulfill the functions of several items recited in the claims. The mere fact that different features are recited in mutually different dependent claims does not indicate that a combination of these features cannot be used to advantage. Any reference signs in the claims should not be construed as limiting the scope of the invention.

[0277] appendix Example 1: Single Image

number

[0278] Example 2: Single image with rotation

number

[0279] Example 3: Single image with rotation and clean aperture

number

[0280] Example 4: Multiple images with the same dimensions but different HEVC configurations

number

[0281] Example 5: Multiple images with the same HEVC composition and dimensions

number

[0282] Example 6: Multiple images with the same HEVC composition and dimensions but with different rotations

number

[0283] Example 7: Multiple images in a grid

number

[0284] Example 8: Multiple images in a rotated grid

number

[0285] Example 9: Multiple Images with Overlay

number

[0286] Example 10: One image and its rotated version

number

[0287] Example 11: Tiled images

number

[0288] Example 12: Auxiliary image with same HEVC composition and dimensions as the master image

number

[0289] Example 13: Image with subsample description

number

[0290] Example 14: HEVC configuration shared between tracks and items

number

Claims

1. A method for outputting data corresponding to one or more types of items based on a file including a metadata portion having a hierarchical structure of boxes storing metadata of the items, and a media data portion, the method comprising: (1) one or more properties listed in a box included in the metadata portion; and (2) association information for associating one or more identities of the one or more properties with one or more identities of one or more types of items; Get a file that contains at least Outputting data corresponding to the item based on the acquired file A method comprising:

2. the file further includes an "Item Information Box" defined in the ISO Base Media File Format indicating information for each of the one or more types of items; The method of claim 1, wherein the association information for associating one or more identification information of the one or more properties with one or more identification information of the one or more types of items is described in a predetermined box different from the box.

3. 3. The method of claim 1, wherein the association information represents a correspondence between an identification of one of the one or more properties and an identification of each of two or more types of items.

4. 3. The method of claim 1, wherein the association information represents a correspondence between identification information of each of two or more properties and identification information of the one or more types of items.

5. The method according to any one of claims 1 to 4, wherein the file is generated based on a number of types of items.

6. 5. The method according to claim 1, wherein the file is generated based on a number of sub-items corresponding to one type of item.

7. The method according to any one of claims 1 to 6, wherein the associated information of the file is described in the metadata portion.

8. the one or more properties include two or more of the transformation operators listed in the box; 8. The method of claim 1, wherein the two or more transformation operators are applied to the associated items in the order in which they are listed in the boxes.

9. A method according to any one of claims 1 to 8, characterized in that the one or more properties listed in the box are shared by multiple items.

10. 10. A method according to any preceding claim, wherein the items are described in the metadata part.

11. 10. A method according to any preceding claim, wherein the items include still images.

12. 12. The method of claim 11, wherein the box includes at least one sub-box "ispe" defined in the ISO Base Media File Format, each of the at least one sub-box "ispe" describing properties including a width and a height of at least one still image.

13. 12. The method of claim 11, wherein the box includes at least one sub-box "hvcC" defined in the ISO Base Media File Format, each of the at least one sub-box "hvcC" describing properties indicating information for decoding at least one still image.

14. Method according to any one of claims 1 to 13, characterized in that the file has a format according to the ISO / IEC 23008-12 standard.

15. 1. A method for generating a file including a metadata portion having a hierarchical structure of boxes storing metadata of one or more types of items, and a media data portion, the method comprising: Get one or more types of items, Obtaining one or more properties associated with each of the one or more types of items; generating said file including said one or more types of items and said one or more properties; The file is (1) one or more properties listed in a box included in the metadata portion; and (2) association information for associating one or more identities of the one or more properties with one or more identities of the one or more types of items; Includes A method comprising:

16. the file further includes an "Item Information Box" defined in the ISO Base Media File Format indicating information for each of the one or more types of items; The method of claim 15, wherein the association information for associating one or more identification information of the one or more properties with one or more identification information of the one or more types of items is described in a predefined box different from the box.

17. 17. The method of claim 15 or 16, wherein the association information represents a correspondence between an identification of one of the one or more properties and an identification of each of two or more types of items.

18. 17. The method of claim 15 or 16, wherein the association information represents an association between an identification of each of two or more properties and an identification of the one or more types of items.

19. 19. The method of any one of claims 15 to 18, wherein the file is generated based on a number of types of items.

20. 19. The method according to any one of claims 15 to 18, wherein the file is generated based on a number of sub-items corresponding to one type of item.

21. 21. The method according to claim 15, wherein the associated information of the file is described in the metadata portion.

22. the one or more properties include two or more of the transformation operators listed in the box; 22. The method of any one of claims 15 to 21, wherein the two or more transformation operators are applied to the associated items in the order they are listed in the box.

23. The method of claim 15, wherein the one or more properties listed in the box are shared by multiple items.

24. 24. A method according to any one of claims 15 to 23, wherein the items are described in the metadata part.

25. 25. A method according to any one of claims 15 to 24, wherein the items include still images.

26. 26. The method of claim 25, wherein the box includes at least one sub-box "ispe" defined in the ISO Base Media File Format, each of the at least one sub-box "ispe" describing properties including width and height of at least one still image.

27. 26. The method of claim 25, wherein the box includes at least one sub-box "hvcC" defined in the ISO Base Media File Format, each of the at least one sub-box "hvcC" describing properties indicating information for decoding at least one still image.

28. 28. The method according to any one of claims 1 to 27, characterized in that the file has a format according to the ISO / IEC 23008-12 standard.

29. An apparatus for outputting data corresponding to one or more types of items based on a file including a metadata portion having a hierarchical structure of boxes storing metadata of the items, and a media data portion, the apparatus comprising: (1) one or more properties listed in a box included in the metadata portion; and (2) association information for associating identification information of each of the one or more properties with identification information of each of the one or more types of items; An acquisition unit that acquires a file including at least an output unit that outputs data corresponding to the item based on the acquired file; Equipped An apparatus comprising:

30. An apparatus for generating a file including a metadata portion having a hierarchical structure of boxes in which metadata of one or more types of items are stored, and a media data portion, the apparatus comprising: A first acquisition unit that acquires one or more types of items; a second acquisition unit that acquires one or more properties associated with each of the one or more types of items; a generator for generating the file including the one or more types of items and the one or more properties; Equipped with The file is (1) one or more properties listed in a box included in the metadata portion; and (2) association information for associating one or more identities of the one or more properties with one or more identities of the one or more types of items; Includes An apparatus comprising:

31. Apparatus for encapsulating a bitstream representing one or more types of items, configured to perform a method according to any one of claims 1 to 14.

32. Apparatus for processing an encapsulated bitstream representing one or more types of items, configured to perform a method as claimed in any one of claims 15 to 28.

33. 32. An apparatus according to claim 31 ; 33. An apparatus according to claim 32 for processing files from said apparatus; A system comprising:

34. A program for causing a computer to execute the method according to any one of claims 1 to 28.

35. A computer-readable storage medium storing the program according to claim 34.

Images