Method and apparatus for processing region-related annotations in image file

Abstract

The present invention relates to a method of encapsulating an image in an ISOBMFF-based media file, where the method comprises: generating a plurality of region items, each region item describing a geometry of a region of the image; generating a data structure representing a region of the image and listing region items included inside the region of the image from among the plurality of region items; and generating a media file comprising the image, the plurality of region items, and the data structure.

Description

Technical Field

[0001] This disclosure relates to methods and apparatus for processing region-related information in image files. Background Technology

[0002] Modern cameras and image analytics services enable the generation of localized metadata for images. Localized metadata is metadata that relates to a region or part of the media content, rather than the entire media content. Media content is typically an image, but it can also be video content or a collection of images. For example, a camera can generate a photograph's focus area or detect faces when taking a picture. As another example, deep learning systems can identify objects within an image. This localized metadata can be viewed as region annotations.

[0003] For example, images captured by a camera or processed by an image analysis service are stored on storage devices such as memory cards. Images are typically encoded to reduce the size of the data on the storage device. Many encoding standards can be used, such as JPEG, AV1, or the more recent HEVC standard.

[0004] The HEVC standard defines a configuration file for encoding still images and describes specific tools for compressing single still images or still images in bursts. An extension to the ISO-based media file format (ISOBMFF) for such image data has been proposed for inclusion in the ISO / IEC 23008 standard (in Part 12, it is named "HEIF" or "High-Efficiency Image File Format").

[0005] HEIF (High-Efficiency Image File Format) is a standard developed by the Moving Picture Experts Group (MPEG) for storing and sharing images and image sequences.

[0006] MIAF (Multi-Image Application Format) is a standard developed by MPEG and included in Part 22 of the ISO / IEC 23000 standard. The MIAF specification defines a multimedia application format, namely the Multi-Image Application Format (MIAF), which achieves precise interoperability points for creating, reading, parsing, and decoding images embedded in the High-Efficiency Image File (HEIF) format. The MIAF specification fully conforms to the HEIF format and defines only additional constraints to ensure greater interoperability.

[0007] The HEIF and MIAF file formats provide mechanisms suitable for linking annotations to regions of an image based on region items. However, for high-resolution images, the number of region items can become very large. When only a portion of a high-resolution image is displayed, the HEIF and MIAF file formats do not provide a means of determining which region items should be parsed and processed. Summary of the Invention

[0008] The present invention has been designed to address one or more of the aforementioned problems.

[0009] According to a first aspect of the present invention, a method for encapsulating an image in an ISOBMFF-based media file is provided, wherein the method comprises:

[0010] - Obtain the image;

[0011] - Obtain annotations for multiple regions associated with the image;

[0012] - Obtain the region partitioning information of the image;

[0013] - Generate multiple data structures representing the region partitions of the image;

[0014] - Associate the region annotation with the data structure based on the region partitioning information;

[0015] - Embed the image, the region annotation, and the multiple data structures in the media file.

[0016] In this embodiment, the region partitioning information includes information related to the area covered by the region partition and rendering scale information.

[0017] In this embodiment, the data structure is a cell representing a spatial partition of an image.

[0018] In this embodiment, cells are organized as a tree representing hierarchical spatial partitions of an image, with the cell at the top of the tree associated with the image, and each cell associated with a child cell in the tree (if any).

[0019] In this embodiment, each layer in the tree represents a version of the image at a given rendering scale.

[0020] In this embodiment, the association between region annotations and cells is based on the region's location and size, and / or the associated rendering scale range and / or priority.

[0021] In this embodiment, the tree is a quadtree.

[0022] In this embodiment, each layer of the tree is associated with an image overview of the image at a given resolution.

[0023] In this embodiment, each overview image, except for the overview image associated with the top of the tree, is divided into tiles, and the tree cells are associated with tiles of the overview images.

[0024] In this embodiment, region annotations are grouped into region items, and a data structure represents a group of region items.

[0025] In this embodiment, the data structure is an entity-to-group box.

[0026] According to another aspect of the present invention, a method for rendering the spatial portion of an image from an ISOBMFF-based media file is provided, wherein the method comprises:

[0027] - Obtain image items describing the image from the media file and the area of ​​the image to be displayed;

[0028] - Obtain multiple data structures representing regional partitions of the image from the media file;

[0029] - Filter the data structure based on the area to be displayed in the image;

[0030] - Further filter the data structure based on the rendering ratio of the area to be displayed in the image;

[0031] - Based on the regional partitioning information, obtain regional items from the media file, including regional annotations associated with the filtered data structure;

[0032] - Render the spatial portion of the image and the region annotations included in the obtained region items.

[0033] In this embodiment, the region partitioning information includes information related to the area covered by the region partition and rendering scale information.

[0034] In this embodiment, the data structure is a cell representing a spatial partition of an image.

[0035] In this embodiment, cells are organized as a tree representing hierarchical spatial partitions of an image, with the cell at the top of the tree associated with the image, and each cell associated with a child cell in the tree (if any).

[0036] In this embodiment, each layer in the tree represents a version of the image at a given rendering scale.

[0037] In this embodiment, the association between region annotations and cells is based on the region's location and size, and / or the associated rendering scale range and / or priority.

[0038] In this embodiment, the tree is a quadtree.

[0039] In this embodiment, each layer of the tree is associated with an image overview of the image at a given resolution.

[0040] In one embodiment, each overview image, except for the overview image associated with the top of the tree, is divided into tiles, and the tree cells are associated with tiles of the overview images.

[0041] In this embodiment, region annotations are grouped into region items, and a data structure represents a group of region items.

[0042] In this embodiment, the data structure is entity-to-box.

[0043] According to another aspect of the invention, a computer program product for a programmable device is provided, the computer program product comprising a sequence of instructions for implementing the method according to the invention when loaded into and executed by the programmable device.

[0044] According to another aspect of the invention, a computer-readable storage medium is provided for storing instructions for implementing a computer program according to the method of the invention.

[0045] According to another aspect of the invention, a computer program is provided that causes the method of the invention to be performed when executed.

[0046] According to another aspect of the present invention, an apparatus is provided for encapsulating an image in an ISOBMFF-based media file, wherein the apparatus includes a processor configured to:

[0047] - Obtain the image;

[0048] - Obtain annotations for multiple regions associated with the image;

[0049] - Obtain the region partitioning information of the image;

[0050] - Generate multiple data structures representing regional partitions of the image;

[0051] - Associate the region annotation with the data structure based on the region partitioning information;

[0052] - Embed the image, the region annotation, and the multiple data structures in the media file.

[0053] According to another aspect of the present invention, an apparatus is provided for rendering a spatial portion of an image from an ISOBMFF-based media file, wherein the apparatus includes a processor configured to:

[0054] - Obtain image items describing the image from the media file and the area of ​​the image to be displayed;

[0055] - Obtain multiple data structures representing regional partitions of the image from the media file;

[0056] - Filter the data structure based on the area to be displayed in the image;

[0057] - Further filter the data structure based on the rendering ratio of the area to be displayed in the image;

[0058] - Based on the regional partitioning information, obtain regional items from the media file, including regional annotations associated with the filtered data structure;

[0059] - Render the spatial portion of the image and the region annotations included in the obtained region items.

[0060] According to another aspect of the present invention, a method for encapsulating an image in an ISOBMFF-based media file is provided, wherein the method includes:

[0061] Multiple region items are generated, each region item describing the geometry of a region in the image;

[0062] Generate a data structure that represents the regions of the image and lists the region items contained within the regions of the image from the plurality of region items;

[0063] The media file is generated, and the generated media file includes the image, the plurality of region items, and the data structure.

[0064] In an embodiment, the data structure includes a reference space and a description of the regions of the image represented by the data structure relative to the reference space.

[0065] In an embodiment, the data structure further includes an indication of whether the data structure includes a reference space and an indication of the description of the region of the image represented by the data structure relative to the reference space.

[0066] In this embodiment, the region of the image represented by the data structure is the region of the image associated with the data structure.

[0067] In an embodiment, the data structure includes an indication of whether a region of the image represented by the data structure is a region of the image associated with the data structure.

[0068] In this embodiment, the image is a grid of multiple input images, and the data structure is associated with the input images among the multiple input images.

[0069] In this embodiment, the data structure is associated with the image.

[0070] In this embodiment, multiple region items are associated with the image.

[0071] In this embodiment, the region of the image represented by the data structure is a rectangular partition of the image.

[0072] In this embodiment, the data structure is an extension of EntityToGroupBox.

[0073] In this embodiment, a reference item is used to associate a data structure with an image.

[0074] According to another aspect of the present invention, a method for processing image-related data from an ISOBMFF-based media file is provided, wherein the method includes:

[0075] Obtain the region of the image;

[0076] A data structure is obtained from the media file, the data structure representing the regions of the image and listing region items contained within the regions of the image from a plurality of region items in the media file, each of the plurality of region items describing the geometry of the regions of the image;

[0077] Based on the listed region items, obtain at least one region item from the plurality of region items; and

[0078] Process at least one region item obtained.

[0079] In this embodiment, the data structure is obtained based on the rendering ratio of the region of the image to be processed.

[0080] In an embodiment, the data structure includes a reference space and a description of the regions of the image represented by the data structure relative to the reference space.

[0081] In an embodiment, the data structure further includes an indication of whether the data structure includes a reference space and an indication of the description of the region of the image represented by the data structure relative to the reference space.

[0082] In this embodiment, the region of the image represented by the data structure is the region of the image associated with the data structure.

[0083] In an embodiment, the data structure includes an indication of whether a region of the image represented by the data structure is a region of the image associated with the data structure.

[0084] In this embodiment, the region of the image represented by the data structure is a rectangular partition of the image.

[0085] In this embodiment, the data structure is an extension of EntityToGroupBox.

[0086] In this embodiment, a reference item is used to associate a data structure with an image.

[0087] In this embodiment, the image is a grid of multiple input images, and the data structure is associated with the input images among the multiple input images.

[0088] At least a portion of the method according to the invention can be implemented by a computer. Therefore, the invention can take the form of a completely hardware embodiment, a completely software embodiment (including firmware, resident software, microcode, etc.), or an embodiment combining software and hardware aspects, which may generally be referred to herein as a “circuit,” “module,” or “system.” Furthermore, the invention can take the form of a computer program product embodied in any tangible medium having computer-usable program code embodied therein.

[0089] Because this invention can be implemented in software, it can be embodied as computer-readable code for provision to a programmable device on any suitable carrier medium. Tangible non-transient carrier media may include storage media such as floppy disks, CD-ROMs, hard disk drives, magnetic tape devices, or solid-state storage devices. Transient carrier media may include signals such as electrical signals, electronic signals, optical signals, acoustic signals, magnetic signals, or electromagnetic signals (e.g., microwave or RF signals). Attached Figure Description

[0090] Embodiments of the invention will now be described by way of example only and with reference to the following figures, in which:

[0091] Figure 1 An embodiment of the invention is illustrated for indicating under which rendering conditions a region item associated with an image is used in a schematic diagram;

[0092] Figure 2 The steps for creating an image file (e.g., HEIF, MIAF, or any other ISOBMFF-based file format) according to an embodiment of the present invention are illustrated.

[0093] Figure 3 This illustrates the steps of using a tree of quadtree cells associated with image items to select region items to be retrieved and rendered in an embodiment of the invention;

[0094] Figure 4 An embodiment of the present invention when using an image overview is illustrated;

[0095] Figure 5a An example is illustrated where region items are grouped into region partition groups;

[0096] Figure 5b Another embodiment is illustrated where region items are grouped into region partition groups;

[0097] Figure 6This is a schematic block diagram of a computing device for implementing one or more embodiments of the present invention. Detailed Implementation

[0098] The HEVC standard defines a configuration file for encoding still images and describes specific tools for compressing single still images or still images in bursts. An extension to the ISO-based media file format (ISOBMFF) for such image data has been proposed for inclusion in the ISO / IEC 23008 standard (in Part 12, it is named "HEIF" or "High-Efficiency Image File Format").

[0099] The HEIF and MIAF standards cover two forms of storage corresponding to different use cases:

[0100] • The storage of image sequences, which can be indicated as a timing sequence or displayed in other ways, wherein images can depend on other images, and

[0101] • Storage of a single coded image or a collection of independent coded images (which may have derived images).

[0102] In the first case, the encapsulation is similar to that of video tracks in the ISO basemedia file format (see document "Information technology — Coding of audiovisual objects — Part 12: ISO basemedia file format", w18855, ISO / IEC 14496-12, 6th edition, October 2019), and uses similar tools and concepts, such as 'trak' boxes and sample groups for describing sample groups. A 'trak' box is a file format box that contains sub-boxes for describing tracks (that is, timing sequences of related samples).

[0103] A box (also called a container) is a hierarchical data structure provided to describe the data in a file. A box is an object-oriented building block defined by a unique type identifier (usually a four-character code, also known as FourCC or 4CC) and a length. All data in a file (media data and metadata describing the media data) is contained within a box. There is no other data within the file. File-level boxes are boxes that are not contained within other boxes.

[0104] In the second case, a collection of ISOBMFF boxes (i.e., 'meta' boxes) is used. These boxes and their hierarchy provide fewer descriptive tools than 'track-related' boxes ('trak' box hierarchy) and involve "information items" or "items" rather than related samples. It should be noted that the terms "box" and "container" can be used interchangeably to refer to the data structure containing metadata about the organization and / or attributes of image data in the description file.

[0105] The HEIF and MIAF standards also enable the annotation of regions from an image. Descriptions of one or more regions are implemented in a "region item." This region item can be associated with an image item via an item reference of type 'cdsc'. Annotations can be associated with the region described by the region item by associating other items with the region item or by associating item attributes with the region item.

[0106] Region tracks can be used to describe annotated regions for a video track or image sequence. A region track is associated with the video track or image sequence using a track reference of type 'cdsc'. Region tracks describe different regions and allow region annotations to be associated with individual regions.

[0107] The following description, by way of example only, applies to HEIF or MIAF files. Other types of files conforming to the ISOBMFF standard or similar structures can also utilize this invention.

[0108] Rendering a HEIF or MIAF file requires several processing steps. First, the file or a portion of a file needs to be read into memory. Then, the file or that portion of the file must be parsed. Specifically, 'meta' boxes can be parsed to obtain information about the items, groups, and item attributes present in the file. Since each box contains its size as its first field and its type as its second field, only a portion of the boxes present in the HEIF or MIAF can be parsed based on the box's type and the information searched. The next step is to process the parsed information to complete the structure specified in the file and connect these structures. For example, an EntityToGroupBox contains a list of identifiers for the entities it contains. Linking an EntityToGroupBox to its contained entities requires finding the entities corresponding to these identifiers. As another example, an 'ipma' box associates item identifiers with item attribute indices. Linking an item to its item attributes requires finding its identifier within the 'ipma' box, obtaining a list of item attribute indices associated with that identifier, and then retrieving the item attributes corresponding to these indices. Finally, for each item, its content is retrieved and decoded. The 'iloc' box indicates the location of the content of different items (e.g., media data corresponding to an image item in the media data box, or, as another example, a description of a region corresponding to a region item in the item data box). Using this location, the content of an item can be read from a HEIF or MIAF file. The content can then be parsed and used to render the item.

[0109] When using region annotations, a region item specifies one or more regions. The geometry of the regions is specified in the content (or payload) of the region item. This means that the location of the regions is stored in the content of the region item. This means that all the above steps are required to render a region item. In particular, determining which regions apply to the regions of an image item requires parsing all region items associated with that image item. The objective of this invention is to provide a means for determining whether to render a region item when performing local rendering of an image item described by a region item without needing to read and parse the content of the region item. In other words, this invention provides a mechanism for splitting region items into groups based on the location of the region items associated with the image item in the image. It is possible that the processing of the parsing information of the region item can be performed only for the region items required for local rendering of the associated image item. For example, the association of a region item with other items describing it or with the attributes of items associated with it can be performed only for the region items required for local rendering of the image item. In other words, considering an image item associated with multiple region items, the goal is to identify the region items of interest for a sub-part of the image item without needing to parse all region items to compare their geometry data with the geometry of the sub-part. When rendering sub-parts, only the identified region items are parsed, which saves computational resources and speeds up rendering.

[0110] According to embodiments of the invention, multiple region items can be grouped into partitions corresponding to regions of an image item containing these region items. This partition can be used to identify the region items contained within the region when rendering the image item. Several partitions can be associated with an image item. These partitions can be organized as a tree, where a parent partition contains one or more child partitions. The number of child units of the parent partition can be constrained, for example, as a quadtree, where the parent partition has four child units.

[0111] Figure 1 An embodiment of the invention is illustrated for indicating under which rendering conditions region items associated with an image are used in a schematic diagram. Image item 100 of type 'hvc1' can be an image with several region annotations. This image can be, for example, a very large picture generated by combining several photographs. For example, it can be an image of a crowd generated from several photographs of the crowd; it can be an aerial or satellite image. The invention is directed towards images with a large number of region annotations, but can also be applied to images with only a few region annotations.

[0112] The region annotations associated with image item 100 are described by region items 130, 131, 140, and 141 of type 'rgan'. These region items are each linked to image item 100 via item references of type 'cdsc'. Each region item can describe one or more regions of its data. Annotations corresponding to region items can be stored in the item attributes associated with that region item or in other items. These annotations are not in... Figure 1 Example in.

[0113] Region items can describe regions that correspond to different parts of an image, or regions that are more useful for a specific rendering scale of the image. For example, a region item might correspond to a large area inside an image and may be most useful when the image is displayed as a whole or at a small scale. On the other hand, another region item might correspond to details inside an image and may be most useful when the image is displayed at a large scale or when only a portion of the image containing details is shown. While each region item describes the precise geometry of the individual regions it contains, it is impossible to determine which part of the image a region item applies to without processing the contents of the region item. When many region items are associated with an image, loading and processing the contents of all region items can adversely impact the image rendering speed and the computational resources required.

[0114] To indicate the regions and rendering scales corresponding to the various region items associated with an image, these region items are grouped into quadtree cells, each quadtree cell being represented as an entity-to-group of type 'qdce' (for 'quadtree cell') (also labeled as an entity group in this disclosure) (the names and four-character codes used are merely examples, and any names or four-character codes that do not conflict with existing or registered names or four-character codes may be used). Each 'qdce' entity-to-group groups the region items that correspond to a region or region partition of an image item (the image is divided into multiple region partitions, e.g., forming a grid of region partitions covering the entire image) and correspond to the range of rendering scales for that image item. The size of the region or region partition of an image item may depend on the range of rendering scales for that image item. These 'qdce' entity-to-groups may be organized as a tree. Figure 1 In the example, the 'qdce' entity-to-group is organized as a quadtree: the first entity-to-group 110 corresponds to a region covering the entire image item. This entity-to-group 110 contains two region items 130 and 131. This entity-to-group is linked to four other entity-to-groups 120, 121, 122, and 123 via item references of type 'qdch' (for "quadtree sub-units") (the names and four-character codes used here are merely examples; any names or four-character codes that do not conflict with existing or registered names or four-character codes can be used). These entity-to-groups correspond to different regions or region partitions of the image item. Entity-to-group 120 corresponds to the top left quarter of the image, entity-to-group 121 corresponds to the top right quarter of the image, and so on. Each of these entity-to-groups contains region items associated with the region corresponding to it. A region item can be associated with a region if all its regions are contained within that region, if at least one of its regions is contained within that region, or if at least one of its regions intersects with that region. Possibly, if a region item is of interest when rendering a region, then that region item can be associated with that region. For example, a region item can describe the future location of an object and can be of interest when displaying that object, even if that future location is outside the display portion of the image. For example, entity-to-group 122 contains region items 140 and 141. These entity-to-groups can be linked to other entity-to-groups corresponding to smaller regions of the image. For example, Figure 1 This shows that entity to group 122 is linked to four other entity to groups of type 'qdce' via item references of type 'qdch'. The same applies to entity to group 123.

[0115] In a first embodiment of the invention, region items are organized using a quadtree, with each quadtree cell containing a region item corresponding to the region or region partition it covers. The quadtree cell is represented by an entity group, for example, having a grouping_type 'qdce' and named QuadtreeCellGroupBox (or any other 4CC or name not yet used in the standard) with the following structure:

[0116] class QuadtreeCellGroupBox(version, flags)

[0117] extends EntityToGroupBox('qdce', version, flags) {

[0118] unsigned int(8) children_presence;

[0119] unsigned int(32) left;

[0120] unsigned int(32) top;

[0121] unsigned int(32) cell_width;

[0122] unsigned int(32) cell_height;

[0123] unsigned int(32) minimum_scale_numerator;

[0124] unsigned int(32) maximum_scale_numerator;

[0125] unsigned int(32) scale_denominator;

[0126] }

[0127] Preferably, the quadtree cell set indicates the region and rendering scale corresponding to the region item associated with the image item. The quadtree cells can be organized as a tree with a root quadtree cell. Preferably, each quadtree cell has up to four child quadtree cells. A quadtree cell can be linked to its child quadtree cells using an item reference of type 'qdch'. Preferably, the child quadtree cells are listed in left-right and top-bottom order in the item reference. The children_presence field of a quadtree cell can be used to indicate which child quadtree cells exist for each quadtree cell. While the exemplary embodiment is based on a quadtree, any other hierarchical spatial decomposition or partitioning may be used, for example, based on the shape of the image or the specific needs of an application using the image.

[0128] Preferably, the root quadtree cell is typically linked to the image item associated with the quadtree cell tree using a 'cdsc' type item reference from the root quadtree cell to the image item. This item reference may have other types, such as 'qdds' (for "quadtree description"). Other quadtree cells may also be linked to the image item using a 'cdsc' type item reference, or a 'qdds' type item reference, or a 'qddc' type item reference (for "quadtree description cell"). Item references may be one or more of the following: from image item to root quadtree cell or from image item to quadtree cell.

[0129] Preferably, each region item associated with an image item is contained within at least one quadtree cell. If a region item is associated with different rendering scales, it may be contained within several quadtree cells at different depths in the tree. A region item may be contained within several quadtree cells at the same depth in the tree, for example, when the region item spans different regions corresponding to these quadtree cells.

[0130] Preferably, the quadtree cell contains only the region items signaled using the item_ID from the entity_id field of the QuadtreeCellGroupBox. Possibly, the quadtree cell may contain other entity groups that contain only region items. Possibly, the quadtree cell may contain other region-related items of the image items associated with the tree of the quadtree cell, either directly in its list of contained items or indirectly by including other entities into groups. For example, a set of quadtree cells can be used to indicate the location of HEVC tile items within an HEVC item.

[0131] Possibly, a quadtree cell set can be used to indicate the region and rendering scale of items associated with entities other than image items. For example, a quadtree cell set could be linked to a bracketing entity to group to indicate the region and rendering scale of a region item associated with an image item contained within that bracketing entity to group. As another example, a quadtree cell set could be linked to a track or image sequence to indicate the region and rendering scale of a region item associated with that track or image sequence (where these region items remain in the same position across all frames).

[0132] The semantics of the fields in QuadtreeCellGroupBox are as follows.

[0133] `children_presence` is a bit field indicating the presence of each of the four possible child cells of a `QuadtreeCellGroupBox`. If a child `QuadtreeCellGroupBox` exists, the corresponding bit is set to 1; otherwise, it is set to 0. The bits in the bit field are ordered according to the left-right and top-bottom order of the `QuadtreeCellGroupBox` child cells in the 'qdch' reference. If all four child cells exist, `children_presence` is 15 (1111 in binary), and the 'qdch' reference lists the identifiers of the four child `QuadtreeCellGroupBox`s. If only the top-right child exists, `children_presence` is 4 (0100 in binary), and the 'qdch' reference lists only the identifier of that child.

[0134] The left and top indicators specify the top-left position of the rectangular area corresponding to the QuadtreeCellGroupBox within the image item it is associated with.

[0135] cell_width and cell_height indicate the size of the rectangular area inside the associated image item that corresponds to the QuadtreeCellGroupBox.

[0136] It is possible to signal the area of ​​the QuadtreeCellGroupBox differently, for example, by using the top, left, bottom, and right positions of the rectangular area.

[0137] The regions of a QuadtreeCellGroupBox may have different shapes. For example, it can be defined as a polygon or a circle.

[0138] `minimum_scale_numerator`, `maximum_scale_numerator`, and `scale_denominator` indicate the minimum and maximum rendering scales corresponding to the region items contained in this `QuadtreeCellGroupBox`. The rendering scale of the image item is calculated as follows:

[0139]

[0140] Where `pixel_size` is the size of the pixels used to render the entire image width, and `image_size` is the width of the entire image. A rendering ratio of 1 means that one display pixel is used to render each image pixel. A rendering ratio of 2 means that one display pixel renders four (2x2) image pixels. A uniform rendering ratio is considered in the exemplary embodiment. Different scaling factors for the horizontal and vertical directions can be used if needed. In this case, the scaling parameters can be replicated for each direction.

[0141] If a number of display pixels are used to render a single image pixel, the rendering scale can range from 0 to 1, or even greater than 1.

[0142] The minimum rendering scale is calculated as follows:

[0143]

[0144] The maximum rendering ratio is calculated as follows:

[0145]

[0146] Possibly, a QuadtreeCellGroupBox can signal a single render scale value. The minimum and maximum render scales can be derived from this single scale value. For example, the minimum and maximum render scales can be calculated by multiplying the specified render scale value by a predefined factor. As another example, the minimum render scale can be a specified render scale, and the maximum render scale can be calculated by multiplying the specified render scale by a predefined factor. As yet another example, the minimum render scale can be a specified render scale, and the maximum render scale can be 1 or the maximum allowed render scale.

[0147] Note that, preferably, the image is rendered proportionally to both width and height to maintain a constant aspect ratio. If this is not the case, the rendering ratio of an image item can be calculated as a combination of the horizontal and vertical rendering ratios. For example, it could be the maximum of these two values, or the geometric mean of them.

[0148] Figure 2The steps for creating an image file (e.g., HEIF, MIAF, or any other ISOBMFF-based file format) according to an embodiment of the present invention are illustrated.

[0149] These steps are described with reference to the fact that the region items are organized as a quadtree; for illustrative purposes only, the region items are grouped into quadtree cells. These steps may be applied to other groups or partitions of region items, where the groups or partitions are organized as trees or not. In the first step 200, the image to be stored in the file is obtained.

[0150] Then, in step 210, one or more region annotations associated with the image are obtained. Region annotations may include locations within the image, such as those defined as geometric shapes. Region annotations may also include metadata linking to that region of the image. This metadata may be, for example, user-defined labels, descriptions, or titles. The metadata may also be machine-generated annotations, such as the results of object detection or face recognition processing.

[0151] Then, at step 220, information for partitioning these region annotations is obtained. This information can be a description of the quadtree used to partition these region annotations. This information can be the depth of the quadtree. In this case, the quadtree is constructed by uniformly splitting each cell into four sub-cells until the obtained depth is reached. This information can be the minimum size in pixels for each quadtree cell. In this case, the quadtree is constructed by uniformly splitting each cell into four sub-cells until further splitting the cells would create sub-cells smaller than the minimum size. This information can also indicate how to split the cells to create sub-cells of different sizes.

[0152] This information allows the rendering scale to be associated with the various depths of the quadtree. These rendering scales can be specified for each depth. The rendering scale for a quadtree cell can be calculated based on the size of the quadtree cell, such as (once scaled) the rendering size of the quadtree cell is a predefined number of pixels. The rendering size of an image or a portion of an image can be obtained by multiplying the size of that portion of the image by the rendering scale. The size can be measured as length or region, and the predefined number of pixels corresponds to the width or height of pixels or the region of pixels. For example, the size of a quadtree cell can be defined as the greater of its width and height. The predefined number of pixels could be 500. Then, the rendering scale corresponding to that region is:

[0153]

[0154] The size of a quadtree cell can be calculated in different ways: it can be the larger of the width and height of the quadtree cell, it can be the average of the height and width, it can be the diagonal of the quadtree cell, it can be the area of ​​the quadtree cell, or any other measure representing the size of the quadtree cell.

[0155] If the size of a quadtree cell is defined as length, then the rendering ratio is a ratio of length. If the size of a quadtree cell is defined as region, then the rendering ratio is a ratio of region. Possibly, when the size of a quadtree cell is defined as region, the rendering ratio can be converted to a ratio of length by calculating the square root of the value obtained as described above. On the other hand, when the size of a quadtree cell is defined as length, the rendering ratio can be converted to a ratio of region by calculating the square of the value obtained as described above. These two different types of rendering ratios can be used for any use of rendering ratios in different embodiments of the invention.

[0156] The information also allows the range of rendering scales to be associated with the various depths of the quadtree. These rendering scale ranges can be specified for each depth. The rendering scale for a quadtree cell can be calculated based on the size of the quadtree cell, such as (once scaled) the rendering size of the quadtree cell being between two predefined pixel numbers. This calculation is similar to the calculation described for the rendering scale, but with two predefined pixel numbers, one for the minimum rendering scale and one for the maximum rendering scale. For example, the predefined pixel numbers could be between 250 and 1000. Then, the rendering scale range for the quadtree cell is within the following range:

[0157]

[0158] The range of rendering scale values ​​can be calculated from the rendering scale values ​​by multiplying the rendering scale by two predefined factors to obtain the minimum and maximum rendering scales.

[0159] The predefined number of pixels and / or predefined factors can depend on the depth of the quadtree cells or on the total depth of the quadtree.

[0160] Each quadtree cell is associated with rendering properties. These properties can include the area that the quadtree cell spans on the image. These properties can include the rendering scale associated with the quadtree cell. These properties can include the range of rendering scales associated with the quadtree cell.

[0161] In step 230, select the region annotation.

[0162] At step 240, rendering conditions for the region annotation are obtained. Rendering conditions may include the region corresponding to the region containing the region annotation. Rendering conditions may include a rendering scale or a range of rendering scales, thereby indicating whether the region annotation should be rendered according to the rendering scale of the image.

[0163] A region containing region annotations can be obtained by calculating the region's bounding box and using that bounding box as the region. This region can also be the region itself. Alternatively, it can be the region's convex outline. The region can also be obtained jointly with the region annotations.

[0164] The rendering scale for a region annotation can be obtained jointly with the region annotation. It can be calculated based on the size of the region, such as (once scaled) the render size of the region being a predefined number of pixels. The size can be measured as length or region, with the predefined number of pixels corresponding to the width in pixels, the height in pixels, or the region in pixels. For example, the size of a region can be defined as the maximum of the width and height of the region's bounding box. The predefined number of pixels could be 20. Then, the rendering scale corresponding to that region is:

[0165]

[0166] The size of a region can be calculated in different ways: it can be the maximum of the width and height of the region's bounding box, it can be the average of the height and width, it can be the diagonal of the bounding box, it can be the region of the region, it can be the area of ​​the bounding box, or any other measure that represents the size of the region.

[0167] The rendering scale range for a region annotation can be obtained jointly with the region annotation. It can be calculated based on the size of the region, such as (once scaled) the region's render size being between two predefined pixel numbers. This calculation is similar to that for a rendering scale description, but with two predefined pixel numbers, one for the minimum rendering scale and one for the maximum. For example, the predefined pixel numbers could be between 5 and 40. Then, the rendering scale range for that region is within the following range:

[0168]

[0169] The range of rendering scale values ​​can be calculated from the rendering scale values ​​by multiplying the rendering scale by two predefined factors to obtain the minimum and maximum rendering scales.

[0170] The predefined number of pixels and / or predefined factors can depend on the size of the region or the number of region annotations. For example, when there are a large number of region annotations, the predefined number of pixels can be larger to reduce the number of region annotations being rendered.

[0171] At step 250, a region annotation is associated with one or more quadtree cells. The region annotation is associated with the quadtree cell whose property matches the region annotation rendering conditions. Possibly, a region annotation can be associated with all quadtree cells whose property matches the region annotation rendering conditions. Possibly, a region annotation can be associated with the quadtree cell whose property best matches the region annotation rendering conditions. Possibly, for some properties, all matches are considered, while for others, only one or more best matches are considered. For example, for a region, all matches may be considered, while for a rendering scale, only the best match may be considered.

[0172] If a region in a region annotation intersects with a region in a quadtree cell, then they are matched. Matches can be compared using the intersection-union ratio (IU) of the regions: the best match is the one with the highest IU between the region in the region annotation and the region in the quadtree cell.

[0173] A region annotation is considered a match if its rendering scale is close to that of the quadtree cell. Matches can be compared using the rendering scale ratio. The rendering scale ratio can be calculated as the largest of the following ratios: the ratio of the region annotation's rendering scale to the quadtree cell's rendering scale, and the ratio of the quadtree cell's rendering scale to the region annotation's rendering scale. The best match is the one with the rendering scale ratio closest to 1.

[0174] If the rendering scale range of a region annotation overlaps with the rendering scale range of a quadtree cell, then they are matched. Matches can be compared using the ratio between the maximum and minimum rendering scale values ​​of the overlapping ranges. The best match is the one with the maximum ratio.

[0175] At step 260, it is checked whether there are any other region annotations to be processed. If so, another region annotation is selected at step 265, and the next step is step 240.

[0176] Otherwise, the next step is step 270. In step 270, regions that share the same annotation and are associated with the same set of quadtree cells are grouped together.

[0177] Other criteria could potentially be used to group regions. For example, there could be a maximum number of regions per group. As another example, regions to be encoded as inline masks may not be grouped with other regions.

[0178] At step 280, a file representing the image, region annotations, and quadtree is generated. This file can be, for example, HEIF, MIAF, or any other ISOBMFF-based file. The image is encoded as an image item. Each region group is encoded as a region item. Each quadtree cell is encoded using a QuadtreeCellGroupBox.

[0179] Figure 3 The steps are illustrated to select the region item to be retrieved and rendered using a tree of quadtree cells associated with the image item.

[0180] These steps are described with reference to the fact that the region items are organized as a quadtree; for illustrative purposes only, the region items are grouped into quadtree cells. These steps may be applied to other groupings or partitions of region items, where the group or partition is organized as a tree or not.

[0181] First, at step 300, the image item is obtained. As an example, the image item can be obtained from HEIF, MIAF, or any other ISOBMFF-based file by reading and parsing the 'meta' boxes. Additionally, the area to be displayed for that image item is obtained. The rendering scale used to display that area is also obtained. For example, the area to be displayed can be a rectangle specified by the user through a graphical user interface (GUI). The rendering scale can also be specified by the user through the same GUI. Possibly, the area and rendering scale can be obtained after a series of interactions between the user and the GUI. For example, the user can repeatedly zoom and pan on the image described by the image item.

[0182] At step 310, the tree of the quadtree cells associated with the image item is retrieved. First, the root of the tree is retrieved by finding the quadtree cell group linked to the image item via the 'cdsc' item reference. Then, other quadtree cell groups are retrieved by following the 'qdch' item reference. For example, the quadtree cell group can be obtained by reading and parsing the EntityToGroupBoxes contained in the 'grpl' box, which itself is contained in the 'meta' box. The 'cdsc' item reference from the quadtree cell group to the image item can be found by parsing the 'iref' box contained in the 'meta' box and searching for the item reference from the EntityToGroupBox of type 'qdce' to the image item.

[0183] For each quadtree cell, retrieve the list of region items it contains, its region, and its minimum and maximum rendering scale from the 'qdce' entity representing it to the group.

[0184] At step 320, the quadtree cells are filtered based on the area to be displayed for the image item. For each quadtree cell, its area is compared with the area to be displayed for the image item. If the two areas have a non-empty intersection, the quadtree cell is retained. Otherwise, the quadtree cell is discarded.

[0185] In step 330, the quadtree cells are filtered based on the rendering ratio. For each quadtree cell, if the rendering ratio is greater than or equal to the minimum rendering ratio of the quadtree cell and less than or equal to the maximum rendering ratio of the quadtree cell, the quadtree cell is retained. Otherwise, the quadtree cell is discarded.

[0186] At step 340, the region items contained in the retained quadtree cells are retrieved. For each region item, the location of its content can be retrieved from the 'iloc' box contained within the 'meta' box. Using this location information, the content of the region item can be loaded and parsed from the file to obtain a description of the region contained within that region item. Additionally, annotations associated with the region item are retrieved. For example, item attributes associated with the region item can be retrieved by searching for 'ipma' boxes contained within the 'iprp' box (which itself is contained within a meta box) for attributes associated with the region item, and then by reading and parsing these attributes from the 'ipco' box contained within the 'iprp' box. As another example, metadata associated with a region item through another item can be retrieved by searching for other items associated with the region item in the 'iref' box. The location of the content of these other items can be obtained from the 'iloc' box. The content of these other items can then be read and parsed from the file.

[0187] At step 350, the obtained area of ​​the image item is displayed. The region items retrieved at step 340 are also rendered. For example, the edges of the regions specified by these region items can be displayed on the image item. As another example, the edges of a region item are only displayed when the mouse is hovering over the region specified by that region item.

[0188] Advantageously, by using these steps, only a subset of the region items associated with the image item is read and parsed from the file. This reduces the memory and CPU required to display the region of the image item and speeds up rendering.

[0189] In some embodiments, steps 320 and / or 330 may be combined with step 310. In some embodiments, the order of steps 320 and 330 may be interchanged.

[0190] Preferably, when using from Figure 3In an embodiment of the steps, when rendering a new region of an image item, information retrieved and processed from the ISOBMFF-based file is stored in memory to avoid re-retrieval and processing. Specifically, information relating to the quadtree cells retrieved and parsed at step 310 is preferably stored in memory. Similarly, information relating to the region items retrieved, parsed, and processed at step 340 is preferably stored in memory. Advantageously, this reduces the processing time for rendering new regions of image items.

[0191] Possibly, in these first embodiments, the number of sub-quadtree cells may differ from four sub-cells organized in 2 rows and 2 columns. For example, a quadtree cell may have six sub-cells organized in 2 rows and 3 columns. As another example, for a wide image, a quadtree cell may have N sub-cells organized in 1 row and N columns. Possibly, different EntityToGroupBoxes may be used to signal quadtree cells with different numbers of sub-cells or different organization of their sub-cells.

[0192] In a second embodiment of the invention, a quadtree is used to organize the region items, and each quadtree cell contains the region item corresponding to the region it covers. The quadtree cell is represented by a QuadtreeCellGroupBox with the following structure:

[0193] class QuadtreeCellGroupBox(version, flags)

[0194] extends EntityToGroupBox('qdce', version, flags) {

[0195] }

[0196] In these embodiments, the characteristics of each QuadtreeCellGroupBox are determined by processing other information.

[0197] A quadtree cell is linked to four child quadtree cells via a 'qdch' item reference. The four child quadtree cells are listed from left to right and top to bottom in the 'qdch' item reference. If a quadtree cell has fewer than four child quadtree cells, the missing child quadtree cells can be replaced by a reference to the quadtree cell itself in the reference list contained in the 'qdch' item reference. For example, if the quadtree cell with identifier 100 only has a top-left child cell with identifier 101 and a bottom-left child cell with identifier 103, the list of identifiers in the 'qdch' item reference could be: 101, 100, 103, 100. Missing child quadtree cells at the end of the reference list may be omitted. In the previous example, the list of identifiers in the 'qdch' item reference could be: 101, 100, 103. It is possible that for a quadtree cell with no children, there may not be a 'qdch' item reference starting from that quadtree cell.

[0198] The root quadtree cell's area covers the entire area of ​​the image item it is associated with via the 'cdsc' item reference. Each child quadtree cell covers a quarter (half width and half height) of its parent quadtree cell's area. The position of a child quadtree cell is determined by its position in the reference list within the 'qdch' item reference that links the parent quadtree cell to that child quadtree cell.

[0199] The rendering scale can be calculated by setting the rendering scale to 1 for the deepest quadtree cell in the tree and by dividing the rendering scale by 2 for each layer in the tree.

[0200] The range of rendering scale for a quadtree cell can be calculated from its rendering scale by setting the maximum value of the range to the rendering scale and by multiplying the rendering scale by a predefined factor to calculate the minimum value of the range.

[0201] The rendering scale range for a quadtree cell can be set to a predefined range for the deepest quadtree cell, such as [1 / 4, 2]. For other quadtree cells, the rendering scale range is calculated by dividing the minimum and maximum values ​​of each level in the tree by 2.

[0202] The first and second embodiments of the present invention can be combined, for example, by specifying some characteristics of the quadtree cell in the QuadtreeCellGroupBox structure and by calculating other characteristics of the quadtree cell.

[0203] In a third embodiment of the invention, region items are organized in a tree structure, with each tree cell containing region items corresponding to the region it covers. A tree cell may have a number of subtree cells other than four. In these embodiments, a tree cell may be represented by a TreeCellGroupBox with the following structure:

[0204] class TreeCellGroupBox(version, flags)

[0205] extends EntityToGroupBox('trce', version, flags) {

[0206] unsigned int(8) rows;

[0207] unsigned int(8) cols;

[0208] unsigned int(32) children_presence;

[0209] unsigned int(32) left;

[0210] unsigned int(32) top;

[0211] unsigned int(32) cell_width;

[0212] unsigned int(32) cell_height;

[0213] unsigned int(32) minimum_scale_numerator;

[0214] unsigned int(32) maximum_scale_numerator;

[0215] unsigned int(32) scale_denominator;

[0216] }

[0217] In this structure, rows and columns indicate the number of rows and columns of the subtree cells of the tree cell specified by the structure.

[0218] The other fields have the same meaning as in the first embodiment.

[0219] The children_presence field could potentially be larger to allow for more than 32 subtree cells. The field size could also depend on the number of subtree cells.

[0220] These third embodiments can be combined with the second embodiments by removing some fields from the TreeCellGroupBox structure and calculating the corresponding properties.

[0221] In these third embodiments, at step 220, the region partitioning information may include the number of sub-cells for each tree cell. The number of sub-cells may be the same for all tree cells. The number of sub-cells may depend on the depth of the tree cell within the tree.

[0222] Possibly, in these embodiments, the number of child cells in each tree cell can be determined dynamically. At step 220, the number of child cells is set to a predefined value. At step 250, the number of child cells in a tree cell can be changed. When a region annotation is associated with a tree cell, if the number of region annotations associated with that tree cell is greater than a predefined threshold, the number of child cells in the parent cell of the tree cell associated with the region annotation is increased. In other words, the tree granularity depends on the number of region annotations. The subtree originating from the parent tree cell is deleted, and it is recreated using the new number of subtree cells associated with the parent tree cell. Any region annotations previously associated with the deleted tree cell are associated with the newly created tree cell. This reassociation process can be accelerated by keeping the individual region annotations associated with the new tree cell at the same depth in the tree as the deleted tree cell associated with them.

[0223] In a fourth embodiment of the invention, a structure corresponding to the third embodiment is used to signal the root of the tree. Specifically, the number of rows and columns of the subtree cells are specified for the root of the tree. Other tree cells have only 4 subtree cells (2 rows and 2 columns). A structure corresponding to the first embodiment can be used to signal other tree cells.

[0224] This fourth embodiment may be combined with the second embodiment for the root of the tree, for other tree cells, or for all tree cells.

[0225] In these embodiments, at step 220, the region partitioning information may include the number of sub-cells for the root tree cell.

[0226] Possibly, in these embodiments, the number of sub-cells of the root tree cell can be dynamically determined in a manner similar to the dynamic determination described for the third embodiment. The difference is that when it is determined at step 250 that the number of region annotations associated with the tree cell is greater than a predefined threshold, the entire tree except for the root cell is deleted, and it is recreated with a different number of sub-cells for the root cell.

[0227] In a fifth embodiment of the present invention, the tree cell is represented by a TreeCellGroupBox having the following structure:

[0228] class QuadtreeCellGroupBox(version, flags)

[0229] extends EntityToGroupBox('trce', version, flags) {

[0230] if (version == 1) {

[0231] unsigned int(8) rows;

[0232] unsigned int(8) cols;

[0233] }

[0234] }

[0235] In these embodiments, the TreeCellGroupBox structure has two versions. A tree cell represented by a TreeCellGroupBox structure with version value 0 has four subtree cells arranged in 2 rows and 2 columns. A tree cell represented by a TreeCellGroupBox structure with version value 1 has multiple tree cells specified by row and column fields.

[0236] Possibly, in these embodiments, two different EntityToGroupBox structures can be used with different names. For example, 'qdc4' and 'qdcy' refer to a structure representing a tree cell with 4 child units and a structure representing a tree cell with a specified number of child units, respectively. As another example, 'qdce' and 'trce' could refer to a structure representing a tree cell with 4 child units and a structure representing a tree cell with a specified number of child units, respectively.

[0237] Image overviews are a proposal to extend HEIF for efficient access to large numbers of images. Under this proposal, the base image is broken down into tiles, all of the same size. These tiles correspond to rectangular regions of the image. They differ from the tile concept in MPEG encoding schemes such as HEVC or VVC. Other images corresponding to the overviews of the base image are created by scaling down the base image. Scaling down the base image is achieved by merging its pixels. For example, merging 2 means that groups of 2×2 pixels are combined into a single pixel, resulting in an image with half the width and half the height of the base image. Each overview is also broken down into tiles of the same size as the tiles of the base image. Each tile should be decodeable independently of the others to allow independent access to sub-sections of the base image at different scaling levels. Still under this proposal, the base image and its overviews are associated to form an image pyramid. In other words, an overview corresponds to a set of image items, all representing the same image at different resolutions.

[0238] Figure 4 An embodiment of the invention when using an image overview is illustrated. Image item 400 is divided into 12 tiles (3 rows and 4 columns of tiles). All tiles have the same size, and each tile can be decoded individually. This makes it possible, for example, to render the center of the image by decoding only tiles 401 and 402. Image item 400 can be encoded, for example, as a 'grid' item, where each tile is encoded as an independent image item. Image item 400 can also be encoded as HEVC tiles. It can be stored as an 'hvc1' image item, where each tile is encoded as an HEVC tile and stored as an item of type 'hvt1'. Image item 400 can also be encoded as VVC, where each tile corresponds to a VVC sub-image stored as an item of type 'vvs'.

[0239] Image item 400 is associated with two other image items that represent an overview of that image item. Image item 410 is half the width and half the height of image item 400. Image item 420 is one-quarter the width and one-quarter the height of image item 400. Both image items 410 and 420 are split into tiles with the same size as the tiles of image item 400. These tiles can also be decoded individually. This makes it possible to efficiently render a scaled-down version of image item 400. For example, image item 420 can be used for rendering with a very small scale.

[0240] Since the tiles are the same size for the three image items 400, 410 and 420, some tiles of image items 410 and 420 cover the area outside the image item.

[0241] Image items 400, 410, and 420 form a pyramid. This pyramid can be signaled in a file storing these items. This file can be, for example, HEIF, MIAF, or any other ISOBMFF-based file. For instance, image items 400, 410, and 420 can be contained within an EntityToGroupBox representing the pyramid.

[0242] The QuadtreeCellGroupBox tree is associated with image item 400 to indicate the area and rendering scale corresponding to the various area items associated with image item 400. The root of the tree 430 is associated with image item 400 using an item reference of type 'cdsc'. This root cell has four child quadtree cells 440, 441, 442, and 443. This root cell is linked to its child quadtree cells using an item reference of type 'qdch'. Quadtree cell 440, corresponding to the upper-left area of ​​image item 400, has four child quadtree cells. However, quadtree cell 442, corresponding to the lower-left area of ​​image item 400, has only two child quadtree cells. Similarly, quadtree cell 441, corresponding to the upper-right area of ​​image item 400, has four child quadtree cells (not shown). And quadtree cell 443, corresponding to the lower-right area of ​​image item 400, has only two child quadtree cells.

[0243] In this example, the quadtree partitioning matches the tile partitioning of image item 400 and its overview. The root quadtree cell covers the same area as the tile of image item 420. The four quadtree cells 440, 441, 442, and 443 each have an area corresponding to one of the tiles of image item 410. The child quadtree cells of quadtree cells 440, 441, 442, and 443 each have an area corresponding to one of the tiles of image item 400.

[0244] In a sixth embodiment of the invention, an image item is associated with an overview that is a scaled-down version of that image item. These image items form an image pyramid ordered by size from the image item to the smallest overview. The image item and its overview are broken down into tiles. A tile of an image in the pyramid is completely contained within one of the tiles of the next smallest image in the pyramid. Figure 4 Image items 400, 410, and 420 illustrate examples of this image pyramid.

[0245] Preferably, in these embodiments, different levels of the quadtree cells are matched with tiles of different overviews. If the minimum overview has a single tile, the root of the quadtree has the same region as that single tile. If the minimum overview has several tiles, the root of the quadtree has a region corresponding to the union of the regions of these tiles. Each quadtree cell in the second level of the tree has a region corresponding to one of the tiles of the minimum overview.

[0246] Then, each level of the quadtree cell corresponds to one of the overviews. Each quadtree cell has an area corresponding to one of the tiles in the overview. The deepest level of the quadtree cell corresponds to the image item itself, and each quadtree cell has an area corresponding to one of the tiles in the image item.

[0247] In these embodiments, at step 200, in addition to the image, information about the overview is also obtained. Possibly, in addition to the image, the overview is also obtained. Possibly, in addition to the image, instructions for constructing the overview are also obtained.

[0248] In these embodiments, at step 220, the tile of the image item and its overview provides region partitioning information related to the area of ​​the quadtree cell.

[0249] Possibly, in these embodiments, at step 340, the retrieved region item may be associated with a retrieved tile of an image item or one of its overviews to enable faster processing during future rendering of the image item or one of its overviews.

[0250] Possibly, in these embodiments, the QuadtreeCellGroupBox corresponding to the root of the tree can be associated with each overview, for example, using an item reference of type 'cdsc'. Possibly, the type of the item reference can be different from the type used to associate it with image items. For example, the QuadtreeCellGroupBox can be associated with image items using an item reference of type 'cdsc', and can be associated with overviews using an item reference of type 'qdov' (for "quadtree overview").

[0251] Possibly, in these embodiments, the QuadtreeCellGroupBox corresponding to the root of the tree can be associated with the structure representing the pyramid, for example, using an item reference of type 'cdsc'. Possibly, this QuadtreeCellGroupBox may not be associated with an image item.

[0252] Possibly, in these embodiments, tiles representing image items and / or their overviews can be associated with their corresponding QuadtreeCellGroupBoxes, for example, using an item reference of type 'qdds' from the QuadtreeCellGroupBox to the entity representing the tile. As another example, tiles can be linked to their corresponding QuadtreeCellGroupBoxes using an item reference of type 'qdre' (for "quadtree regions") from the entity representing the tile to the QuadtreeCellGroupBox.

[0253] Possibly, in these embodiments, tiles representing image items and / or their overviews may be associated with a collection of corresponding QuadtreeCellGroupBoxes. For example, a tile may be linked to a collection of corresponding QuadtreeCellGroupBoxes using an item reference of type 'qdre' from the entity representing the tile to the QuadtreeCellGroupBoxes.

[0254] In these embodiments, the quadtree may be directly represented by tiles of image items and their overviews. This means there is no QuadtreeCellGroupBox structure in the file. Tiles can be linked to the region items they contain using a 'reco' type (for "contained region") item reference from the entity representing the tile to the region item it contains. One or more item attributes may be associated with the entity representing the tile to describe information related to the region items it contains. For example, item attributes may describe the minimum and maximum rendering scale of the region annotation associated with the tile. If no tile corresponding to the entire image exists in any overview, an image item can be linked to the region item associated with the root of the quadtree using a 'reco' type item reference from the image item to the region item. Similarly, if no tile corresponding to the entire image exists in any overview, an overview can be linked to the region item associated with the root of the quadtree using a 'reco' type item reference from the overview to the region item.

[0255] Possibly, in these embodiments, region items can be grouped into entities to groups of TileGroupBoxes, which have a structure similar to QuadtreeCellGroupBox, and the quadtree organization can be directly represented by tiles of image items and their overviews. Tiles can be linked to TileGroupBoxes containing the region items contained within them, using, for example, an item reference of type 'reco' from the entity representing the tile to the TileGroupBox. Possibly, if no tile corresponding to the entire image exists in any overview, an image item can be linked to a TileGroupBox containing the region item associated with the root of the quadtree, using an item reference of type 'reco' from the image item to the TileGroupBox. Possibly, if no tile corresponding to the entire image exists in any overview, an overview can be linked to a TileGroupBox containing the region item associated with the root of the quadtree, using an item reference of type 'reco' from the overview to the TileGroupBox.

[0256] In a variation of the sixth embodiment, image items are encoded using tiles, as are image overviews, but where no image item has an associated overview. For example, image items can be encoded as grid items using HEVC tiles or VVC sub-pictures. Preferably, in these variations, the boundaries of quadtree cells match the boundaries of the tiles for the image item. Possibly, there may be only two levels in the quadtree, where the root cell corresponds to a region of the entire image, and the leaf cells correspond to tiles or sets of tiles.

[0257] Possibly, in these embodiments, the tiles of an image item can be associated with the corresponding QuadtreeCellGroupBox, for example, using an item reference of type 'qdds' from the QuadtreeCellGroupBox to the entity representing the tile. As another example, a tile can be linked to its corresponding QuadtreeCellGroupBox using an item reference of type 'qdre' from the entity representing the tile to the QuadtreeCellGroupBox.

[0258] Possibly, in these embodiments, the tiles of an image item may be associated with a collection of corresponding QuadtreeCellGroupBoxes. For example, a tile may be linked to a collection of corresponding QuadtreeCellGroupBoxes using an item reference of type 'qdre' from the entity representing the tile to the QuadtreeCellGroupBoxes.

[0259] Possibly, in these embodiments, the leaves of a quadtree can be directly represented by tiles representing image items and their overviews. This means there is no QuadtreeCellGroupBox structure in the file. Tiles can be linked to the region items contained in the tile using a 'reco' type item reference from the entity representing the tile to the region item. Possibly, one or more item attributes can be associated with the entity representing the tile to describe information related to the region items it contains. For example, an item attribute can describe the minimum and maximum rendering scale of the region annotation associated with the tile. Possibly, image items can be linked to the region item associated with the root of the quadtree using a 'reco' type item reference from the image item to the region item.

[0260] Possibly, in these embodiments, the quadtree can be directly represented by tiles of image items. Tiles can be linked to entities to groups of TileGroupBoxes, having a structure similar to a QuadtreeCellGroupBox containing the region items contained within the tiles, using, for example, an item reference of type 'reco' from the entity representing the tile. Possibly, images can be linked to TileGroupBoxes containing region items associated with the root of the quadtree, using, for example, an item reference of type 'reco' from the image item to the TileGroupBox.

[0261] In the seventh embodiment, the entire quadtree can be represented as a single QuadtreeGroupBox with the following structure:

[0262] class QuadtreeGroupBox(version, flags)

[0263] extends EntityToGroupBox('qdce', version, flags) {

[0264] unsigned int(16) cell_count;

[0265] for (i = 0; i < count; i++) {

[0266] unsigned int(16) region_count;

[0267] unsigned int(8) children_presence;

[0268] unsigned int(32) left;

[0269] unsigned int(32) top;

[0270] unsigned int(32) cell_width;

[0271] unsigned int(32) cell_height;

[0272] unsigned int(32) minimum_scale_numerator;

[0273] unsigned int(32) maximum_scale_numerator;

[0274] unsigned int(32) scale_denominator;

[0275] }

[0276] }

[0277] Advantageously, according to the invention, this embodiment limits the number of entity-to-group boxes in the document. Therefore, it reduces the document size.

[0278] The cells of the quadtree represented by the QuadtreeGroupBox are described in breadth-first order, and the sub-cells are described in left-right and top-bottom order.

[0279] The semantics of the fields in QuadtreeGroupBox are as follows.

[0280] cell_count indicates the number of quadtree cells in the quadtree.

[0281] The `region_count` indicates the number of region items contained in the quadtree cell. Using the `entity_id` field of the `EntityToGroupBox`, the identifiers of these region items are listed in the `QuadtreeGroupBox` after the region items in the previously described quadtree cell.

[0282] For each quadtree cell, the children_presence, left, top, cell_width, cell_height, minimum_scale_numerator, maximum_scale_numerator, and scale_denomiator fields have the same meaning as in the QuadtreeCellGroupBox structure of the first embodiment.

[0283] Other embodiments can be combined with these seventh embodiments, for example, to provide different descriptions for each quadtree cell. Possibly, the QuadtreeGroupBox structure can allow for different descriptions for different quadtree cells.

[0284] In the eighth embodiment, the region groups are not organized in a tree structure. Instead, to indicate the area and rendering scale corresponding to each region item associated with the image, these region items are grouped into one or more region partition groups (also denoted as region partition entity groups).

[0285] A region partitioning group (or region partitioning entity group, such as one with grouping_type 'rgpa' or any other 4CC not yet used by the standard) lists the region entries corresponding to the regions of an image entry. For example, a region partitioning group can list the region entries contained within a region of an image. The region is the area covered by the region partitioning group and can be defined as a rectangle in the image. The region can have another shape.

[0286] A region covered by a region partitioning group can be defined within the region partitioning group structure itself, or it can be a region of an image item associated with the region partitioning group via an item reference of type 'rpds' (or any other 4CC not yet used by the standard), for example, representing a "region partitioning description." This item reference proceeds from the image item to the region partitioning group. The names and four-character codes used here are merely examples; any name or four-character code that does not conflict with existing or registered names or four-character codes can be used. For example, an item reference of type 'cdsc' from the region partitioning group to the image item could be used instead of an item reference of type 'rpds'.

[0287] The region partition group associated with an image item preferably contains only region items associated with that image item or with another image item that is part of that image item. For example, the region partition group associated with an image item that is input to a grid image item may contain region items associated with the grid image item.

[0288] Preferably, when a region item is included in a region partition group, at least one of its regions intersects with a region covered by the region partition group. Other relationships between region items and regions covered by region partitions can be used, as described above for quadtree cells or tree cells.

[0289] A region partition group preferably lists only the region items it contains. A region partition group may list all the region items it contains. When the rendering size of the area covered by the region partition group is greater than or equal to the display area, the region partition group may list only the region items it contains and is suitable for rendering. In other words, a region partition group may list region items only if the rendering scale associated with a region item is greater than or equal to the rendering scale associated with the region partition group. Other constraints on the rendering scale of region items and region partition groups may be used, such as those described above.

[0290] When rendering a portion of an image item, the renderer can use region partitioning groups to select which region items to parse and render. The renderer can render using a region partitioning group associated with the image item, or using a region partitioning group associated with the item corresponding to a portion of the image item. For example, when rendering a mesh image item, the renderer can use a region partitioning group associated with the input image item of that mesh image item. As another example, when rendering an HEVC image item, the renderer can use a region partitioning group associated with the HEVC tile item that is part of that HEVC image item.

[0291] A regional partition group can be represented by the following structure.

[0292] aligned(8) class RegionPartitionGroupBox

[0293] extends EntityToGroupBox('rgpa', version, flags) {

[0294] unsigned int field_size = ((flags & 1) + 1) * 16;

[0295] if (flags & 2 == 2) {

[0296] unsigned int(field_size) reference_width;

[0297] unsigned int(field_size) reference_height;

[0298] unsigned int(field_size) top;

[0299] unsigned int(field_size) left;

[0300] unsigned int(field_size) width;

[0301] unsigned int(field_size) height;

[0302] }

[0303] }

[0304] The semantics of the fields in RegionPartitionGroupBox are as follows.

[0305] Define the following flag values ​​to describe the regions covered by a set of region annotation entries included in a region partition group:

[0306] The value 0x000001, when set, specifies that the lengths of the fields reference_width, reference_height, top, left, width, and height are 32 bits. When not set, it specifies that the lengths of the fields reference_width, reference_height, top, left, width, and height are 16 bits.

[0307] The value 0x000002, when set, specifies the existence of the fields reference_width, reference_height, top, left, width, and height. When not set, it specifies that the fields reference_width, reference_height, top, left, width, and height do not exist.

[0308] `reference_width` and `reference_height` specify the width and height of the reference space in pixels, respectively, and specify the region partitioning group in the reference space.

[0309] `top` and `left` specify the coordinates of the top-left corner of the region covered by the region partition group relative to the reference space.

[0310] width and height specify the coordinates of the width and height of the area covered by the area partition group relative to the reference space.

[0311] In these embodiments, Figure 2 Steps 200 to 280 shown Figure 3 Steps 300 to 350 shown can be used by applying them to region partition groups instead of quadtree cells.

[0312] In these embodiments, at step 220, the region partitioning information may indicate the existence of a group of region partitions for the image obtained at 200 and a group of region partitions for each portion of the image obtained at 200. For example, there may be a group of region partitions for a grid image item and a group of region partitions for each input item of the grid image item. The region partitioning information may also indicate the structure of the group of region partitions, which is similar to the structure described above for a quadtree cell.

[0313] In these embodiments, at step 310, a region partition group is obtained by retrieving all region partition groups associated with the image item obtained at step 300. Additionally, region partition groups associated with items corresponding to portions of the image item obtained at step 300 can also be retrieved. For example, if the image item obtained at step 300 is a grid image item, then region partition groups associated with the input image item of that grid image item can also be retrieved.

[0314] Other embodiments may be combined with these eighth embodiments, for example, to provide different descriptions of regional partition groups.

[0315] Figure 5a An embodiment corresponding to the eighth embodiment is illustrated. In this example, a mesh image item 500 is constructed from six HEVC image items. The mesh image item is associated with these input image items via an item reference of type 'dimg'. The mesh image item is also associated with several region items of type 'rgan'. Two region partition groups 520 and 521 of type 'rgpa' are associated with mesh image item 500 and image item 510, which is one of the input image items as mesh image items, respectively. Preferably, the regions of these two region partition groups are signaled as regions of the respective image items associated with them. The regions associated with region partition groups 520 and 521 are indicated by reference numerals 522 and 523, respectively. Region partition group 520 contains region items that should always be rendered. Region partition group 521 contains region items contained in the regions of input image item 510, and these region items should be rendered when the size of input image item 510 is approximately the size of the rendering area. For example, when the size of the input image item is between half and twice the size of the rendering area, it is preferable to render the region item contained in the region partition group 521.

[0316] Figure 5b Another embodiment corresponding to the eighth embodiment is illustrated. An HEVC image item 550 of type 'hvc1' is associated with several region items of type 'rgan'. Two region partition groups 560 and 561 of type 'rgpa' are associated with the image item. The region of region partition group 560 is the entire image. Preferably, the region is signaled as the region of the image item associated with region partition group 560. Region partition group 560 contains region items that should always be rendered. The region of region partition group 561 is associated with… Figure 5aThe lower central portion of image item 550 corresponding to input image item 510. The region associated with region group 560 is indicated by reference numeral 562. Preferably, the region is signaled within the structure of region group 561. Region group 561 contains region items contained in the lower central region of image item 550, and these region items should be rendered when the size of the lower central region is approximately the size of the rendering area. For example, when the size of the lower central region is between two-thirds and one and a half times the size of the rendering area, the region items contained in region group 561 should be rendered. The region associated with region group 561 is indicated by reference numeral 563.

[0317] In some embodiments, at step 220, the rendering conditions for region annotations may include a priority. This priority can be defined for each region annotation. The priority can depend on the type of annotation. The priority can be indicated as a list of high-priority region annotations.

[0318] At step 250, when a region annotation is associated with a quadtree cell, if the number of region annotations associated with that quadtree cell exceeds a predefined threshold, the region annotation with the lowest priority associated with that quadtree cell is removed from the association. If several region annotations have the same lowest priority, one of them is selected. For example, the smallest region can be selected. The region annotation removed from the association can be associated with one or more sub-cells of the quadtree cell. A region annotation can be associated with a sub-quadtree cell whose properties match the region annotation rendering conditions.

[0319] If the quadtree cell for which the associated region annotation has been removed has no child cells, one or more new child quadtree cells may be created for that quadtree cell.

[0320] If a quadtree cell that removes an associated region annotation does not have a corresponding subcell, one or more new subquadtree cells may be created for that quadtree cell.

[0321] If a quadtree cell with more than a predefined threshold of associated region annotations has no child cells, then the associated region annotations may not be removed from the association.

[0322] If a quadtree cell has no child cells and the number of associated region annotations exceeds a predefined threshold, the associated region annotation is removed from the association without associating it with any other quadtree cell.

[0323] In these embodiments, at step 250, preferably, the region annotation is associated with one or more quadtree cells belonging to the lowest layer (i.e., the layer closest to the root) that matches the rendering scale.

[0324] Possibly, in these embodiments, only a portion of the quadtree is constructed at step 220. When the number of region annotations associated with a quadtree cell becomes greater than a predefined threshold, the remaining portion of the quadtree is constructed at step 250 by expanding the quadtree.

[0325] In variations of these embodiments, the rendering conditions for a region annotation may include a occupancy value corresponding to that region annotation. This occupancy value represents the space required to render the region annotation. A larger occupancy value corresponds to a region annotation that requires a larger area for its rendering. For example, if a region annotation corresponds to a label or text to be displayed on an image, a longer label or text would correspond to a larger occupancy value. As another example, the occupancy value may be the area of ​​the rendered region (considering the width of the rendered edges).

[0326] Possibly, in these embodiments, priority and / or occupancy values ​​can be described in a file generated at step 280 (e.g., within the item attributes). Possibly, a single value can be described for a region item by combining different values ​​of the regions it contains. Possibly, values ​​corresponding to individual regions can be described.

[0327] At the start of step 340, the priority and / or occupancy values ​​of the region items contained in the retained quadtree cells can be retrieved from the file. These priority and / or occupancy values ​​can be used to further filter the region items. For example, only a predefined number of region items can be retained, thus retaining only the region items with the highest priority.

[0328] In some embodiments, the rendering scale range is defined only by the minimum value, not the maximum value. A quadtree cell has a minimum rendering scale. A region annotation has a minimum rendering scale. At step 250, if the rendering scale of a region annotation is greater than or equal to the rendering scale of a quadtree cell, then the rendering scale of the region annotation matches the rendering scale of the quadtree cell. Alternatively, the rendering scale of a region annotation may match only the minimum rendering scale among the quadtree cells that satisfy the condition that the rendering scale of the region annotation is greater than or equal to the rendering scale of the quadtree cell.

[0329] In some embodiments of the invention, an item attribute can be associated with a region item to indicate its bounding box. This attribute can be a RegionBoundingBox item attribute of type 'rbox' with the following structure:

[0330] aligned(8) class RegionBoundingBox

[0331] extends ItemFullProperty('rbox', version = 0, flags) {

[0332] unsigned int field_size = ((flags & 1) + 1) * 16;

[0333] unsigned int(field_size) top;

[0334] unsigned int(field_size) left;

[0335] unsigned int(field_size) width;

[0336] unsigned int(field_size) height;

[0337] }

[0338] The semantics of the 'rbox' item attribute are as follows.

[0339] (flags & 1) When equal to 0, it specifies that the length of the top, left, width, and height fields is 16 bits. (flags & 1) When equal to 1, it specifies that the length of the top, left, width, and height fields is 32 bits.

[0340] `top` and `left` are the positions of the top-left corner of the bounding box of any area item associated with the item attribute.

[0341] `width` and `height` are the sizes of the bounding box of any area item associated with the item attribute.

[0342] In these embodiments, at step 280, when generating a region item representing a region annotation, an 'rbox' item attribute can be created to specify the bounding box containing all regions described by and associated with the region item.

[0343] In these embodiments, at step 340, the region items contained in the retained quadtree cells are filtered for the first time based on their bounding boxes. Region items whose bounding boxes intersect with the region to be displayed are retained. Region items whose bounding boxes do not intersect with the region to be displayed are discarded.

[0344] Possibly, the top, left, width, and height values ​​are expressed as the coordinates of the image item associated with the region item.

[0345] It is possible to express the top, left, width, and height values ​​in the reference space of the region item.

[0346] Possibly, in these embodiments, the size of the minimum quadtree cell is larger than in other embodiments. The quadtree cell can be used for the first filtering of region items, while the bounding box information can be used for the second filtering of region items.

[0347] Advantageously, specifying the bounding box of a region item can help determine whether it is needed when rendering the region of an image item associated with that region item.

[0348] Alternatively, the 'ispe' and 'rloc' item attributes can be used to provide the size and position of the bounding boxes of all regions contained in the containing region item, respectively.

[0349] In some embodiments, the item attribute can be used instead of RegionPartitionGroupBox to describe region partitions.

[0350] Different embodiments or combinations thereof can be adapted to store files of 3D media. For example, if the file stores 3D point cloud items using, for example, G-PCC (ISO / IEC 23080-18) or V3C (ISO / IEC 23090-10), then regions can be defined as 3D volumes. In this case, region items can be grouped into OctreeCellGroupBoxes, whose structure or description is similar to QuadtreeCellGroupBox.

[0351] As an example, similar to the first embodiment, an octree cell can have eight sub-cells organized into 2 rows, 2 columns, and 2 depths. An OctreeCellGroupBox can be represented by the following structure:

[0352] class OctreeCellGroupBox(version, flags)

[0353] extends EntityToGroupBox('occe', version, flags) {

[0354] unsigned int(8) children_presence;

[0355] unsigned int(32) x;

[0356] unsigned int(32) y;

[0357] unsigned int(32) z;

[0358] unsigned int(32) cell_width;

[0359] unsigned int(32) cell_height;

[0360] unsigned int(32) cell_depth;

[0361] unsigned int(32) minimum_scale_numerator;

[0362] unsigned int(32) maximum_scale_numerator;

[0363] unsigned int(32) scale_denominator;

[0364] }

[0365] An OctreeCellGroupBox can be associated with its child cells using an item reference of type 'occh'. The child cells are listed in ascending order of X, Y, and Z.

[0366] The semantics of the fields in OctreeCellGroupBox, which are shared with QuadtreeCellGroupBox, are similar. The semantics of the other fields in OctreeCellGroupBox are as follows.

[0367] x, y, z indicate the position of the angle closest to the origin of the volume corresponding to the OctreeCellGroupBox.

[0368] cell_width, cell_height, and cell_depth indicate the size of the volume corresponding to the OctreeCellGroupBox.

[0369] All these embodiments can be appropriately combined together.

[0370] Possibly, in some embodiments, a region item of type 'rgan' is not associated with an image item using an item reference of type 'cdsc', but rather through a grouping structure such as one of the grouping structures described above. For example, a region item can be associated with an image item by being included in a QuadtreeCellGroupBox that is part of a quadtree associated with the image item. As another example, a region item can be associated with an image item by being included in a RegionPartitionGroupBox associated with the image item.

[0371] Preferably, in these embodiments, the contents of region items contained in the same grouping structure are stored close to each other in a HEIF, MIAF, or ISOBMF-based file. For example, at step 280, when the file is generated, the region items can be sorted according to the grouping structure containing them. The contents of these region items are then stored following this order. Advantageously, this allows the contents of region items contained in the grouping structure to be retrieved at step 340 by accessing a single portion of the file.

[0372] In these embodiments, the information signaled in the EntityToGroupBox can be signaled in other structures of HEIF, MIAF, or any other ISOBMFF-based file. For example, some fields of the QuadtreeCellGroupBox can be signaled in the item properties associated with the QuadtreeCellGroupBox.

[0373] These embodiments can be used in conjunction with an overview to indicate which region items correspond to which resolution. In particular, region partition groups are useful when combined with an overview. In effect, region partition groups can describe which region items are suitable for rendering at a given scale and at a given location in an image item.

[0374] Figure 6 This is a schematic block diagram of a computing device 600 for implementing one or more embodiments of the present invention. The computing device 600 may be a device such as a microcomputer, workstation, or lightweight portable device. The computing device 600 includes a communication bus connected to:

[0375] - The central processing unit 601, labeled as CPU, such as a microprocessor;

[0376] - Random access memory 602, labeled RAM, is used to store executable code of the methods of embodiments of the present invention and registers adapted to record variables and parameters required to implement the methods according to embodiments of the present invention. Its memory capacity can be expanded, for example, by an optional RAM connected to an expansion port.

[0377] - A read-only memory 603, labeled ROM, is used to store a computer program for implementing embodiments of the present invention;

[0378] - Network interface 604, which is typically connected to a communication network for transmitting or receiving digital data to be processed. Network interface 604 can be a single network interface or a group of different network interfaces (e.g., wired and wireless interfaces, or different types of wired or wireless interfaces). Under the control of a software application running in CPU 601, data packets are written to the network interface for transmission or read from the network interface for reception.

[0379] - The graphical user interface 605 can be used to receive input from the user or display information to the user;

[0380] - The HD-marked 606 hard drive can be provided as a high-capacity storage device;

[0381] - I / O module 607 can be used to receive / send data to external devices such as 3D video sources or displays.

[0382] The executable code can be stored in read-only memory 603, hard disk 606, or a removable digital medium such as a disk. According to a variation, the executable code of the program can be received via a communication network through network interface 604 and stored in one of the storage components (such as hard disk 606) of the communication device 600 before being executed.

[0383] The central processing unit 601 can be adapted to control and direct the execution of instructions or portions of software code of one or more programs according to embodiments of the present invention, these instructions being stored in one of the aforementioned storage components. For example, upon power-up, the CPU 601 is capable of executing software application-related instructions from the main RAM memory 602 after the instructions have been loaded from the program ROM 603 or the hard disk (HD) 606. Such a software application, when executed by the CPU 601, enables the steps of the flowchart of the present invention to be performed.

[0384] Any step of the algorithm of the present invention may be implemented in software by executing a set of instructions or a program by a programmable computing machine (such as a PC (“personal computer”), DSP (“digital signal processor”) or microcontroller); or in hardware by a machine or special-purpose component (such as a FPGA (“field-programmable gate array”) or ASIC (“application-specific integrated circuit”)).

[0385] Although the present invention has been described above with reference to specific embodiments, the present invention is not limited to the specific embodiments, and modifications falling within the scope of the present invention will be apparent to those skilled in the art.

[0386] Many further modifications and variations will arise for those skilled in the art when referring to the foregoing illustrative embodiments. These embodiments are given by way of example only and are not intended to limit the scope of the invention, which is defined only by the appended claims. In particular, different features from different embodiments may be interchanged where appropriate.

[0387] The embodiments of the invention described above can each be implemented individually or as a combination of multiple embodiments. Furthermore, features from different embodiments can be combined where necessary, or where a combination of elements or features from various embodiments in a single embodiment is advantageous.

[0388] In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite articles "a" or "an" do not exclude plural. The fact that different features are defined only in mutually different dependent claims does not mean that combinations of these features cannot be used advantageously.

Claims

1. A method for encapsulating an image in an ISOBMFF-based media file, wherein, The method includes: Multiple region items are generated, each region item describing the geometry of a region in the image; Generate a data structure that represents the regions of the image and lists the region items contained within the regions of the image from the plurality of region items; The media file is generated, and the generated media file includes the image, the plurality of region items, and the data structure.

2. The method according to claim 1, wherein, The data structure includes a reference space and a description of the regions of the image represented by the data structure relative to the reference space.

3. The method according to claim 2, wherein, The data structure also includes an indication of whether the data structure includes the reference space and the description of the region of the image represented by the data structure relative to the reference space.

4. The method according to claim 1, wherein, The region of the image represented by the data structure is the region of the image associated with the data structure.

5. The method according to claim 4, wherein, The data structure includes an indication of whether a region of the image represented by the data structure is a region of the image associated with the data structure.

6. The method according to any one of claims 1 to 5, wherein, The image is a grid of multiple input images, and the data structure is associated with one of the input images.

7. The method according to any one of claims 1 to 5, wherein, The data structure is associated with the image.

8. The method according to any one of claims 1 to 7, wherein, The plurality of region items are associated with the image.

9. The method according to any one of claims 1 to 8, wherein, The region of the image represented by the data structure is a rectangular partition of the image.

10. The method according to any one of claims 1 to 9, wherein, The data structure is an extension of EntityToGroupBox.

11. The method according to any one of claims 1 to 10, wherein, The data structure uses references associated with images.

12. A method for processing image-related data from ISOBMFF-based media files, wherein, The method includes: Obtain the region of the image; A data structure is obtained from the media file, the data structure representing the regions of the image and listing region items contained within the regions of the image from a plurality of region items in the media file, each of the plurality of region items describing the geometry of the regions of the image; Based on the listed region items, obtain at least one region item from the plurality of region items; and Process at least one region item obtained.

13. The method according to claim 12, wherein, The data structure is obtained based on the rendering ratio of the region of the image to be processed.

14. The method according to claim 12 or 13, wherein, The data structure includes a reference space and a description of the regions of the image represented by the data structure relative to the reference space.

15. The method according to claim 14, wherein, The data structure also includes an indication of whether the data structure includes the reference space and the description of the region of the image represented by the data structure relative to the reference space.

16. The method according to claim 12 or 13, wherein, The region of the image represented by the data structure is the region of the image associated with the data structure.

17. The method according to claim 16, wherein, The data structure includes an indication of whether a region of the image represented by the data structure is a region of the image associated with the data structure.

18. The method according to any one of claims 12 to 17, wherein, The region of the image represented by the data structure is a rectangular partition of the image.

19. The method according to any one of claims 12 to 18, wherein, The data structure is an extension of EntityToGroupBox.

20. The method according to any one of claims 12 to 19, wherein, The data structure uses a reference item associated with the image.

21. The method according to any one of claims 12 to 19, wherein, The image is a grid of multiple input images, and the data structure is associated with one of the input images.

22. A computer program product for a programmable device, the computer program product comprising a sequence of instructions configured to implement, when loaded into and executed by the programmable device, the steps of the method according to any one of claims 1 to 21.

23. A non-transitory computer-readable storage medium storing instructions for a computer program that implements the steps of the method according to any one of claims 1 to 21.

24. An apparatus including a processing unit configured to perform various steps of the method according to any one of claims 1 to 21.

Images