Data encapsulation method and electronic device

Abstract

Embodiments of the present application provide a data encapsulation method and an electronic device. The method comprises: first, obtaining T.35 information; wherein the T.35 information comprises a country code, a terminal provider code and a terminal provider orientation code; then, encapsulating the T.35 information into an image file format; wherein the image file format comprises a first box, and the first box comprises the T.35 information. In this way, the file can carry the T.35 information; when the T.35 information carries metadata (including HDR metadata) of multimedia, the HDR image can be correctly displayed by restoring and correctly using the HDR metadata at a decoding end.

Description

Data packaging method and electronic device Technical Field

[0001] The embodiments of the present application relate to the field of data processing, and in particular to a data encapsulation method and electronic device. Background Art

[0002] Currently, some metadata can be encapsulated in image file formats such as High Efficiency Image File Format (HEIF) and AV1 Image File Format (AVIF), but these image file formats do not support the encapsulation of HDR metadata that is commonly found in the T.35 information of the High Dynamic Range (HDR) standard. The T.35 information can carry information specified by the T.35 specification "Procedure for the Allocation of ITU-defined Non-standard Extension Codes" published by the ITU-T (International Telecommunication Union Telecommunication Standardization Branch). Therefore, existing image file formats have no way to carry T.35 information, nor can they recover and use the correct HDR metadata at the decoding end (for example, performing tone mapping and other operations based on the correct HDR metadata) to complete the correct display of HDR images.

[0003] Summary of the Invention

[0004] In view of this, the present application provides a data packaging method and an electronic device.

[0005] In a first aspect, an embodiment of the present application provides a data encapsulation method, which includes: first, obtaining T.35 information; wherein the T.35 information includes: a country code (Country code), a terminal provider code (Terminal provider code) and a terminal provider oriented code (Terminal provider oriented code); then, encapsulating the T.35 information into an image file format; wherein the image file format includes a first box, and the first box includes the T.35 information.

[0006] Among them, the present application can pre-define T.35 information and define the position of the T.35 information in the image file format according to the regulations / specifications of the image file format; then, the T.35 information can be encapsulated into the image file format according to the pre-defined T.35 information and the position of the T.35 information; in this way, the image file format can carry T.35 information; when the T.35 information carries multimedia metadata (which may include HDR metadata), the HDR metadata can be restored and correctly used at the decoding end to complete the correct display of the HDR image.

[0007] For example, the present application can add a custom box based on the existing image file format, and the newly added box can be used to store T.35 information; and define the location of the newly added box. In order to distinguish the newly added box from the existing box in the existing image file format, the newly added box can be called the first box (in some scenarios, it can also be called it35 box, or other names, which are not limited by this application). In other words, the present application can encapsulate the T.35 information into the first box of the image file format.

[0008] For example, image file formats may include, but are not limited to, JPEG, WebP, PNG, HEIF, AVIF, and JXL, etc., which are not limited in this application. This application uses HEIF as an example for illustration.

[0009] For example, a box is a general mechanism for organizing and storing data, enabling files to contain a variety of information types in a flexible manner. The hierarchical structure and contents of a box are defined by corresponding standard specifications to ensure correct parsing and processing of files. The term "box" may be used by other names in other image file formats or in future image file formats, and this application does not impose any restrictions on the name of the box.

[0010] According to the first aspect, the image file format further includes a second box, the second box including any one of the following: a metadata box or a stream data box; the first box is nested in the second box.

[0011] Exemplarily, a metadata box may be used to store metadata of multimedia and may be referred to as MetaBox (meta).

[0012] Exemplarily, a stream data box may be used to store an image sequence and may be referred to as a MovieBox (moov).

[0013] That is to say, the first box can be nested in MetaBox or MovieBox, and this application does not impose any restrictions on this.

[0014] According to the first aspect, or any implementation of the first aspect above, the second box includes multiple third boxes, and the multiple third boxes have a hierarchical relationship; the first box is nested in the third box at any level.

[0015] Exemplarily, when the second box is a metadata box, the metadata box may include multiple third boxes, including but not limited to: item Information Box (iinf), item Information Entry Box (infe), item Property Container Box (ipco), item Properties Assocation Box (ipma), and item Location Box (iloc), etc.; the hierarchical relationship of these multiple third boxes can be as shown in Structure 1 in the specific implementation manner; the first box can be nested in the third box at any level.

[0016] When the second box is a stream data box, the stream data box may include multiple third boxes, including but not limited to: SampleDescriptionBox (stsd), Sample Group Description Box (sgpd), metadata Sample Entry Box, timed metadata Track Box, Track Box, etc. The hierarchical relationship of the multiple third boxes can be as shown in Structure 2 in the specific implementation method; the first box can be nested in the third box at any level.

[0017] According to the first aspect, or any implementation of the first aspect above, the image file format also includes a project identifier and a media box, the project identifier indicates that the T.35 information is a project; the T.35 information includes header information and a payload, the header information includes a country code, a terminal provider code and a terminal provider orientation code, and the payload includes multimedia metadata; the first box includes the header information of the T.35 information, and the media box includes the payload of the T.35 information.

[0018] For example, an item is data that does not require regular processing (ie, untimed data), as opposed to sample data. An item identifier may be referred to as an item ID.

[0019] For example, the effective load may be referred to as payload; and the header information may be referred to as header.

[0020] Exemplarily, the metadata of the multimedia may include HDR metadata.

[0021] Exemplarily, HDR metadata may include: static metadata and / or dynamic metadata.

[0022] For example, static metadata, including the mastering display's color space, the master display's maximum and minimum brightness, the video sequence's maximum brightness, and the maximum average brightness of the video sequence, is functionally used to ensure accurate display on the reference display and restore the creator's intent. In a video sequence, this is static data that doesn't change over time. For example, the maximum brightness of a video sequence is the maximum value of the maximum brightness across all frames.

[0023] For example, dynamic metadata includes most static metadata and also includes metadata related to tone mapping on the display device. Functionally, this information is closely related to display and tone mapping processes, such as indicating the selection of tone mapping algorithms and parameters for shifting from a high dynamic range to a relatively low dynamic range. In video sequences, dynamic metadata often changes over time. For example, when a scene changes, the maximum brightness of the image and the maximum brightness of the primary display also change accordingly.

[0024] Exemplarily, a media box may be used to store image data and may be referred to as MediaDataBox (mdat).

[0025] This method is applicable to the case where the T.35 information carries static metadata.

[0026] It should be understood that multimedia metadata can be image metadata, audio metadata, video metadata, etc., and the embodiments of the present application do not limit this.

[0027] According to the first aspect, or any implementation of the first aspect, the T.35 information is an item, the metadata box includes an image information box, and the first box is nested in the image information box, and the first box is an item information entry box.

[0028] Illustratively, an image information box may be used to store item information and may be referred to as an item information box (iinfBox).

[0029] For example, an item information entry box can be used to store item information entries and can be called an item information entry box (infe Box). In other words, the first box is an item information entry box.

[0030] This method is applicable to the case where the T.35 information carries static metadata.

[0031] According to the first aspect, or any implementation of the first aspect above, the T.35 information is a project attribute, and the metadata box includes a project attribute container box and a project attribute association box; the first box is nested in the project attribute container box, and the project attribute association box includes association information between the T.35 information and the corresponding project.

[0032] For example, the project attribute can be referred to as item property.

[0033] Illustratively, an item property container box may be used to store item properties and may be referred to as an item property container box (ipco Box).

[0034] Illustratively, the item properties association box may be used to store item properties association information, and may be referred to as an item properties association box (ipma box).

[0035] For example, Property is an attribute related to item.

[0036] This method is applicable to the case where the T.35 information carries static metadata.

[0037] According to the first aspect, or any implementation of the first aspect above, the T.35 information is a sample entry, the stream data box includes a track box, the track box includes a sample description box; and the first box is nested in the sample description box.

[0038] For example, a sample entry can be called a Sample Entry. A sample is all data associated with a single time. In other words, a sample is data at a certain point in time, or a type of timed data.

[0039] For example, a track box, also known as a track box, is a timed sequence of related samples in an ISO base media file. For multimedia, a track corresponds to a sequence of images or sampled audio; for a hint track, a track corresponds to a streaming media channel.

[0040] It should be noted that in HEIF, continuous or timed media or metadata streams form a track, while static media or metadata are stored as items.

[0041] Illustratively, a sample description box may be used to store a sample entry (Sample Entry), and may be referred to as a SampleDescriptionBox (stsd Box).

[0042] This method is applicable to the case where the T.35 information carries static metadata.

[0043] According to the first aspect, or any implementation of the first aspect above, the T.35 information is a sample group entry, the stream data box includes a track box, the track box includes a sample group description box; and the first box is nested in the sample group description box.

[0044] Illustratively, the sample group description box may store attributes describing a sample group (Sample Group) and may be referred to as a Sample Group Description Box (sgpd Box).

[0045] Since all entries in a SampleToGroupBox or CompactSampleToGroupBox with the same grouping_type_parameter value should map to the same type of T.35 message, which means at least T.35 sample group entries with the same ITU-T T.35 country code, terminal provider code, terminal provider orientation code, and T.35 specific type and version information (if applicable), this is useful when the metadata carried by the T.35 message is not frequent (i.e., the amount of identical metadata is not large) and does not cause problems due to the size of the MovieBox.

[0046] This approach can reduce the redundancy of storing the same metadata in each sample and is applicable to situations where the T.35 information carries static metadata or dynamic metadata.

[0047] According to the first aspect, or any implementation of the first aspect above, the T.35 information is a sample entry, the stream data box includes a track box, the track box includes a sample description box and a sample group description box; the T.35 information includes header information and a payload, the header information includes a country code, a terminal provider code and a terminal provider orientation code, and the payload includes multimedia metadata; the first box includes the header information of the T.35 information, and the first box is nested in the sample group description box; the sample description box includes the payload of the T.35 information.

[0048] This allows very frequent (very large amounts of identical) T.35 metadata to be processed.

[0049] This method is applicable to the case where the T.35 message carries static metadata or dynamic metadata, where the dynamic metadata is part of the payload of the T.35 message.

[0050] According to the first aspect, or any implementation of the first aspect above, the T.35 information is a sample entry, the stream data box includes a timed metadata track box, the timed metadata track box includes a metadata sample entry box, and the first box is nested in the metadata sample entry box.

[0051] Illustratively, a timed metadata track box may be used to store timed metadata and may be referred to as a timed metadata Track Box.

[0052] Illustratively, a metadata sample entry box may be used to store timed metadata entries and may be referred to as a metadata Sample Entry Box.

[0053] This method is applicable to the case where the T.35 information carries dynamic metadata.

[0054] According to the first aspect, or any implementation of the first aspect above, the method further includes: obtaining the encoded multimedia data; and encapsulating the encoded multimedia data into an image file format, wherein the image file format further includes a media box or a stream data box, wherein the media box or the stream data box includes the encoded multimedia data. In this way, the encoded multimedia data and the T.35 information can be encapsulated into the same image file format.

[0055] According to the first aspect, or any implementation of the first aspect above, the T.35 information further includes multimedia metadata.

[0056] In a second aspect, an embodiment of the present application provides a data acquisition method, which includes: first, receiving an image file format; wherein the image file format includes a first box, and the first box includes T.35 information; then, reading the T.35 information from the first box of the image file format; wherein the T.35 information includes: a country code, a terminal provider code, and a terminal provider orientation code.

[0057] According to the second aspect, the image file format further includes a second box, the second box including any one of the following: a metadata box or a stream data box; the first box is nested in the second box.

[0058] According to the second aspect, or any implementation of the second aspect, the second box includes multiple third boxes, and the multiple third boxes have a hierarchical relationship; the first box is nested in the third box at any level.

[0059] According to the second aspect, or any implementation of the above second aspect, the image file format also includes a project identifier and a media box, the project identifier indicates that the T.35 information is a project; the T.35 information includes header information and a payload, the header information includes a country code, a terminal provider code and a terminal provider orientation code, and the payload includes multimedia metadata; the first box includes the header information of the T.35 information, and the media box includes the payload of the T.35 information.

[0060] According to the second aspect, or any implementation of the second aspect, the T.35 information is an item, the metadata box includes an image information box, and the first box is nested in the image information box, and the first box is an item information entry box.

[0061] According to the second aspect, or any implementation method of the above second aspect, the T.35 information is a project attribute, and the metadata box includes a project attribute container box and a project attribute association box; the first box is nested in the project attribute container box, and the project attribute association box includes association information between the T.35 information and the corresponding project.

[0062] According to the second aspect, or any implementation of the second aspect, the T.35 information is a sample entry, the stream data box includes a track box, the track box includes a sample description box; and the first box is nested in the sample description box.

[0063] According to the second aspect, or any implementation of the second aspect, the T.35 information is a sample group entry, the stream data box includes a track box, the track box includes a sample group description box; and the first box is nested in the sample group description box.

[0064] According to the second aspect, or any implementation of the above second aspect, the T.35 information is a sample entry, the stream data box includes a track box, the track box includes a sample description box and a sample group description box; the T.35 information includes header information and a payload, the header information includes a country code, a terminal provider code and a terminal provider orientation code, and the payload includes multimedia metadata; the first box includes the header information of the T.35 information, and the first box is nested in the sample group description box; the sample description box includes the payload of the T.35 information.

[0065] According to the second aspect, or any implementation of the second aspect above, the T.35 information is a sample entry, the stream data box includes a timed metadata track box, the timed metadata track box includes a metadata sample entry box, and the first box is nested in the metadata sample entry box.

[0066] According to the second aspect, or any implementation of the second aspect above, the image file format also includes a media box or a stream data box, and the media box or the stream data box includes encoded multimedia data; the method also includes: obtaining the encoded multimedia data from the media box or the stream data box in the image file format; decoding the encoded multimedia data to obtain reconstructed multimedia data.

[0067] The second aspect and any implementation of the second aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the second aspect and any implementation of the second aspect can be referred to the technical effects corresponding to the first aspect and any implementation of the first aspect, and will not be repeated here.

[0068] In a third aspect, an embodiment of the present application provides an image file format, which is encapsulated according to the above-mentioned first aspect or any one of the implementation methods of the first aspect.

[0069] The third aspect and any implementation of the third aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the third aspect and any implementation of the third aspect can be referred to the technical effects corresponding to the first aspect and any implementation of the first aspect, and will not be repeated here.

[0070] In a fourth aspect, an embodiment of the present application provides a data encapsulation device, the data encapsulation device comprising:

[0071] An acquisition module is used to acquire T.35 information, wherein the T.35 information includes: a country code, a terminal provider code, and a terminal provider directional code;

[0072] The encapsulation module is used to encapsulate the T.35 information into an image file format; wherein the image file format includes a first box, and the first box includes the T.35 information.

[0073] Illustratively, the above-mentioned data encapsulation device can be used to execute the method in the first aspect or any possible implementation of the first aspect.

[0074] The fourth aspect and any implementation of the fourth aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the fourth aspect and any implementation of the fourth aspect can be referred to the technical effects corresponding to the first aspect and any implementation of the first aspect, and will not be repeated here.

[0075] In a fifth aspect, an embodiment of the present application provides a data reading device, the data reading device comprising:

[0076] A receiving module, configured to receive an image file format; wherein the image file format includes a first box, and the first box includes T.35 information;

[0077] The reading module is used to read T.35 information from the first box of the image file format; wherein the T.35 information includes: country code, terminal provider code and terminal provider orientation code.

[0078] The fifth aspect and any implementation of the fifth aspect correspond to the second aspect and any implementation of the second aspect, respectively. The technical effects corresponding to the fifth aspect and any implementation of the fifth aspect can be referred to the technical effects corresponding to the above-mentioned second aspect and any implementation of the second aspect, and will not be repeated here.

[0079] In a sixth aspect, an embodiment of the present application provides an electronic device comprising: a memory and a processor, wherein the memory is coupled to the processor; the memory stores program instructions, and when the program instructions are executed by the processor, the electronic device executes the method in the first aspect or any possible implementation of the first aspect.

[0080] The sixth aspect and any implementation of the sixth aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the sixth aspect and any implementation of the sixth aspect can be referred to the technical effects corresponding to the first aspect and any implementation of the first aspect, and will not be repeated here.

[0081] In the seventh aspect, an embodiment of the present application provides an electronic device, comprising: a memory and a processor, the memory being coupled to the processor; the memory storing program instructions, which, when executed by the processor, enables the electronic device to execute the method in the second aspect or any possible implementation of the second aspect.

[0082] The seventh aspect and any implementation of the seventh aspect correspond to the second aspect and any implementation of the second aspect, respectively. The technical effects corresponding to the seventh aspect and any implementation of the seventh aspect can be referred to the technical effects corresponding to the above-mentioned second aspect and any implementation of the second aspect, and will not be repeated here.

[0083] In an eighth aspect, an embodiment of the present application provides a chip comprising one or more interface circuits and one or more processors; the one or more processors receive or send data through the one or more interface circuits, and when the one or more processors execute computer instructions, the first aspect or any possible implementation of the first aspect is executed.

[0084] The eighth aspect and any implementation of the eighth aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the eighth aspect and any implementation of the eighth aspect can be referred to the technical effects corresponding to the first aspect and any implementation of the first aspect, and will not be repeated here.

[0085] In the ninth aspect, an embodiment of the present application provides a chip comprising one or more interface circuits and one or more processors; the one or more processors receive or send data through the one or more interface circuits, and when the one or more processors execute computer instructions, the second aspect or any possible implementation of the second aspect is executed.

[0086] The ninth aspect and any implementation of the ninth aspect correspond to the second aspect and any implementation of the second aspect, respectively. The technical effects corresponding to the ninth aspect and any implementation of the ninth aspect can be referred to the technical effects corresponding to the above-mentioned second aspect and any implementation of the second aspect, and will not be repeated here.

[0087] In the tenth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a computer or a processor, it enables the computer or the processor to execute the method in the first aspect or any possible implementation of the first aspect.

[0088] The tenth aspect and any implementation of the tenth aspect correspond to the first aspect and any implementation of the first aspect, respectively. The technical effects corresponding to the tenth aspect and any implementation of the tenth aspect can be referred to the technical effects corresponding to the first aspect and any implementation of the first aspect, and will not be repeated here.

[0089] In the eleventh aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a computer or a processor, it enables the computer or the processor to execute the method in the second aspect or any possible implementation of the second aspect.

[0090] The eleventh aspect and any implementation of the eleventh aspect correspond to the second aspect and any implementation of the second aspect, respectively. The technical effects corresponding to the eleventh aspect and any implementation of the eleventh aspect can be referred to the technical effects corresponding to the above-mentioned second aspect and any implementation of the second aspect, and will not be repeated here.

[0091] In the twelfth aspect, an embodiment of the present application provides a computer program product, which includes computer instructions. When the computer instructions are executed by a computer or a processor, the computer or the processor executes the method in the first aspect or any possible implementation of the first aspect.

[0092] The twelfth aspect and any implementation of the twelfth aspect respectively correspond to the first aspect and any implementation of the first aspect. The technical effects corresponding to the twelfth aspect and any implementation of the twelfth aspect can be referred to the technical effects corresponding to the above-mentioned first aspect and any implementation of the first aspect, and will not be repeated here.

[0093] In a thirteenth aspect, an embodiment of the present application provides a computer program product, which includes computer instructions. When the computer instructions are executed by a computer or a processor, the computer or the processor executes the method in the second aspect or any possible implementation of the second aspect.

[0094] The thirteenth aspect and any implementation of the thirteenth aspect correspond to the second aspect and any implementation of the second aspect, respectively. The technical effects corresponding to the thirteenth aspect and any implementation of the thirteenth aspect can be referred to the technical effects corresponding to the above-mentioned second aspect and any implementation of the second aspect, and will not be repeated here.

[0095] In a fourteenth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores the image file format in the above-mentioned third aspect or any one of the implementation methods of the third aspect.

[0096] The fourteenth aspect and any implementation of the fourteenth aspect correspond to the third aspect and any implementation of the third aspect, respectively. The technical effects corresponding to the fourteenth aspect and any implementation of the fourteenth aspect can be referred to the technical effects corresponding to the above-mentioned third aspect and any implementation of the third aspect, and will not be repeated here.

[0097] In the fifteenth aspect, an embodiment of the present application provides a device for storing an image file format, the device comprising: a receiver and at least one storage medium, the receiver being used to receive the image file format; and at least one storage medium being used to store the image file format in the above-mentioned third aspect or any one of the implementations of the third aspect.

[0098] The fifth aspect and any implementation of the fifth aspect correspond to the third aspect and any implementation of the third aspect, respectively. The technical effects corresponding to the fifth aspect and any implementation of the fifth aspect can be referred to the technical effects corresponding to the third aspect and any implementation of the third aspect, and will not be repeated here.

[0099] In the sixteenth aspect, an embodiment of the present application provides a device for transmitting an image file format, the device comprising: a transmitter and at least one storage medium, the at least one storage medium being used to store the image file format in the above-mentioned third aspect or any one of the implementation methods of the third aspect; the transmitter being used to obtain the image file format from the storage medium and send the image file format to the end-side device via the transmission medium.

[0100] The sixteenth aspect and any implementation of the sixteenth aspect correspond to the third aspect and any implementation of the third aspect, respectively. The technical effects corresponding to the sixteenth aspect and any implementation of the sixteenth aspect can be referred to the technical effects corresponding to the third aspect and any implementation of the third aspect, and will not be repeated here.

[0101] In the seventeenth aspect, an embodiment of the present application provides a system for distributing image file formats, the system comprising: at least one storage medium for storing the image file format in the above-mentioned third aspect or any one of the implementation methods of the third aspect, a streaming media device for obtaining the target image file format from at least one storage medium and sending the target image file format to the end-side device, wherein the streaming media device comprises a content server or a content distribution server.

[0102] The seventeenth aspect and any implementation of the seventeenth aspect correspond to the third aspect and any implementation of the third aspect, respectively. The technical effects corresponding to the seventeenth aspect and any implementation of the seventeenth aspect can be referred to the technical effects corresponding to the third aspect and any implementation of the third aspect, and will not be repeated here.

[0103] In an eighteenth aspect, an embodiment of the present application provides a data structure, which includes a first box, and the first box includes T.35 information; wherein the T.35 information includes: a country code (Country code), a terminal provider code (Terminal provider code) and a terminal provider oriented code (Terminal provider oriented code).

[0104] Exemplarily, the data structure may be in an image file format.

[0105] According to the eighteenth aspect, the data structure further includes a second box, the second box including any one of the following: a metadata box or a stream data box; the first box is nested in the second box.

[0106] According to the eighteenth aspect, or any implementation of the eighteenth aspect, the second box includes multiple third boxes, and the multiple third boxes have a hierarchical relationship; the first box is nested in the third box at any level.

[0107] According to the eighteenth aspect, or any implementation of the above eighteenth aspect, the data structure also includes a project identifier and a media box, the project identifier indicates that the T.35 information is a project; the T.35 information includes header information and a payload, the header information includes a country code, a terminal provider code and a terminal provider orientation code, and the payload includes multimedia metadata; the first box includes the header information of the T.35 information, and the media box includes the payload of the T.35 information.

[0108] According to the eighteenth aspect, or any implementation of the eighteenth aspect, the T.35 information is a project, the metadata box includes an image information box; the first box is nested in the image information box, and the first box is a project information entry box.

[0109] According to the eighteenth aspect, or any implementation method of the above eighteenth aspect, the T.35 information is a project attribute, and the metadata box includes a project attribute container box and a project attribute association box; the first box is nested in the project attribute container box, and the project attribute association box includes association information between the T.35 information and the corresponding project.

[0110] According to the eighteenth aspect, or any one of the implementations of the eighteenth aspect, the T.35 information is a sample entry, the stream data box includes a track box, the track box includes a sample description box; and the first box is nested in the sample description box.

[0111] According to the eighteenth aspect, or any implementation of the eighteenth aspect, the T.35 information is a sample group entry, the stream data box includes a track box, the track box includes a sample group description box; the first box is nested in the sample group description box.

[0112] According to the eighteenth aspect, or any implementation of the above eighteenth aspect, the T.35 information is a sample entry, the stream data box includes a track box, the track box includes a sample description box and a sample group description box; the T.35 information includes header information and a payload, the header information includes a country code, a terminal provider code and a terminal provider orientation code, and the payload includes multimedia metadata; the first box includes the header information of the T.35 information, and the first box is nested in the sample group description box; the sample description box includes the payload of the T.35 information.

[0113] According to the eighteenth aspect, or any implementation of the eighteenth aspect above, the T.35 information is a sample entry, the stream data box includes a timed metadata track box, the timed metadata track box includes a metadata sample entry box, and the first box is nested in the metadata sample entry box.

[0114] According to the eighteenth aspect, or any one of the implementations of the eighteenth aspect, the data structure further includes a media box or a stream data box, and the media box or the stream data box includes the encoded multimedia data. BRIEF DESCRIPTION OF THE DRAWINGS

[0115] FIG1A is a schematic diagram illustrating an exemplary application scenario;

[0116] FIG1B is a schematic diagram illustrating an end-to-end process of HDR video;

[0117] FIG2A is a schematic diagram illustrating an exemplary data encapsulation process 200;

[0118] FIG2B is a schematic diagram illustrating an exemplary image file format;

[0119] FIG2C is a schematic diagram illustrating an exemplary image file format;

[0120] FIG3 is a schematic diagram illustrating an exemplary data acquisition process 300;

[0121] FIG4 is a schematic diagram of an exemplary data encapsulation device 400;

[0122] FIG5 is a schematic diagram of an exemplary data reading device 500;

[0123] FIG6 is a schematic block diagram of a coding and decoding system used in an embodiment of the present application;

[0124] FIG7A is a block diagram of a content providing system for implementing a content distribution service according to an embodiment of the present application;

[0125] FIG7B is a schematic diagram of an example structure of the terminal device 2106 in FIG7A ;

[0126] FIG8A is a schematic diagram of a workflow of a streaming media system used in an embodiment of the present application;

[0127] FIG8B is a schematic diagram of a streaming media system architecture used in an embodiment of the present application;

[0128] FIG9A is a schematic diagram of a possible system architecture applicable to an embodiment of the present application;

[0129] FIG9B is a schematic diagram of the structure of an image processing system provided in an embodiment of the present application;

[0130] FIG10 is a schematic structural diagram of an exemplary device. DETAILED DESCRIPTION

[0131] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0132] The term "and / or" in this article is merely a description of the association relationship between associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0133] In the description and claims of the embodiments of this application, the terms "first" and "second" are used to distinguish different objects, rather than to describe a specific order of objects. For example, the terms "first target object" and "second target object" are used to distinguish different objects, rather than to describe a specific order of objects.

[0134] In the embodiments of this application, words such as "exemplary" or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of this application should not be interpreted as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary" or "for example" is intended to present the relevant concepts in a concrete manner.

[0135] In the description of the embodiments of this application, unless otherwise specified, "multiple" means two or more. For example, "multiple processing units" means two or more processing units; "multiple systems" means two or more systems.

[0136] FIG1A is a schematic diagram illustrating an exemplary application scenario.

[0137] 1A , exemplarily, the first electronic device may include an image generating module, an information generating module, an image encoding module (or an image encoder), an encapsulating module, or a transmitting module.

[0138] Exemplarily, the image encoding module may be a software module or a hardware module, and the embodiments of the present application do not limit this.

[0139] Exemplarily, the image generation module may include but is not limited to: an image acquisition module, an image editing module, etc., which is not limited in the embodiment of the present application.

[0140] It should be understood that FIG1A is only an example of the first electronic device. The first electronic device in other embodiments of the present application may have more or fewer modules than those shown in FIG1A , and the embodiments of the present application are not limited to this.

[0141] 1A , illustratively, the second electronic device may include a display module, an image decoding module (or image decoder), a decapsulation module, and a receiving module. Exemplarily, the image decoding module may be a software module or a hardware module, and the present application does not limit this. It should be understood that FIG1A is only an example of the second electronic device, and the second electronic device in other embodiments of the present application has more modules than shown in FIG1A , and the present application does not limit this.

[0142] Continuing with reference to Figure 1A, illustratively, after the image generation module of the first electronic device generates an image, the generated image can be output to the image encoding module and the information generation module. Subsequently, the information generation module can generate T.35 information associated with the image and output the T.35 information to the encapsulation module. In addition, the image encoding module can encode the image to obtain a code stream of the image (wherein the code stream can also be referred to as a bit stream or bit stream), and output the encoded image code stream to the encapsulation module. Afterwards, the encapsulation module can encapsulate the code stream of the image and the T.35 information into an image file format, and output the image file format to the sending module. The sending module can then send the image file format to the second electronic device.

[0143] It should be noted that T.35 information is also binary; thus, it can be directly encapsulated / embedded into the image file format.

[0144] Afterwards, the receiving module of the second electronic device can receive the image file format and output the image file format to the decapsulation module. Subsequently, the decapsulation module can decapsulate the image file format to obtain the image code stream and T.35 information. Next, the decapsulation module can output the image code stream to the image decoding module and output the T.35 information to the display module. Then, the image decoding module can decode the image code stream to obtain a reconstructed image and output the reconstructed image to the display module, which displays the reconstructed image according to the T.35 information.

[0145] Exemplarily, the first electronic device includes but is not limited to: a server, a PC (Personal Computer), a laptop, a tablet computer, a mobile phone, and a watch.

[0146] Exemplarily, the second electronic device includes, but is not limited to: a PC, a laptop, a tablet computer, a mobile phone, a watch, a head-mounted display (such as a virtual reality (VR) / augmented reality (AR) helmet, etc.).

[0147] For example, when the first electronic device is used for encoding and the second electronic device is used for decoding, the first electronic device can be called the encoding end and the second electronic device can be called the decoding end. When the second electronic device is used for encoding and the first electronic device is used for decoding, the second electronic device can be called the encoding end and the first electronic device can be called the decoding end.

[0148] For example, the T.35 information may include the information required for multimedia to be carried as specified in the T.35 specification published by the ITU-T (i.e., "Procedure for the Allocation of Non-standard Extension Codes Defined by the ITU"): country code, terminal provider code, and terminal provider oriented code. For example, the multimedia involved in this application may include, but is not limited to, images, video, audio, and text (such as subtitles and bullet comments in the screen).

[0149] Exemplarily, the T.35 information may also include metadata of the image. In one possible embodiment, the image encoded by the first electronic device in FIG1A may be an HDR image; the corresponding T.35 information may also include metadata of the HDR.

[0150] For easier understanding, let’s first introduce SDR and HDR.

[0151] Dynamic range is used in many fields to express the ratio of a variable's maximum to minimum values. In digital images, dynamic range represents the ratio between the maximum and minimum grayscale values ​​within the image's displayable range. The dynamic range in nature is quite large. The brightness of a starry night sky is approximately 0.001 cd / m², while the sun itself has a brightness of 1,000,000,000 cd / m², resulting in a dynamic range of 1,000,000,000 / 0.001 = 1013. However, in real-world scenes, the brightness of the sun and starlight are not simultaneously present. For real-world scenes, the dynamic range is between 10-3 and 106. In most current color digital images, each R, G, and B channel is stored using an 8-bit byte. This means each channel represents a grayscale range of 0 to 255. This range is the image's dynamic range. Since the dynamic range for the same scene in the real world ranges from 10⁻³ to 10⁶, this is called high dynamic range (HDR). The dynamic range of an ordinary image is referred to as low dynamic range (LDR). The imaging process of a digital camera is essentially a mapping from the real-world high dynamic range to the low dynamic range of a photograph.

[0152] Standard dynamic range images correspond to high dynamic range images. Traditionally used 8-bit images in formats such as JPEG can be considered standard dynamic range images. Before the advent of cameras capable of capturing HDR images, traditional cameras could only record captured light information within a certain range by controlling the exposure value. Since the maximum illumination information of a display device cannot match the brightness information of the real world, and we browse images through a display device, a photoelectric transfer function is required. Early display devices used cathode ray tube (CRT) displays, whose photoelectric transfer function was the gamma function. This photoelectric transfer function, based on the "gamma" function, is defined in the ITU-R Recommendation BT.1886 standard.

[0153] However, with the upgrade of display devices, the illumination range of display devices continues to increase. Existing consumer-grade HDR displays are at 600cd / m2, while high-end HDR displays can reach 2000cd / m2, far exceeding the illumination information of SDR display devices. The photoelectric conversion function in the ITU-R Recommendation BT.1886 standard cannot well express the display performance of HDR display devices. Therefore, an improved electro-optical transfer function is needed to adapt to the upgrade of display devices. The concept of the photoelectric transfer function is derived from the mapping function in the tone mapping algorithm. The photoelectric transfer function is appropriately adjusted to form the photoelectric transfer function. Currently, there are three common photoelectric conversion functions: perception quantization (PQ) function, gamma, logarithm (log) function, and hybrid log-gamma (HLG) function. These three photoelectric conversion functions are the conversion functions specified in the AVS standard.

[0154] Among them, if HDR images and videos need to obtain a better experience, it is an end-to-end process. Please refer to Figure 1B, which describes the end-to-end process of HDR video.

[0155] Because the dynamic range of an image acquisition module is limited under specific shooting conditions, to obtain an image with a higher dynamic range, multiple images with different exposures captured simultaneously are typically combined to create an image with a higher dynamic range, known as an HDR image. The resulting HDR image typically has a bit width greater than 10 bits, allowing it to accommodate scenes with a higher dynamic range.

[0156] 1B , illustratively, material production may refer to the process of synthesizing an HDR image using multiple images with different exposures captured at the same time; and synthesizing the temporally continuous HDR images into an original HDR video; the original HDR video obtained from the material production may be referred to as material.

[0157] Afterwards, the HDR image / video production module of the image generation module can perform editing and color grading on the footage to produce the final HDR video (also known as the master). Subsequently, the metadata generation module can generate metadata based on the final HDR video. Partial metadata (referred to as metadata 1) can be used along with the country code, terminal provider code, and terminal provider targeting code to generate T.35 information. Another portion of metadata (referred to as metadata 2) can be output to the video encoding module, which encodes the HDR video and metadata 2 to generate a bitstream and outputs it to the encapsulation module. The encapsulation module then encapsulates the bitstream and T.35 information into an image file format and distributes / transmits this image file format over the network. After receiving the image file format, the decoder can invoke the decapsulation module to decapsulate the image file format, obtaining the bitstream and T.35 information. The video decoding module can then decode the bitstream to obtain the reconstructed HDR video and reconstructed metadata 2. The display module can then display the reconstructed HDR video based on the reconstructed metadata and metadata carried in the T.35 information.

[0158] For example, the T.35 information may include dynamic metadata or dynamic metadata. For another example, the T.35 information may also include HDR display mapping information. It should be understood that other information may also be carried by the T.35 information, and this application does not limit this.

[0159] For example, static metadata, including the mastering display's color space, the master display's maximum and minimum brightness, the video sequence's maximum brightness, and the maximum average brightness of the video sequence, is functionally used to ensure accurate display on the reference display and restore the creator's intent. In a video sequence, this is static data that doesn't change over time. For example, the maximum brightness of a video sequence is the maximum value of the maximum brightness across all frames.

[0160] For example, dynamic metadata includes most static metadata and also includes metadata related to tone mapping on the display device. Functionally, this information is closely related to display and tone mapping processes, such as indicating the selection of tone mapping algorithms and parameters for shifting from a high dynamic range to a relatively low dynamic range. In video sequences, dynamic metadata often changes over time. For example, the maximum brightness of the image or the primary display changes during scene changes.

[0161] For example, this application can be applied to various image business scenarios (e.g., mobile phone photography, cloud photo albums, image transcoding services, etc.), various video / audio business scenarios (e.g., live broadcast, on-demand, etc.), etc., and the embodiments of this application are not limited to this. This application takes multimedia as an example for explanation.

[0162] The following describes the process of encapsulating T.35 information into an image file format.

[0163] First, according to the provisions / specifications of the image file format, the T.35 information and the location of the T.35 information in the image file format may be defined.

[0164] Exemplarily, the image file format may include, but is not limited to, JPEG, WebP, PNG, HEIF, AVIF, and JXL, etc., which is not limited in this application.

[0165] For example, the present application may add a custom box based on the existing image file format, and the newly added box may be used to store T.35 information; and define the location of the newly added box. In order to distinguish the newly added box from the existing box in the existing image file format, the newly added box may be referred to as the first box (in some scenarios, it may also be referred to as the T35 box (which may be represented by the four characters "it35", which can be understood as "it35" representing a box carrying T.35 information), or other names, which are not limited by the present application).

[0166] For example, a box is a general mechanism for organizing and storing data, enabling files to contain multiple types of information in a flexible manner. The hierarchy and content of boxes are defined by corresponding standard specifications to ensure correct parsing and processing of files.

[0167] This application uses the HEIF image file format as an example for illustration.

[0168] First, let's explain some of the syntax in HEIF:

[0169] Box: An object-oriented building block defined by a unique type identifier and length.

[0170] Container Box: A box whose sole purpose is to contain and group a set of related boxes.

[0171] Item: is data that does not require regular processing, as opposed to sample data.

[0172] Property: It is the property related to item.

[0173] A sample is all the data associated with a single time. In other words, a sample refers to dynamic data.

[0174] A track is a timed sequence of related samples in an ISO base media file. For multimedia, a track corresponds to a sequence of images or sampled audio; for a hint track, a track corresponds to a streaming media channel.

[0175] Sample grouping is the process of assigning each sample in a track to a sample group based on a grouping criterion.

[0176] It should be noted that in HEIF, continuous or timed media or metadata streams form a track, while static media or metadata are stored as items.

[0177] The definitions of the various syntaxes in HEIF can be found in ISO / IEC 23008-12:2022, Second Edition, and will not be repeated here.

[0178] For example, HEIF has the following basic design:

[0179] 1. Still images are stored as items. Typically, an image item is independently encoded and does not depend on any other items for decoding. If there are predictively coded image items with encoding dependencies, these are explicitly indicated. Any number of image items can be included in the same file.

[0180] 2. Image sequences are stored as tracks. Image sequence tracks can be instructed to be displayed in a timed sequence (video or burst animation) or in a non-timed manner, such as an image gallery. When there are coding dependencies between images, image sequence tracks can be used instead of image items.

[0181] Each static element in a HEIF file is an item. For example, an "item" is data that does not require timed processing, as opposed to sample data (time-related), and is described by the Boxes contained in a MetaBox.

[0182] For items, HEIF files can contain coded items, such as an HEVC-encoded image; HEIF can also have derived items, such as image overlays; HEIF can also have metadata items, such as EXIF ​​information.

[0183] Each item can also have one or more properties associated with it.

[0184] There are structures connecting items and between items and their attributes.

[0185] For example, HEIF specifies a box structure format from which codec-specific image formats can be derived.

[0186] For example, in HEIF, a box is a data structure used to organize and store information within a file. Boxes are hierarchical, each with a unique identifier and length. Boxes can contain other boxes, forming a nested structure. Different types of boxes are used to store different types of information, such as image data, metadata, and configuration information.

[0187] From a structural point of view, Box can be divided into Full Box and Container Box; Container Box can nest Box, while Full Box can only contain specific definition content and cannot nest Box.

[0188] From the perspective of content, Box can be divided into:

[0189] Box used to describe file types and compatibility: FileTypeBox (ftyp) (also known as file type box);

[0190] Box used to store multimedia metadata: MetaBox (meta) (also known as metadata box);

[0191] Box used to store image data: MediaDataBox (mdat) (also called media data box);

[0192] Box used to store image sequences: MovieBox (moov) (also known as streaming data box);

[0193] Box used to store item information, such as item information box (iinf) (also called image information box);

[0194] A box used to store item information entries, such as an item information entry box (infe) (also known as an item information entry box);

[0195] Box used to store item properties, for example, item Property Container Box (ipco) (item property container box);

[0196] A box used to store item property association information, such as the item properties association box (ipma) (also known as the item properties association box);

[0197] A box used to store samples, such as a Track Box.

[0198] A box used to store sample entries (Sample Entry), for example, SampleDescriptionBox (stsd);

[0199] A box used to store properties describing a sample group, such as a Sample Group Description Box (sgpd) (also known as a sample group description box).

[0200] A box used to store timed metadata, for example, a timed metadata track box.

[0201] A box used to store timed metadata entries, for example, a metadata Sample Entry Box.

[0202] Among them, the ftyp Box, meta Box, mdat Box, and moov Box are the four top-level boxes of HEIF; among them, the ftyp Box and mdat Box are full boxes, and the meta Box and moov Box are container boxes. It should be noted that the moov Box is an optional box in HEIF.

[0203] Illustratively, the item Information Box (iinf), the item Information Entry Box (infe), the item Property Container Box (ipco), the item Properties Assocation Box (ipma), and the item Location Box (iloc), etc., are nested in the meta Box.

[0204] For example, the SampleDescriptionBox (stsd) and SampleGroupDescriptionBox (sgpd) are nested within the TrackBox. The MetadataSampleEntryBox is nested within the TimedMetadataTrackBox. The TrackBox and TimedMetadataTrackBox are nested within the MoovBox.

[0205] For example, a simplified HEIF hierarchical structure may be as follows:

[0206] Structure 1:

[0207] For example, a simplified HEIF hierarchical structure may be as follows:

[0208] Structure 2:

[0209] It should be understood that Structure 1 and Structure 2 are merely examples of the hierarchical structure of HEIF. The hierarchical structure of HEIF in this application may include more or fewer boxes, or more or fewer levels, than those shown in Structure 1 and Structure 2; this application does not impose any restrictions on this. In addition, this application does not impose any restrictions on the position of each box in the hierarchical structure of HEIF.

[0210] For example, it35 can be nested into any existing HEIF Box.

[0211] For example, it35 can be nested in any existing Box at any level of HEIF.

[0212] For example, the definition of it35 and its location in HEIF may be as follows:

[0213] Definition method 1: Add a custom metadata element called "T35Information" to the image file format. This element can be defined as an item (that is, T.35 information is defined as an item), such as a metadata item. The type of the metadata item can be defined as the newly added "it35" value (item_type = 'it35').

[0214] Definition method 1 (1):

[0215] T35Information (also known as it35) can have a syntax similar to the following:

[0216] Syntax 1:

[0217] For example, the semantic interpretation of Syntax 1 can be as follows:

[0218] itu_t_t35_country_code shall be one octet, whose value shall be the country code as defined in Rec.ITU-T T.35 Annex A, or the country code extension value 0xFF.

[0219] If itu_t_t35_country_code_extension_byte is present, it shall be one byte whose value is the country code as defined in Rec.ITU-T T.35 Annex B.

[0220] itu_t_t35_payload shall be a payload containing data registered according to Rec.ITU-T T.35 (which may include multimedia metadata, such as dynamic metadata for HDR images or static metadata for HDR images).

[0221] The ITU-T T.35 terminal provider code and terminal provider orientation code shall be contained in the first octet or octets of the itu_t_t35_payload, in a format specified by the authority issuing the terminal provider code. Any remaining itu_t_t35_payload data shall be data with the specific syntax and semantics specified for the entity identified by the ITU-T T.35 country code, terminal provider code, and terminal provider orientation code.

[0222] The length of itu_t_t35_payload is the number of bytes remaining in the project.

[0223] For example, in HEIF, when T.35 information is defined as a metadata item and the syntax of the T.35 Box is Syntax 1, based on the above "Structure 1", a simplified HEIF possible hierarchical structure is given:

[0224] Structure 3:

[0225] Among them, the structural differences between Structure 3 and Structure 1 are marked, as shown in bold font.

[0226] Referring to Structure 3, the T.35 Box is an infe Box; the T.35 Box can be nested in iinf.

[0227] Definition method 1 (2):

[0228] T35Information (also known as it35) can have a syntax similar to the following:

[0229] Syntax 2:

[0230] It should be noted that in definition method 1 (2), a data block T35DataBlock needs to be defined to store the T.35 payload. T35DataBlock can have a syntax similar to the following:

[0231] Syntax 3:

[0232] aligned(8)class T35DataBlock(){

[0233] bit(8)itu_t_t35_payload[];

[0234] }

[0235] The semantic interpretation of Grammar 2 and Grammar 3 can refer to the semantic interpretation of Grammar 1 and will not be repeated here.

[0236] For example, in HEIF, when T.35 information is defined as a metadata item and the syntax of the T.35 Box is Syntax 2, based on the above "Structure 1", a simplified HEIF possible hierarchical structure is given:

[0237] Structure 4:

[0238] The T.35 information includes a header and a payload. The header includes a country code, a terminal provider code, and a terminal provider orientation code, and the payload may include multimedia metadata, such as dynamic metadata or static metadata of an HDR image.

[0239] Referring to Structure 4, the T.35 Box (including only the header) is the infe Box. The T.35 Box can be nested in the iinf Box, and the payload can be nested in the mdat Box.

[0240] Definition method 2: Add a custom metadata element called "T35Information" to the image file format. This element can be defined as an item property (that is, T.35 information is defined as an item property). For example, this item property can be attributed to and associated with an item (image). In HEIF, an image is also an item, such as a derived item or a coded image item. Both can have properties. Defined properties can be specified in a property box (for example, the T.35 property is placed in the itemPropertyContainerBox container box) and assigned to a specific image. Since each item property is a box (Box) or a full box (FullBox), the item property's box type (Boxtype) specifies the property type. Therefore, the metadata element (item property) can be defined with the newly added "it35" value (Box_type = 'it35'). Furthermore, according to the property definition, this item property box should be located within the itemPropertyContainerBox container.

[0241] The definition of T.35 Box can be as follows:

[0242] Box Type: 'it35'

[0243] Property type: Descriptive item property

[0244] Container: itemPropertyContainerBox

[0245] For example, T35Information (i.e. it35) may have a syntax similar to the following:

[0246] Syntax 3:

[0247] The semantic interpretation of Grammar 3 can refer to the semantic interpretation of Grammar 1 and will not be repeated here.

[0248] For example, in HEIF, when T.35 information is defined as an item property, based on the above "Structure 1", a simplified HEIF possible hierarchical structure is given:

[0249] Structure 5:

[0250] Among them, the structural differences between Structure 5 and Structure 1 are marked, as shown in bold font.

[0251] In other words, the T.35 box is nested in the ipco box, and the association information between the T.35 information and the corresponding items is added to the ipma box.

[0252] Definition method three: Add a custom element to the image file format and define it as a sample entry. In other words, define the T.35 information as a sample entry.

[0253] A Sample Entry is a box type that contains information describing a sample. Each Sample Entry corresponds to a sample in a track. This box primarily describes the format and encoding of media data.

[0254] Since Sample Entry describes different data formats and encoding methods, the T.35 information defined here is codec-independent.

[0255] Since the Sample Entry is a Box, this sample entry can be named "T35InformationBox." The box type of the Sample Entry Box of this Sample Entry is defined as the newly added "it35" value (Box_type = 'it35'). The Sample Entry can be placed in a SampleDescriptionBox ('stsd').

[0256] For example, T35InformationBox (also known as it35) may have a syntax similar to the following:

[0257] Syntax 4:

[0258] Among them, the semantic interpretation of Grammar 4 can be the semantic interpretation of Grammar 1, which will not be repeated here.

[0259] For example, in HEIF, when T.35 information is defined as Sample Entry, a simplified HEIF possible hierarchical structure is given based on the above "Structure 2":

[0260] Structure 6:

[0261] Among them, the structural differences between Structure 6 and Structure 2 are marked, as shown in bold font.

[0262] That is to say, nest the T.35 box in the stsd.

[0263] For example, for structure 6: When there is an image sequence, a track needs to be defined to store it. This allows for a stsd file to the end of the hierarchy. This stsd file contains codec information, such as hvc1. This solution places it35, making metadata codec-independent (parallel boxes).

[0264] It should be noted that in solution three, it35 (Sample Entry) is independent of other Sample Entry boxes and does not limit the codec to achieve applicability.

[0265] It should be noted that Solutions 1, 2, and 3 are applicable to T.35 information that carries static metadata.

[0266] Definition method 4: Add a custom element to the image file format and define it as a Sample Group Entry. In other words, define T.35 as a Sample Group Entry.

[0267] The T.35 information defined by the Sample Group Entry can carry dynamic metadata. Since the Sample Group Entry is a Box, this sample entry can be named "T35SampleGroupEntry." The Sample Group Box defines the grouping criteria (grouping type) for the newly added "it35" value (grouping_type = 'it35'). This Sample Group Entry can be placed in a SampleGroupDescriptionBox ('sgpd') to associate the sample group with a Track.

[0268] Definition method 4 (1): All data carried by the T.35 message (including header and payload) are stored in the SampleGroupDescriptionBox ('sgpd') rather than in the sample (i.e., the Sample Entry in stsd).

[0269] For example, each sample in a track may be associated with zero or more sample group descriptions, each of which defines a different type of T.35 information record. The same T.35 information may be applicable to different samples.

[0270] grouping_type='it35' is defined as a grouping criterion for T.35 metadata.

[0271] A track's SampleTableBox ('stbl') or TrackFragmentBox ('traf') may contain zero or more SampleToGroupBox ('sbgp') or CompactSampleToGroupBox ('csgp') with grouping_type = 'it35' but with different grouping_type_parameter entries.

[0272] Exemplarily, T35SampleGroupEntry (i.e. it35) may have the following syntax:

[0273] Syntax 5:

[0274] Among them, the semantic interpretation in Grammar 5 can refer to the semantic interpretation in Grammar 1 and will not be repeated here.

[0275] For example, in HEIF, when the T.35 information is defined as SampleGroupEntry, based on the above "Structure 2", a simplified HEIF possible hierarchical structure is given:

[0276] Structure 7:

[0277] In other words, the T.35 box is embedded into sgpd.

[0278] All entries in a SampleToGroupBox or CompactSampleToGroupBox with the same grouping_type_parameter value shall map to the same type of T.35 message; this means T.35 sample group entries with at least the same ITU-T T.35 country code, terminal provider code and terminal provider orientation code, and T.35 specific type and version information (if applicable).

[0279] For example, the five T.35 messages with ids = 1, 2, 3, 4, and 5 in sgpd come from HDR Vivid, HDR Vivid, Dolby Vision, Dolby Vision, and Dolby Vision respectively.

[0280] Definition method 4 (1) can reduce the redundancy of storing the same metadata in each sample and is very useful when the metadata carried by the T.35 information is not frequent and does not cause problems due to the size of the MovieBox.

[0281] Definition method 4 (2): The payload carried by the T.35 message is stored in the sample, and the SampleGroupDescriptionBox is only used to indicate the header of the T.35 message. The T.35 message is carried in the sample. The sample grouping mechanism is used to identify the sample holding a specific T.35 message. This method allows processing very frequent (large amounts of identical metadata) T.35 metadata.

[0282] Each sample in a track can be associated with zero or more sample group descriptions, each of which defines the record of a different type of T.35 information. The same T.35 information may apply to different samples.

[0283] grouping_type='it35' is defined as a grouping criterion for T.35 metadata.

[0284] A track's SampleTableBox or TrackFregmentBox can contain zero or more SampleToGroupBoxes or CompactSampleToGroupBoxes with grouping_type='it35'.

[0285] Each Sample Group Description shall record exactly one T.35 message, i.e. shall map to a T.35 Sample Group Entry with at least the same ITU-T T.35 Country Code, Terminal Provider Code and Terminal Provider Direction Code, and T.35 Type and Version (if applicable).

[0286] The syntax of T35SampleGroupEntry (i.e. it35) in definition method 4 (2) can be as follows:

[0287] Syntax 6:

[0288] For example, in HEIF, when the T.35 information is defined as SampleGroupEntry, based on the above "Structure 2", a simplified HEIF possible hierarchical structure is given:

[0289] Structure 8:

[0290] For example, the first sbgp has two sub-idxs pointing to id=1 and id=2 in sgpd. grouping_type_parameter=0 (this means grouping HDR Vivid together).

[0291] The second sbgp has three sub-idxs pointing to id=3, id=4, id=5 in sgpd, and grouping_type_parameter=1 (this groups the three Dolby Vision images together).

[0292] That is, in definition method 4 (1), the header and payload of the T.35 information are all placed in the Sample Group Description Box. In definition method 4 (2), the Sample Group Description Box only stores the header of the T.35 information, and the payload of the T.35 information is stored in the sample (i.e., the Sample Entry of the stsd).

[0293] Definition method 5: A timed metadata track can be used to store dynamic metadata carried by T.35 messages, such as HDR dynamic tone mapping or film granularity metadata that changes per frame. It can also be used to store less frequent T.35 metadata (smaller amounts of identical metadata). The T.35 sample duration can be set to cover multiple samples of the reference track, which are the tracks referenced by the T.35 message.

[0294] When using a timed metadata track, the reference track shall not contain a T35InformationBox in a SampleEntry with the same ITU-T T.35 country code, terminal provider code, and terminal provider orientation code.

[0295] For example, T.35 information may be defined as a sample entry (Sample Entry), named "T35InfoSampleEntry", and coded with 'it35'.

[0296] For example, a sample entry of T.35 information (Sample Entry) should contain a T35InfoConfigurationBox to describe the T.35 information present in each sample, and its type can be represented by ''T35C''.

[0297] For example, the syntax of T.35 may be as follows:

[0298] Syntax 7:

[0299] For example, the syntax of T35CommmonHeaderInfoBox may be as follows:

[0300] Syntax 8:

[0301] For example, the syntax of T35InfoSample may be as follows:

[0302] Syntax 9:

[0303] Semantic explanation in Grammar 7 to Grammar 9:

[0304] t35_common is the common T.35 information for all samples associated with this sample entry. It shall include the ITU-T T.35 country code, the terminal provider code, and the terminal provider guidance code. It shall also include the portion of the ITU-T T.35 message that specifies the message version. It may also include any other common data following the terminal provider guidance code for all samples associated with this sample entry.

[0305] itu_t_t35_country_code shall be one octet whose value is specified by the country code specification in Annex A of ITU-T T.35, or the country code extension value 0xFF.

[0306] itu_t_t35_country_code_extension_byte, if present, shall be a byte whose value is specified by the country code specification in Annex B of ITU-T T.35.

[0307] itu_t_t35_common shall be a payload containing the ITU-T T.35 terminal provider code and the terminal provider guidance code, as specified in the ITU-T T.35 specification. It shall also include part of the version of the ITU-T T.35 message. In all samples related to this sample entry, it may also include any common data following the terminal provider guidance code.

[0308] t35_remaining_payload shall be the remaining payload containing data registered according to the ITU-T T.35 Country Code, Terminal Provider Code, and Terminal Provider Direction Code specifications in T35InfoSampleEntry. Concatenating the data from T35CommmonHeaderInfoBox with the data from each T35InfoSample will result in a complete ITU-T T.35 message.

[0309] For example, the T.35 information is defined as a sample entry. Based on the above "Structure 2", a simplified HEIF possible hierarchical structure is given:

[0310] Structure 9:

[0311] Similarly, the present application may also define the T35 Box and the location of the T35 Box in the JPEG Universal Metadata Box Format (JUMBF). For example, JUMBF may refer to the JPEG Systems Part 5 document; JPEG Systems Part 5 defines a framework for JPEG standards to help JPEG add universal metadata and allow future extensions. Extensions may be, for example, metadata, supplementary images, or other elements in addition to the base image. (specifies a framework for JPEG standards to add universal metadata allowing future extensions using metadata, supplementary images, or other elements in addition to the base image.) For example, JUMBF also defines the basic concept of the box structure; therefore, in JUMBF, T.35 information may also be defined as a box, and the location of the T35 box may be defined according to the JUMBF regulations (i.e., the T35 box may be set at the appropriate location specified by JUMBF); details will not be given here.

[0312] In this way, the T.35 information can be encapsulated into the image file format (or called image file) according to the image file format with the T.35 information definition and the T.35 information location definition added; the following data encapsulation process 200 can be referred to.

[0313] FIG. 2A is a schematic diagram illustrating an exemplary data encapsulation process 200 .

[0314] S201, obtaining T.35 information; wherein the T.35 information includes: country code, terminal provider code and terminal provider directional code.

[0315] Exemplarily, the T.35 message may also carry multimedia metadata. It should be understood that multimedia metadata is optional information that the T.35 message can carry, while the country code, terminal provider code, and terminal provider orientation code are mandatory information that the T.35 message can carry.

[0316] S202, encapsulate the T.35 information into an image file format; wherein the image file format includes a first box, and the first box includes the T.35 information.

[0317] Exemplarily, the T.35 information may be encapsulated into the image file format according to any one of the above-mentioned definition methods 1 to 5.

[0318] For example, according to any one of the above definition methods 1 to 5, the image file format obtained in S202 may include a first box, and the first box includes T.35 information.

[0319] Exemplarily, when the image file format obtained in S202 is a HEIF file, according to any one of the above-mentioned definition methods one (1) and definition methods two to five, the image file format obtained in S202 can also include a second box, which can be a metadata box or a stream data box, and the first box can be nested in the second box; as shown in Figure 2B.

[0320] Exemplarily, the second box includes multiple third boxes, and there is a hierarchical relationship between the multiple third boxes; the first box is nested in the third box at any level.

[0321] That is to say, when the second box is a metadata box, the metadata box may include multiple third boxes, including but not limited to: item Information Box (iinfBox), item Information Entry Box (infe Box), item Property Container Box (ipco Box), item Properties Assocation Box (ipma Box), and item Location Box (iloc Box), etc.; the hierarchical relationship of these multiple third boxes can be as shown in Structure 1; the first box can be nested in the third box of any level (as shown in Structure 3, Structure 4 or Structure 5).

[0322] When the second box is a stream data box, the stream data box may include multiple third boxes, including but not limited to: SampleDescriptionBox (stsd Box), Sample Group Description Box (sgpd Box), metadata Sample Entry Box, timed metadata Track Box, Track Box, etc. The hierarchical relationship of multiple third boxes can be as shown in Structure 2; the first box can be nested in the third box of any level (as shown in Structure 6 or Structure 7 or Structure 8 or Structure 9).

[0323] Exemplarily, when the definition method of the T.35 box and the position of the T.35 box in the image file format is as shown in definition method 1 (2), the image file format obtained in S202 also includes a project identifier and a media box, the project identifier indicates that the T.35 information is a project, the first box includes the header information of the T.35 information; the media box includes the payload of the T.35 information, and the first box is nested in the image information box; as shown in Figure 2C.

[0324] For example, when the T.35 box and the position of the T.35 box in the image file format are defined as shown in definition method 1 (1), the T.35 information is a project, the first box is nested in the image information box, and the first box is a project information entry box.

[0325] Exemplarily, when the T.35 box and the position of the T.35 box in the image file format are defined as shown in definition method 2, the T.35 information is a project attribute; the first box is nested in the project attribute container box, and the project attribute association box includes association information between the T.35 information and the corresponding project.

[0326] Exemplarily, when the T.35 box and the position of the T.35 box in the image file format are defined as shown in definition method three, the T.35 information is a sample entry, and the first box is nested in the sample description box.

[0327] For example, when the T.35 box and the position of the T.35 box in the image file format are defined as shown in definition method four (1), the T.35 information is a sample group entry, and the first box is nested in the sample group description box.

[0328] For example, when the T.35 box and the position of the T.35 box in the image file format are defined as shown in definition method four (2), the T.35 information is a sample entry, the first box includes the header information of the T.35 information, and the first box is nested in the sample group description box; the sample description box includes the payload of the T.35 information.

[0329] Exemplarily, when the T.35 box and the position of the T.35 box in the image file format are defined as shown in definition method five, the T.35 information is a sample entry, and the first box is nested in the metadata sample entry box.

[0330] Exemplarily, the encoded multimedia data may be obtained and encapsulated into an image file format, where the image file format further includes a media box or a stream data box, and the media box or the stream data box includes the encoded multimedia data.

[0331] FIG3 is a schematic diagram illustrating an exemplary data acquisition process 300 .

[0332] S301, receiving an image file format; wherein the image file format includes a first box, and the first box includes T.35 information.

[0333] S302, read T.35 information from the first box of the image file format; wherein the T.35 information includes: country code, terminal provider code and terminal provider orientation code.

[0334] Illustratively, the T.35 information may be read from the first box of the image file format.

[0335] For example, when the T.35 box and the position of the T.35 box in the image file format are defined as shown in definition method 1 (2), the project identifier can be read from the image file format, the payload of the T.35 information can be read from the media box according to the project identifier, and the header information of the T.35 information can be read from the first box nested in the image information box.

[0336] For example, when the T.35 box and the position of the T.35 box in the image file format are defined as shown in definition method 1 (1), the T.35 information can be read from the first box nested in the image information box.

[0337] For example, when the T.35 box and its location in the image file format are defined as shown in definition method 2, the T.35 information can be read from the first box nested within the item attribute container box. Furthermore, the association information between the T.35 information and the corresponding item can be read from the item attribute association box to determine the item associated with the T.35 information.

[0338] For example, when the T.35 box and the position of the T.35 box in the image file format are defined as shown in definition method three, the T.35 information can be read from the first box nested in the sample description box.

[0339] For example, when the T.35 box and the position of the T.35 box in the image file format are defined as shown in definition method four (1), the T.35 information can be read from the first box nested in the sample group description box.

[0340] Exemplarily, when the T.35 box and the position of the T.35 box in the image file format are defined as shown in definition method four (2), the header information of the T.35 information can be read from the first box nested in the sample group description box, and the payload of the T.35 information can be read from the sample description box.

[0341] Exemplarily, when the T.35 box and the position of the T.35 box in the image file format are defined as shown in definition method five, the T.35 information can be read from the first box nested in the metadata sample entry box.

[0342] In addition, the encoded multimedia data may be read from the image file format including the media box or the stream data box, and then the encoded multimedia data may be decoded to obtain reconstructed multimedia data. The reconstructed multimedia data may then be played according to the T.35 information.

[0343] Illustratively, an embodiment of the present application further provides an image file format, which can be encapsulated according to the data encapsulation process 200 .

[0344] 4 is a schematic diagram of an exemplary data encapsulation device 400. The data encapsulation device 400 can be used to execute the method of the aforementioned embodiment, and therefore, the beneficial effects achieved by the data encapsulation device 400 can refer to the beneficial effects of the corresponding method provided above, which will not be repeated here.

[0345] Exemplarily, the data encapsulation device 400 may include:

[0346] The acquisition module 401 is configured to acquire T.35 information, wherein the T.35 information includes: a country code, a terminal provider code, and a terminal provider directional code;

[0347] The encapsulation module 402 is configured to encapsulate the T.35 information into an image file format; wherein the image file format includes a first box, and the first box includes the T.35 information.

[0348] 5 is a schematic diagram of an exemplary data reading device 500. The data encapsulation device 500 can be used to execute the method of the aforementioned embodiment, and therefore, the beneficial effects that can be achieved can refer to the beneficial effects of the corresponding method provided above, which will not be repeated here.

[0349] Exemplarily, the data encapsulation device 500 may include:

[0350] The receiving module 501 is configured to receive an image file format, wherein the image file format includes a first box, and the first box includes T.35 information;

[0351] The reading module 502 is configured to read T.35 information from the first box in the image file format; wherein the T.35 information includes: a country code, a terminal provider code, and a terminal provider orientation code.

[0352] The following describes the codec system used in the present application in conjunction with FIG6 , which is a schematic block diagram of a codec system used in an embodiment of the present application, such as a video codec system 10 (or simply codec system 10 ) that can utilize the techniques of the present application. The video encoder 20 (or simply encoder 20 ) and video decoder 30 (or simply decoder 30 ) in the video codec system 10 represent devices that can be used to perform various techniques according to the various examples described in this application.

[0353] As shown in FIG6 , the encoding and decoding system 10 includes a source device 12 , which is used to provide encoded data 21 such as an encoded image to a destination device 14 for decoding the encoded data.

[0354] The source device 12 includes an encoder 20 and, optionally, may further include an image source 16 , an image preprocessor 18 (or a preprocessing unit), and a communication interface or communication unit 22 .

[0355] The image source 16 may include or may be any type of image capture device for capturing real-world images, etc., and / or any type of image generation device, such as a computer graphics processor for generating computer-animated images, or any type of device for acquiring and / or providing real-world images, computer-generated images (e.g., screen content, virtual reality (VR) images, and / or any combination thereof (e.g., augmented reality (AR) images). The image source may be any type of memory or storage that stores any of the above images.

[0356] In order to distinguish the processing performed by the pre-processor 18 or the pre-processing unit 18 , the image or image data 17 may also be referred to as a raw image or raw image data 17 .

[0357] The preprocessor 18 is configured to receive (raw) image data 17 and preprocess the image data 17 to obtain a preprocessed image 19 or preprocessed image data 19. For example, the preprocessing performed by the preprocessor 18 may include cropping, color format conversion (e.g., from RGB to YCbCr), color grading, or denoising. It will be appreciated that the preprocessor 18 may be an optional component.

[0358] The video encoder 20 is configured to receive the pre-processed image data 19 and provide encoded image data 21 .

[0359] Communication interface 22 in source device 12 may be used to receive encoded image data 21 and transmit the encoded image data 21 (or any other processed version thereof) to another device, such as destination device 14, or any other device, via communication channel 13 for storage or direct reconstruction.

[0360] Destination device 14 includes a decoder 30 (eg, video decoder 30 ) and, in addition and optionally, may include a communication interface or communication unit 28 , a post-processor 32 (or post-processing unit 32 ), and a display device 34 .

[0361] The communication interface 28 in the destination device 14 is used to receive the encoded image data 21 (or any other processed version) directly from the source device 12 or from any other source device such as a storage device, for example, a storage device that is an encoded image data storage device, and provide the encoded image data 21 to the decoder 30.

[0362] The communication interface 22 and the communication interface 28 may be used to send or receive the encoded image data 21 or the encoded data via a direct communication link between the source device 12 and the destination device 14, such as a direct wired or wireless connection, or via any type of network, such as a wired network, a wireless network, or any combination thereof, any type of private network and public network, or any combination thereof.

[0363] For example, the communication interface 22 may be used to encapsulate the encoded image data 21 into a suitable format such as a message, and / or process the encoded image data using any type of transmission coding or processing for transmission over a communication link or network.

[0364] The communication interface 28 corresponds to the communication interface 22 , for example, and can be used to receive transmission data and process the transmission data using any type of corresponding transmission decoding or processing and / or decapsulation to obtain encoded image data 21 .

[0365] Both the communication interface 22 and the communication interface 28 can be configured as a unidirectional communication interface as indicated by the arrow pointing from the source device 12 to the corresponding communication channel 13 of the destination device 14 in Figure 6, or a bidirectional communication interface, and can be used to send and receive messages, etc. to establish a connection, confirm and exchange any other information related to the communication link and / or data transmission such as encoded image data transmission, etc.

[0366] The decoder 30 is configured to receive the encoded image data 21 and provide decoded image data 31 or a decoded image 31 .

[0367] The post-processor 32 in the destination device 14 is configured to post-process the decoded image data 31 (also referred to as reconstructed image data), such as the decoded image 31, to obtain post-processed image data 33, such as the post-processed image 33. The post-processing performed by the post-processing unit 32 may include, for example, color format conversion (e.g., from YCbCr to RGB), color grading, cropping, or resampling, or any other processing for generating the decoded image data 31 for display on a display device 34, etc.

[0368] A display device 34 in the destination device 14 is configured to receive the post-processed image data 33 and display the image to a user or viewer. The display device 34 may be or include any type of display for displaying the reconstructed image, such as an integrated or external display screen or monitor. For example, the display screen may include a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a plasma display, a projector, a micro-LED display, a liquid crystal on silicon (LCoS) display, a digital light processor (DLP), or any other type of display screen.

[0369] Although FIG6 shows source device 12 and destination device 14 as separate devices, device embodiments may also include both source device 12 and destination device 14 or the functionality of both source device 12 and destination device 14, that is, both source device 12 or the corresponding functionality and destination device 14 or the corresponding functionality. In these embodiments, source device 12 or the corresponding functionality and destination device 14 or the corresponding functionality may be implemented using the same hardware and / or software or through separate hardware and / or software, or any combination thereof.

[0370] Based on the description, it will be apparent to those skilled in the art that the existence and (precise) division of different units or functions in the source device 12 and / or the destination device 14 shown in FIG. 6 may vary depending on the actual device and application.

[0371] The content providing system of the content distribution service used in the present application is described below in conjunction with Figure 7A. Figure 7A is a block diagram of a content providing system for implementing the content distribution service used in an embodiment of the present application. The content providing system 2100 includes a capture device 2102, a terminal device 2106 and an (optional) display 2126. The capture device 2102 communicates with the terminal device 2106 via a communication link 2104. The communication link may include the above-mentioned communication channel 13. The communication link 2104 includes but is not limited to WiFi, Ethernet, wired, wireless (3G / 4G / 5G), USB or any combination thereof.

[0372] Capture device 2102 generates data and can use the coding method shown in the above embodiment to encode the data. Alternatively, capture device 2102 can distribute data to a streaming media server (not shown), which encodes the data and transmits the encoded data to terminal device 2106. Capture device 2102 includes but is not limited to a camera, a smart phone or a tablet computer, a computer or a notebook computer, a video conferencing system, a PDA, a vehicle-mounted device, or any combination thereof. For example, capture device 2102 can include above-mentioned source device 12. When data includes video, the video encoder 20 in capture device 2102 can actually perform video encoding. When data includes audio (i.e., sound), the audio encoder 20 in capture device 2102 can actually perform audio encoding. In some actual scenarios, capture device 2102 distributes encoded video data and encoded audio data by multiplexing the encoded video data and the encoded audio data together. In other actual scenarios, for example, in a video conferencing system, the encoded audio data and the encoded video data are not multiplexed. Capture device 2102 distributes the encoded audio data and the encoded video data to terminal device 2106 respectively.

[0373] The terminal device 2106 in the content providing system 2100 receives and regenerates the encoded data. The terminal device 2106 can be a device with data reception and recovery capabilities, such as a smartphone or tablet computer 2108, a computer or laptop computer 2110, a network video recorder (NVR) / digital video recorder (DVR) 2112, a television 2114, a set-top box (STB) 2116, a video conferencing system 2118, a personal digital assistant (PDA) 2122, an in-vehicle device 2124, or any combination thereof, or any other device capable of decoding the encoded data. For example, the terminal device 2106 can include the destination device 14 described above. When the encoded data includes video, the video decoder 30 in the terminal device prioritizes video decoding. When the encoded data includes audio, the audio decoder in the terminal device prioritizes audio decoding. The terminal device 2106 can be a video playback application, a streaming media playback application, a streaming media playback platform, or a live broadcast platform running on the terminal device.

[0374] For end devices with displays, such as smartphones or tablets 2108, computers or laptops 2110, NVRs / DVRs 2112, TVs 2114, PDAs 2122, or in-vehicle devices 2124, the end device can send the decoded data to its display. For end devices without displays, such as STBs 2116 and video conferencing systems 3118, the device can be connected to an external display 2126 to receive and display the decoded data.

[0375] When performing encoding or decoding, each device in this system may use the image encoding device or image decoding device shown in the above embodiments.

[0376] FIG7B is a schematic diagram of an example structure of the terminal device 2106 in FIG7A . After the terminal device 2106 receives the bitstream from the capture device 2102, the protocol processing unit 2202 analyzes the transmission protocol of the bitstream. The protocol includes, but is not limited to, Real Time Streaming Protocol (RTSP), Hyper Text Transfer Protocol (HTTP), HTTP Live Streaming Protocol (HLS), MPEG Dynamic Adaptive Streaming over HTTP (MPEG-DASH), Real-time Transport Protocol (RTP), Real Time Messaging Protocol (RTMP), or any combination thereof.

[0377] After processing the stream, protocol processing unit 2202 generates a stream file. This file is output to demultiplexing unit 2204. Demultiplexing unit 2204 can separate the multiplexed data into coded audio data and coded video data. As mentioned above, in other practical scenarios, such as video conferencing systems, coded audio data and coded video data are not multiplexed. In this case, the coded data is not transmitted to video decoder 3206 and audio decoder 2208 via demultiplexing unit 2204.

[0378] Through the demultiplexing process, a video elementary stream (ES), an audio ES, and optional subtitles are generated. The video decoder 2206, including the video decoder 30 described in the above embodiment, decodes the video ES by the decoding method shown in the above embodiment to generate video frames, and sends the data to the synchronization unit 2212. The audio decoder 2208 decodes the audio ES to generate audio frames, and sends the data to the synchronization unit 2212. Alternatively, the video frames can be stored in a buffer (not shown in the figure) before being sent to the synchronization unit 2212. Similarly, the audio frames can be stored in a buffer (not shown in the figure) before being sent to the synchronization unit 2212.

[0379] The synchronization unit 2212 synchronizes the video frames and audio frames and provides the video / audio to the video / audio display 3214. For example, the synchronization unit 2212 synchronizes the presentation of the video and audio information. The information can be encoded in the syntax using the timestamps associated with the representation of the coded audio and visual data and the timestamps associated with the transmission of the data stream.

[0380] If subtitles are included in the bitstream, the subtitle decoder 2210 decodes the subtitles, synchronizes them with the video frames and audio frames, and provides the video / audio / subtitles to the video / audio / subtitle display 2216.

[0381] The present invention is not limited to the above-mentioned system. The image encoding device or the image decoding device in the above-mentioned embodiments can be used in other systems such as automobiles.

[0382] The following describes a streaming media system applicable to an embodiment of the present application in conjunction with Figure 8A. Figure 8A is a schematic diagram of a workflow of a streaming media system applicable to an embodiment of the present application.

[0383] The streaming media system includes a content creation module that generates the required content data, such as video or audio. The streaming media system also includes a video encoding module that encodes the generated content through an encoder. The streaming media system also includes a video stream transmission module that transmits the encoded video in the form of a code stream. Optionally, the format of the video stream can be converted into a code stream format of a commonly used transmission protocol for OTT (over-the-top) devices, for example, the protocol includes but is not limited to Real Time Streaming Protocol (RTSP), Hyper Text Transfer Protocol (HTTP), HTTP Live streaming protocol (HLS), MPEG HTTP Dynamic Adaptive Streaming over HTTP (MPEG-DASH), Real-time Transport Protocol (RTP), Real Time Messaging Protocol (RTMP) or any combination thereof. Optionally, the video stream can be stored to store the original format of the video stream and / or the converted multiple code stream formats for easy use. Furthermore, the streaming system also includes a video stream encapsulation module for encapsulating the video stream to generate an encapsulated video stream. The encapsulated video stream can be referred to as a video streaming package. Exemplarily, the video streaming package can be generated based on a transcoded video stream or a stored video stream. Furthermore, the streaming system also includes a content distribution network (CDN) for distributing the video streaming package to multiple OTT devices, such as mobile phones, computers, tablets, and home projectors.

[0384] It should be noted that video encoding, video streaming transmission, video stream transcoding, video stream storage, video streaming package generation and content distribution network can all be implemented on cloud servers.

[0385] An exemplary streaming media system architecture of the present application is described below in conjunction with FIG8B . The streaming media system architecture includes: a client device, a content distribution network, and a cloud server.

[0386] The user on the client device sends a play or playback request to the cloud platform. Optionally, the content of the request can be the title of the movie or TV program to be played.

[0387] The cloud platform makes a decision and responds to the client, sending the client the address of the requested content on the CDN. Optionally, the content sent to the client can be a URL link (uniform resource locator). Specifically, the playback application service in the cloud platform checks user authorization and permissions, and then considers individual client characteristics and current network conditions to determine which specific files are needed to process the playback request. It should be noted that the content delivery network (CDN) regularly reports its operating status, learned routes, and available content (files) to the cache control service in the cloud platform.

[0388] The client then requests the CDN to play the content based on the address, and the CDN provides the content to the client, ultimately completing the client's request.

[0389] The following describes the system architecture applicable to the embodiment of the present application in conjunction with Figure 9A. Figure 9A is a schematic diagram of a possible system architecture applicable to the embodiment of the present application. The system architecture of the embodiment of the present application includes: front-end equipment, transmission links, and terminal display equipment.

[0390] Among them, the front-end equipment is used to capture or produce HDR / SDR content (for example, HDR / SDR video or images).

[0391] In a possible embodiment, the front-end device can also be used to extract corresponding metadata from the HDR content. The metadata may include global mapping information, local mapping information, and dynamic metadata and static metadata corresponding to the HDR content. The front-end device can send the HDR content and metadata to the terminal display device via a transmission link. Specifically, the HDR content and metadata can be transmitted in the form of one data packet, or respectively in two data packets, which is not specifically limited in the embodiment of the present application.

[0392] Optionally, the terminal display device can be used to receive metadata and HDR content, and extract the global mapping information, local mapping information, and the terminal display device information contained in the corresponding metadata according to the HDR content, obtain a mapping curve to perform global tone mapping and local tone mapping on the HDR content, convert it into display content adapted for the HDR display device or SDR device in the terminal display device, and display it. It should be understood that in different embodiments, the terminal display device may include a display device with a display capability of a lower dynamic range or a higher dynamic range than the HDR content generated by the front-end device, and this application is not limited thereto.

[0393] Optionally, the front-end device and the terminal display device in this application can be independent and different physical devices. For example, the front-end device can be a video capture device or a video production device, where the video capture device can be a video camera, a camera, an image rendering machine, etc. The terminal display device can be a device with video playback function, such as virtual reality (VR) glasses, a mobile phone, a tablet, a television, a projector, etc.

[0394] Optionally, the transmission link between the front-end device and the terminal display device can be a wireless connection or a wired connection, wherein the wireless connection can adopt technologies such as long term evolution (LTE), fifth generation (5G) mobile communications, and future mobile communications. Wireless connections can also include wireless fidelity (WiFi), Bluetooth, near field communication (NFC), and other technologies. Wired connections can include Ethernet connections, local area network connections, etc. There is no specific limitation on this.

[0395] The present application may also integrate the functions of the front-end device and the terminal display device into the same physical device, for example, a terminal device such as a mobile phone or tablet with a video capture function. The present application may also integrate some functions of the front-end device and some functions of the terminal display device into the same physical device. This is not specifically limited.

[0396] The following describes an end-to-end image processing system provided by an embodiment of the present application in conjunction with Figure 9B . This system can be applied to the system architecture shown in Figure 9A . Figure 9B is a schematic diagram of the structure of an image processing system provided by an embodiment of the present application. In Figure 8B , HDR / SDR content is exemplified by HDR video. The image processing system includes: an HDR preprocessing module, an HDR video encoding module, an HDR video decoding module, and a tone mapping module.

[0397] Among them, the HDR preprocessing module and the HDR video encoding module can be located in the front-end device shown in Figure 9A, and the HDR video decoding module and the tone mapping module can be located in the terminal display device shown in Figure 9A.

[0398] HDR pre-processing module: This module is used to extract dynamic metadata (e.g., maximum, minimum, average, and range of brightness) from HDR video, determine mapping curve parameters based on the dynamic metadata and the display capabilities of the target display device, write the mapping curve parameters into the dynamic metadata to obtain HDR metadata, and transmit the metadata. HDR video can be either captured or processed by a colorist; the display capabilities of the target display device are the brightness range that the target display device can display.

[0399] HDR video encoding module: used to encode HDR video and HDR metadata according to the video compression standard (for example, AVS or HEVC standard) (for example, embedding HDR metadata into the user-defined part of the bitstream) and output the corresponding bitstream (AVS or HEVC bitstream).

[0400] HDR video decoding module: used to decode the generated bitstream (AVS bitstream or HEVC bitstream) according to the standard corresponding to the bitstream format, and output the decoded HDR video and HDR metadata.

[0401] Tone mapping module: used to generate a mapping curve according to the parameters of the mapping curve in the decoded HDR metadata, and perform tone mapping on the decoded HDR video (i.e. HDR adaptation processing or SDR adaptation processing), and send the HDR adapted video after tone mapping to the HDR display terminal for display, or send the SDR adapted video to the SDR display terminal for display.

[0402] Exemplarily, the HDR pre-processing module may exist in a video acquisition device or a video production device.

[0403] Exemplarily, the HDR video encoding module may exist in a video acquisition device or a video production device.

[0404] Exemplarily, the HDR video decoding module may be present in a set-top box, a television display device, a mobile terminal display device, and a video conversion device for live broadcasting, online video applications, and the like.

[0405] For example, the tone mapping module may be present in a set-top box, a television display device, a mobile terminal display device, or a video conversion device for a live webcast or online video application. More specifically, the tone mapping module may be present in the form of a chip or software program in the set-top box, television display, or mobile terminal display, and may be present in the form of a software program in the video conversion device for a live webcast or online video application.

[0406] In a possible embodiment, when the tone mapping module and the HDR video decoding module are both present in a set-top box, the set-top box can complete the functions of receiving, decoding, and tone mapping the video stream. The set-top box sends the decoded video data to a display device through a high-definition multimedia interface (HDMI) for display, so that the user can enjoy the video content.

[0407] In an example, FIG10 shows a schematic block diagram of a device 1000 according to an embodiment of the present application. The device 1000 may include: a processor 1001 and a transceiver / transceiver pin 1002 , and optionally, a memory 1003 .

[0408] The various components of the device 1000 are coupled together via a bus 1004. The bus 1004 includes a data bus, a power bus, a control bus, and a status signal bus. However, for clarity, all buses are referred to as bus 1004 in the figure.

[0409] Optionally, the memory 1003 may be used to store instructions in the aforementioned method embodiment. The processor 1001 may be used to execute the instructions in the memory 1003 and control the receiving pin to receive a signal and control the transmitting pin to send a signal.

[0410] The apparatus 1000 may be the electronic device or a chip of the electronic device in the above method embodiment.

[0411] The electronic device may be a terminal device or a server.

[0412] Among them, all relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional module and will not be repeated here.

[0413] The present application also provides a chip including one or more interface circuits and one or more processors; the one or more processors receive or send data via the one or more interface circuits, and when the one or more processors execute computer instructions, the above-mentioned related method steps are implemented to implement the steps of the method in the above-mentioned embodiment. The interface circuit is a transceiver / transceiver pin 1002.

[0414] This embodiment further provides a computer-readable storage medium, in which computer instructions are stored. When the computer instructions are executed on an electronic device, the electronic device executes the above-mentioned related method steps to implement the method in the above-mentioned embodiment.

[0415] This embodiment further provides a computer program product, which includes computer instructions. When the computer instructions are executed by a computer or a processor, the computer executes the above-mentioned related steps to implement the method in the above-mentioned embodiment.

[0416] In addition, an embodiment of the present application also provides a device, which can specifically be a chip, component or module, and the device may include a connected processor and memory; wherein the memory is used to store computer-executable instructions, and when the device is running, the processor can execute the computer-executable instructions stored in the memory to enable the chip to execute the methods in the above-mentioned method embodiments.

[0417] Among them, the electronic device, computer-readable storage medium, computer program product or chip provided in this embodiment are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0418] Through the description of the above implementation methods, technical personnel in the relevant field can understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0419] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely schematic. For example, the division of modules or units is only a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0420] Units described as separate components may or may not be physically separate, and components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0421] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0422] Any content of each embodiment of this application, as well as any content of the same embodiment, can be freely combined. Any combination of the above content is within the scope of this application.

[0423] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to execute all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0424] The steps of the method or algorithm described in conjunction with the disclosure of the embodiments of the present application can be implemented in a hardware manner, or can be implemented by a processor executing a software instruction. The software instruction can be composed of corresponding software modules, and the software module can be stored in a random access memory (Random Access Memory, RAM), a flash memory, a read-only memory (Read Only Memory, ROM), an erasable programmable read-only memory (Erasable Programmable ROM, EPROM), an electrically erasable programmable read-only memory (Electrically EPROM, EEPROM), a register, a hard disk, a mobile hard disk, a read-only compact disc (CD-ROM) or any other form of storage medium well known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and can write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC.

[0425] Those skilled in the art will appreciate that in one or more of the above examples, the functions described in the embodiments of the present application can be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer-readable storage media and communication media, wherein communication media include any media that facilitates the transmission of computer programs from one place to another. The storage medium can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0426] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of this application, ordinary technicians in this field can also make many forms without departing from the purpose of this application and the scope of protection of the claims, all of which are within the protection of this application.

Claims

1. A data encapsulation method, characterized in that, The method includes: Obtaining T.35 information; wherein, the T.35 information includes: country code, terminal provider code, and terminal provider orientation code; Encapsulating the T.35 information into an image file format; wherein, the image file format includes a first box, and the first box includes the T.35 information.

2. The method according to claim 1, wherein The image file format further includes a second box, and the second box includes any one of the following: metadata box or stream data box; The first box is nested in the second box.

3. The method according to claim 2, wherein The second box includes a plurality of third boxes, and there is a hierarchical relationship between the plurality of third boxes; The first box is nested in any level of the third boxes.

4. The method according to claim 1, wherein The image file format further includes a project identifier and a media box, and the project identifier indicates that the T.35 information is a project; The T.35 information includes header information and payload, the header information includes the country code, the terminal provider code, and the terminal provider orientation code, and the payload includes metadata of multimedia; The first box includes the header information of the T.35 information, and the media box includes the payload of the T.35 information.

5. The method according to claim 2 or 3, characterized in that, The T.35 information is a project, and the metadata box includes an image information box; The first box is nested in the image information box, and the first box is a project information entry box.

6. The method according to claim 2 or 3, characterized in that, The T.35 information is a project attribute, and the metadata box includes a project attribute container box and a project attribute association box; The first box is nested in the project attribute container box, and the project attribute association box includes the association information between the T.35 information and the corresponding project.

7. The method according to claim 2 or 3, characterized in that, The T.35 information is a sample entry, the stream data box includes a track box, and the track box includes a sample description box; The first box is nested in the sample description box.

8. The method according to claim 2 or 3, characterized in that, The T.35 information is a sample group entry, the stream data box includes a track box, and the track box includes a sample group description box; The first box is nested in the sample group description box.

9. The method according to claim 2 or 3, characterized in that, The T.35 information is a sample entry, the stream data box includes a track box, and the track box includes a sample description box and a sample group description box; the T.35 information includes header information and payload, the header information includes the country code, the terminal provider code, and the terminal provider orientation code, and the payload includes metadata of multimedia; The first box includes the header information of the T.35 information, and the first box is nested in the sample group description box; The sample description box includes the payload of the T.35 information.

10. The method according to claim 2 or 3, characterized in that, The T.35 information is a sample entry, the stream data box includes a timing metadata track box, and the timing metadata track box includes a metadata sample entry box, The first box is nested in the metadata sample entry box.

11. The method according to any one of claims 1 to 10, characterized in that, The method further includes: Obtaining encoded multimedia data; Encapsulate the encoded multimedia data into the image file format, which further includes a media box or a stream data box, and the media box or the stream data box includes the encoded multimedia data.

12. The method according to any one of claims 1 to 11, characterized in that, The T.35 information further includes metadata of the multimedia.

13. A data acquisition method, characterized in that, The method includes: Receiving an image file format; wherein, the image file format includes a first box, and the first box includes T.35 information; Reading the T.35 information from the first box of the image file format; wherein, the T.35 information includes: a country code, a terminal provider code, and a terminal provider orientation code.

14. The method according to claim 13, characterized in that, The image file format further includes a second box, and the second box includes any one of the following: a metadata box or a stream data box; The first box is nested in the second box.

15. The method according to claim 14, wherein The second box includes a plurality of third boxes, and there is a hierarchical relationship among the plurality of third boxes; The first box is nested in a third box at any level.

16. The method according to claim 13, wherein, The image file format further includes an item identifier and a media box, and the item identifier indicates that the T.35 information is an item; The T.35 information includes header information and a payload, the header information includes the country code, the terminal provider code, and the terminal provider orientation code, and the payload includes metadata of the multimedia; The first box includes the header information of the T.35 information, and the media box includes the payload of the T.35 information.

17. The method according to claim 14 or 15, characterized in that The T.35 information is an item, and the metadata box includes an image information box; The first box is nested in the image information box, and the first box is an item information entry box.

18. The method according to claim 14 or 15, characterized in that, The T.35 information is an item attribute, and the metadata box includes an item attribute container box and an item attribute association box; The first box is nested in the item attribute container box, and the item attribute association box includes the association information between the T.35 information and the corresponding item.

19. The method according to claim 14 or 15, characterized in that, The T.35 information is a sample entry, the stream data box includes a track box, and the track box includes a sample description box; The first box is nested in the sample description box.

20. The method according to claim 14 or 15, characterized in that, The T.35 information is a sample group entry, the stream data box includes a track box, and the track box includes a sample group description box; The first box is nested in the sample group description box.

21. The method according to claim 14 or 15, characterized in that, The T.35 information is a sample entry, the stream data box includes a track box, and the track box includes a sample description box and a sample group description box; the T.35 information includes header information and a payload, the header information includes the country code, the terminal provider code, and the terminal provider orientation code, and the payload includes metadata of the multimedia; The first box includes the header information of the T.35 information, and the first box is nested in the sample group description box; The sample description box includes the payload of the T.35 information.

22. The method according to claim 14 or 15, characterized in that, The T.35 information is a sample entry, and the stream data box includes a timing metadata track box, and the timing metadata track box includes a metadata sample entry box. The first box is nested in the metadata sample entry box.

23. The method according to any one of claims 13 to 22, characterized in that, The image file format further includes a media box or a stream data box, and the media box or the stream data box includes encoded multimedia data. The method further includes: Obtaining the encoded multimedia data from the media box or the stream data box of the image file format. Decoding the encoded multimedia data to obtain reconstructed multimedia data.

24. The method according to any one of claims 14 to 23, characterized in that, The T.35 information further includes metadata of the multimedia.

25. An image file format, characterized in that, The image file format is obtained by encapsulating according to the data encapsulation method described in any one of claims 1 to 12 above.

26. An electronic device, characterized in that, Including: A memory and a processor, the memory being coupled to the processor. The memory stores program instructions, and when the program instructions are executed by the processor, the electronic device is caused to execute the method described in any one of claims 1 to 12.

27. An electronic device, characterized in that, Including: A memory and a processor, the memory being coupled to the processor. The memory stores program instructions, and when the program instructions are executed by the processor, the electronic device is caused to execute the method described in any one of claims 13 to 24.

28. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, and when the computer program runs on a computer or a processor, the computer or the processor is caused to execute the method described in any one of claims 1 to 24.

29. A computer program product, characterized in that, The computer program product contains computer instructions, and when the computer instructions are executed by a computer or a processor, the steps of the method described in any one of claims 1 to 24 are caused to be executed.

30. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores an image file format, and the image file format is obtained by encapsulating according to the data encapsulation method described in any one of claims 1 to 12 above.

Images