Live photo encapsulation method and apparatus, device, storage medium, and program product

Abstract

The present application discloses a live photo encapsulation method and apparatus, a device, a storage medium, and a program product. The method comprises: acquiring a captured first picture and a filmed first video; encoding attribute information of the first picture and the first video to generate extensible metadata platform information in a preset picture format; and encapsulating the first picture and the first video on the basis of the preset picture format to generate a live photo file. In this way, it can be ensured that a live photo can be correctly identified and parsed during end-to-end sharing, and using a unified encapsulation format can prompt further improvement the industrial ecology for live photos.

Description

Dynamic picture packaging method, device, equipment, storage medium and program product

[0001] Cross-reference to Related Applications

[0002] The present application claims priority to the Chinese patent application No. 202411254827.5, filed on September 6, 2024, and entitled "Dynamic picture packaging method, device, equipment, storage medium and program product", the whole content of which is incorporated herein by reference. TECHNICAL FIELD

[0003] The present application relates to the technical field of picture processing, and in particular to a dynamic picture packaging method, device, equipment, storage medium and program product. BACKGROUND

[0004] The current dynamic picture is widely used in electronic devices of various manufacturers and various operating systems. The dynamic picture, also known as a live photo, is displayed in the form of a static picture in the gallery, and the user can quickly play the video by long pressing, etc. It is often used to capture exciting moments, making the photo more lively.

[0005] However, there is no uniform standard for packaging dynamic pictures at present, so that the existing dynamic picture scheme may have some defects. For example, due to the different packaging formats of various manufacturers, when using the picture viewing software of other manufacturers, the dynamic picture may not be recognized, so that the video cannot be normally played. Moreover, the non-uniform packaging standard also causes serious ecological fragmentation, which is not conducive to the development of the dynamic picture industry ecology. SUMMARY

[0006] The technical scheme of the present application is implemented as follows:

[0007] In a first aspect, the embodiments of the present application provide a dynamic picture packaging method, which comprises:

[0008] obtaining a first picture and a first video taken;

[0009] encoding attribute information of the first picture and the first video to generate extensible metadata platform information in a preset picture format;

[0010] packaging the first picture and the first video based on the preset picture format to generate a dynamic picture file.

[0011] In a second aspect, the embodiments of the present application provide a dynamic picture packaging method, which comprises:

[0012] obtaining a dynamic picture file;

[0013] Decode the animated image file to determine the first image and the extensible metadata platform information;

[0014] The dynamic image file is decoded based on information from the extensible metadata platform to determine the first video and / or extended information.

[0015] Thirdly, embodiments of this application provide a dynamic image encapsulation device, which includes an acquisition unit, an encoding unit, and an encapsulation unit, wherein:

[0016] The acquisition unit is configured to acquire the first captured image and the first video.

[0017] The encoding unit is configured to encode the attribute information of the first image and the first video to generate extensible metadata platform information in a preset image format;

[0018] The encapsulation unit is configured to encapsulate the first image and the first video based on a preset image format to generate a dynamic image file.

[0019] Fourthly, embodiments of this application provide a dynamic image encapsulation device, which includes an acquisition unit and a decoding unit, wherein:

[0020] The acquisition unit is configured to acquire animated image files;

[0021] The decoding unit is configured to decode a dynamic image file to determine a first image and extensible metadata platform information; and to decode the dynamic image file based on the extensible metadata platform information to determine a first video and / or extended information.

[0022] Fifthly, embodiments of this application provide an electronic device, which includes a memory and a processor, wherein:

[0023] Memory is used to store computer programs that can run on a processor;

[0024] A processor, configured to, while running a computer program, perform the method described in the first aspect or the method described in the second aspect.

[0025] In a sixth aspect, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the first aspect or the method described in the second aspect.

[0026] In a seventh aspect, embodiments of this application provide a computer program product, including a computer program or instructions that, when executed by a processor, implement the method described in the first aspect or the method described in the second aspect. Attached Figure Description

[0027] FIG. 1 is a flow diagram of a dynamic picture packaging method according to an embodiment of the present application;

[0028] FIG. 2 is a storage structure diagram of a dynamic picture file according to an embodiment of the present application;

[0029] FIG. 3 is a flow diagram of a dynamic picture packaging method according to an embodiment of the present application;

[0030] FIG. 4 is a packaging definition diagram of a dynamic picture file according to an embodiment of the present application;

[0031] FIG. 5 is a packaging definition diagram of a dynamic picture file according to an embodiment of the present application;

[0032] FIG. 6 is a flow diagram of a dynamic picture packaging method according to an embodiment of the present application;

[0033] FIG. 7 is a packaging definition diagram of a dynamic picture file according to an embodiment of the present application;

[0034] FIG. 8 is a packaging definition diagram of a dynamic picture file according to an embodiment of the present application;

[0035] FIG. 9 is a flow diagram of a dynamic picture packaging method according to an embodiment of the present application;

[0036] FIG. 10 is a structural diagram of a dynamic picture packaging apparatus according to an embodiment of the present application;

[0037] FIG. 11 is a structural diagram of a dynamic picture packaging apparatus according to an embodiment of the present application;

[0038] FIG. 12 is a specific hardware structure diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0039] In a first aspect, an embodiment of the present application provides a dynamic picture packaging method, which comprises:

[0040] acquiring a first picture and a first video taken;

[0041] encoding attribute information of the first picture and the first video to generate extensible metadata platform information in a preset picture format;

[0042] packaging the first picture and the first video based on the preset picture format to generate a dynamic picture file.

[0043] In some embodiments, the preset picture format comprises a static picture part and a video part, and the video part is located after the static picture part; the encapsulating the first picture and the first video based on the preset picture format comprises:

[0044] storing the first picture in the static picture part;

[0045] storing the first video in the video part.

[0046] In some embodiments, the video part comprises a video data subpart and an extensible subpart; the method further comprises:

[0047] storing the first video in the video data subpart;

[0048] storing the extension information in the extensible subpart after obtaining the extension information of the dynamic picture file.

[0049] In some embodiments, the preset picture format further comprises an extensible part, and the extensible part is independent of the video part; the method further comprises:

[0050] storing the extension information in the extensible part after obtaining the extension information of the dynamic picture file.

[0051] In some embodiments, the method further comprises:

[0052] storing the data content in the extension information in the form of key-value pairs; or,

[0053] storing the data content in the extension information in the form of multi-layer key-value pairs.

[0054] In some embodiments, the attribute information comprises at least one of the following: offset information of the first video, length information of the first video, offset information of the extension information, indication information of whether the dynamic picture file is a dynamic picture, file format version of the dynamic picture file, and presentation timestamp of a video frame corresponding to the first picture.

[0055] In some embodiments, the picture format of the first picture comprises at least one of the following: JPEG format, HEIF / AVIF format; the video format of the first video comprises at least one of the following: MP4 format, MOV format.

[0056] In some embodiments, in the case that the first picture is in JPEG format, the encapsulating the first picture and the first video based on the preset picture format comprises:

[0057] In the case that the first picture is a standard dynamic range type picture, encapsulating based on the picture start bit in the static picture part, extensible metadata platform information, the first picture and the picture end bit, and the first video and / or the extension information in the video part; or,

[0058] In a case where the first picture is a high dynamic range type picture, the first picture and the first video are encapsulated based on a main graph part and a gain graph part in a static picture part and the first video and / or the extension information in a video part; wherein the main graph part comprises a main picture start bit, the extensible metadata platform information, the first picture and a main picture end bit, and the gain graph part comprises a gain picture start bit, private extensible metadata platform information, the gain graph of the first picture and a gain picture end bit.

[0059] In some embodiments, in a case where the first picture is in a HEIF / AVIF format, the first picture and the first video are encapsulated based on a preset picture format, comprising:

[0060] In a case where the first picture is a standard dynamic range type picture, the first picture and the first video are encapsulated based on an mdat data block, a meta data block and a free data block in a static picture part and the first video and / or the extension information in a video part; wherein the first picture and the extensible metadata platform information are both located in the mdat data block; or,

[0061] In a case where the first picture is a high dynamic range type picture, the first picture and the first video are encapsulated based on an mdat data block, a meta data block and a free data block in a static picture part and the first video and / or the extension information in a video part; wherein the first picture, the gain graph of the first picture and the extensible metadata platform information are all located in the mdat data block.

[0062] In some embodiments, after the dynamic picture file is obtained, the method further comprises:

[0063] The dynamic picture file is decoded to determine the first picture and the extensible metadata platform information;

[0064] The dynamic picture file is decoded according to the extensible metadata platform information to determine the first video and / or the extension information.

[0065] In some embodiments, the extensible metadata platform information at least comprises offset information of the first video and offset information of the extension information; wherein the offset information of the first video is used to determine the storage location of the first video in the dynamic picture file, and the offset information of the extension information is used to determine the storage location of the extension information in the dynamic picture file.

[0066] In some embodiments, in a case where the first video and the extension information are included in the dynamic picture file, the dynamic picture file is decoded according to the extensible metadata platform information, comprising:

[0067] The dynamic picture file is decoded according to the offset information of the first video to determine the first video;

[0068] According to the offset information of the extension information, the dynamic picture file is decoded to determine the extension information.

[0069] In a second aspect, the embodiments of the present application provide a dynamic picture packaging method, which comprises:

[0070] obtaining a dynamic picture file;

[0071] decoding the dynamic picture file to determine a first picture and extensible metadata platform information;

[0072] decoding the dynamic picture file according to the extensible metadata platform information to determine a first video and / or extension information.

[0073] In some embodiments, the dynamic picture file comprises a static picture part and a video part, and the video part is located after the static picture part; wherein the static picture part is used to store the first picture, and the video part is used to store the first video and the extension information.

[0074] In some embodiments, when the first video and the extension information are included in the dynamic picture file, decoding the dynamic picture file according to the extensible metadata platform information comprises:

[0075] determining offset information of the first video and offset information of the extension information based on the extensible metadata platform information;

[0076] decoding the dynamic picture file according to the offset information of the first video to determine the first video;

[0077] decoding the dynamic picture file according to the offset information of the extension information to determine the extension information.

[0078] In a third aspect, the embodiments of the present application provide a dynamic picture packaging device, which comprises an obtaining unit, an encoding unit and a packaging unit, wherein:

[0079] the obtaining unit is configured to obtain a first picture and a first video taken;

[0080] the encoding unit is configured to encode attribute information of the first picture and the first video to generate extensible metadata platform information in a preset picture format;

[0081] the packaging unit is configured to package the first picture and the first video based on the preset picture format to generate a dynamic picture file.

[0082] In a fourth aspect, the embodiments of the present application provide a dynamic picture packaging device, which comprises an obtaining unit and a decoding unit, wherein:

[0083] the obtaining unit is configured to obtain a dynamic picture file;

[0084] a decoding unit configured to decode the dynamic picture file to determine the first picture and the extensible metadata platform information, and decode the dynamic picture file according to the extensible metadata platform information to determine the first video and / or the extended information.

[0085] In a fifth aspect, an electronic device is provided, which includes a memory and a processor, wherein:

[0086] a memory for storing a computer program capable of running on the processor;

[0087] the processor is configured to execute the method according to the first aspect or the method according to the second aspect when running the computer program.

[0088] In a sixth aspect, a computer readable storage medium is provided, which stores a computer program, and the computer program is executed by a processor to implement the method according to the first aspect or the method according to the second aspect.

[0089] In a seventh aspect, a computer program product is provided, which includes a computer program or instructions, and the computer program or instructions are executed by a processor to implement the method according to the first aspect or the method according to the second aspect.

[0090] In order to enable more detailed understanding of the features and technical contents of the embodiments of the present application, the implementation of the embodiments of the present application is described in detail below with reference to the accompanying drawings, and the accompanying drawings are only used for reference and are not intended to limit the embodiments of the present application.

[0091] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0092] It should be noted that the terms "first", "second", and "third" used in the embodiments of the present application are only used to distinguish similar objects, and do not represent a specific order or sequence of the objects. It can be understood that the "first", "second", and "third" can be interchanged in a specific order or sequence as allowed, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0093] It should also be noted that in the following description, "some embodiments" are described, which describe a subset of all possible embodiments, but it can be understood that "some embodiments" can be the same subset or different subsets of all possible embodiments, and can be combined with each other without conflict.

[0094] It should be understood that the term "and / or" used herein is only a description of the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B can represent the following three cases: A exists alone, A and B exist together, and B exists alone. In addition, the character " / " herein generally represents an "or" relationship between the associated objects before and after it.

[0095] It should also be understood that before the embodiments of the present application are further described in detail, the terms and terms involved in the embodiments of the present application are explained, which are applicable to the following explanations:

[0096] Joint Photographic Experts Group (JPEG), JPEG is a widely used image format, but it will introduce more distortion at a higher compression rate;

[0097] High Efficiency Image File Format (HEIF), HEIF is a modern image format that uses an efficient compression algorithm to maintain high-quality images at a smaller file size; sometimes also called HEIC (High Efficiency Image File Coding), the file name usually has a.heif or.heic suffix.

[0098] AV1 Image File Format (AVIF), which is an image format based on the AV1 codec developed by the Alliance for Open Media (AOM);

[0099] Start of Image (SOI), which is a marker bit in a JPEG image used to describe the beginning;

[0100] End of Image (EOI), which is a marker bit in a JPEG image used to describe the end;

[0101] ISO Base Media File Format (ISO-BMFF), BMFF-Box is a data unit conforming to the BMFF format standard and used to encapsulate specific types of data;

[0102] Extensible Metadata Platform (XMP);

[0103] Exchangeable image file format (EXIF);

[0104] Primary;

[0105] Gainmap;

[0106] Field of View (FOV);

[0107] Standard Dynamic Range (SDR), the details in SDR picture are relatively weak, and there may be details loss in bright and dark areas;

[0108] High Dynamic Range (HDR), HDR can better show the details in the picture, whether in bright or dark area, HDR picture can better present the texture of the picture;

[0109] MOV, i.e. QuickTime encapsulation format, is an audio, video file encapsulation, used to store common digital media types;

[0110] MPEG-4 or MP4 for short, is a video encapsulation standard proposed by Moving Picture Experts Group, which is a set of compression and encoding standards for audio and video information.

[0111] It should also be understood that the current dynamic picture is widely used in electronic devices of various manufacturers and operating systems. Dynamic picture, also known as dynamic image, refers to a picture that can play video. When the dynamic picture is not triggered, the dynamic picture displays the cover picture content, and when the dynamic picture is triggered, the dynamic picture will play the video content.

[0112] Live Photo technology is a scheme of encapsulating video in a picture file. If the Live Photo function of the camera is turned on, the phone will record a fixed length (such as 1.5s) of image data before and after taking a picture. Users can choose different cover photos, add interesting effects, edit Live Photo photos, or share them with family and friends. It can be seen that Live Photo actually has the characteristics of both pictures and videos, and cannot be simply attributed to pictures or videos. Since there is no uniform Live Photo standard in the industry, this function mainly relies on the private implementation of mobile phone manufacturers.

[0113] The key to the implementation of the Live Photo function lies in the encapsulation of the video. However, there is no uniform standard for encapsulating dynamic pictures at present, so that the existing dynamic picture scheme may have some defects. For example, when the dynamic picture of a manufacturer A is transmitted to the device of a manufacturer B, the dynamic picture may not be recognized and is parsed as a normal static photo, so that the video cannot be normally played, and the user experience is reduced. Due to the different encapsulation formats of various manufacturers, the subsequent three-party application (App) ecological support is relatively complex, the standard is not unified, the ecological fragmentation problem is serious, and it is not conducive to the development of the dynamic picture industry ecology.

[0114] Based on this, an embodiment of the present application provides a dynamic picture encapsulation method. After obtaining a first picture and a first video, the attribute information of the first picture and the first video is encoded to generate extensible metadata platform information in a preset picture format. The first picture and the first video are encapsulated based on the preset picture format to generate a dynamic picture file. In this way, the dynamic picture is encapsulated and packaged based on the preset picture format. Since a unified dynamic picture encapsulation format is adopted, the dynamic picture can be correctly recognized and parsed when shared end to end, and the extensible metadata platform information includes offset information of the first video and offset information of the extension information, so that the first video and the extension information can be accurately read. In this way, the presentation effects of various manufacturers are consistent, the further improvement of the dynamic picture industry ecology is promoted, and the universality of the dynamic picture is improved.

[0115] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0116] In an embodiment of the present application, Fig. 1 is a flowchart of a dynamic picture encapsulation method provided by an embodiment of the present application. As shown in Fig. 1, the method can include:

[0117] S101, obtaining a first picture and a first video.

[0118] S102, encoding the attribute information of the first picture and the first video to generate extensible metadata platform information in a preset picture format.

[0119] It should be noted that in the embodiment of the present application, the method is applied to a first device. The first picture can be a normally taken picture, or can also be a frame image in the first video. The first video can be obtained from a video stream, or can also come from a video file of a dynamic picture to be processed, which is not limited here.

[0120] It should be further explained that in the embodiments of the present application, the attribute information of the first picture and the first video is decoded by using the extensible metadata platform information (i.e. XMP information) in the preset picture format, so that the first picture and the first video can be accurately decoded subsequently.

[0121] In some embodiments, the attribute information can include at least one of the following: offset information of the first video, length information of the first video, offset information of the extension information, indication information of whether the dynamic picture file is a dynamic picture, file format version of the dynamic picture file, and presentation timestamp of the video frame corresponding to the first picture.

[0122] In the embodiments of the present application, in addition to the first picture and the first video, the dynamic picture file can also include extension information for storing private data information of various manufacturers, such as video interpolation information and FOV view clipping information.

[0123] Here, for the attribute information, the offset information of the first video specifically refers to the offset starting position of the first video relative to the dynamic picture file, which can be represented by VideoOffset; the length information of the first video can be represented by VideoLength; the offset information of the extension information specifically refers to the offset starting position of the extension information relative to the dynamic picture file, which can be represented by ExtensionOffset; the indication information of whether the dynamic picture file is a dynamic picture can be represented by MotionPhoto; the file format version of the dynamic picture file can be represented by MotionPhotoVersion; and the presentation timestamp of the video frame corresponding to the first picture can be represented by MotionPhotoPresentationTimestampUs.

[0124] Exemplarily, the default namespace prefix of the dynamic picture XMP information is GCamera, and Table 1 shows the related attribute information that can be displayed in the XMP information.

[0125] Table 1

[0126] S103, encapsulating the first picture and the first video based on the preset picture format to generate a dynamic picture file.

[0127] In the embodiments of the present application, the first picture and the first video can be packaged into a media file according to the preset picture format, i.e. a dynamic picture file (which can also be referred to as a "dynamic picture"), and stored in the electronic device. After the dynamic picture file is generated, the dynamic picture is displayed in the gallery in the form of a static picture, and the user can quickly play the video by long pressing, etc., which is commonly used for capturing exciting moments, making the photo more lively, such as fireworks, diving, etc.

[0128] It should be noted that, in the embodiment of the present application, as shown in FIG. 2, the preset picture format can include two parts: a static picture part 201 and a video part 202. The static picture part 201 is used to store the main picture content, and the video part 202 is used to store the video file and the extension information, and the video part 202 is located after the static picture part 201. It should be noted that the attribute information in Table 1 is set in the XMP information of the main picture.

[0129] In some embodiments, encapsulating the first picture and the first video based on the preset picture format can include: storing the first picture in the static picture part; and storing the first video in the video part.

[0130] That is, in the embodiment of the present application, the first picture is stored in the static picture part 201 as the main picture content in the dynamic picture, and the first video is stored in the video part 202.

[0131] It should be further noted that, still taking FIG. 2 as an example, the video part 202 can include a video data subpart and an extensible subpart. The method further includes: storing the first video in the video data subpart; and storing the extension information in the extensible subpart after the extension information of the dynamic picture file is obtained.

[0132] That is, in the embodiment of the present application, the first video is stored in the video data subpart as the video file in the dynamic picture, and the extension information is stored in the extensible subpart. In this way, when the first video and the extension information are included in the dynamic picture file at the same time, the offset information of the first video and the offset information of the extension information are included in the XMP information at this time, so that the first video and the extension information can be accurately read.

[0133] In some embodiments, the video part can be represented by a box, and the video data subpart and the extensible subpart can also be represented by a box. The data storage mode of the extension information is shown in Table 2.

[0134] Table 2

[0135] As shown in Table 2, the related content is explained as follows:

[0136] size: 4 bytes of UInt, indicating the size of the box, including the size of the placeholder;

[0137] type: indicating the type of the box, usually consisting of four printable ASCII characters;

[0138] largeSize: 8 bytes of ULong, when size is 1, it indicates that the box is large, wherein the box size is defined in this area.

[0139] In the embodiment of the present application, the data content in the extension information is stored in the form of key-value pairs. If there is much information to be added, the information can be saved in the form of multiple layers of key-value pairs.

[0140] It can be understood that, in the embodiment of the present application, the extensible part (box) for storing the extension information can be independent of the video part for the extension information, at this time the dynamic picture file includes a still picture part, a video part and an extensible part; or it can also be a subpart under the video part, that is, the aforementioned extensible subpart. Considering the actual application scenario, the embodiment of the present application is described by taking the extension information as a subpart (subbox) under the video part as an example, but is not specifically limited.

[0141] It can also be understood that, in the embodiment of the present application, the total length (size) of the dynamic picture file can include the length of the still picture part and the length of the video part. Here, the length of the still picture part 201 is equal to the offset of the video relative to the file, and the length of the video part is composed of the length of the video file and the length of the extension information. Moreover, it can be seen from FIG. 2 that the extension information should be stored at the end of the dynamic picture (after the video file), and the "MotionPhoto" length in the main picture XMP of the dynamic picture should be the length of the original video file plus the length of the extension information.

[0142] The embodiment of the present application provides a dynamic picture packaging method. After generating XMP information in a preset picture format according to attribute information of a first picture and a first video, the dynamic picture is packaged and packed based on the preset picture format. Here, a unified dynamic picture packaging format is adopted, so that the dynamic picture can be correctly recognized and parsed when shared end to end, and when the dynamic picture file includes the first video and the extension information, the XMP information correspondingly includes offset information of the first video and offset information of the extension information, so that the first video and the extension information can also be accurately read. In this way, the presentation effects of various manufacturers can be consistent, and the further improvement of the dynamic picture industry ecology can be promoted.

[0143] In another embodiment of the present application, based on the dynamic picture packaging method of the foregoing embodiment, the picture format of the first picture can at least include JPEG format and HEIF / AVIF format, and the video format of the first video can at least include MP4 format and MOV format, but is not limited thereto.

[0144] That is, in the embodiment of the present application, for the packaging of dynamic pictures, in addition to the video interpolation information, FOV view cutting information mentioned above, the embodiment of the present application can theoretically be extended to any information, such as video super-resolution, post-processing information, etc. In addition, in addition to the MP4 video format mentioned in the technical solution, the embodiment of the present application can also be extended to other video file formats such as MOV, etc. without any limitation.

[0145] In a possible implementation, FIG. 3 is a flow diagram of a dynamic picture packaging method provided by an embodiment of the present application. As shown in FIG. 3, the method can include the following steps:

[0146] S301, determining that the first picture is in JPEG format.

[0147] S302, when the first picture is a standard dynamic range type picture, packaging based on a picture start bit in a static picture part, extensible metadata platform information, the first picture and a picture end bit, and a first video and / or extension information in a video part.

[0148] In the embodiment of the present application, it is assumed that the first picture is an SDR type in the JPEG format, and the first video is an SDR / HDR type in the MP4 format. The preset picture format at this time is shown in FIG. 4. For the static picture part, there are a picture start bit (SOI), extensible metadata platform information (APP2: XMP) in the exchangeable image file format (APP1: EXIF), a start of scan (Start of Scan), the first picture and a picture end bit (EOI), etc. Then, video data (i.e. MotionPhoto) is added after the EOI of the JPEG image data, which is represented by "mpvd" and used to describe the video file and related extension information.

[0149] In addition, in the embodiment of the present application, manufacturers are also allowed to add customized BMFF-Box, such as the extensible subpart in FIG. 4, for storing private data information. The customized Box needs to comply with the BMFF standard and can be stored as a subbox in the'mpvd' box.

[0150] S303, when the first picture is a high dynamic range type picture, packaging based on a main picture part and a gain picture part in a static picture part, and a first video and / or extension information in a video part; wherein the main picture part includes a main picture start bit, extensible metadata platform information, the first picture and a main picture end bit.

[0151] In the embodiments of the present application, it is assumed that the first picture is of the HDR type in the JPEG format, and the first video is of the SDR / HDR type in the MP4 format, and the preset picture format at this time is shown in FIG. 5. For the static picture part, there are a main picture part and a gain picture part. The main picture part can include a start of image (SOI), an extensible metadata platform information (APP1: XMP), a multi-data file (APP2: MPF), the first picture, and an end of image (EOI), etc. The gain picture part includes a start of image (SOI), a private extensible metadata platform information (APP1: XMP), a start of scan (Start of Scan), a gain picture of the first picture, and an end of image (EOI), etc. Then, video data (i.e., MotionPhoto) is added after the EOI of the gain picture, which can also be represented by "mpvd" and is used to describe the video file and related extension information.

[0152] As shown in FIG. 5, the MotionPhoto field is added in the XMP file of the main picture based on the Ultra-HDR format of Google, and is used to describe the video file related information. Video data is added after the gain picture, and the customized Box of each manufacturer is allowed to store private information at the end of the file. The customized Box needs to comply with the BMFF standard and is stored as a sub-box in the'mpvd' box.

[0153] For example, the code description of the extensible metadata platform information (APP1: XMP) 501 in FIG. 5 is as follows:

[0154] For example, the code description of the private extensible metadata platform information (APP1: XMP) 502 in FIG. 5 is as follows:

[0155] In another possible implementation, FIG. 6 is a flow diagram of a dynamic picture packaging method provided by an embodiment of the present application. As shown in FIG. 6, the method can include the following steps:

[0156] S601, determining that the first picture is in the HEIF / AVIF format.

[0157] S602, when the first picture is a standard dynamic range type picture, packaging based on the mdat data block, the meta data block, and the free data block in the static picture part, and the first video and / or extension information in the video part, wherein the first picture and the extensible metadata platform information are both located in the mdat data block.

[0158] In the embodiments of the present application, it is assumed that the first picture is of SDR type in HEIF / AVIF format, and the first video is of SDR / HDR type in MP4 format, for example. The preset picture format at this time is shown in FIG. 7. For the static picture part, there are mdat data block, meta data block and free data block. Then, video data (i.e., MotionPhoto) is added after the free data block, which can also be represented by "mpvd" and used to describe the video file and related extension information. The mdat data block includes tiles1, tiles2, EXIF and XMP, etc.

[0159] That is, in the embodiments of the present application, MotionPhoto video data is added after the end of the main picture file (HEIF / AVIF) to describe the video file and related extension information. In addition, manufacturers are allowed to add customized BMFF-Box to store private data information. The customized Box needs to comply with the BMFF standard and is stored as a sub-box in the'mpvd' box.

[0160] For example, for XMP 701 in the mdat data block in FIG. 7, the code description is as follows:

[0161] For example, for XMP 702 in the mdat data block in FIG. 7, the code description is as follows:

[0162] For example, for the extensible (subpart) 703 in the mpvd part in FIG. 7, the code description is as follows, taking 8 extension information entries as an example:

[0163] [

[0164] {"length":437368,"name":"local.hdr.linear.mask","offset":478741,"version":1},

[0165] {"length":144,"name":"local.hdr.meta.data","offset":41373,"version":1},

[0166] {"length":4,"name":"private.emptyspace","offset":478927,"version":1},

[0167] {"length":41209,"name":"watermark","offset":41229,"version":1},

[0168] {"length":72,"name":"watermark.color","offset":478813,"version":1},

[0169] {"length":20,"name":"watermark.config","offset":20,"version":1},

[0170] {"length":47,"name":"watermark.device","offset":478923,"version":1},

[0171] {"length":63,"name":"watermark.params","offset":478876,"version":1}

[0172] ]jxrs-

[0173] S603, when the first picture is a high dynamic range type picture, encapsulating based on the mdat data block, the meta data block and the free data block in the static picture part, and the first video and / or extended information in the video part; wherein the first picture, the gain picture corresponding to the first picture and the extensible metadata platform information are all located in the mdat data block.

[0174] In the embodiment of the present application, it is assumed that the first picture is an HDR type in HEIF / AVIF format, and the first video is an SDR / HDR type in MP4 format, for example. The preset picture format at this time is as shown in FIG. 8. The static picture part includes the mdat data block, the meta data block and the free data block. Then, the video data (i.e. MotionPhoto) is added after the free data block, which can also be represented by “mpvd” and is used to describe the video file and related extended information. The mdat data block includes tiles1, tiles2, gain map (Gainmap), EXIF and XMP, etc.

[0175] That is, in the embodiments of the present application, MotionPhoto video data is added after the end of the main graph file (HEIF / AVIF), and the video file and related extension information are described. In addition, vendors are allowed to add customized BMFF-Box to store private data information. The customized Box needs to comply with the BMFF standard and is stored as a sub-box in the'mpvd' box.

[0176] Exemplarily, for XMP 801 in the mdat data block in FIG. 8, the code description is as follows:

[0177] Exemplarily, for XMP 802 in the mdat data block in FIG. 8, the code description is as follows:

[0178] Exemplarily, for the extensible (subpart) 803 in the mpvd part in FIG. 8, taking 8 extension information entries as an example, the code description is as follows:

[0179] [

[0180] {"length":437368,"name":"local.hdr.linear.mask","offset":478741,"version":1},

[0181] {"length":144,"name":"local.hdr.meta.data","offset":41373,"version":1},

[0182] {"length":4,"name":"private.emptyspace","offset":478927,"version":1},

[0183] {"length":41209,"name":"watermark","offset":41229,"version":1},

[0184] {"length":72,"name":"watermark.color","offset":478813,"version":1},

[0185] {"length":20,"name":"watermark.config","offset":20,"version":1},

[0186] {"length": 47, "name": "watermark.device", "offset": 478923, "version": 1},

[0187] {"length": 63, "name": "watermark.params", "offset": 478876, "version": 1}

[0188] ]jxrs-

[0189] It can also be understood that, after obtaining the dynamic picture file, the dynamic picture file can also be decoded. In some embodiments, the method can further include: decoding the dynamic picture file to determine the first picture and the extensible metadata platform information; decoding the dynamic picture file according to the extensible metadata platform information to determine the first video and / or the extension information.

[0190] In the embodiments of the present application, the dynamic picture file here can be generated by the self device, or can be a dynamic picture file from the opposite device (for example, the second device). Whether the dynamic picture file is generated by the self device or from the second device, the dynamic picture file can be decoded to obtain the first video and / or the extension information.

[0191] In the embodiments of the present application, if the dynamic picture file does not include the extension information, decoding the dynamic picture file according to the extensible metadata platform information can also determine the first video. If the dynamic picture file includes the extension information, decoding the dynamic picture file according to the extensible metadata platform information can determine the first video and the extension information.

[0192] In the embodiments of the present application, the extensible metadata platform information at least includes the offset information of the first video and the offset information of the extension information, at this time, the storage position of the first video in the dynamic picture file can be accurately read according to the offset information of the first video, and the storage position of the extension information in the dynamic picture file can be accurately read according to the offset information of the extension information, so that the first video and the extension information can be accurately read.

[0193] It can also be understood that, in the embodiments of the present application, the application scenario here is not limited to dynamic cars, and can be applied to any field of picture / photograph and video packaging scenarios; in addition, JPEG, HEIF, AVIF and other various picture formats can be applied here; and the embodiments of the present application store the extension information through the extensible dynamic information XMP mode, which is more flexible and has strong extensibility.

[0194] That is, the embodiment of the present application provides a dynamic picture packaging method, specifically proposes a packaged dynamic picture packaging standard, that is, a picture file (i.e. the aforementioned "first picture") and a video file (i.e. the aforementioned "first video") are packaged into a media file and stored in an electronic device. When packaging the file, the extension information (such as video interpolation information, FOV view angle clipping information, etc.) can be added according to the standard, and the application end obtains the extension information by analyzing the dynamic picture file according to the standard when using, so that the dynamic picture has a more extensive and high-quality use experience, thereby improving the universality of the dynamic picture.

[0195] In another embodiment of the present application, based on the dynamic picture packaging method described in the foregoing embodiment, Fig. 9 is a flow diagram of a dynamic picture packaging method provided by the embodiment of the present application. As shown in Fig. 9, the method can include:

[0196] S901, obtaining a dynamic picture file.

[0197] It should be noted that in the embodiment of the present application, the method is applied to a second device. Wherein, the first device and the second device can interact with each other. After the first device generates the dynamic picture file, the dynamic picture file can be transmitted to the second device, and then decoded and processed by the second device. That is, the dynamic picture file obtained here can be sent by the first device, or when the second device has the encoding and packaging function, the dynamic picture file can also be packaged and generated by the second device itself.

[0198] It should also be noted that in the embodiment of the present application, the dynamic picture file obtained here is stored in the gallery in the form of a static picture, and the user can quickly play the video through long pressing and the like. That is, the dynamic picture file can include two parts: a static picture part and a video part, the static picture part is used to store the main picture content, the video part is used to store the video file and the extension information, and the video part is located after the static picture part.

[0199] It should also be noted that in the embodiment of the present application, the dynamic picture file is packaged and generated by the first picture and the first video according to a preset picture format. Specifically, the first picture is stored in the static picture part as the main picture content in the dynamic picture, and the first video is stored in the video part.

[0200] S902, decoding the dynamic picture file to determine the first picture and the extensible metadata platform information.

[0201] S903, decoding the dynamic picture file according to the extensible metadata platform information to determine the first video and / or the extension information.

[0202] In the embodiments of the present application, the dynamic picture file is first decoded to obtain a first picture and extensible metadata platform information, wherein the extensible metadata platform information stores attribute information of the first picture and the first video. Here, the attribute information can include at least one of the following: offset information of the first video, length information of the first video, offset information of the extension information, indication information of whether the dynamic picture file is a dynamic picture, file format version of the dynamic picture file, and presentation timestamp of the video frame corresponding to the first picture, such as shown in Table 1 described above. According to the attribute information in the extensible metadata platform information, the first video and / or the extension information can be accurately decoded.

[0203] It should be noted that the dynamic picture file can include extension information or can not include extension information. Here, if the dynamic picture file does not include extension information, the dynamic picture file is decoded according to the extensible metadata platform information, and the first video can be determined at this time. If the dynamic picture file also includes extension information, the dynamic picture file is decoded according to the extensible metadata platform information, and the first video and the extension information can be determined at this time.

[0204] In some embodiments, when the first video and the extension information are included in the dynamic picture file, decoding the dynamic picture file according to the extensible metadata platform information can include: determining offset information of the first video and offset information of the extension information based on the extensible metadata platform information; decoding the dynamic picture file according to the offset information of the first video to determine the first video; and decoding the dynamic picture file according to the offset information of the extension information to determine the extension information.

[0205] That is, in the embodiments of the present application, the extensible metadata platform information at least includes offset information of the first video and offset information of the extension information, and at this time, according to the offset information of the first video, i.e., the offset starting position of the first video relative to the dynamic picture file, the storage position of the first video in the dynamic picture file can be accurately read, and according to the offset information of the extension information, i.e., the offset starting position of the extension information relative to the dynamic picture file, the storage position of the extension information in the dynamic picture file can be accurately read, so that the first video and the extension information can be accurately read.

[0206] It should also be noted that in the embodiments of the present application, if the video part is located at the end of the dynamic picture file, and the length of the video part is equal to the length of the first video plus the length of the extension information, then the difference between the total length of the dynamic picture file corresponding to the file and the length of the video part can also be used to determine the offset information of the first video, which is not limited here.

[0207] The embodiment of the application provides a dynamic picture packaging method, which decodes a dynamic picture file according to extensible metadata platform information, for example, decodes the dynamic picture file according to offset information of a first video to determine the first video, and decodes the dynamic picture file according to offset information of extension information to determine the extension information, so that the first video and the extension information can be accurately read; that is, a second device (namely, an application end) acquires the extension information by analyzing the dynamic picture file according to the standard when in use, so that the dynamic picture has more extensive and high-quality use experience, and the universality of the dynamic picture is improved.

[0208] In another embodiment of the application, based on the dynamic picture packaging method described in the foregoing embodiment, the embodiment of the application aims to establish a general dynamic picture packaging standard, unify the packaging format of the dynamic picture, and promote the further improvement of the dynamic picture industry ecology. Here, four technologies are mainly involved: an extension information data storage mode, an extension information storage method in a dynamic picture, a picture video packaging storage mode, and a general XMP information standard definition of the dynamic picture.

[0209] In a specific implementation manner, the technical solution can include the following parts:

[0210] (1) Dynamic picture extension information data storage mode.

[0211] The extension information needs to comply with the ISO-BMFF format standard, for example, ISO / IEC 14496-12, and the data storage mode is as shown in Table 2.

[0212] (2) Extension information storage method in a dynamic picture.

[0213] As shown in FIG. 2, the extension information of the dynamic picture should be stored at the end of the dynamic picture (specifically, after the video file), and the "MotionPhoto" length in the main picture XMP information of the dynamic picture should be equal to the length of the original video plus the length of the extension information.

[0214] (3) Dynamic picture packaging standard definition (namely, picture video packaging storage mode).

[0215] According to mainstream picture formats, the standard mainly describes the following packaging schemes:

[0216] (a) JPEG (SDR) + MP4 (SDR / HDR): The picture is of the SDR type in the JPEG format, and the video is of the SDR / HDR type in the MP4 format.

[0217] As shown in FIG. 4, MotionPhoto video data is added after the EOI of JPEG image data, for describing video file and related extension information. In addition, custom BMFF-Box is allowed to be added by each vendor to store private data information. The custom Box needs to comply with BMFF standard, as a sub-box in'mpvd' box.

[0218] (b) JPEG (HDR) + MP4 (SDR / HDR): Picture is HDR type in JPEG format, video is SDR / HDR type in MP4 format.

[0219] As shown in FIG. 5, MotionPhoto field is added in the main image XMP file based on Google Ultra-HDR format, for describing video file and related extension information. Video data is added after the Gainmap image, allowing each vendor to customize Box to store private information at the end of the file. The custom Box needs to comply with BMFF standard, as a sub-box in'mpvd' box.

[0220] (c) HEIF / AVIF (SDR) + MP4 (SDR / HDR): Picture is SDR type in HEIF / AVIF format, video is SDR / HDR type in MP4 format.

[0221] As shown in FIG. 7, MotionPhoto video data is added after the end of the main image file (HEIF / AVIF), for describing video file and related extension information. In addition, custom BMFF-Box is allowed to be added by each vendor to store private data information. The custom Box needs to comply with BMFF standard, as a sub-box in'mpvd' box.

[0222] (d) HEIF / AVIF (HDR) + MP4 (SDR / HDR): Picture is HDR type in HEIF / AVIF format, video is SDR / HDR type in MP4 format.

[0223] As shown in FIG. 8, MotionPhoto video data is added after the end of the main image file (HEIF / AVIF), for describing video file and related extension information. In addition, custom BMFF-Box is allowed to be added by each vendor to store private data information. The custom Box needs to comply with BMFF standard, as a sub-box in'mpvd' box.

[0224] (4) Dynamic picture general XMP information standard definition.

[0225] The dynamic picture general XMP information is shown in Table 1 as described above. The camera metadata encodes information about how to present the main picture part and the video part in the dynamic picture. The namespace URI can be found at http: / / ns.google.com / photos / 1.0 / camera / ; the default namespace prefix is GCamera. The attribute information shown in Table 1 can be displayed in the XMP metadata of the static picture.

[0226] Briefly, the core modules of the embodiments of the present application are divided according to the main functions, and mainly include: an extended information data storage mode, an extended information storage method in a dynamic picture, a picture video packaging storage mode, and a dynamic picture general XMP information standard definition.

[0227] Here, the functions of the modules are described as follows:

[0228] Extended information data storage mode: the dynamic picture extended information (such as video interpolation information, FOV view cutting information, etc.) is saved in the form of key-value pairs, and if the information is more, a multi-layer key-value pair mode can be used for storage.

[0229] Extended information storage method in a dynamic picture: the extended information is encapsulated into a BMFF-Box according to the ISO-BMFF format standard, and stored in the file tail. The length of the video plus the length of the extended information is used as the "MotionPhoto" length in the XMP information.

[0230] Picture video packaging storage mode: the picture and the video are packaged into a media file according to the present specification and stored in the device.

[0231] Dynamic picture general XMP information standard definition: the metadata information about how to present the main picture part and the video part in the dynamic picture is encoded.

[0232] It should be noted that, for the encapsulation of the dynamic picture, in addition to the video interpolation information and the FOV view cutting information mentioned above, the technical solution can theoretically be extended to any information, such as video super-resolution, post-processing information, etc. In addition, in addition to the MP4 video format mentioned in the present text, other video file formats such as MOV can also be extended, which is not limited here.

[0233] The embodiment of the present application provides a packaged dynamic picture packaging standard. The specific implementation of the foregoing embodiment is described in detail through the foregoing embodiment. It can be seen that the technical scheme of the foregoing embodiment mainly describes how to perform packaged dynamic picture packaging, and unifies the packaging mode of extension information. Therefore, the dynamic picture can be correctly recognized when sharing end to end. When the video and the extension information are simultaneously included in the dynamic picture file, the offset information of the video and the offset information of the extension information are included in the XMP information, so that the video and the extension information can be accurately read. In addition, the three-party application can correctly analyze the dynamic picture packaged according to the standard, ensures the consistency of effects of various manufacturers, promotes the further improvement of the dynamic picture industry ecology, and improves the universality of the dynamic picture.

[0234] In another embodiment of the present application, based on the same inventive concept of the foregoing embodiment, FIG. 10 is a structural schematic diagram of a dynamic picture packaging device provided by the embodiment of the present application. As shown in FIG. 10, the dynamic picture packaging device 100 applied to a first device can include an acquisition unit 1001, an encoding unit 1002 and a packaging unit 1003, wherein:

[0235] The acquisition unit 1001 is configured to acquire the first picture and the first video.

[0236] The encoding unit 1002 is configured to encode the attribute information of the first picture and the first video, and generate extensible metadata platform information in a preset picture format.

[0237] The packaging unit 1003 is configured to package the first picture and the first video based on the preset picture format, and generate a dynamic picture file.

[0238] In some embodiments, the preset picture format includes a static picture part and a video part, and the video part is located after the static picture part. Correspondingly, the packaging unit 1003 is further configured to store the first picture in the static picture part, and store the first video in the video part.

[0239] In some embodiments, the video part includes a video data subpart and an extensible subpart. The packaging unit 1003 is further configured to store the first video in the video data subpart, and store the extension information in the extensible subpart after acquiring the extension information of the dynamic picture file.

[0240] In some embodiments, the attribute information includes at least one of the following: offset information of the first video, length information of the first video, offset information of the extension information, indication information of whether the dynamic picture file is a dynamic picture, file format version of the dynamic picture file, and presentation timestamp of a video frame corresponding to the first picture.

[0241] In some embodiments, the picture format of the first picture comprises at least a JPEG format and a HEIF / AVIF format; and the video format of the first video comprises at least an MP4 format and a MOV format.

[0242] In some embodiments, when the first picture is in the JPEG format, the encapsulation unit 1003 is further configured to, when the first picture is a standard dynamic range type picture, encapsulate based on a picture start bit, extensible metadata platform information, the first picture and a picture end bit in the static picture part, and the first video and / or the extension information in the video part; or, when the first picture is a high dynamic range type picture, encapsulate based on a main picture part and a gain picture part in the static picture part, and the first video and / or the extension information in the video part; wherein the main picture part comprises a main picture start bit, the extensible metadata platform information, the first picture and a main picture end bit, and the gain picture part comprises a gain picture start bit, private extensible metadata platform information, a gain picture of the first picture and a gain picture end bit.

[0243] In some embodiments, when the first picture is in the HEIF / AVIF format, the encapsulation unit 1003 is further configured to, when the first picture is a standard dynamic range type picture, encapsulate based on an mdat data block, a meta data block and a free data block in the static picture part, and the first video and / or the extension information in the video part, wherein the first picture and the extensible metadata platform information are both located in the mdat data block; or, when the first picture is a high dynamic range type picture, encapsulate based on the mdat data block, the meta data block and the free data block in the static picture part, and the first video and / or the extension information in the video part, wherein the first picture, the gain picture of the first picture and the extensible metadata platform information are all located in the mdat data block.

[0244] In some embodiments, referring to FIG. 10, the dynamic picture encapsulation apparatus 100 can further comprise a decoding unit 1004 configured to, after obtaining the dynamic picture file, decode the dynamic picture file to determine the first picture and the extensible metadata platform information; and decode the dynamic picture file according to the extensible metadata platform information to determine the first video and / or the extension information.

[0245] In another embodiment of the present application, based on the same inventive concept as the foregoing embodiments, FIG. 11 is a schematic structural diagram of a dynamic picture encapsulation apparatus provided by an embodiment of the present application. As shown in FIG. 11, applied to a second device, the dynamic picture encapsulation apparatus 110 can comprise an obtaining unit 1101 and a decoding unit 1102, wherein:

[0246] The obtaining unit 1101 is configured to obtain a dynamic picture file.

[0247] The decoding unit 1102 is configured to decode the dynamic picture file to determine the first picture and the extensible metadata platform information, and decode the dynamic picture file according to the extensible metadata platform information to determine the first video and / or the extension information.

[0248] In some embodiments, when the first video and the extension information are included in the dynamic picture file, the decoding unit 1102 is further configured to determine offset information of the first video and offset information of the extension information based on the extensible metadata platform information, and decode the dynamic picture file according to the offset information of the first video to determine the first video, and decode the dynamic picture file according to the offset information of the extension information to determine the extension information.

[0249] In the embodiments of the present application, the "unit" can be part of a circuit, part of a processor, part of a program or software, etc., and of course can be a module, and can also be non-modular. Moreover, the components in the embodiments can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware, or in the form of a software functional module.

[0250] In addition, those skilled in the art should understand that the above description of the dynamic picture packaging apparatus of the embodiments of the present application can be understood with reference to the description of the dynamic picture packaging method of the embodiments of the present application.

[0251] In still another embodiment of the present application, based on the same inventive concept as the foregoing embodiments, FIG. 12 is a specific hardware structure schematic diagram of an electronic device provided by the embodiments of the present application. As shown in FIG. 12, the electronic device 120 can include a communication interface 1201, a memory 1202 and a processor 1203; each component is coupled together through a bus system 1204. It can be understood that the bus system 1204 is used to realize the connection communication between the components. The bus system 1204 includes a data bus, a power bus, a control bus and a status signal bus. However, for the purpose of clear illustration, all kinds of buses are marked as the bus system 1204 in FIG. 9. Among them, the communication interface 1201 is used for receiving and sending signals in the process of transceiving information with other external network elements; the memory 1202 is used to store computer programs capable of running on the processor 1203.

[0252] In some embodiments, when the electronic device 120 is the first device, the processor 1203 is configured to, when the computer program is running, perform: obtaining the first picture and the first video; encoding attribute information of the first picture and the first video to generate extensible metadata platform information in a preset picture format; and packaging the first picture and the first video based on the preset picture format to generate a dynamic picture file.

[0253] In some embodiments, when the electronic device 120 is the second device, the processor 1203 is configured to, when the computer program is running, perform: obtaining the dynamic picture file; decoding the dynamic picture file to determine the first picture and the extensible metadata platform information; and decoding the dynamic picture file according to the extensible metadata platform information to determine the first video and / or the extended information.

[0254] It can be understood that the memory 1202 in the embodiments of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (Read-Only Memory, ROM), a programmable read-only memory (Programmable ROM, PROM), an erasable programmable read-only memory (Erasable PROM, EPROM), an electrically erasable programmable read-only memory (Electrically EPROM, EEPROM) or a flash memory. The volatile memory can be a random access memory (Random Access Memory, RAM) used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (Static RAM, SRAM), dynamic random access memory (Dynamic RAM, DRAM), synchronous dynamic random access memory (Synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (Double Data Rate SDRAM, DDR SDRAM), enhanced synchronous dynamic random access memory (Enhanced SDRAM, ESDRAM), synchronous link dynamic random access memory (Synchronous link DRAM, SLDRAM) and direct memory bus random access memory (Direct Rambus RAM, DRRAM). The memory 1202 of the system and method described herein is intended to include, but not limited to, these and any other suitable types of memory.

[0255] The processor 1203 can be an integrated circuit chip logic circuit, or an instruction in a form of software. In implementation process, each step of the above method can be completed by the integrated logic circuit or the instruction in the form of software in the processor 1203. The processor 1203 described above can be a general processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component. Each method, step and logic block in the embodiments of the present application can be realized or executed. The general processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as a hardware code processor to execute, or be executed by a combination of hardware and software modules in the code processor. The software module can be located in a random access memory, a flash memory, a read only memory, a programmable read only memory or an electrically erasable programmable memory, a register, or other mature storage medium in the art. The storage medium is located in the storage 1202, and the processor 1203 reads the information in the storage 1202 and combines the hardware to complete the steps of the above method.

[0256] It can be understood that the embodiments described herein can be realized in hardware, software, firmware, middleware, microcode or a combination thereof. For hardware implementation, the processing unit can be implemented in one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), general purpose processors, controllers, micro-controllers, microprocessors, other electronic units designed to perform the functions described in the embodiments of the present application, or a combination thereof.

[0257] For software implementation, the techniques described herein can be implemented with a module (for example, procedures, functions, and so on) that performs the functions described herein. The software code can be stored in a memory and executed by a processor. The memory can be implemented in the processor or outside the processor.

[0258] Optionally, as another embodiment, the processor 1203 is further configured to execute the steps of the method according to any one of the preceding embodiments when the computer program is run.

[0259] In some embodiments, the electronic device according to the embodiments of the present application can comprise the dynamic picture packaging apparatus 100 according to any one of the preceding embodiments, or can comprise the dynamic picture packaging apparatus 110 according to any one of the preceding embodiments.

[0260] The embodiments of the present application further provide a computer readable storage medium for storing a computer program.

[0261] In some embodiments, the computer readable storage medium can be applied to the electronic device according to the embodiments of the present application, and the computer program is executed by at least one processor to realize the corresponding processes implemented by the electronic device in the various methods according to the embodiments of the present application, which will not be repeated here for brevity.

[0262] The embodiments of the present application further provide a computer program product comprising a computer program or instructions.

[0263] In some embodiments, the computer program product can be applied to the electronic device according to the embodiments of the present application, and the computer program or instructions enable the computer to execute the corresponding processes implemented by the electronic device in the various methods according to the embodiments of the present application, which will not be repeated here for brevity.

[0264] The embodiments of the present application further provide a computer program.

[0265] In some embodiments, the computer program can be applied to the electronic device according to the embodiments of the present application, and when the computer program is run on the computer, the computer is enabled to execute the corresponding processes implemented by the electronic device in the various methods according to the embodiments of the present application, which will not be repeated here for brevity.

[0266] Those skilled in the art can clearly understand that the units and algorithm steps of the examples described in combination with the embodiments disclosed in the present application can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0267] Those skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working processes of the above-described apparatus and units can refer to the corresponding processes in the preceding method embodiments, which will not be repeated here.

[0268] In several embodiments provided in the present application, it should be understood that the disclosed apparatus and method can be implemented by other manners. For example, the apparatus embodiments described above are merely illustrative, for example, the division of the units is merely a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units or components shown or discussed can be indirect coupling or communication connection through some interfaces, apparatuses or units, and can be electrical, mechanical or other forms.

[0269] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e., can be located in one place or can be distributed to a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0270] In addition, the functional units in each embodiment of the present application can be integrated into one processing unit, or each unit can exist physically, or two or more units can be integrated into one unit. If the functions are realized in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the parts that make contributions to the prior art or parts of the technical solutions can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for making a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in the embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (Read-Only Memory, ROM), a random access memory (Random Access Memory, RAM), a magnetic disk or an optical disk, and various program code storage media.

[0271] It should be noted that in the present application, the terms "include", "contain" or any other variants thereof are intended to cover non-exclusive inclusion, so that the processes, methods, articles or apparatuses including a series of elements not only include those elements, but also include other elements not explicitly listed or inherent to such processes, methods, articles or apparatuses. Without more limitations, the element defined by the statement "including a" does not exclude the presence of another identical element in the process, method, article or apparatus including the element.

[0272] The serial numbers of the embodiments of the present application are only for description, and do not represent the advantages or disadvantages of the embodiments.

[0273] The methods disclosed in the several method embodiments of the present application can be combined arbitrarily to obtain new method embodiments, without conflict.

[0274] The features disclosed in the several product embodiments of the present application can be combined arbitrarily to obtain new product embodiments, without conflict.

[0275] The features disclosed in the several method or device embodiments of the present application can be combined arbitrarily to obtain new method or device embodiments, without conflict.

[0276] The above is merely a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims. Industrial applicability

[0277] In the embodiments of the present application, first, a first picture and a first video are acquired, and then attribute information of the first picture and the first video is encoded to generate extensible metadata platform information in a preset picture format; and then the first picture and the first video are encapsulated based on the preset picture format to generate a dynamic picture file. Alternatively, after the dynamic picture file is acquired, the dynamic picture file is decoded to determine the first picture and the extensible metadata platform information; and the dynamic picture file is decoded according to the extensible metadata platform information to determine the first video and / or extension information. In this way, after the extensible metadata platform information in the preset picture format is generated according to the attribute information of the first picture and the first video, the dynamic picture is encapsulated and packaged based on the preset picture format, and since a unified dynamic picture encapsulation format is adopted, it can be ensured that the dynamic picture can be correctly recognized and parsed when sharing end to end, and the offset information of the first video is included in the extensible metadata platform information to accurately read the first video; in this way, the presentation effects of various manufacturers are consistent, the further improvement of the dynamic picture industry ecology is promoted, and the universality of the dynamic picture is improved.

Claims

A dynamic picture packaging method, the method comprising: acquiring a first picture and a first video; encoding attribute information of the first picture and the first video to generate extensible metadata platform information in a preset picture format; packaging the first picture and the first video based on the preset picture format to generate a dynamic picture file. The method of claim 1, wherein The preset picture format comprises a static picture part and a video part, and the video part is located after the static picture part; The packaging of the first picture and the first video based on the preset picture format comprises: storing the first picture in the static picture part; storing the first video in the video part. The method of claim 2, wherein, The video part comprises a video data subpart and an extensible subpart; the method further comprises: storing the first video in the video data subpart; after acquiring extension information of the dynamic picture file, storing the extension information in the extensible subpart. The method of claim 2, wherein The preset picture format further comprises an extensible part, and the extensible part is independent of the video part; The method further comprises: after acquiring extension information of the dynamic picture file, storing the extension information in the extensible part. The method according to claim 3 or 4, wherein The method further comprises: storing data content in the extension information in the form of key-value pairs; or storing data content in the extension information in the form of multi-layer key-value pairs. The method according to claim 3 or 4, wherein The attribute information comprises at least one of the following: offset information of the first video, length information of the first video, offset information of the extension information, indication information of whether the dynamic picture file is a dynamic picture, file format version of the dynamic picture file, and presentation timestamp of a video frame corresponding to the first picture. The method according to claim 3 or 4, wherein The picture format of the first picture comprises at least one of the following: JPEG format, HEIF / AVIF format; The video format of the first video comprises at least one of the following: MP4 format, MOV format. The method of claim 7, wherein, In the case where the first picture is in JPEG format, the packaging of the first picture and the first video based on the preset picture format comprises: In the case where the first picture is a standard dynamic range type picture, packaging based on a picture start bit in the static picture part, the extensible metadata platform information, the first picture, and a picture end bit, and the first video and / or the extension information in the video part; or In the case where the first picture is a high dynamic range type picture, packaging based on a main picture part and a gain picture part in the static picture part, and the first video and / or the extension information in the video part; wherein the main picture part comprises a main picture start bit, the extensible metadata platform information, the first picture, and a main picture end bit, and the gain picture part comprises a gain picture start bit, private extensible metadata platform information, a gain picture of the first picture, and a gain picture end bit. The method of claim 7, wherein, In a case that the first picture is in a HEIF / AVIF format, the encapsulating the first picture and the first video based on the preset picture format comprises: In a case that the first picture is a standard dynamic range type picture, the encapsulating is based on mdat data blocks, meta data blocks and free data blocks in the static picture part, and the first video and / or the extension information in the video part, wherein the first picture and the extensible metadata platform information are both located in the mdat data blocks; or, In a case that the first picture is a high dynamic range type picture, the encapsulating is based on mdat data blocks, meta data blocks and free data blocks in the static picture part, and the first video and / or the extension information in the video part, wherein the first picture, a gain map of the first picture and the extensible metadata platform information are all located in the mdat data blocks. The method of any one of claims 3 to 9, wherein, After the dynamic picture file is acquired, the method further comprises: decoding the dynamic picture file to determine the first picture and the extensible metadata platform information; decoding the dynamic picture file according to the extensible metadata platform information to determine the first video and / or the extension information. The method of claim 10, wherein, The extensible metadata platform information at least comprises offset information of the first video and offset information of the extension information, wherein the offset information of the first video is used to determine a storage position of the first video in the dynamic picture file, and the offset information of the extension information is used to determine a storage position of the extension information in the dynamic picture file. The method of claim 11, wherein, In a case that the dynamic picture file comprises the first video and the extension information, the decoding the dynamic picture file according to the extensible metadata platform information comprises: decoding the dynamic picture file according to the offset information of the first video to determine the first video; decoding the dynamic picture file according to the offset information of the extension information to determine the extension information. A dynamic picture encapsulating method, the method comprising: acquiring a dynamic picture file; decoding the dynamic picture file to determine a first picture and extensible metadata platform information; decoding the dynamic picture file according to the extensible metadata platform information to determine a first video and / or extension information. The method of claim 13, wherein, The dynamic picture file comprises a static picture part and a video part, and the video part is located after the static picture part; wherein the static picture part is used to store the first picture, and the video part is used to store the first video and the extension information. The method according to claim 13 or 14, wherein In a case that the dynamic picture file comprises the first video and the extension information, the decoding the dynamic picture file according to the extensible metadata platform information comprises: determining offset information of the first video and offset information of the extension information based on the extensible metadata platform information; decoding the dynamic picture file according to the offset information of the first video to determine the first video; decoding the dynamic picture file according to the offset information of the extension information to determine the extension information. According to offset information of the extension information, the dynamic picture file is decoded to determine the extension information. A dynamic picture packaging apparatus, comprising an acquisition unit, an encoding unit and a packaging unit, wherein: The acquisition unit is configured to acquire the first picture and the first video; The encoding unit is configured to encode attribute information of the first picture and the first video to generate extensible metadata platform information in a preset picture format; The packaging unit is configured to package the first picture and the first video based on the preset picture format to generate a dynamic picture file. A dynamic picture packaging apparatus, comprising an acquisition unit and a decoding unit, wherein: The acquisition unit is configured to acquire a dynamic picture file; The decoding unit is configured to decode the dynamic picture file to determine a first picture and extensible metadata platform information; and decode the dynamic picture file according to the extensible metadata platform information to determine a first video and / or extension information. An electronic device, comprising a memory and a processor, wherein: The memory is configured to store a computer program capable of running on the processor; The processor is configured to execute the method according to any one of claims 1 to 12, or the method according to any one of claims 13 to 15 when the computer program is run. A computer-readable storage medium having stored thereon a computer program, wherein, The computer program is executed by the processor to implement the method according to any one of claims 1 to 12, or the method according to any one of claims 13 to 15. A computer program product comprising computer programs or instructions, wherein, The computer program or instructions are executed by the processor to implement the method according to any one of claims 1 to 12, or the method according to any one of claims 13 to 15.

Images