MP4-based data processing method

Abstract

The invention relates to a data processing method based on MP4. The method comprises the following steps: converting an MP4 format file into an fMP4 format file; the converted file in the fMP4 format is converted into a file in the fakeMP4 format; and transmitting the file in the fakeMP4 format to a playing end, and playing the file in the fakeMP4 format. The technical problem that the first screen is slow can be solved, the data format is reversibly converted and easy to explain, and wide equipment compatibility is achieved.

Description

Technical Field

[0001] This invention belongs to the field of audio and video data processing, and specifically relates to a data processing method based on MP4. Background Technology

[0002] The advantage of the MP4 format lies in its wide playback compatibility, such as support for precise dragging and various encoding formats. However, it suffers from a slow initial playback speed. The audio and video frame index data of an MP4 file is stored in a moov box data block. The general process for a player to play an MP4 video is as follows: It retrieves all audio and video frame index data from the moov box data block at the beginning of the file; it finds the file offset position of the next audio and video frame data to be decoded based on the index data; and it reads the corresponding audio and video frame data for decoding and rendering.

[0003] For a movie, the moov box data block is typically several megabytes in size. This means that the player needs to download these megabytes of index data before it can start rendering the video, which slows down the first screen playback of MP4 videos. Summary of the Invention

[0004] This invention provides a data processing method based on MP4, which aims to solve at least one of the technical problems existing in the prior art.

[0005] The technical solution of the present invention relates to an MP4-based data processing method, which includes the following steps: S100. Convert MP4 format files to fMP4 format files; S200: Convert the converted fMP4 format file to a fakeMP4 format file; S300: Transfer the fakeMP4 format file to the playback device and play it.

[0006] Furthermore, step S100 includes: S110. Create a new fMP4 format file with empty content, and obtain audio and video data from the MP4 format file. The audio and video data includes at least audio and video compatibility information, audio and video media metadata, audio and video frame index data, and audio and video frame data. S120. Create an ftyp box data block in the fMP4 format file to store audio and video compatibility information, and write the audio and video compatibility information into the ftyp box data block of the fMP4 format file; S130. Create a moov box data block in the fMP4 format file to store audio and video media metadata, and write the audio and video media metadata into the moov box data block of the fMP4 format file. S140. The audio and video frame index data and audio and video frame data are split into multiple audio and video segments according to time logic. Each audio and video segment stores part of the audio and video frame index data and part of the audio and video frame data in the order of segment timestamp. S150. Create a sidx box data block to store audio and video segment index data, and write the audio and video segment index data into the sidx box data block of the fMP4 format file; S160. Write the audio and video segments into an fMP4 format file in the order of segment timestamps.

[0007] Furthermore, in step S130, the audio and video media metadata includes the resolution and encoder type of the audio and video media; In step S140, the audio and video frame index data includes the audio and video frame timestamp, the audio and video frame size, and the data index address of the audio and video frame in the fMP4 format file; In step S150, the sidx box data block includes a first sidx box data block for locating video segments containing video frame data and a second sidx box data block for locating audio segments containing audio frame data. The audio and video segment index data includes at least the audio and video segment timestamp, the audio and video segment size, and the data index address of the audio and video segment in the file.

[0008] Further, in step S140, the audio / video frame index data and audio / video frame data are split into several audio / video segments using temporal logic. Each audio / video segment includes: Create a moof box data block for storing the audio and video frame index information of the audio and video segments, and write the audio and video frame index information into the moof box data block of the fMP4 format file; Create an mdat box data block for storing the audio and video frame data of the audio and video segment, and write the audio and video frame data into the mdat box data block of the fMP4 format file.

[0009] Further, step S200 includes: S210. Create a new fakeMP4 format file with empty content, and obtain audio and video data from the fakeMP4 format file. The audio and video data includes audio and video compatibility information, audio and video media metadata, audio and video segment index data, audio and video frame index data, and audio and video frame data. S220. Create an ftyp box data block in the fakeMP4 format file to store audio and video compatibility information and an mdat box data block to store the entire content of the fMP4 file. Write the audio and video compatibility information to the ftyp box data block of the fakeMP4 format file and write the entire content of the fMP4 file to the mdat box data block of the fakeMP4 format file. S230. Create a moov box data block in the fakeMP4 format file to store the audio and video media metadata and the frame index information corresponding to the audio and video frame data, and write it to the fakeMP4 format file. The frame index information includes the timestamp of the audio and video frame, the size of the audio and video frame, and the data index address of the audio and video frame in the fakeMP4 format file. In steps S220 and S230, the ftyp box data block in the fakeMP4 format file is set at the beginning of the fakeMP4 format file, and the moov box data block in the fakeMP4 format file is set before or after the mdatbox data block in the fakeMP4 format file.

[0010] Furthermore, for players that support fakeMP4 format files, when a fakeMP4 format file is detected, the audio and video will be played as an fMP4 format file. For players that do not support fakeMP4 format files, fakeMP4 format files will be treated as ordinary MP4 format and played as MP4 format videos.

[0011] Furthermore, for players that support fakeMP4, step S300 includes the following steps: S310. Identify fakeMP4 format files and determine the content range of the fakeMP4 file; S320. Obtain all audio and video segment index data from the fMP4 content data block of the fakeMP4 format file; S330. Find the offset position of the required audio and video segment based on the audio and video segment index data; S340. Read the moof box data block of the audio and video segment at the corresponding position and obtain the index data of all audio and video frames in the audio and video segment; S350: Based on the audio and video frame index data, find the file offset position of the next audio and video frame data that needs to be decoded, and read the audio and video frame data at the corresponding position for decoding and rendering; S360. Repeat steps S330 to S350 until playback ends.

[0012] Furthermore, for players that do not support fakeMP4, step S300 includes the following steps: S310. Obtain all audio and video frame index data from the moov box data block at the end of the fakeMP4 format file; S320. Find the file offset position of the next audio / video frame data to be decoded based on the audio / video frame index data. S330: Read the audio and video frame data at the corresponding position, and perform decoding and rendering; S340. Repeat steps S320 to S330 until playback ends.

[0013] Furthermore, in step S300, the MP4-based data processing method further includes the following steps: S400. If you need to reverse convert the received fakeMP4 format file to an MP4 format file or an fMP4 format file, perform the following conversion: Converting a received fakeMP4 file to an MP4 file involves the following steps: Create a new MP4 file with empty content, and read audio and video data from a fakeMP4 file. The audio and video data includes audio and video compatibility information, audio and video media metadata, audio and video frame index data, and audio and video frame data. Create an ftyp box data block in the MP4 file to store the audio and video compatibility information, and write the audio and video compatibility information into the MP4 file; Create a moov box data block in the MP4 format file to store the audio and video media metadata and audio and video frame index data, and write the audio and video media metadata and audio and video frame index data into the MP4 format file; Create an mdat box data block in the MP4 format file to store the audio and video frame data, and write the audio and video frame data into the MP4 format file; The MP4 file containing the data is obtained and sent to the playback device for playback. The process of converting a received fakeMP4 file to an fMP4 file includes the following steps: Create a new empty fMP4 file, read the fMP4 file content from the mdat box data block in the fakeMP4 file, and write it to the fMP4 file; The fMP4 format file containing the data is obtained and sent to the playback device for playback.

[0014] Furthermore, the present invention also proposes a computer-readable storage medium storing program instructions thereon, which, when executed by a processor, implement the MP4-based data processing method described above.

[0015] Compared with existing technologies, the present invention has the following characteristics: This invention provides a data processing method based on MP4, which stores fMP4 content in the MP4's mdat box while inheriting the playback compatibility of MP4, thus solving the problem of slow first screen display. Its data format is reversibly convertible and easy to interpret, and it has broad device compatibility. Attached Figure Description

[0016] Figure 1 This is a flowchart of an MP4-based data processing method.

[0017] Figure 2 This is a schematic diagram of the existing MP4 format.

[0018] Figure 3 This is a schematic diagram of the existing fMP4 format.

[0019] Figure 4 This is a schematic diagram of the existing fMP4 format (including the sidx box).

[0020] Figure 5 This is a diagram of the fakeMP4 format.

[0021] Figure 6 This is a diagram illustrating the process of converting an MP4 file to a fakeMP4 file.

[0022] Figure 7 A diagram illustrating the splitting of audio and video index data and frame data in an MP4 file (based on an MP4 file).

[0023] Figure 8 A diagram illustrating the splitting of audio and video index data and frame data in an MP4 file (based on an fMP4 file).

[0024] Figure 9 This is a diagram illustrating the process of converting a fakeMP4 file to an fMP4 file.

[0025] Figure 10 This is a diagram illustrating the process of converting a fakeMP4 file to an MP4 file. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] The following will provide a clear and complete description of the concept, specific structure, and technical effects of the present invention in conjunction with the embodiments and accompanying drawings, so as to fully understand the purpose, solution, and effects of the present invention.

[0028] It should be noted that, unless otherwise specified, when a feature is referred to as "fixed" or "connected" to another feature, it can be directly fixed or connected to the other feature, or indirectly fixed or connected to the other feature. The singular forms "a," "described," and "the" used herein are also intended to include the plural forms, unless the context clearly indicates otherwise. Furthermore, unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in this specification is for the purpose of describing particular embodiments only and not for limiting the invention. The term "and / or" as used herein includes any combination of one or more of the associated listed items.

[0029] It should be understood that although the terms first, second, third, etc., may be used in this disclosure to describe various elements, these elements should not be limited to these terms. These terms are only used to distinguish elements of the same type from one another. For example, a first element may also be referred to as a second element without departing from the scope of this disclosure, and similarly, a second element may also be referred to as a first element. Any and all instances or exemplary language (“e.g.,” “such as,” etc.) provided herein are intended only to better illustrate embodiments of the invention and, unless otherwise required, do not impose a limitation on the scope of the invention.

[0030] Reference Figures 1 to 10 This invention provides an MP4-based data processing method, which includes the following steps: S100. Convert MP4 format files to fMP4 format files; S200: Convert the converted fMP4 format file to a fakeMP4 format file; S300: Transfer the fakeMP4 format file to the playback device and play it.

[0031] Compared with existing technologies, the present invention has the following characteristics: This MP4-based data processing method, while maintaining MP4's playback compatibility, stores fMP4 content in the MP4's mdat box, solving the problem of slow initial screen loading. Its data format is reversibly convertible and easy to interpret, with broad device compatibility.

[0032] Video container formats, also known as container formats, are file formats used to store multiple media streams (such as video, audio, subtitles, and metadata) in a single file. They can package video streams with audio streams and other related data together for playback as a synchronized multimedia experience. Common video container formats include MP4 (MPEG-4 Part 14), TS (MPEG transport stream), and Matroska.

[0033] Video frames constitute the basic static image units of a video. A video frame can be simply considered as a static picture. By playing these frames quickly and continuously, the persistence of vision of the human eye is used to generate continuous dynamic images, thus forming the video we see.

[0034] The first screen refers to the first frame a user sees when playing a video. First screen time typically refers to the time interval between when a user clicks to start playing the video and when they see the first frame. Generally, a shorter first screen time results in a better user experience.

[0035] ISOBMFF, or ISO Base Media File Format, is a container file format that defines a common structure for storing time-based media data, such as video and audio. It is standardized in ISO / IEC 14496-12, also known as MPEG-4 Part 12. Designed as a flexible and scalable format, it facilitates the exchange, management, editing, and presentation of media. This format has been widely used for media file storage and serves as the basis for various other media file formats, such as MP4 and 3GP container formats.

[0036] In terms of existing technology, refer to Figure 2 MP4 is a widely used standard video container format in the industry, based on the ISOBMFF file format. The MP4 format is composed of multiple box structural units, and each box consists of the following three parts: 1) size: box size, occupying 4 bytes; 2) type: box type, represented by 4 characters, for example "ftyp", occupying 4 bytes; 3) data: The main data content stored in the box. Different types of boxes are used to store different data content.

[0037] Looking at the outermost layer of an MP4 file, it mainly consists of three boxes: ftyp (File Type), moov (Movie), and mdat (Media Data). The ftyp box is located at the very beginning of the file, while the order of the latter two is not important. The moov box can be placed either before or after the mdat box.

[0038] 1) ftyp box: Stores MP4 format version compatibility information, typically occupying tens of bytes; 2) moov box: Stores all audio and video frame index data. This index data can be used to find the offset position of a specific audio or video frame in the file based on the playback time point. The index data generally occupies several megabytes in size. 3) mdat box: Stores all audio and video frame data, which is the main data of the file.

[0039] Reference Figure 3 fMP4 (fragmented MP4) is another video container format based on the ISOBMFF file format. It can be simply regarded as an MP4 format that logically partitions audio and video data.

[0040] fMP4 divides the MP4 mdat box into numerous distinct mdat boxes. Each mdat box stores only a portion of the audio / video frame data, typically 1-5 seconds of audio / video data. The audio / video frame index data in fMP4 is not stored in the moov box, but rather in the moof box (Movie Fragment). Each moof box corresponds to one mdat box, and each moof / mdat box pair forms a segment.

[0041] Reference Figure 4Each segment's moof box can only index audio and video frame data within the same segment's mdat box. If you need to find the file offset of a specific frame of audio or video data based on playback time, you first need to find the corresponding segment based on the playback time, and then find the file offset of the audio and video frame data based on the segment's moof box. fMP4 uses sidx boxes (Segment Indexes) to store the index data of all these segments, used to find the corresponding segment based on playback time. For single-track video files, there are generally two sidx boxes: one for locating the segment containing video frame data, and the other for locating the segment containing audio frame data.

[0042] Therefore, compared to existing technologies like MP4 and fMP4, since the player primarily locates audio and video frame data using the MP4's audio and video frame index data, and the MP4 format does not restrict the mdat box to only store audio and video frame data, nor does it limit the position of audio and video frame data within the mdat box, this can be leveraged to merge MP4 and fMP4 into a fakeMP4 format. FakeMP4 can be simply viewed as an MP4 format that uses the mdat box to store fMP4 content.

[0043] Furthermore, step S100 includes: S110. Create a new fMP4 format file with empty content, and obtain audio and video data from the MP4 format file. The audio and video data includes at least audio and video compatibility information, audio and video media metadata, audio and video frame index data, and audio and video frame data. S120. Create an ftyp box data block in the fMP4 format file to store audio and video compatibility information, and write the audio and video compatibility information into the ftyp box data block of the fMP4 format file; S130. Create a moov box data block in the fMP4 format file to store audio and video media metadata, and write the audio and video media metadata into the moov box data block of the fMP4 format file. S140. The audio and video frame index data and audio and video frame data are split into multiple audio and video segments according to time logic. Each audio and video segment stores part of the audio and video frame index data and part of the audio and video frame data in the order of segment timestamp. S150. Create a sidx box data block to store audio and video segment index data, and write the audio and video segment index data into the sidx box data block of the fMP4 format file; S160. Write the audio and video segments into an fMP4 format file in the order of segment timestamps.

[0044] Furthermore, in step S130, the audio and video media metadata includes the resolution and encoder type of the audio and video media; In step S140, the audio and video frame index data includes the audio and video frame timestamp, the audio and video frame size, and the data index address of the audio and video frame in the fMP4 format file; In step S150, the sidx box data block includes a first sidx box data block for locating video segments containing video frame data and a second sidx box data block for locating audio segments containing audio frame data. The audio and video segment index data includes at least the audio and video segment timestamp, the audio and video segment size, and the data index address of the audio and video segment in the file.

[0045] Specifically, the sidx box data block stores the index data of audio and video segments, including the first sidx box data block and the second sidx box data block. This index data is used to locate audio and video segments, and the audio and video segments store audio and video frame index data and audio and video frame data.

[0046] Furthermore, refer to Figure 7 and Figure 8 In step S140, the audio / video frame index data and audio / video frame data are split into several audio / video segments using temporal logic. Each audio / video segment includes: Create a moof box data block for storing the audio and video frame index information of the audio and video segments, and write the audio and video frame index information into the moof box data block of the fMP4 format file; Create an mdat box data block for storing the audio and video frame data of the audio and video segment, and write the audio and video frame data into the mdat box data block of the fMP4 format file.

[0047] Further, step S200 includes: S210. Create a new fakeMP4 format file with empty content, and obtain audio and video data from the fakeMP4 format file. The audio and video data includes audio and video compatibility information, audio and video media metadata, audio and video segment index data, audio and video frame index data, and audio and video frame data. S220. Create an ftyp box data block in the fakeMP4 format file to store audio and video compatibility information and an mdat box data block to store the entire content of the fMP4 file. Write the audio and video compatibility information to the ftyp box data block of the fakeMP4 format file and write the entire content of the fMP4 file to the mdat box data block of the fakeMP4 format file. S230. Create a moov box data block in the fakeMP4 format file to store the audio and video media metadata and the frame index information corresponding to the audio and video frame data, and write it to the fakeMP4 format file. The frame index information includes the timestamp of the audio and video frame, the size of the audio and video frame, and the data index address of the audio and video frame in the fakeMP4 format file. In steps S220 and S230, the ftyp box data block in the fakeMP4 format file is set at the beginning of the fakeMP4 format file, and the moov box data block in the fakeMP4 format file is set before or after the mdatbox data block in the fakeMP4 format file.

[0048] Reference Figure 5 and Figure 6 From the outermost layer, fakeMP4, like MP4, mainly consists of three boxes: ftyp, moov, and mdat. The only difference is that the mdat box of fakeMP4 stores the content of an actual fMP4 file, while the index data of the moov box is generated based on the fMP4 audio and video frame data in its mdat box. If the moov box of fakeMP4 is placed before the mdat box, it is inconvenient for the player to quickly retrieve the fMP4 content from the beginning of the file (the moov box of fakeMP4 is generally several megabytes in size). To take advantage of the fast first-screen display of fMP4, the moov box of fakeMP4 can be designed to be placed at the end of the file.

[0049] Furthermore, for players that support fakeMP4 format files, when a fakeMP4 format file is detected, the audio and video will be played as an fMP4 format file. For players that do not support fakeMP4 format files, fakeMP4 format files will be treated as ordinary MP4 format and played as MP4 format videos.

[0050] Specifically, the process of playing fakeMP4 videos using a player that supports fakeMP4 involves an additional step compared to playing regular fakeMP4 videos: confirming the content range of the fakeMP4 video.

[0051] Furthermore, for players that support fakeMP4, step S300 includes the following steps: S310. Identify fakeMP4 format files and determine the content range of the fakeMP4 file; S320. Obtain all audio and video segment index data from the fMP4 content data block of the fakeMP4 format file; S330. Find the offset position of the required audio and video segment based on the audio and video segment index data; S340. Read the moof box data block of the audio and video segment at the corresponding position and obtain the index data of all audio and video frames in the audio and video segment; S350: Based on the audio and video frame index data, find the file offset position of the next audio and video frame data that needs to be decoded, and read the audio and video frame data at the corresponding position for decoding and rendering; S360. Repeat steps S330 to S350 until playback ends.

[0052] Specifically, for steps S320 and S330, locating a specific frame of audio / video data in the fMP4 content requires a two-layer location search process. The specific steps are as follows: (1) Obtain the index data of all audio and video segments from the fMP4 sidx box, and locate the required audio and video segments based on the index data; (2) Obtain the index data of all audio and video frames from the moof box in the audio and video clip, and locate the required audio and video frame based on the index data.

[0053] Furthermore, for players that do not support fakeMP4, step S300 includes the following steps: S310. Obtain all audio and video frame index data from the moov box data block at the end of the fakeMP4 format file; S320. Find the file offset position of the next audio / video frame data to be decoded based on the audio / video frame index data. S330: Read the audio and video frame data at the corresponding position, and perform decoding and rendering; S340. Repeat steps S320 to S330 until playback ends.

[0054] In other embodiments, for players that support the fakeMP4 format, the content of the fMP4 file in the fakeMP4 file can be identified, and the fakeMP4 video can be played directly according to the logic of playing the fMP4 format (the content of the fMP4 file is placed in the mdat box data block of the fakeMP4 file without any changes. Players that support the fakeMP4 format only need to know the size of the fMP4 file content and the offset position of the fMP4 file content in the fakeMP4 file to easily read the content of the fMP4 file).

[0055] When third-party players that only support MP4 format play fakeMP4 files, the fakeMP4 format achieves compatibility with MP4 format. The player recognizes the fakeMP4 file as a normal MP4 file and plays it. (The principle of third-party players playing MP4 files is to locate and read the audio and video frame data in the mdatbox based on the audio and video frame index data in the moov box, and then render and play the data. The fakeMP4 format also stores audio and video frame index data in the moov box and audio and video frame data in the mdat box. The principle is the same as MP4. The difference is that the mdat box of MP4 format generally only stores fragmented audio and video frame data, while the mdat box of fakeMP4 format stores the complete content of the fakeMP4 file. In essence, the fakeMP4 format is a specially designed MP4 format. It is compatible with MP4 format. Players that support MP4 format can directly play fakeMP4 files, but they do not play fakeMP4 files as fakeMP4 files, but rather as normal MP4 files.) The logical playback format has the disadvantage of a slow first screen.

[0056] Furthermore, refer to Figure 10 The MP4-based data processing method further includes the following steps: S400. If you need to reverse convert the received fakeMP4 format file to an MP4 format file or an fMP4 format file, perform the following conversion: Converting a received fakeMP4 file to an MP4 file involves the following steps: Create a new MP4 file with empty content, and read audio and video data from a fakeMP4 file. The audio and video data includes audio and video compatibility information, audio and video media metadata, audio and video frame index data, and audio and video frame data. Create an ftyp box data block in the MP4 file to store the audio and video compatibility information, and write the audio and video compatibility information into the MP4 file; Create a moov box data block in the MP4 format file to store the audio and video media metadata and audio and video frame index data, and write the audio and video media metadata and audio and video frame index data into the MP4 format file; Create an mdat box data block in the MP4 format file to store the audio and video frame data, and write the audio and video frame data into the MP4 format file; The MP4 file containing the data is obtained and sent to the playback device for playback. Among them, reference Figure 9 Converting a received fakeMP4 file to an fMP4 file involves the following steps: Create a new empty fMP4 file, read the fMP4 file content from the mdat box data block in the fakeMP4 file, and write it to the fMP4 file; The fMP4 format file containing the data is obtained and sent to the playback device for playback.

[0057] Furthermore, the present invention also proposes a computer-readable storage medium storing program instructions thereon, which, when executed by a processor, implement the MP4-based data processing method described above.

[0058] It should be understood that the method steps in the embodiments of the present invention can be implemented or carried out by computer hardware, a combination of hardware and software, or by computer instructions stored in a non-transitory computer-readable storage medium. The method can use standard programming techniques. Each program can be implemented in a high-level procedural or object-oriented programming language to communicate with the computer system. However, if necessary, the program can be implemented in assembly or machine language. In any case, the language can be a compiled or interpreted language. Furthermore, for this purpose, the program can run on a programmed application-specific integrated circuit (ASIC).

[0059] Furthermore, the procedures described herein may be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by the context. The procedures described herein (or variations and / or combinations thereof) may be executed under the control of one or more computer systems configured with executable instructions, and may be implemented by hardware or a combination thereof as code (e.g., executable instructions, one or more computer programs, or one or more applications) that commonly executes on one or more processors. The computer program comprises a plurality of instructions executable by one or more processors.

[0060] Furthermore, the method can be implemented in any suitable type of computing platform, including but not limited to personal computers, minicomputers, mainframes, workstations, networked or distributed computing environments, standalone or integrated computer platforms, or in communication with charged particle tools or other imaging devices, etc. Aspects of the invention can be implemented as machine-readable code stored on a non-transitory storage medium or device, whether removable or integrated into a computing platform, such as a hard disk, optical read and / or write storage medium, RAM, ROM, etc., such that it is readable by a programmable computer, and when the storage medium or device is read by the computer, it can be used to configure and operate the computer to perform the processes described herein. Furthermore, the machine-readable code, or portions thereof, can be transmitted via wired or wireless networks. The invention described herein includes these and other different types of non-transitory computer-readable storage media when such media comprises instructions or programs that implement the steps described above in conjunction with a microprocessor or other data processor. When programmed according to the methods and techniques described in the invention, the invention may also include the computer itself.

[0061] A computer program can be applied to input data to perform the functions described herein, thereby transforming the input data to generate output data stored in non-volatile memory. The output information can also be applied to one or more output devices, such as a display. In a preferred embodiment of the invention, the transformed data represents physical and tangible objects, including specific visual depictions of physical and tangible objects generated on the display.

[0062] The above description is merely a preferred embodiment of the present invention. The present invention is not limited to the above-described embodiments. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention, as long as they achieve the technical effects of the present invention by the same means, should be included within the scope of protection of the present invention. Within the scope of protection of the present invention, the technical solutions and / or implementation methods can have various modifications and variations.

Claims

1. A data processing method based on MP4, characterized in that, The MP4-based data processing method includes the following steps: S100. Convert MP4 format files to fMP4 format files; S200: Convert the converted fMP4 format file to a fakeMP4 format file; S300: Transfer the fakeMP4 format file to the playback device and play it.

2. The MP4-based data processing method according to claim 1, characterized in that, Step S100 includes: S110. Create a new fMP4 format file with empty content, and obtain audio and video data from the MP4 format file. The audio and video data includes at least audio and video compatibility information, audio and video media metadata, audio and video frame index data, and audio and video frame data. S120. Create an ftyp box data block in the fMP4 format file to store audio and video compatibility information, and write the audio and video compatibility information into the ftyp box data block of the fMP4 format file; S130. Create a moov box data block in the fMP4 format file to store audio and video media metadata, and write the audio and video media metadata into the moov box data block of the fMP4 format file. S140. The audio and video frame index data and audio and video frame data are split into multiple audio and video segments according to time logic. Each audio and video segment stores part of the audio and video frame index data and part of the audio and video frame data in the order of segment timestamp. S150. Create a sidx box data block to store audio and video segment index data, and write the audio and video segment index data into the sidx box data block of the fMP4 format file; S160. Write the audio and video segments into an fMP4 format file in the order of segment timestamps.

3. The MP4-based data processing method according to claim 2, characterized in that, In step S130, the audio and video media metadata includes the resolution and encoder type of the audio and video media; In step S140, the audio and video frame index data includes the audio and video frame timestamp, the audio and video frame size, and the data index address of the audio and video frame in the fMP4 format file; In step S150, the sidx box data block includes a first sidx box data block for locating video segments containing video frame data and a second sidx box data block for locating audio segments containing audio frame data. The audio and video segment index data includes at least the audio and video segment timestamp, the audio and video segment size, and the data index address of the audio and video segment in the file.

4. The MP4-based data processing method according to claim 2, characterized in that, In step S140, the audio / video frame index data and audio / video frame data are split into several audio / video segments using temporal logic. Each audio / video segment includes: Create a moof box data block for storing the audio and video frame index information of the audio and video segments, and write the audio and video frame index information into the moof box data block of the fMP4 format file; Create an mdat box data block for storing the audio and video frame data of the audio and video segment, and write the audio and video frame data into the mdat box data block of the fMP4 format file.

5. The MP4-based data processing method according to claim 1, characterized in that, Step S200 includes: S210. Create a new fakeMP4 format file with empty content, and obtain audio and video data from the fakeMP4 format file. The audio and video data includes audio and video compatibility information, audio and video media metadata, audio and video segment index data, audio and video frame index data, and audio and video frame data. S220. Create an ftyp box data block in the fakeMP4 format file to store audio and video compatibility information and an mdat box data block to store the entire content of the fMP4 file. Write the audio and video compatibility information to the ftyp box data block of the fakeMP4 format file and write the entire content of the fMP4 file to the mdat box data block of the fakeMP4 format file. S230. Create a moov box data block in the fakeMP4 format file to store the audio and video media metadata and the frame index information corresponding to the audio and video frame data, and write it to the fakeMP4 format file. The frame index information includes the timestamp of the audio and video frame, the size of the audio and video frame, and the data index address of the audio and video frame in the fakeMP4 format file. In steps S220 and S230, the ftyp box data block in the fakeMP4 format file is set at the beginning of the fakeMP4 format file, and the moov box data block in the fakeMP4 format file is set before or after the mdat box data block in the fakeMP4 format file.

6. The MP4-based data processing method according to claim 1, characterized in that, In step S300, For players that support fakeMP4 format files, when a fakeMP4 format file is detected, the audio and video will be played as an fMP4 format file. For players that do not support fakeMP4 format files, fakeMP4 format files will be treated as ordinary MP4 format and played as MP4 videos.

7. The MP4-based data processing method according to claim 1, characterized in that, For players that support fakeMP4, step S300 includes the following steps: S310. Identify fakeMP4 format files and determine the content range of the fakeMP4 file; S320. Obtain all audio and video segment index data from the fMP4 content data block of the fakeMP4 format file; S330. Find the offset position of the required audio and video segment based on the audio and video segment index data; S340. Read the moof box data block of the audio and video segment at the corresponding position and obtain the index data of all audio and video frames in the audio and video segment; S350: Based on the audio and video frame index data, find the file offset position of the next audio and video frame data that needs to be decoded, and read the audio and video frame data at the corresponding position for decoding and rendering; S360. Repeat steps S330 to S350 until playback ends.

8. The MP4-based data processing method according to claim 1, characterized in that, For players that do not support fakeMP4, step S300 includes the following steps: S310. Obtain all audio and video frame index data from the moov box data block at the end of the fakeMP4 format file; S320. Find the file offset position of the next audio / video frame data to be decoded based on the audio / video frame index data. S330: Read the audio and video frame data at the corresponding position, and perform decoding and rendering; S340. Repeat steps S320 to S330 until playback ends.

9. The MP4-based data processing method according to claim 1, characterized in that, The MP4-based data processing method further includes the following steps: S400. If you need to reverse convert the received fakeMP4 format file to an MP4 format file or an fMP4 format file, perform the following conversion: Converting a received fakeMP4 file to an MP4 file involves the following steps: Create a new MP4 file with empty content, and read audio and video data from a fakeMP4 file. The audio and video data includes audio and video compatibility information, audio and video media metadata, audio and video frame index data, and audio and video frame data. Create an ftyp box data block in the MP4 file to store the audio and video compatibility information, and write the audio and video compatibility information into the MP4 file; Create a moovbox data block in the MP4 format file to store the audio and video media metadata and audio and video frame index data, and write the audio and video media metadata and audio and video frame index data into the MP4 format file; Create an mdat box data block in the MP4 format file to store the audio and video frame data, and write the audio and video frame data into the MP4 format file; The MP4 file containing the data is obtained and sent to the playback device for playback. The process of converting a received fakeMP4 file to an fMP4 file includes the following steps: Create a new empty fMP4 file, read the fMP4 file content from the mdat box data block in the fakeMP4 file, and write it to the fMP4 file; The fMP4 format file containing the data is obtained and sent to the playback device for playback.

10. A computer-readable storage medium having program instructions stored thereon, characterized in that, When the program instructions are executed by the processor, they implement the MP4-based data processing method as described in any one of claims 1 to 9.

Images