A method for processing data based on MP4

Abstract

The present application relates to a data processing method based on MP4, which comprises the following steps: converting an MP4 format file into an fMP4 format file; converting the converted fMP4 format file into a fakeMP4 format file; transmitting the fakeMP4 format file to a playing end and playing the fakeMP4 format file. The present application can solve the technical problem of slow first screen, and its data format can be reversely converted and easily interpreted, and has wide device compatibility.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of audio and video data processing, and particularly relates to a data processing method based on MP4. BACKGROUND

[0002] The advantage of MP4 format lies in wide playing compatibility, such as supporting accurate dragging and supporting various encoding formats, however, it has the problem of slow first screen. The audio and video frame index data of MP4 are all stored in the moov box data block, and the general process of the player playing MP4 video is as follows: obtaining all the audio and video frame index data from the moov box data block at the beginning of the file; finding out the file offset position of the next audio and video frame data to be decoded according to the index data; reading the audio and video frame data at the corresponding position to decode and render.

[0003] For a movie, the moov box data block is basically several megabytes in size, which means that the player needs to download the index data of several megabytes before starting to render the video picture, which leads to slow first screen when playing MP4 video. SUMMARY

[0004] The present application provides a data processing method based on MP4, aiming to at least solve one of the technical problems existing in the prior art.

[0005] The technical scheme of the present application relates to a data processing method based on MP4, which comprises the following steps:

[0006] S100, converting an MP4 format file into an fMP4 format file;

[0007] S200, converting the converted fMP4 format file into a fakeMP4 format file;

[0008] S300, transmitting the fakeMP4 format file to a playing end and playing.

[0009] Further, the step S100 comprises:

[0010] S110, newly creating an fMP4 format file with empty content, obtaining audio and video data from the MP4 format file, wherein the audio and video data at least comprises audio and video compatibility information, audio and video media metadata, audio and video frame index data and audio and video frame data;

[0011] S120, creating an ftyp box data block for storing the audio and video compatibility information in the fMP4 format file, and writing the audio and video compatibility information into the ftyp box data block of the fMP4 format file;

[0012] S130, creating a moov box data block for storing audio and video media metadata in the fMP4 format file, and writing the audio and video media metadata into the moov box data block of the fMP4 format file;

[0013] S140, time-logical splitting the audio and video frame index data and the audio and video frame data into a plurality of audio and video segments, each of the audio and video segments storing part of the audio and video frame index data and part of the audio and video frame data in sequence of segment time stamps;

[0014] S150, creating a sidx box data block for storing audio and video segment index data, and writing the audio and video segment index data into the sidx box data block of the fMP4 format file;

[0015] S160, writing the audio and video segments in sequence of segment time stamps into the fMP4 format file.

[0016] Further, in the step S130, the audio and video media metadata includes resolution and encoder type of the audio and video media;

[0017] In the step S140, the audio and video frame index data includes audio and video frame time stamp, audio and video frame size, and data index address of the audio and video frame in the fMP4 format file;

[0018] In the step S150, the sidx box data block includes a first sidx box data block for locating a video segment containing video frame data and a second sidx box data block for locating an audio segment containing audio frame data, and the audio and video segment index data at least includes audio and video segment time stamp, audio and video segment size, and data index address of the audio and video segment in the file.

[0019] Further, in the step S140, the audio and video frame index data and the audio and video frame data are time-logical split into a plurality of audio and video segments, each of the audio and video segments including:

[0020] creating a moof box data block for storing audio and video frame index information of the audio and video segment, and writing the audio and video frame index information into the moof box data block of the fMP4 format file;

[0021] creating a mdat box data block for storing audio and video frame data of the audio and video segment, and writing the audio and video frame data into the mdat box data block of the fMP4 format file.

[0022] Further, the step S200 includes:

[0023] S210, create a fakeMP4 file with empty content, get audio and video data from the fMP4 file, the audio and video data including audio and video compatibility information, audio and video media metadata, audio and video segment index data, audio and video frame index data and audio and video frame data;

[0024] S220, create an ftyp box data block in the fakeMP4 file to store audio and video compatibility information and an mdat box data block to store the entire fMP4 file content, and write the audio and video compatibility information to the ftyp box data block of the fakeMP4 file and the entire fMP4 file content to the mdat box data block of the fakeMP4 file;

[0025] S230, create a moov box data block in the fakeMP4 file to store the audio and video media metadata and the corresponding frame index information of the audio and video frame data, and write to the fakeMP4 file, the frame index information including the timestamp of the audio and video frame, the size of the audio and video frame and the data index address of the audio and video frame in the fakeMP4 file;

[0026] In step S220 and step S230, the ftyp box data block in the fakeMP4 file is set at the head of the fakeMP4 file, and the moov box data block in the fakeMP4 file is set in front of or behind the mdat box data block in the fakeMP4 file.

[0027] Further, for a player supporting the fakeMP4 file, when the fakeMP4 file is identified, the audio and video are played in the manner of the fMP4 file;

[0028] For a player not supporting the fakeMP4 file, the fakeMP4 file is treated as a normal MP4 file, and the video is played in the MP4 format.

[0029] Further, for a player supporting the fakeMP4, step S300 includes the following steps:

[0030] S310, identify the fakeMP4 file and determine the fMP4 content range;

[0031] S320, get all audio and video segment index data from the fMP4 content data block of the fakeMP4 file;

[0032] S330, find the offset position of the required audio and video segment according to the audio and video segment index data;

[0033] S340, reading the moof box data block of the audio and video segment corresponding to the position, obtaining all the audio and video frame index data in the audio and video segment;

[0034] S350, finding the file offset position of the next audio and video frame data to be decoded according to the audio and video frame index data, reading the audio and video frame data at the corresponding position for decoding and rendering;

[0035] S360, repeating steps S330 to S350 until the end of the playback.

[0036] Further, for the player that does not support fakeMP4, step S300 includes the following steps:

[0037] S310, obtaining all the audio and video frame index data from the moov box data block at the end of the fakeMP4 format file;

[0038] S320, finding the file offset position of the next audio and video frame data to be decoded according to the audio and video frame index data;

[0039] S330, reading the audio and video frame data at the corresponding position for decoding and rendering;

[0040] S340, repeating steps S320 to S330 until the end of the playback.

[0041] Further, in the step S300, the MP4-based data processing method further includes the following steps:

[0042] S400, if it is necessary to reversely convert the received fakeMP4 format file into an MP4 format file or an fMP4 format file, the following conversion is performed:

[0043] Reversely converting the received fakeMP4 format file into an MP4 format file includes the following steps:

[0044] Creating a new MP4 format file with empty content, reading the audio and video data in the fakeMP4 format file, wherein the audio and video data includes audio and video compatibility information, audio and video media metadata, audio and video frame index data, and audio and video frame data;

[0045] Creating an ftyp box data block in the MP4 format file for storing the audio and video compatibility information, and writing the audio and video compatibility information into the MP4 format file;

[0046] Creating a moov box data block in the MP4 format file for storing the audio and video media metadata and the audio and video frame index data, and writing the audio and video media metadata and the audio and video frame index data into the MP4 format file;

[0047] creating mdat box data block in MP4 format file to store the audio and video frame data, and writing the audio and video frame data into the MP4 format file;

[0048] the MP4 format file containing data is transmitted to a playing terminal for playing;

[0049] wherein, the reverse conversion of the received fakeMP4 format file into fMP4 format file comprises the following steps:

[0050] creating a fMP4 format file with empty content, reading the fMP4 file content in the mdat box data block in the fakeMP4 format file, and writing the fMP4 file content into the fMP4 format file;

[0051] the fMP4 format file containing data is transmitted to a playing terminal for playing.

[0052] Further, the present application also provides a computer readable storage medium, which stores program instructions, and the program instructions are executed by a processor to implement the MP4-based data processing method.

[0053] Compared with the prior art, the present application has the following characteristics:

[0054] The present application provides an MP4-based data processing method, which stores fMP4 content in the mdat box of MP4 under the condition of inheriting the playing compatibility of MP4, solves the problem of slow first screen, and has wide device compatibility due to the reversible conversion and easy interpretation of the data format. BRIEF DESCRIPTION OF DRAWINGS

[0055] Figure 1 The flowchart of the MP4-based data processing method.

[0056] Figure 2 The schematic diagram of the prior art MP4 format.

[0057] Figure 3 The schematic diagram of the prior art fMP4 format.

[0058] Figure 4 The schematic diagram of the prior art fMP4 format (containing sidx box).

[0059] Figure 5 The schematic diagram of the fakeMP4 format.

[0060] Figure 6 The schematic diagram of the conversion of the MP4 format file into the fakeMP4 format file.

[0061] Figure 7 A schematic diagram for splitting audio and video index data and frame data of an MP4 file (based on an MP4 file).

[0062] Figure 8 A schematic diagram for splitting audio and video index data and frame data of an MP4 file (based on an fMP4 file).

[0063] Figure 9 A schematic diagram for converting a fakeMP4 file to an fMP4 file.

[0064] Figure 10 A schematic diagram for converting a fakeMP4 file to an MP4 file. DETAILED DESCRIPTION

[0065] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0066] The concept, specific structure and generated technical effects of the present application will be described clearly and completely below with reference to the embodiments and drawings, so as to fully understand the objects, solutions and effects of the present application.

[0067] It should be noted that, unless otherwise specified, when a certain feature is referred to as being "fixed" or "connected" to another feature, it can be directly fixed or connected to the other feature, or indirectly fixed or connected to the other feature. The singular forms "a", "an" and "the" used in this text are also intended to include the plural forms, unless the context clearly indicates otherwise. In addition, unless otherwise defined, all technical and scientific terms used in this text have the same meanings as those commonly understood by the person skilled in the art. The terms used in the specification of this text are only used to describe the specific embodiments, and are not intended to limit the present application. The term "and / or" used in this text includes any combination of one or more related listed items.

[0068] It should be understood that, although the terms first, second, third, etc. can be employed in this disclosure to describe various elements, these elements should not be limited to these terms. These terms are only used to distinguish one element from another. For example, a first element could also be termed a second element, and, similarly, a second element could also be termed a first element, without departing from the scope of the present disclosure. The use of any and all examples, or exemplary language (e.g., "such as", "for instance", etc.) provided herein is intended merely to better illuminate embodiments of the present application and does not pose a limitation on the scope of the present application unless otherwise claimed.

[0069] With reference to Figures 1 to 10 The embodiments of the present application provide a data processing method based on MP4, which comprises the following steps:

[0070] S100, converting an MP4 format file into an fMP4 format file;

[0071] S200, converting the converted fMP4 format file into a fakeMP4 format file;

[0072] S300, transmitting the fakeMP4 format file to a playing end and playing.

[0073] Compared with the prior art, the present application has the following characteristics:

[0074] The data processing method based on MP4 stores the fMP4 content in the mdat box of MP4 under the condition of inheriting the playing compatibility of MP4, solves the problem of slow first screen, and has wide device compatibility due to reversible conversion and easy interpretation of the data format.

[0075] Video packaging format, also known as container format, is a file format used to store multiple media streams (such as video, audio, subtitles and metadata) in one file. Video streams can be packaged with audio streams and other related data together to be played as a synchronized multimedia experience. Common video packaging formats include MP4 (MPEG-4 Part 14), TS (MPEG transport stream), Matroska, etc.

[0076] Video frames constitute the basic static image unit of a video. A video frame can be simply considered as a static picture. By playing these frames in quick succession, the visual persistence effect of the human eye is utilized to produce continuous dynamic images, thus forming the video seen.

[0077] The first screen refers to the first picture seen by a user when playing a video. The first screen time generally refers to the time interval between the user clicking to play the video and seeing the first frame of the video. The shorter the first screen time, the better the user's playback experience.

[0078] ISOBMFF, or ISO Base Media File Format, is a container file format that defines a general structure for storing time-based media data such as video and audio. It is standardized in ISO / IEC 14496-12, also known as MPEG-4 Part 12. It is designed as a flexible and extensible format that facilitates the exchange, management, editing, and presentation of media. The format has been widely used for media file storage and serves as the basis for various other media file formats such as MP4 and 3GP container formats.

[0079] In the prior art, with reference to Figure 2 , MP4 is a widely used standard video packaging format in the industry, based on the ISOBMFF file format. The MP4 format is composed of multiple box structure units, each box consisting of the following 3 parts:

[0080] 1) size: box size, occupying 4 bytes;

[0081] 2) type: box type, represented by 4 characters, such as "ftyp", occupying 4 bytes;

[0082] 3) data: the main data content stored in the box, different types of boxes are used to store different data content.

[0083] From the outermost layer of the MP4 file, the MP4 file is mainly composed of ftyp (File Type), moov (Movie), and mdat (Media Data) boxes. The ftyp box is located at the beginning of the file, and the order of the latter two is not required. The moov box can be placed before or after the mdat box.

[0084] 1) ftyp box: stores version compatibility information of MP4 format, generally occupying tens of bytes;

[0085] 2) moov box: stores all audio and video frame index data, which can be used to find the offset position of a specific frame of audio and video data in the file according to the playback time point. The index data generally occupies several megabytes;

[0086] 3) mdat box: stores all audio / video frame data, is the main data of the file.

[0087] Referring to Figure 3 fMP4 (fragmented MP4) is another video encapsulation format based on ISOBMFF file format, which can be simply regarded as a kind of MP4 format for logically partitioning audio / video data.

[0088] fMP4 cuts the mdat box of MP4 into very many different mdat boxes, each of which stores only part of the audio / video frame data, usually 1-5s of audio / video data. The audio / video frame index data of fMP4 is not stored in the moov box, but in the moof box (Movie Fragment). Each moof box corresponds to an mdat box, and each moof / mdat box pair forms a segment.

[0089] Referring to Figure 4 The moof box of each segment can only index the audio / video frame data in the mdat box of the same segment. If it is necessary to find the file offset position of a specific frame of audio / video data according to the playback time point, it is necessary to first find the corresponding segment according to the playback time point, and then find the file offset position of the audio / video frame data according to the moof box in the segment. fMP4 can use sidx box (Segment Index) to store the index data of all segments, which is used to find the corresponding segment according to the playback time point. For a single audio track video file, there are usually 2 sidx boxes, one for locating the segment containing video frame data, and the other for locating the segment containing audio frame data.

[0090] Therefore, corresponding to the prior art MP4 and fMP4, since the player mainly finds the position of the audio / video frame data through the audio / video frame index data of MP4, and the MP4 format does not constrain that the mdat box can only store audio / video frame data, nor does it limit the position of the audio / video frame data in the mdat box. Taking advantage of this, MP4 and fMP4 can be fused to generate a fakeMP4 format, which can be simply regarded as a kind of MP4 format for storing fMP4 content in the mdat box.

[0091] Further, the step S100 comprises:

[0092] S110, creating a fMP4 format file with empty content, obtaining audio and video data from the MP4 format file, wherein the audio and video data at least comprises audio and video compatibility information, audio and video media metadata, audio and video frame index data and audio and video frame data;

[0093] S120, creating an ftyp box data block in the fMP4 format file for storing the audio and video compatibility information, and writing the audio and video compatibility information into the ftyp box data block of the fMP4 format file;

[0094] S130, creating a moov box data block in the fMP4 format file for storing the audio and video media metadata, and writing the audio and video media metadata into the moov box data block of the fMP4 format file;

[0095] S140, time logically splitting the audio and video frame index data and the audio and video frame data into a plurality of audio and video segments, wherein each of the audio and video segments stores part of the audio and video frame index data and part of the audio and video frame data in sequence according to segment timestamp;

[0096] S150, creating an sidx box data block for storing audio and video segment index data, and writing the audio and video segment index data into the sidx box data block of the fMP4 format file;

[0097] S160, writing the audio and video segments into the fMP4 format file in sequence according to segment timestamp.

[0098] Further, in the step S130, the audio and video media metadata comprises resolution and encoder type of the audio and video media;

[0099] In the step S140, the audio and video frame index data comprises audio and video frame timestamp, audio and video frame size and data index address of the audio and video frame in the fMP4 format file;

[0100] In the step S150, the sidx box data block comprises a first sidx box data block for locating a video segment containing video frame data and a second sidx box data block for locating an audio segment containing audio frame data, and the audio and video segment index data at least comprises audio and video segment timestamp, audio and video segment size and data index address of the audio and video segment in the file.

[0101] Specifically, the sidx box data block stores index data of the audio / video segment, including a first sidx box data block and a second sidx box data block, the index data is used to locate the audio / video segment, and the audio / video segment stores audio / video frame index data and audio / video frame data.

[0102] Further, referring to Figure 7 and Figure 8 , in step S140, the audio / video frame index data and the audio / video frame data are time logically split into a plurality of audio / video segments, each of which includes:

[0103] creating a moof box data block for storing the audio / video frame index information of the audio / video segment, and writing the audio / video frame index information into the moof box data block of the fMP4 format file;

[0104] creating an mdat box data block for storing the audio / video frame data of the audio / video segment, and writing the audio / video frame data into the mdat box data block of the fMP4 format file.

[0105] Further, the step S200 includes:

[0106] S210, a fakeMP4 format file is newly created as empty, audio / video data is obtained from the fMP4 format file, the audio / video data includes audio / video compatibility information, audio / video media metadata, audio / video segment index data, audio / video frame index data and audio / video frame data;

[0107] S220, an ftyp box data block for storing the audio / video compatibility information and an mdat box data block for storing the entire fMP4 file content are created in the fakeMP4 format file, the audio / video compatibility information is written into the ftyp box data block of the fakeMP4 format file, and the entire fMP4 file content is written into the mdat box data block of the fakeMP4 format file;

[0108] S230, a moov box data block for storing the audio / video media metadata and the frame index information corresponding to the audio / video frame data is created in the fakeMP4 format file, and is written into the fakeMP4 format file, the frame index information includes a timestamp of the audio / video frame, a size of the audio / video frame and a data index address of the audio / video frame in the fakeMP4 format file;

[0109] In step S220 and step S230, the ftyp box data block in the fakeMP4 format file is set in the head of the fakeMP4 format file, and the moov box data block in the fakeMP4 format file is set in front of or behind the mdat box data block in the fakeMP4 format file.

[0110] Referring to Figure 5 and Figure 6 From the outermost layer, fakeMP4 and MP4 are mainly composed of ftyp, moov, mdat three boxes. The only difference from MP4 is that the mdat box of fakeMP4 stores an fMP4 content, and the index data of the moov box of fakeMP4 can be generated according to the fMP4 audio and video frame data in the mdat box. If the moov box of fakeMP4 is placed before the mdat box, it is not convenient for the player to quickly obtain the fMP4 content from the beginning of the file (the moov box of fakeMP4 is generally several megabytes in size). In order to be able to play the advantage of fMP4 first screen fast, the moov box of fakeMP4 can be designed to be placed at the end of the file.

[0111] Further, for the player supporting the fakeMP4 format file, when the fakeMP4 format file is identified, the audio and video are played in the manner of the fMP4 format file;

[0112] For the player not supporting the fakeMP4 format file, the fakeMP4 format file is regarded as a normal MP4 format, and the video is played in the MP4 format.

[0113] Specifically, the flow of the player supporting fakeMP4 to play fakeMP4 video has an additional step of confirming the fMP4 content range.

[0114] Further, for the player supporting fakeMP4, step S300 includes the following steps:

[0115] S310, identify the fakeMP4 format file, and determine the fMP4 content range;

[0116] S320, obtain all audio and video segment index data from the fMP4 content data block of the fakeMP4 format file;

[0117] S330, find the offset position of the required audio and video segment according to the audio and video segment index data;

[0118] S340, reading the moof box data block of the audio and video segment corresponding to the position, obtaining all the audio and video frame index data in the audio and video segment;

[0119] S350, finding the file offset position of the next audio and video frame data to be decoded according to the audio and video frame index data, reading the corresponding audio and video frame data for decoding and rendering;

[0120] S360, repeating steps S330 to S350 until the end of the play.

[0121] Specifically, for steps S320 and S330, locating a specific frame of audio and video data in the fMP4 content needs to go through two layers of positioning and searching process, and the specific steps are:

[0122] (1) obtaining all the index data of the audio and video segment from the sidx box of the fMP4, and locating the required audio and video segment according to the index data;

[0123] (2) obtaining all the index data of the audio and video frame from the moof box in the audio and video segment, and locating the required audio and video frame according to the index data.

[0124] Further, for the player not supporting fakeMP4, step S300 includes the following steps:

[0125] S310, obtaining all the audio and video frame index data from the moov box data block at the end of the fakeMP4 format file;

[0126] S320, finding the file offset position of the next audio and video frame data to be decoded according to the audio and video frame index data;

[0127] S330, reading the corresponding audio and video frame data for decoding and rendering;

[0128] S340, repeating steps S320 to S330 until the end of the play.

[0129] In some other embodiments, for the player supporting fakeMP4 format, the fMP4 file content in the fakeMP4 file can be identified, and the fakeMP4 video can be played directly according to the logic of playing fMP4 format (the fMP4 file content is put into the mdat box data block of the fakeMP4 file without change, and the player supporting fakeMP4 format only needs to know the size of the fMP4 file content and the offset position of the fMP4 file content in the fakeMP4 file, so that the fMP4 file content can be read simply).

[0130] For the third-party player supporting only MP4 format to play fakeMP4 file, fakeMP4 format achieves compatibility with MP4 format, and the player identifies fakeMP4 file as a normal MP4 file to play (the principle of the third-party player playing MP4 file is to locate and read audio and video frame data in mdat box according to audio and video frame index data in moov box, and then to data rendering and play; fakeMP4 format also stores audio and video frame index data in moov box, and stores audio and video frame data in mdat box, and the principle is the same as MP4. The difference is that the mdat box of MP4 format generally only stores pure audio and video frame data, while the mdat box of fakeMP4 format stores a complete fMP4 file content. Essentially, fakeMP4 format is a specially designed MP4 format, which is compatible with MP4 format, and the player supporting MP4 format can directly play fakeMP4 file. However, it is not to play fakeMP4 file in fMP4 format, but to play it in MP4 format, which has the disadvantage of slow first screen).

[0131] Further, with reference to Figure 10 , the MP4-based data processing method further includes the following steps:

[0132] S400, if it is needed to reversely convert the received fakeMP4 format file into MP4 format file or fMP4 format file, the following conversion is performed:

[0133] Reversely converting the received fakeMP4 format file into MP4 format file includes the following steps:

[0134] Creating a new MP4 format file with empty content, reading audio and video data in the fakeMP4 format file, wherein the audio and video data includes audio and video compatibility information, audio and video media metadata, audio and video frame index data, and audio and video frame data;

[0135] Creating ftyp box data block in the MP4 format file to store the audio and video compatibility information, and writing the audio and video compatibility information into the MP4 format file;

[0136] Creating moov box data block in the MP4 format file to store the audio and video media metadata and the audio and video frame index data, and writing the audio and video media metadata and the audio and video frame index data into the MP4 format file;

[0137] An mdat box data block used to store the audio and video frame data is created in the MP4 format file, and the audio and video frame data is written into the MP4 format file;

[0138] The MP4 format file containing the data is obtained and transmitted to a playing terminal for playing;

[0139] The fakeMP4 format file is converted into an fMP4 format file by the following steps: Figure 9

[0140] The fMP4 format file is newly created with an empty content, the fMP4 file content in the mdat box data block of the fakeMP4 format file is read and written into the fMP4 format file;

[0141] The fMP4 format file containing the data is obtained and transmitted to a playing terminal for playing.

[0142] Further, the present application also provides a computer readable storage medium, which stores program instructions, and the program instructions are executed by a processor to implement the MP4-based data processing method.

[0143] It should be appreciated that the method steps in the embodiments of the present application can be realized or implemented by computer hardware, a combination of hardware and software, or through computer instructions stored in a non-transitory computer readable storage medium. The method can use standard programming techniques. Each program can be implemented in a high-level procedural or object-oriented programming language to communicate with a computer system. However, if necessary, the program can be implemented in assembly or machine language. In any case, the language can be a compiled or interpreted language. In addition, the program can run on a programmed special integrated circuit for this purpose.

[0144] In addition, the operations of the processes described herein can be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context. The processes described herein (or variations and / or combinations thereof) can be performed under the control of one or more computer systems configured with executable instructions (e.g., executable instructions, one or more computer programs or one or more applications), by hardware or combinations thereof. The computer programs include a plurality of instructions that can be executed by one or more processors.

[0145] ​Further, the methods can be implemented in any type of computing platform operably connected to a suitable computing platform, including but not limited to a personal computer, mini-computer, mainframe, workstation, network or distributed computing environment, separate or integrated computer platforms, or in communication with charged particle tools or other imaging devices, and the like. Aspects of the present application can be implemented in machine-readable code stored on a non-transitory storage medium or device, whether removable or integrated to the computing platform, such as a hard disk, optical read and / or write storage media, RAM, ROM, and the like, such that it can be read by a programmable computer to configure and operate the computer to perform the processes described herein when the storage medium or device is read by the computer. In addition, the machine-readable code, or portions thereof, can be transmitted over wired or wireless networks. The present application described herein includes these and other different types of non-transitory computer readable storage media when such media include instructions or programs that implement the steps described above in conjunction with a microprocessor or other data processor. The present application can also include the computer itself when programmed in accordance with the methods and techniques described herein.

[0146] The computer program can be applied to input data to perform the functions described herein to transform the input data to generate output data that is stored to non-volatile memory. The output information can also be applied to one or more output devices such as a display. In a preferred embodiment of the present application, the transformed data represents a physical and tangible object, including a particular visual depiction of the physical and tangible object produced on a display.

[0147] The above description is only preferred embodiments of the present application, and the present application is not limited to the above-described embodiments, but any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the present application. The technical solutions and / or embodiments of the present application can have various modifications and changes within the scope of the present application.

Claims

1. A method for processing MP4-based data, characterized by, The MP4-based data processing method comprises the following steps: S100, converting an MP4 format file into an fMP4 format file; S200, converting the converted fMP4 format file into a fakeMP4 format file; The step S200 comprises: S210, newly creating a fakeMP4 format file with empty content, obtaining audio and video data from the fMP4 format file, wherein the audio and video data comprises audio and video compatibility information, audio and video media metadata, audio and video segment index data, audio and video frame index data and audio and video frame data; S220, creating an ftyp box data block for storing the audio and video compatibility information and an mdat box data block for storing the entire fMP4 file content in the fakeMP4 format file, and writing the audio and video compatibility information into the ftyp box data block of the fakeMP4 format file and writing the entire fMP4 file content into the mdat box data block of the fakeMP4 format file; S230, creating a moov box data block for storing frame index information corresponding to the audio and video media metadata and the audio and video frame data in the fakeMP4 format file, and writing into the fakeMP4 format file, wherein the frame index information comprises a timestamp of the audio and video frame, a size of the audio and video frame and a data index address of the audio and video frame in the fakeMP4 format file; In the step S220 and the step S230, the ftyp box data block in the fakeMP4 format file is arranged at the head of the fakeMP4 format file, and the moov box data block in the fakeMP4 format file is arranged in front of or behind the mdat box data block in the fakeMP4 format file; S300, transmitting the fakeMP4 format file to a playing end and playing.

2. The MP4-based data processing method of claim 1, wherein, The step S100 comprises: S110, newly creating an fMP4 format file with empty content, obtaining audio and video data from an MP4 format file, wherein the audio and video data at least comprises audio and video compatibility information, audio and video media metadata, audio and video frame index data and audio and video frame data; S120, creating an ftyp box data block for storing the audio and video compatibility information in the fMP4 format file, and writing the audio and video compatibility information into the ftyp box data block of the fMP4 format file; S130, creating a moov box data block for storing the audio and video media metadata in the fMP4 format file, and writing the audio and video media metadata into the moov box data block of the fMP4 format file; S140, time logically splitting the audio and video frame index data and the audio and video frame data into a plurality of audio and video segments, and storing part of the audio and video frame index data and part of the audio and video frame data in each audio and video segment according to the segment timestamp sequence; S150, creating an sidx box data block for storing audio and video segment index data, and writing the audio and video segment index data into the sidx box data block of the fMP4 format file; S160, write the audio and video segments in the fMP4 format file according to the segment timestamp order.

3. The MP4-based data processing method of claim 2, characterized in that, In step S130, the audio and video media metadata includes the resolution and encoder type of the audio and video media; In step S140, the audio and video frame index data includes the audio and video frame timestamp, the audio and video frame size, and the data index address of the audio and video frame in the fMP4 format file; In step S150, the sidx box data block includes a first sidx box data block for locating a video segment containing video frame data and a second sidx box data block for locating an audio segment containing audio frame data, and the audio and video segment index data at least includes the audio and video segment timestamp, the audio and video segment size, and the data index address of the audio and video segment in the file.

4. The MP4-based data processing method of claim 2, wherein, In step S140, the audio and video frame index data and the audio and video frame data are time logically split into a plurality of audio and video segments, each of which includes: Create a moof box data block for storing the audio and video frame index information of the audio and video segment, and write the audio and video frame index information into the moof box data block of the fMP4 format file; Create an mdat box data block for storing the audio and video frame data of the audio and video segment, and write the audio and video frame data into the mdat box data block of the fMP4 format file.

5. The MP4-based data processing method of claim 1, wherein, In step S300, For a player supporting the fakeMP4 format file, when the fakeMP4 format file is identified, the audio and video are played in the fMP4 format file mode; For a player not supporting the fakeMP4 format file, the fakeMP4 format file is treated as a normal MP4 format, and the video is played in the MP4 format.

6. The MP4-based data processing method of claim 1, wherein, For a player supporting fakeMP4, step S300 includes the following steps: S310, identify the fakeMP4 format file and determine the fMP4 content range; S320, obtain all audio and video segment index data from the fMP4 content data block of the fakeMP4 format file; S330, find the offset position of the required audio and video segment according to the audio and video segment index data; S340, read the moof box data block of the audio and video segment at the corresponding position to obtain all the audio and video frame index data in the audio and video segment; S350, find the file offset position of the next audio and video frame data to be decoded according to the audio and video frame index data, read the corresponding audio and video frame data for decoding and rendering; S360, repeat steps S330 to S350 until the end of the play.

7. The MP4-based data processing method of claim 1, wherein, For a player not supporting fakeMP4, step S300 includes the following steps: S310, obtain all audio and video frame index data from the moov box data block at the end of the fakeMP4 format file; S320, find the file offset position of the next audio and video frame data to be decoded according to the audio and video frame index data; S330, reading the audio and video frame data of the corresponding position, decoding and rendering; S340, repeating steps S320 to S330 until the end of playing.

8. The MP4-based data processing method of claim 1, wherein, The MP4-based data processing method further comprises the following steps: S400, if it is needed to reversely convert the received fakeMP4 format file into an MP4 format file or an fMP4 format file, the following conversion is performed: The reverse conversion of the received fakeMP4 format file into an MP4 format file comprises the following steps: creating an empty MP4 format file, reading the audio and video data in the fakeMP4 format file, wherein the audio and video data comprises audio and video compatibility information, audio and video media metadata, audio and video frame index data and audio and video frame data; creating an ftyp box data block in the MP4 format file for storing the audio and video compatibility information, and writing the audio and video compatibility information into the MP4 format file; creating a moov box data block in the MP4 format file for storing the audio and video media metadata and the audio and video frame index data, and writing the audio and video media metadata and the audio and video frame index data into the MP4 format file; creating an mdat box data block in the MP4 format file for storing the audio and video frame data, and writing the audio and video frame data into the MP4 format file; transmitting the MP4 format file containing the data to a playing terminal for playing; The reverse conversion of the received fakeMP4 format file into an fMP4 format file comprises the following steps: creating an empty fMP4 format file, reading the fMP4 file content in the mdat box data block in the fakeMP4 format file, and writing into the fMP4 format file; transmitting the fMP4 format file containing the data to a playing terminal for playing.

9. A computer readable storage medium having stored thereon program instructions, wherein, The program instructions are executed by the processor to implement the MP4-based data processing method according to any one of claims 1 to 8.

Images