Method, device and equipment for transmitting mp4 segmented file and storage medium

Abstract

The invention discloses a method, a device and equipment for transmitting an mp4 segmented file. The method comprises the steps that a request, sent by an opposite terminal, for transmitting an mp4 segmented file is received, and the request comprises the starting moment and the duration of the mp4 segmented file in a corresponding source mp4 file; opening and explaining the source mp4 file to obtain the position of each box in the moov in the source mp4 file; according to the starting moment and the duration, matching one or more corresponding tags in additional information of a source mp4 file; according to the matched tag and the position of each box in the moov in the source mp4 file, generating moov index data of the required part of the segmented file in the mp4 in a memory; and sending the moov index and the corresponding audio and video data in the mdat to an opposite terminal. According to the invention, the user can only download the required part of the mp4 video, so that the waste of bandwidth and storage is reduced.

Description

Technical Field

[0001] The present invention relates to the technical field of network communications, and more specifically, to a method, device, equipment and storage medium for transmitting MP4 segmented files. Background Art

[0002] With the development of Internet technology and the popularization of broadband networks, video content has become a major component of network traffic. At present, the playback of MP4 video files requires downloading the file header information of the video file first, parsing the file header information to obtain the file information and audio and video frame related information required for video playback, and then playback can be realized.

[0003] The current method of downloading mp4 video files generally requires the entire video file to be downloaded to the local computer before it can be played. Even if the user only wants to watch a part of the content, the entire file still needs to be downloaded. However, mp4 video files are generally large, and downloading the entire mp4 video file will have some significant disadvantages, such as: 1. Bandwidth waste: The user may only need a part of the video, but the entire video file is still downloaded, resulting in a waste of bandwidth resources; 2. Storage pressure: For downloading users, the complete video file takes up too much space on mobile devices with limited storage space. 3. Cumbersome steps: After the video file is completely downloaded, it needs to be edited using non-linear editing software (non-linear editing software). Summary of the invention

[0004] In order to overcome at least one of the above-mentioned defects of the prior art, the present invention provides a method, device, equipment and storage medium for transmitting mp4 segmented files. The technical solution adopted by the present invention is as follows.

[0005] In a first aspect, the present invention provides a method for transmitting mp4 segmented files, comprising:

[0006] Receive a request for transmitting an mp4 segmented file sent by a peer end, wherein the request includes: a starting time and a duration of the mp4 segmented file in a corresponding source mp4 file;

[0007] Open and interpret the source mp4 file to obtain the position of each box in the moov in the source mp4 file;

[0008] According to the starting time and duration, one or more corresponding tags are matched in the additional information of the source mp4 file; wherein the additional information is a record file additionally generated during the recording of the source mp4 file, and the additional information includes at least one tag, each of which has its own starting time point and duration, as well as the state of the index data corresponding to the starting time point and the offset of mdat, and the starting time point is the time point when each I frame is written during the recording of the mp4 file;

[0009] Generate the moov index data of the required part of the segmented file in the mp4 in the memory according to the matched tag and the position of each box in the moov in the source mp4 file;

[0010] According to the request, read and send the moov index data corresponding to the source mp4 file to the other end, read and send the moov index data of the mp4 segmented file in the memory to the other end, read and send the corresponding audio and video data in the mdat in the source mp4 file to the other end according to the request, and instruct the other end to generate an mp4 segmented file based on the received audio and video data and moov index data.

[0011] In one embodiment, the process of matching one or more corresponding tags in the additional information of the source mp4 file according to the start time and duration includes:

[0012] Find two tags whose starting time points are closest to the starting time according to the starting time, and select one of them as the starting tag;

[0013] Determine the end time according to the start time and duration;

[0014] According to the end time, find a tag whose end time point is closest to the end time and after the end time, and select it as the end tag;

[0015] The start tag, the end tag, and the tags between the start tag and the end tag are listed as matched tags.

[0016] In one embodiment, each tag includes: the video track moment avDurationOffset corresponding to the start point of the recording segment, the duration avDuration of the video track included in the recording segment, the audio track moment auDurationOffset corresponding to the start point of the recording segment, and the duration auDuration of the audio track included in the recording segment;

[0017] The process of generating the required part of the moov index data of the segmented file in the mp4 in the memory according to the matched tag and the position of each box in the moov in the source mp4 file includes:

[0018] According to the avDurationOffset of the end tag, the avDuration of the end tag and the avDurationOffset of the start tag, the duration of the video track is obtained, and the corresponding index data in moov is rewritten according to the duration of the video track;

[0019] According to the auDurationOffset of the end tag, the auDuration of the end tag and the auDurationOffset of the start tag, the duration of the audio track is obtained, and the corresponding index data in moov is rewritten according to the duration of the audio track;

[0020] The larger value between the duration of the video track and the duration of the audio track is taken as the duration of the mp4 segment file, and the corresponding index data in moov is rewritten according to the duration of the mp4 segment file.

[0021] In one implementation, each tag includes: an mdat offset mdatOffset corresponding to the start point of the recording segment;

[0022] The process of generating the required part of the moov index data of the segmented file in the mp4 in the memory according to the matched tag and the position of each box in the moov in the source mp4 file includes:

[0023] According to the moovSize of the mp4 segment file, the moovSize of the source mp4 file and the mdatOffset of the starting tag, the change of the stco element is obtained, and the corresponding index data in moov is rewritten according to the change of the stco element.

[0024] In one embodiment, each tag includes: a stss box element number stssEntriesOffset corresponding to the start point of the record segment;

[0025] The process of generating the required part of the moov index data of the segmented file in the mp4 in the memory according to the matched tag and the position of each box in the moov in the source mp4 file includes:

[0026] The number of video frames skipped in stss is obtained according to the avEntriesOffset of the starting tag, and the corresponding index data in moov is rewritten according to the number of video frames skipped.

[0027] In one implementation, during recording of the source mp4 file, the mapping relationship between chunks and samples is controlled so that one chunk corresponds to one sample.

[0028] In a second aspect, the present invention provides a device for transmitting mp4 segmented files, comprising:

[0029] The server receives a request for transmitting an mp4 segmented file sent by a client, wherein the request includes: a starting time and a duration of the mp4 segmented file in a corresponding source mp4 file;

[0030] The server opens and interprets the source mp4 file, and obtains the position of each box in the moov in the source mp4 file;

[0031] The server matches one or more corresponding tags in the additional information of the source mp4 file according to the starting time and duration; wherein the additional information is a record file additionally generated during the recording of the source mp4 file, and the additional information includes at least one tag, each of which has its own starting time point and duration, as well as the state of the index data corresponding to the starting time point and the offset of mdat, and the starting time point is the time point when each I frame is written during the recording of the mp4 file;

[0032] The server generates the required moov index data of the segmented file in the mp4 file in the memory according to the matched tag and the position of each box in the moov in the source mp4 file;

[0033] The server reads and sends the moov index data corresponding to the source mp4 file to the client according to the request, reads and sends the moov index data of the mp4 segment file in the memory to the client, and reads and sends the corresponding audio and video data in the mdat in the source mp4 file to the client according to the request;

[0034] The client generates mp4 segment files based on the received audio and video data and moov index data.

[0035] In a third aspect, the present invention provides a device for transmitting mp4 segmented files, characterized in that it comprises: a server and a client;

[0036] The server side includes:

[0037] The server receiving module is used to receive a request for transmitting an mp4 segmented file sent by a client, wherein the request includes: a starting time and a duration of the mp4 segmented file in a corresponding source mp4 file;

[0038] An interpretation module is used to open and interpret the source MP4 file to obtain the position of each box in the moov in the source MP4 file;

[0039] A matching module, for matching one or more corresponding tags in the additional information of the source mp4 file according to the starting time and duration; wherein the additional information is a record file additionally generated during the recording of the source mp4 file, and the additional information includes at least one tag, each of which has its own starting time point and duration, as well as the state of the index data corresponding to the starting time point and the offset of mdat, and the starting time point is the time point when each I frame is written during the recording of the mp4 file;

[0040] A rewriting module, for generating in memory the moov index data of the required part of the segmented file in the mp4 file according to the matched tag and the position of each box in the moov in the source mp4 file;

[0041] The server sending module reads and sends the moov index data corresponding to the source mp4 file to the client according to the request, reads and sends the moov index data of the mp4 segmented file in the memory to the client, and reads and sends the corresponding audio and video data in the mdat in the source mp4 file to the client according to the request;

[0042] The client includes:

[0043] The client sending module is used to send the request for transmitting the mp4 segmented file;

[0044] The client receiving module is used to receive the audio and video data and moov index data sent by the server;

[0045] The generation module is used to generate mp4 segment files according to the received audio and video data and moov index data.

[0046] In a fourth aspect, the present invention provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the above-mentioned methods when executing the program.

[0047] In a fifth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, characterized in that when the program is executed by a processor, the method of any of the above-mentioned embodiments is implemented.

[0048] In the present invention, additional information for recording the start time point and duration of GOP is first added in units of GOP during the recording process, and when it is necessary to download segmented mp4 files, moov index data corresponding to the segmented mp4 files is generated according to the additional information, making it possible to download segmented mp4 files. The present invention enables users to download only the required part of the mp4 video, thereby reducing the waste of bandwidth and storage. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 It is a schematic diagram of the overall process of an implementation method of Example 1 of the present invention.

[0050] Figure 2 It is the box storage information table of the mp4 file.

[0051] Figure 3 It is a schematic diagram of the overall process of another implementation method of the first embodiment of the present invention.

[0052] Figure 4 It is a schematic diagram of the overall structure of an implementation method of the second embodiment of the present invention.

[0053] Figure 5 It is a schematic diagram of the overall structure of another implementation method of the second embodiment of the present invention. DETAILED DESCRIPTION

[0054] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0055] It should be noted that the terms "first\second\..." involved in the embodiments of the present invention are only used to distinguish similar objects, and do not represent a specific order for the objects. It is understandable that the specific order or sequence of "first\second\..." can be interchanged where permitted. It should be understood that the objects distinguished by "first\second\..." can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein.

[0056] Embodiment 1

[0057] See also Figure 1 , Figure 1 The flowchart of a method for transmitting an MP4 segmented file provided in the first embodiment of the present invention includes step S110, step S120, step S130, step S140 and step S150. It should be noted that step S110, step S120, step S130, step S140 and step S150 are only reference numerals for clarifying the embodiment and the appendix. Figure 1 The corresponding relationship does not limit the order of the steps in this embodiment.

[0058] Step S110, receiving a request for transmitting an mp4 segmented file sent by the opposite end, wherein the request includes: a starting time and a duration of the mp4 segmented file in the corresponding source mp4 file;

[0059] Step S120, opening and interpreting the source mp4 file, and obtaining the position of each box in the moov in the source mp4 file;

[0060] Step S130, according to the starting time and duration, matching one or more corresponding tags in the additional information of the source mp4 file; wherein the additional information is a record file additionally generated during the recording of the source mp4 file, and the additional information includes at least one tag, each of which has its own starting time point and duration, as well as the state of the index data corresponding to the starting time point and the offset of mdat, and the starting time point is the time point when each I frame is written during the recording of the mp4 file;

[0061] Step S140, generating the moov index data of the required part of the segmented file in the mp4 in the memory according to the matched tag and the position of each box in the moov in the source mp4 file;

[0062] Step S150, reads and sends the moov index data corresponding to the source mp4 file to the peer end according to the request, reads and sends the moov index data of the mp4 segmented file in the memory to the peer end, reads and sends the corresponding audio and video data in the mdat in the source mp4 file to the peer end according to the request, and instructs the peer end to generate an mp4 segmented file based on the received audio and video data and moov index data.

[0063] The following uses a server and a client as examples to illustrate this method, and this method is used in the server. Before the transmission of this method, some preparations must be done first, specifically, the preparations are done during the recording process.

[0064] like Figure 2 As shown, Figure 2 It is the box storage information table of mp4 files. MP4 files are generally composed of "boxes". Small boxes are stored in large boxes, and media information is stored one level at a time. Among them, the most basic and largest boxes include: file type container ftpy, metadata container moov and media data container mdat.

[0065] During the recording process, it is necessary to record additional necessary information during the recording process. This information is additional information. The file of this additional information can be recorded as slice.txt. slice.txt contains recording units, and each recording unit is recorded as a tag. The function of slice.txt is to record the state of the index data and the offset of mdat at the time point when the I frame is written during the recording of the mp4 file. The reason for using the time point of the I frame as the timing of recording is that after the subsequent generation of mp4, the video needs to be parsed to the I frame before the subsequent non-key frames can be parsed normally. Of course, this also limits the granularity of the segmented file. The generated segmented file is a multiple of gop, but it is usually at the level of several seconds, which is acceptable for video viewing experience and video analysis.

[0066] Preferably, each tag needs to contain at least the following information:

[0067] (1) Record the video track time corresponding to the starting point of the segment, recorded as avDurationOffset;

[0068] (2) The audio track time corresponding to the start point of the recording segment, recorded as auDurationOffset;

[0069] (3) Record the duration of the video track contained in the segment, recorded as avDuration;

[0070] (4) The duration of the audio track contained in the recording segment, recorded as auDuration;

[0071] (5) Record the stss box element number corresponding to the starting point of the segment, recorded as stssEntriesOffset;

[0072] (6) Record the video track box element number corresponding to the starting point of the segment, recorded as avEntriesOffset;

[0073] (7) Record the audio track box element number corresponding to the starting point of the segment, recorded as auEntriesOffset;

[0074] (8) Record the number of video track box elements contained in the segment, recorded as avEntriesCount;

[0075] (9) The number of audio track box elements contained in the record segment, recorded as auEntriesCount;

[0076] (10) Record the mdat offset corresponding to the starting point of the segment, recorded as mdatOffset;

[0077] (11) The mdat size contained in the record segment.

[0078] Preferably, during the recording process, the mapping relationship between chunks and samples is controlled so that one chunk corresponds to one sample, which can simplify the element types of tag records. Otherwise, the number of elements in the above (8) and (9) may be multiple, and the above tag element types should be increased as appropriate.

[0079] In addition, as the recording progresses, the recorded tags show the following pattern:

[0080] 1) The recorded starting point of the audio and video track is monotonically increasing;

[0081] 2) The recorded box element number and mdat offset are also monotonically increasing;

[0082] 3) The index information recorded by a tag marks a segment.

[0083] After recording is completed and a complete mp4 file is obtained, the transmission process begins. It should be noted that in the following description, the complete mp4 file is referred to as the source mp4 file. When a user wants to download an mp4 file, he sends a request to download the mp4 file through the browser (client), and passes in the starting time of the mp4 file to be downloaded and the desired download duration from the starting time. At this time, in step S110, the server receives the request for transmitting the mp4 segmented file sent by the client, and the request includes: the starting time and duration of the mp4 segmented file in the corresponding source mp4 file.

[0084] In step S120, the source mp4 file is opened and interpreted to obtain the position of each box in the moov in the source mp4 file.

[0085] In step S110, the starting time and duration of the source mp4 file are known, that is, the time interval is known. In step S130, the time interval is used to match the corresponding tag, that is, the time interval is compared with the time index of the tag to find the appropriate tag. In this step, we will get the starting tag (recorded as startTag) and the ending tag (endTag); there may be only one tag, which is both the starting tag and the ending tag.

[0086] For example, by reading tags for matching, assuming a 20s video with a frame rate of 30 and a GOP (Group of Pictures) with 120 frames, a tag is recorded every 4 seconds. The starting point of the video recorded by tag0 is 0s, and the duration of the recording is 4s; the starting point of the video recorded by tag1 is 4s, and the duration of the recording is 4s, and so on.

[0087] If we need to download the video starting at the 4th second, and hope to download for 12 seconds, then the start time of the download is the 4th second, and the end time of the download is the 16th second. At this point, we know from this example that the appropriate startTag is tag1, and the endTag is tag3.

[0088] Assume that, for the 20-second video above, the starting time to be downloaded is the 16th second of the video, and the desired download duration is 4 seconds. By reading the tag for matching, we know that the appropriate startTag is tag4, and the endTag is also tag4. Through tag4, we can get the last segment of the mp4 file.

[0089] Since there is a certain correspondence between the metadata container moov and the media data container mdat, if a new mp4 segment file is to be regenerated, a moov dedicated to the new mp4 segment file must be regenerated. In fact, because the target file is a segment of the source mp4, some of the moov index data is the same as the source mp4, and some of it needs to be adjusted based on the original moov index data, such as the content indicating the duration in moov, and the header and element content of some boxes. For these parts that need to be adjusted, the source file is not directly modified, but only generated in memory.

[0090] First, rewrite the box in moov that contains duration information, such as Figure 2 As shown, at least the mvhd box, tkhd box, and mdhd box are involved.

[0091] The new duration is calculated as follows:

[0092] Calculate the video track duration:

[0093] AvDuration=endTag’s avDurationOffset+endTag’s avDuration-startTag’s avDurationOffset;

[0094] Audio track duration:

[0095] AuDuration=endTag’s auDurationOffset+endTag’s auDuration-startTag’s auDurationOffset;

[0096] mp4 duration:

[0097] Duration=MAX(AvDuration,AuDuration).

[0098] Rewrite the duration field in the video tkhd box based on AvDuration, rewrite the duration field in the audio tkhd box based on AuDuration, and rewrite the duration field in the video mvhd box based on Duration.

[0099] Secondly, the data recorded by stss needs special processing. stss records the serial number of the key frame. Since the video frames before the starting time need to be skipped when downloading segments, the recorded key frame serial number will change. At the same time, its box size and the number of entries also change.

[0100] In the MP4 file structure, stss (Sync Sample Box) mainly consists of two parts: box header and entries list. The following is a detailed explanation of these two parts:

[0101] Box Header:

[0102] Box header is a structure common to every box, including:

[0103] Box Size (size): indicates the size of the entire box, including header and entries. This is a 32-bit integer, and its size does not include the 4 bytes themselves.

[0104] Box Type (type): Indicates the type of the box. For the stss box, the type is a four-character code, i.e. "stss".

[0105] For 64-bit files, this header information may be slightly different to support larger box sizes, but the basic box size and type fields remain the same.

[0106] Entries:

[0107] The Entries section is unique to the stss box and includes:

[0108] Version: Version number, one byte.

[0109] Flags: occupies 3 bytes and is used to define specific box properties.

[0110] Entry Count: Indicates the number of keyframe indexes in the following list.

[0111] Next is the Entry Count number of entries, each of which defines the position of a key frame (synchronous sample).

[0112] Sample Number: Indicates the index of the key frame in the sample table, usually starting from 1.

[0113] For example, suppose an SSS box has the following content:

[0114] Box Size: 24 bytes (including box header and entries)

[0115] Box Type:"stss"

[0116] Version:0

[0117] Flags: 0 (This is an example, the specific value depends on the file implementation)

[0118] Entry Count: 3 (indicates that there are three keyframe indexes)

[0119] Followed by three Sample Number entries, for example:

[0120] Sample Number:5,15,25

[0121] This means that in the video stream, samples 5, 15, and 25 are key frames.

[0122] Thirdly, special processing is required for the data recorded by stco.

[0123] The change in the stco element is: moovSize of the mp4 segment file - moovSize of the source mp4 - the offset of mdat of startTag.

[0124] Finally, the contents of other box entries are not sensitive to the increase or decrease of the previous and next elements and can be read directly from the source mp4; you only need to pay attention to returning the correct box size and number of entries when reading the box header data.

[0125] Accordingly, the stsz box of the audio track should also be modified accordingly.

[0126] After the moov index data of the mp4 segmented file is generated in the memory in step 140, the mp4 segmented file is ready and can be transmitted to the client. In step S150, the server starts to transmit the file. When downloading to moov, the moov index data in the memory is read and sent to the browser; when transmitting to the mdat part, the audio and video data content of the corresponding position in the mdat of the source mp4 file is read and sent to the browser. The browser receives and saves it to obtain the mp4 segmented file.

[0127] It should be noted here that in order to allow the other party to obtain the complete moov index data of the segmented file, it is necessary to read and send the moov index data corresponding to the source mp4 file and the moov index of the mp4 segmented file in the memory to the other party. However, the process of reading and sending the source mp4 file and the moov index in the memory is carried out alternately, and the reading is carried out in the order of moov reading, instead of reading the source mp4 file first and then reading the memory.

[0128] In this method, the receiving end is not sensitive to the transmission process, that is, except for specifying the start time and download duration when initiating a request, it does not need to cooperate with the sending end to do other special operations, and the data of the sending end can be received as a normal byte stream. The sending end needs to generate the data required for the mp4 segment based on the additional information recorded in the recording and the source mp4.

[0129] In this method, additional information for recording the start time point and duration of GOP is first added in units of GOP during the recording process. When it is necessary to download segmented mp4 files, the moov index data corresponding to the segmented mp4 files is generated according to the additional information, making it possible to download segmented mp4 files. This method allows users to download only the required part of the mp4 video, thereby reducing the waste of bandwidth and storage.

[0130] In the meantime, see Figure 3 , Figure 3 A flowchart of another method for transmitting mp4 segmented files provided in Embodiment 1 of the present invention is provided, the method comprising step S210, step S220, step S230, step S240, step S250 and step S260. It should be noted that step S210, step S220, step S230, step S240, step S250 and step S260 are merely reference numerals for the purpose of clearly explaining the embodiment and the appendix. Figure 1 The corresponding relationship does not limit the order of the steps in this embodiment.

[0131] Step S210, the server receives a request for transmitting an mp4 segmented file sent by the client, wherein the request includes: a starting time and a duration of the mp4 segmented file in the corresponding source mp4 file;

[0132] Step S220, the server opens and interprets the source mp4 file, and obtains the position of each box in the moov in the source mp4 file;

[0133] Step S230: the server matches one or more corresponding tags in the additional information of the source mp4 file according to the starting time and duration; wherein the additional information is a record file additionally generated during the recording of the source mp4 file, and the additional information includes at least one tag, each of which has its own starting time point and duration, as well as the state of the index data corresponding to the starting time point and the offset of mdat, and the starting time point is the time point when each I frame is written during the recording of the mp4 file;

[0134] Step S240, the server generates the moov index data of the required part of the segmented file in the mp4 in the memory according to the matched tag and the position of each box in the moov in the source mp4 file;

[0135] Step S250, the server reads and sends the moov index data corresponding to the source mp4 file to the client according to the request, reads and sends the moov index data of the mp4 segment file in the memory to the client, and reads and sends the corresponding audio and video data in the mdat in the source mp4 file to the client according to the request;

[0136] Step S260: The client generates an mp4 segment file according to the received audio and video data and moov index data.

[0137] This method corresponds to the above method and is used between the server and the client.

[0138] Embodiment 2

[0139] Corresponding to the method of embodiment 1, as Figure 4 As shown, the present invention also provides a device for transmitting mp4 segmented files, including: a receiving module 410, an interpretation module 420, a matching module 430, a rewriting module 440 and a sending module 450.

[0140] The receiving module 410 is used to receive a request for transmitting an mp4 segmented file sent by a peer end, wherein the request includes: a starting time and a duration of the mp4 segmented file in a corresponding source mp4 file;

[0141] An interpretation module 420 is used to open and interpret the source MP4 file to obtain the position of each box in the moov in the source MP4 file;

[0142] A matching module 430 is used to match one or more corresponding tags in the additional information of the source mp4 file according to the starting time and duration; wherein the additional information is a record file additionally generated during the recording of the source mp4 file, and the additional information includes at least one tag, each of which has its own starting time point and duration, as well as the state of the index data corresponding to the starting time point and the offset of mdat, and the starting time point is the time point when each I frame is written during the recording of the mp4 file;

[0143] The rewriting module 440 is used to generate the moov index data of the required part of the segmented file in the mp4 in the memory according to the matched tag and the position of each box in the moov in the source mp4 file;

[0144] The sending module 450 reads and sends the moov index data corresponding to the source mp4 file to the other end according to the request, reads and sends the moov index data of the mp4 segmented file in the memory to the other end, reads and sends the corresponding audio and video data in the mdat in the source mp4 file to the other end according to the request, and instructs the other end to generate an mp4 segmented file based on the received audio and video data and moov index data.

[0145] In one embodiment, the matching module performs the process of matching one or more corresponding tags in the additional information of the source mp4 file according to the starting time and duration, including:

[0146] Find two tags whose starting time points are closest to the starting time according to the starting time, and select one of them as the starting tag;

[0147] Determine the end time according to the start time and duration;

[0148] According to the end time, find a tag whose end time point is closest to the end time and after the end time, and select it as the end tag;

[0149] The start tag, the end tag, and the tags between the start tag and the end tag are listed as matched tags.

[0150] In one embodiment, each tag includes: the video track moment avDurationOffset corresponding to the start point of the recording segment, the duration avDuration of the video track included in the recording segment, the audio track moment auDurationOffset corresponding to the start point of the recording segment, and the duration auDuration of the audio track included in the recording segment;

[0151] The rewriting module executes the process of generating the required part of the moov index data of the segmented file in the mp4 in the memory according to the matched tag and the position of each box in the moov in the source mp4 file, including:

[0152] According to the avDurationOffset of the end tag, the avDuration of the end tag and the avDurationOffset of the start tag, the duration of the video track is obtained, and the corresponding index data in moov is rewritten according to the duration of the video track;

[0153] According to the auDurationOffset of the end tag, the auDuration of the end tag and the auDurationOffset of the start tag, the duration of the audio track is obtained, and the corresponding index data in moov is rewritten according to the duration of the audio track;

[0154] The larger value between the duration of the video track and the duration of the audio track is taken as the duration of the mp4 segment file, and the corresponding index data in moov is rewritten according to the duration of the mp4 segment file.

[0155] In one implementation, each tag includes: an mdat offset mdatOffset corresponding to the start point of the recording segment;

[0156] The rewriting module executes the process of generating the required part of the moov index data of the segmented file in the mp4 in the memory according to the matched tag and the position of each box in the moov in the source mp4 file, including:

[0157] According to the moovSize of the mp4 segment file, the moovSize of the source mp4 file and the mdatOffset of the starting tag, the change of the stco element is obtained, and the corresponding index data in moov is rewritten according to the change of the stco element.

[0158] In one embodiment, each tag includes: a stss box element number stssEntriesOffset corresponding to the start point of the record segment;

[0159] The rewriting module executes the process of generating the required part of the moov index data of the segmented file in the mp4 in the memory according to the matched tag and the position of each box in the moov in the source mp4 file, including:

[0160] The number of video frames skipped in stss is obtained according to the avEntriesOffset of the starting tag, and the corresponding index data in moov is rewritten according to the number of video frames skipped.

[0161] like Figure 5 As shown, the present invention also provides another device for transmitting mp4 segmented files, including: a server 5 and a client 6;

[0162] The server side includes:

[0163] The server receiving module 510 is used to receive a request for transmitting an mp4 segmented file sent by a client, wherein the request includes: a starting time and a duration of the mp4 segmented file in a corresponding source mp4 file;

[0164] An interpretation module 520 is used to open and interpret the source MP4 file to obtain the position of each box in the moov in the source MP4 file;

[0165] A matching module 530 is used to match one or more corresponding tags in the additional information of the source mp4 file according to the starting time and duration; wherein the additional information is a record file additionally generated during the recording of the source mp4 file, and the additional information includes at least one tag, each of which has its own starting time point and duration, as well as the state of the index data corresponding to the starting time point and the offset of mdat, and the starting time point is the time point when each I frame is written during the recording of the mp4 file;

[0166] The rewriting module 540 is used to generate the moov index data of the required part of the segmented file in the mp4 in the memory according to the matched tag and the position of each box in the moov in the source mp4 file;

[0167] The server sending module 550 reads and sends the moov index data corresponding to the source mp4 file to the client according to the request, reads and sends the moov index data of the mp4 segment file in the memory to the client, and reads and sends the corresponding audio and video data in the mdat in the source mp4 file to the client according to the request;

[0168] The client includes:

[0169] The client sending module 610 is used to send the request for transmitting the mp4 segmented file;

[0170] The client receiving module 620 is used to receive the audio and video data and moov index data sent by the server;

[0171] The generation module 630 is used to generate an mp4 segment file according to the received audio and video data and moov index data.

[0172] In the device, additional information for recording the start time point and duration of GOP is first added in units of GOP during the recording process. When it is necessary to download the segmented mp4 file, the moov index data corresponding to the segmented mp4 file is generated according to the additional information, making it possible to download the segmented mp4 file. The device allows users to download only the required part of the mp4 video, thereby reducing the waste of bandwidth and storage.

[0173] Embodiment 3

[0174] An embodiment of the present invention further provides a storage medium on which computer instructions are stored. When the instructions are executed by a processor, the method for transmitting MP4 segmented files in any of the above embodiments is implemented.

[0175] Those skilled in the art can understand that: all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above method embodiments; and the aforementioned storage medium includes: mobile storage devices, random access memories (RAM), read-only memories (ROM), magnetic disks or optical disks, etc., various media that can store program codes.

[0176] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention can be essentially or partly reflected in the form of a software product that contributes to the relevant technology. The computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, terminal, or network device, etc.) to execute all or part of the methods of each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, RAM, ROM, magnetic disks or optical disks.

[0177] Corresponding to the above-mentioned computer storage medium, in one embodiment, a computer device is also provided, which includes a memory, an encoder, and a computer program stored in the memory and executable on the encoder, wherein when the encoder executes the program, it implements any one of the methods for transmitting mp4 segmented files in the above-mentioned embodiments.

[0178] The computer device first adds additional information for recording the start time point and duration of the GOP in units of GOP during the recording process, and when it is necessary to download the segmented mp4 file, generates moov index data corresponding to the segmented mp4 file according to the additional information, making it possible to download the segmented mp4 file. The computer device allows users to download only the required part of the mp4 video, thereby reducing the waste of bandwidth and storage.

[0179] The technical features of the above-described embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above-described embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0180] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. For those skilled in the art, other different forms of changes or modifications can be made based on the above description. It is not necessary and impossible to list all the embodiments here. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the claims of the present invention.

Claims

1. A method for transmitting mp4 segmented files, characterized in that: include: Receive a request for transmitting an mp4 segmented file sent by a peer end, wherein the request includes: a starting time and a duration of the mp4 segmented file in a corresponding source mp4 file; Open and interpret the source mp4 file to obtain the position of each box in the moov in the source mp4 file; According to the starting time and duration, one or more corresponding tags are matched in the additional information of the source mp4 file; wherein the additional information is a record file additionally generated during the recording of the source mp4 file, and the additional information includes at least one tag, each of which has its own starting time point and duration, as well as the state of the index data corresponding to the starting time point and the offset of mdat, and the starting time point is the time point when each I frame is written during the recording of the mp4 file; Generate the moov index data of the required part of the segmented file in the mp4 in the memory according to the matched tag and the position of each box in the moov in the source mp4 file; According to the request, read and send the moov index data corresponding to the source mp4 file to the other end, read and send the moov index data of the mp4 segmented file in the memory to the other end, read and send the corresponding audio and video data in the mdat in the source mp4 file to the other end according to the request, and instruct the other end to generate an mp4 segmented file based on the received audio and video data and moov index data.

2. The method for transmitting mp4 segmented files according to claim 1, characterized in that: The process of matching one or more corresponding tags in the additional information of the source mp4 file according to the starting time and duration includes: Find two tags whose starting time points are closest to the starting time according to the starting time, and select one of them as the starting tag; Determine the end time according to the start time and duration; According to the end time, find a tag whose end time point is closest to the end time and after the end time, and select it as the end tag; The start tag, the end tag, and the tags between the start tag and the end tag are listed as matched tags.

3. The method for transmitting mp4 segmented files according to claim 2, characterized in that: Each tag includes: the video track moment avDurationOffset corresponding to the start point of the recording segment, the duration avDuration of the video track contained in the recording segment, the audio track moment auDurationOffset corresponding to the start point of the recording segment, and the duration auDuration of the audio track contained in the recording segment; The process of generating the required part of the moov index data of the segmented file in the mp4 in the memory according to the matched tag and the position of each box in the moov in the source mp4 file includes: According to the avDurationOffset of the end tag, the avDuration of the end tag and the avDurationOffset of the start tag, the duration of the video track is obtained, and the corresponding index data in moov is rewritten according to the duration of the video track; According to the auDurationOffset of the end tag, the auDuration of the end tag and the auDurationOffset of the start tag, the duration of the audio track is obtained, and the corresponding index data in moov is rewritten according to the duration of the audio track; The larger value between the duration of the video track and the duration of the audio track is taken as the duration of the mp4 segment file, and the corresponding index data in moov is rewritten according to the duration of the mp4 segment file.

4. The method for transmitting mp4 segmented files according to claim 2, characterized in that: Each tag includes: mdat offset mdatOffset corresponding to the starting point of the record segment; The process of generating the required part of the moov index data of the segmented file in the mp4 in the memory according to the matched tag and the position of each box in the moov in the source mp4 file includes: According to the moovSize of the mp4 segment file, the moovSize of the source mp4 file and the mdatOffset of the starting tag, the change of the stco element is obtained, and the corresponding index data in moov is rewritten according to the change of the stco element.

5. The method for transmitting mp4 segmented files according to claim 2, characterized in that: Each tag includes: stss box element number stssEntriesOffset corresponding to the starting point of the record segment; The process of generating the required part of the moov index data of the segmented file in the mp4 in the memory according to the matched tag and the position of each box in the moov in the source mp4 file includes: The number of video frames skipped in stss is obtained according to the avEntriesOffset of the starting tag, and the corresponding index data in moov is rewritten according to the number of video frames skipped.

6. The method for transmitting mp4 segmented files according to claim 1, characterized in that: During recording of the source mp4 file, the mapping relationship between chunks and samples is controlled so that one chunk corresponds to one sample.

7. A device for transmitting mp4 segmented files, characterized in that: include: The server receives a request for transmitting an mp4 segmented file sent by a client, wherein the request includes: a starting time and a duration of the mp4 segmented file in a corresponding source mp4 file; The server opens and interprets the source mp4 file, and obtains the position of each box in the moov in the source mp4 file; The server matches one or more corresponding tags in the additional information of the source mp4 file according to the starting time and duration; wherein the additional information is a record file additionally generated during the recording of the source mp4 file, and the additional information includes at least one tag, each of which has its own starting time point and duration, as well as the state of the index data corresponding to the starting time point and the offset of mdat, and the starting time point is the time point when each I frame is written during the recording of the mp4 file; The server generates the required moov index data of the segmented file in the mp4 file in the memory according to the matched tag and the position of each box in the moov in the source mp4 file; The server reads and sends the moov index data corresponding to the source mp4 file to the client according to the request, reads and sends the moov index data of the mp4 segment file in the memory to the client, and reads and sends the corresponding audio and video data in the mdat in the source mp4 file to the client according to the request; The client generates mp4 segment files based on the received audio and video data and moov index data.

8. A device for transmitting mp4 segmented files, characterized in that: include: Server and client side; The server side includes: The server receiving module is used to receive a request for transmitting an mp4 segmented file sent by a client, wherein the request includes: a starting time and a duration of the mp4 segmented file in a corresponding source mp4 file; An interpretation module is used to open and interpret the source MP4 file to obtain the position of each box in the moov in the source MP4 file; A matching module, for matching one or more corresponding tags in the additional information of the source mp4 file according to the starting time and duration; wherein the additional information is a record file additionally generated during the recording of the source mp4 file, and the additional information includes at least one tag, each of which has its own starting time point and duration, as well as the state of the index data corresponding to the starting time point and the offset of mdat, and the starting time point is the time point when each I frame is written during the recording of the mp4 file; A rewriting module, for generating in memory the moov index data of the required part of the segmented file in the mp4 file according to the matched tag and the position of each box in the moov in the source mp4 file; The server sending module reads and sends the moov index data corresponding to the source mp4 file to the client according to the request, reads and sends the moov index data of the mp4 segmented file in the memory to the client, and reads and sends the corresponding audio and video data in the mdat in the source mp4 file to the client according to the request; The client includes: The client sending module is used to send the request for transmitting the mp4 segmented file; The client receiving module is used to receive the audio and video data and moov index data sent by the server; The generation module is used to generate mp4 segment files according to the received audio and video data and moov index data.

9. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 6 is implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images