Fast in-place FMP4 to MP4 conversion

Abstract

Converting fragmented MP4 (FMP4) files in place to MP4 format without having to create separate files is discussed herein. Audio and video data is captured by a multimedia application and stored in an FMP4 file having an initial moov atom and one or more fragmented headers assigned to portions of the audio / video data. Once the capture session is complete (e.g., the user stops recording audio and video), the FMP4 file is converted to an MP4 file by attaching a final moov atom to the FMP4 file and changing the header name of the initial moov (i) atom from a "moov" name to an "mdat" name. This change in name makes the initial moov (i) atom of the FMP4 file opaque to media players and converts the FMP4 file to an MP4 format.

Description

Background Technology

[0001] The Moving Picture Experts Group (MPEG) has defined several multimedia standards, including the widely popular MPEG-4 Part 14 standard (often simply referred to as "MP4"). MP4 is a digital multimedia container format used to store audiovisual files, and with more and more users accessing the internet, the MP4 standard has become the preferred standard for storing multimedia files. Many multimedia applications allow users to play and record audiovisual files as easily shareable MP4 files. Most users today do not know or have little knowledge of how these applications create MP4 files.

[0002] To record content as an MP4 file, users typically click the recording button to start capturing audio and video (audio / video) data. When complete, the user stops recording, triggering a multimedia application to perform several complex operations to finalize the MP4 file. One operation performed during the finalization phase is creating "moov" atoms (often called "movie" atoms). These moov atoms include a header containing various data used by the multimedia player. In traditional MP4 recording, if the application crashes before the moov atoms are written, the audio / video will be unplayable. However, the average user creating an MP4 file is unaware of the additional processing that must be performed after recording stops. Therefore, the user will close the recording application before the moov atoms have been written—before finalization is complete.

[0003] This becomes even more problematic for longer records that require more complex moov atoms, often causing header write timeouts before the moov atoms complete. Records may be lost if the moov atoms are not created before the application closes, if the moov atom write times out, or if there isn't enough disk space to store the finalized MP4 record. Clearly, lost records can be disastrous for the user experience, especially when the record is critical.

[0004] A relatively new variant of the MP4 format is Fragmented MP4 (FMP4). Instead of writing a single moov atom after the record is created, FMP4 generates multiple segment headers (called "moof" atoms) for different segments of the multimedia file. These moof atoms are scattered throughout the record. However, FMP4 is not yet widely adopted, so FMP4 files must be converted back to traditional MP4 files before being used with multimedia players—otherwise, various player functions (such as playback, search, etc.) will not work properly. This conversion process from FMP4 to MP4 is very cumbersome and resource-intensive. Audio / video data must be copied from the FMP4 file to the newly created MP4 file, creating two instances of the data. Two copies of the same audio / video data put pressure on processing and storage resources, or worse, if there is not enough storage available for both copies of the audio / video data, the conversion may fail and the audio / video data may be lost. Summary of the Invention

[0005] The disclosed examples are described in detail below with reference to the accompanying drawings. The following overview is provided to illustrate some of the examples disclosed herein. However, this is not intended to limit all examples to any particular configuration or order of operation.

[0006] The examples disclosed herein pertain to technologies and devices configured to perform in-place conversion of FMP4 files to MP4 format without having to create separate files. For this purpose, audio / video data is recorded by a multimedia application or otherwise accessed. The recorded audio / video data is stored in an FMP4 file and assigned an initial moov atom (referred to herein as the "moov(i)" atom). Additionally, a segment header (moof) is assigned to a portion of the audio / video data. Thus, if the multimedia application is closed before the FMP4 file is converted, or if a system failure occurs, at least the FMP4 file with the segment header is stored. However, if recording is complete (e.g., the user stops recording), the FMP4 file is converted to an MP4 file. This conversion is achieved by generating a final moov atom (referred herein as the "moov(f)" atom) and attaching it to the FMP4 file, and also by changing the designation of the initial moov(i) atom from the "moov" name to the "mdat" name. Changing the header name to mdat makes the initial moov(i) atom and fragment headers opaque to the multimedia player, thus effectively converting FMP4 files to MP4 files without having to make multiple copies of the audio / video data. Attached Figure Description

[0007] The disclosed implementation and embodiments will be described in detail below with reference to the accompanying drawings:

[0008] Figure 1 A block diagram of a client computing device configured to perform in-situ conversion of FMP4 files to MP4 format, according to some publicly available implementations, is shown.

[0009] Figure 2 A block diagram is shown of an FMP4 version of an audio / video file being recorded and converted to MP4 format, based on some publicly available implementations;

[0010] Figure 3 A block diagram is shown showing the audio / video files after recording, according to some publicly available implementations of FMP4.

[0011] Figure 4 A block diagram is shown showing an audio / video file, based on some publicly available implementations of FMP4, after being in-situ converted to MP4 format;

[0012] Figure 5 The data file of an example FMP4 audio / video file is shown being converted to MP4 format using the disclosed in-place conversion according to some disclosed implementations;

[0013] Figure 6 The data file of an example FMP4 audio / video file, modified to MP4 format according to some publicly available implementations, is shown.

[0014] Figure 7 A flowchart illustrating a workflow for performing in-situ conversion of FMP4 files to MP4 files, based on some publicly available implementations, is shown; and

[0015] Figure 8 A flowchart illustrating a workflow for converting FMP4 files to MP4 files, based on some publicly available implementations, is shown. Detailed Implementation

[0016] Various implementations will be described in detail with reference to the accompanying drawings. Where possible, the same reference numerals will be used throughout the drawings to refer to the same or similar parts. References relating to specific examples and implementations throughout this disclosure are provided for illustrative purposes only and are not intended to limit all examples unless otherwise indicated.

[0017] The implementations and examples disclosed herein provide systems, methods, and computer storage devices with instructions for performing in-situ conversions from FMP4 files to MP4 files. As referenced herein, "in-situ" conversion refers to modifying an existing FMP4 file to convert it to MP4 format. In some implementations, in-situ conversion is accomplished by writing new moov atoms based on the fragment header of the FMP4 file and then changing the header information of the initial moov atoms from "moov" to "mdat" to make the initial moov atoms opaque to the multimedia player. Using the disclosed techniques and devices, only one file—the FMP4 file—is needed to generate the MP4 file. However, conventional techniques for converting FMP4 to MP4 files require creating a new MP4 file and copying the audio / video data of the FMP4 file to the new MP4 file, which creates two copies of the audio / video data, doubling the storage and processing requirements. First recording the multimedia file as an FMP4 file and then converting that FMP4 file to MP4 using the disclosed in-situ conversion preserves the recording—or at least most of the recording—from being lost before the MP4 finalization stage. Besides the FMP4 file used for conversion, it requires only a small amount of additional storage, thus reducing the usual processing and storage requirements for FMP4 to MP4 conversion.

[0018] An FMP4 file comprises numerous segment headers (moof atoms) scattered and assigned to different portions (or segments) of the recorded audio / video data. For example, an FMP4 record may include a first moof atom assigned to a first block of audio / video data, a second moof atom assigned to a second block of audio / video data, and so on. If the multimedia application recording the audio / video data (e.g., by recording) closes, fails, or some other problem prevents the FMP4 file from being finalized, at least some of the audio / video data has already been stored along with its corresponding segment moof atoms. Because the FMP4 file generates segment moof atoms and stores them along with the portions of audio / video data being recorded, those portions are effectively finalized once the moof atoms are assigned to the FMP4 file. Therefore, if an error occurs before the FMP4 is finalized, at least all audio / video data except the last segment has been saved. In other words, for a record with N segments, at least N-1 segments have been saved because the segment moof atoms are assigned while the audio / video data is being recorded. Therefore, using the FMP4 format ensures that the entire record is not lost simply because finalization was not completed correctly—at most, only the last segment might be lost.

[0019] Furthermore, using the disclosed implementation as an example, only a minimal number of resources are required to perform the in-situ conversion of an FMP4 file to MP4 format. Once recording has stopped, a finalization sequence is performed. In some examples, the finalization sequence involves generating MP4 moov atoms and attaching them to a re-encoded FMP4 file containing information from fragment moof atoms as well as the initial moov atoms of the FMP4 file. It should be noted that this generated MP4 moov atom is attached to the FMP4 file to facilitate in-situ conversion, rather than being attached to a new file generated from the FMP4 file data. Once the MP4 moov atom is attached to the FMP4 file, some implementations change the header of the initial moov atoms of the FMP4 file, and the header of the optional fragment moof, from the name "moof" to the name "mdat". The mdat name identifies the atoms as audio / video data, rather than moov or moof atoms. Changing the header of the initial moov atom and the fragment moof header from moof to mdat names will change the initial moov atom and fragment moof header of the FMP4 file to opaque mdat atoms that are not visible or processed by multimedia applications. While this obfuscation of the initial moov and moof atoms may add some unnecessary data bytes to the converted FMP4 file, the resource cost is much lower than copying the FMP4 audio / video data to a new MP4 file.

[0020] In some implementations, the new MP4 moov atoms added to the FMP4 file include appropriate chunk offsets. A chunk offset is a numerical value describing the time offset of audio data, video data, media data, frames, and / or samples of the MP4 file. In an MP4 file, audio, video, and media data are written into so-called "blurbs" or "chunks." For example, video frames might be written; then audio frames; then metadata; then additional audio, video, or media data. This interleaving pattern creates chunks of data for each media type. Disclosed FMP4 and MP4 files may include a chunk offset table describing the offset of each data chunk for a given media track.

[0021] Once the MP4 moov atom is attached and the initial FMP4 moov atom is changed to mdat, the FMP4 file has been effectively converted into a usable MP4 file. The new MP4 file can then be stored and transmitted, played, or otherwise used by various multimedia applications. Using these publicly available techniques, FMP4 files can be quickly converted to MP4 without the need for costly copying and processing. Furthermore, the newly created MP4 file is compatible with traditional players, and the MP4 file can be converted back to FMP4 at any time by overwriting the modified mdat header to change the initial moov atom back to moov atom.

[0022] An overview of some of the disclosed examples has been provided; please refer to the accompanying drawings for further details. The illustrated configurations and sequences of operation are provided to help the reader understand certain aspects of the disclosed examples. The accompanying drawings are not intended to limit all examples, and therefore some examples may include different components, devices, or sequences of operations without departing from the scope of the disclosed examples discussed herein. In other words, some examples may be embodied or operate in a different manner than those shown.

[0023] Figure 1 This is a block diagram of a client computing device 100 configured to perform in-situ conversion of FMP4 files to MP4 format, based on some publicly available implementations. The client computing device 100 includes one or more processors 102, input / output (I / O) ports 104, a communication interface 106, computer storage memory (RAM) 108, I / O components 110, and a communication path 112. The client computing device 100 is capable of communicating with other devices or cloud computing resources via a network 114.

[0024] Client computing device 100 can be any of a variety of computing devices, such as, but not limited to, laptops, smartphones, tablets, virtual reality (VR) or augmented reality (AR) headsets, smartwatches, wearable devices, electronic kiosks, smart cars, etc. While client computing device 100 is depicted as a single device, multiple client computing devices 100 can work together and share the depicted device resources. For example, various processors 102 and memories 108 can be housed and distributed across multiple client computing devices 100. Client computing device 100 is merely an example of a suitable computing environment and is not intended to imply any limitation on the scope or functionality of the invention.

[0025] Processor 102 includes any number of microprocessors, microcontrollers, analog circuit systems, etc., programmed to execute computer-executable instructions to implement aspects of this disclosure. In some examples, processor 102 is programmed to execute instructions such as those shown in the other figures discussed herein. In some implementations, processor 102 is programmed with instructions for a dedicated purpose of performing in-situ conversion of FMP4 files to MP4 format, as disclosed herein.

[0026] I / O port 104 connects various hardware I / O components 110 to the client computing device 100. Example I / O components 110 include, for example, but not limited to, one or more microphones 110a, cameras 110b, and speakers 110c, operating to capture and present audio / video content. The client computing device 100 may additionally or alternatively be equipped with other hardware I / O components 110, such as, for example, but not limited to, displays, touchscreens, AR and VR headsets, peripherals, joysticks, scanners, printers, etc. Such components are well known to those skilled in the art and do not need to be discussed in detail herein.

[0027] Communication interface 106 allows software and data to be transferred between client computer device 100 and external devices via network 114. Communication interface 106 may include a modem, network interface (such as an Ethernet card), communication port, PCMCIA slot and card, wireless adapter, etc. Software and data transmitted via communication interface 106 are in the form of signals, which may be electronic signals, electromagnetic signals, optical signals, or other signals that can be received by communication interface 106. Such signals are provided to communication interface 106 via a communication path (e.g., channel) 112. Communication path 112 carries signals and can be implemented using wired, wireless, fiber optic, telephone, cellular, radio frequency (RF), or other communication channels.

[0028] Network 114 may include any computer network or combination thereof. Examples of computer networks that can be configured to operate as network 114 include, but are not limited to, wireless networks; landline telephones; cable; digital subscriber line (DSL); fiber optic lines; cellular networks (e.g., 3G, 4G, 5G, etc.); local area networks (LANs); wide area networks (WANs); metropolitan area networks (MANs); and so on. However, network 114 is not limited to connections that couple separate computer units. Rather, network 114 may also include subsystems that transmit data between servers or computing devices. For example, network 114 may also include point-to-point connections, the Internet, Ethernet, electrical buses, neural networks, or other internal systems. Such network architectures are well known and do not need to be discussed in detail herein.

[0029] Computer storage memory 108 includes any number of memory devices associated with or accessible by client computing device 100. Computer storage memory 108 may take the form of computer storage media referenced below and is operatively provided with storage for computer-readable code, data structures, program modules, and other code for client computing device 100 to store and access instructions configured to perform the various operations disclosed herein. Computer storage memory 108 may include memory devices in the form of volatile and / or non-volatile memory, removable or non-removable memory, data disks in a virtual environment, or combinations thereof. Examples of client computing device 100 include, but are not limited to, random access memory (RAM); read-only memory (ROM); electronically erasable programmable read-only memory (EEPROM); flash memory or other memory technologies; CD-ROM, digital versatile optical disc (DVD), or other optical or holographic media; magnetic cassettes, magnetic tapes, disk storage devices, or other magnetic storage devices; memory connected to analog computing devices; or any other computer memory.

[0030] Computer storage memory 108 can be located inside client computing device 100 (e.g., Figure 1 The computer storage medium 108 may be located outside the client computing device 100 (not shown) or both (not shown). Alternatively or additionally, the computer storage medium 108 may be distributed across multiple client computing devices 100 and / or servers, for example, in a virtualized environment providing distributed processing. For the purposes of this disclosure, “computer storage medium,” “computer storage memory,” “memory,” and “memory device” are synonymous terms for the computer storage medium 108, and none of these terms include a carrier wave or propagating signal.

[0031] In some examples, computer storage memory 108 stores executable computer instructions for operating system (OS) 116 and various software applications 118. OS 116 can be any OS designed to control the functions of client computing device 100, including but not limited to: MICROSOFT of Redmond, Washington. Developed By Apple of Cupertino, California Developed MAC By Google in Mountain View, California Developed ANDROID TM Open source wait.

[0032] In other applications, application 118 includes multimedia application 120, which is configured to play and capture (e.g., record) audio and / or video captured by microphone 110a and camera 110b of I / O component 110, respectively. Multimedia application 120 may be a video or audio player, a video or audio recorder, or a video conferencing application (e.g., Microsoft). etc.), video sharing applications (e.g.) This can be either an audio / video player or any other audio / video player. In some implementations, the multimedia application 120 provides recording functionality that allows a user to capture audio and / or video captured by the I / O component 110. Alternatively, the multimedia application 120 may also provide playback functionality for playing back audio and / or video. The multimedia application 120 may include one or separate applications for recording audio / video data (e.g., a multimedia recorder) and one application for playing back audio / video data (e.g., a multimedia player). For clarity, implementations and examples herein are discussed as capturing both audio and video (audio / video), but the same implementation can be used to perform in-situ conversion of audio-only or video-only FMP4 files.

[0033] In operation, multimedia application 120 records an FMP4 file containing various moof headers, which are generated during recording and distributed throughout the FMP4 file. As described above, the recorded FMP4 file with moof headers ensures that if a failure occurs before MP4 finalization, at least a portion of the recorded FMP4 file (e.g., N-1 moofs) is present. For example, if a captured FMP4 file is 60 minutes long, has 10 separate 6-minute moofs, and only 5 moofs have sufficient time to generate and distribute their moof headers before a system crash or multimedia application 120 closes, the resulting FMP4 file, saved before the crash or closure, will have at least those 5 moofs and their corresponding 5 moof headers. Therefore, the user has an FMP4 file containing at least 50 minutes of the total 60 minutes, which is far better than losing the entire file because the MP4 moof atoms could not be written before the failure / closure.

[0034] In some implementations, multimedia application 120 includes an in-place converter 122 for converting FMP4 files to MP4 using the techniques disclosed herein. The in-place converter 122 includes executable code instructions or firmware that perform the in-place conversion of audio / video file 124 from FMP4 to MP4 format without creating a new MP4 file. For clarity, the same audio / video file 124 is displayed at two different times: before finalization (time A) and after finalization (time A'). In other words, only one audio / video file 124 is generated, even if two instances are displayed at different times A and A'. The audio / video file 124 is not copied or recreated. Instead, the audio / video file 124 is recorded originally in FMP4 format, and the FMP4 version of the audio / video file 124 is modified using the techniques disclosed herein for conversion to MP4 files. Similarly, although two instances of the audio / video file are displayed... Figure 1 Only one file is being used.

[0035] As described above, once the user begins recording, audio / video file 120 is captured in FMP4 format, as shown at time A. The FMP4 version of audio / video file 124, as shown at time A on the left, originally includes audio / video data labeled with the name mdat (shown as mdat section 130), initial FMP4 moov atoms 126, and one or more segment headers 128a-n assigned to different portions of the audio / video data. Similarly, recording the audio / video data and assigning segment headers 128a-n ensures that at least a portion of the FMP4 (e.g., N-1 segments) survives a crash or malfunction.

[0036] When recording is stopped, the in-place converter 122 modifies (automatically or upon user prompting) the FMP4 version of the audio / video file 124 to create the MP4 version shown at time A'. This modification involves writing and appending the finalized MP4 moov atom (referred to herein as "moov(f)") 132 to the FMP4 version of the audio / video file 124 (e.g., by appending it to the end). In some implementations, the MP4 moov(f) atom 132 is generated at least partially from the initial FMP4 moov(i) atom 126 and fragment headers 128a-n. After the MP4 moov(f) atom 132 is generated, the in-place converter 122 hides the FMP4 moov(i) atom 126 and fragment headers 128a-n by changing their "moov" and "moof" headers to "mdat" headers. This can be done by searching for the moov and moof headers and changing both (or at least the moov name of the moov(i) atom 126) to the mdat name.

[0037] Marking FMP4 moov(i) atom 126 and fragment headers 128a-n as mdat—replacing moov and moof respectively—makes these atoms 126 and 128a-n opaque to the multimedia application 120. In other words, FMP4 moov(i) atom 126 and fragment headers 128a-n are used to create the FMP4 record of the audio / video file 124, but once the final MP4 moov(f) 132 is written, the headers for these atoms 126 and 128a-n are changed. This effectively converts the FMP4 version of the audio / video file 124 to MP4 format, despite the added atoms 126 and 128a-n being changed to mdat headers, but with a minimal size and requiring far fewer processing and memory resources than if audio / video data needed to be written to a new file.

[0038] Figures 2 to 4 Block diagrams are shown illustrating the in-situ conversion of an audio / video file 124 from FMP4 to MP4 format performed by the in-situ converter 122. These diagrams show the audio / video file 124 at different times, from being recorded as FMP4 to being converted to MP4. Specifically, Figure 2 The FMP4 version of the audio / video file 124 during recording is displayed. Figure 3 This displays the FMP4 version of the audio / video file 124 once recording has stopped. Figure 4 This displays the MP4 version of the audio / video file 124 created from a modified FMP4 version.

[0039] At first glance Figure 2 Audio / video file 124 is shown as being captured as an FMP4 file. A user or application 118 may have already begun recording audio and / or visual data. For example, a user may have selected the recording option during a video conference, a video feed from a home security camera may have automatically started recording once a doorbell is pressed or movement is detected, or recording may have started in countless other ways. During recording, audio / video file 124 is initially recorded as an FMP4 file, and an initial FMP4 moov(i) atom 126 is generated and assigned to it. Additionally, since the recording is captured in FMP4, various fragment headers 128a-n are generated during recording and assigned to different audio / video data in the mdat 130 payload. As recording continues, additional fragment headers 128 are generated and added to the audio / video data.

[0040] Figure 3The image shows an FMP4 version of the audio / video file 124 once recording has stopped. When recording stops, the in-situ converter 122 begins finalization by creating a final moov(f) atom 132 from the initial moov(i) atom 126 and fragment moov atoms 128a-n. Once the final moov(f) 132 is generated and appended to the FMP4 version of the audio / video file 124, the FMP4 to MP4 conversion is accomplished by blurring the initial moov(i) atom 126 and fragment headers 128a-n, as shown. Figure 4 As shown. To this end, some implementations change the header of the initial moov(i) atom 126 and the fragment headers 128a-n from "moov" and "moof" to "mdat". By overwriting these headers with mdat, the initial moov(i) atom 126 and the fragment headers 128a-n become opaque to the multimedia application 120 and are not processed during playback. This effectively converts the FMP4 version of the audio / video file 124 to MP4 without creating a new file. Additionally, by changing only the header of the initial moov(i) atom 126, the initial moov(i) atom 126 and the fragment headers 128a-n of the FMP4 are hidden. In other words, some implementations only change the bytes in the initial moov(i) atom 126 to hide both it and the fragment headers 128a-n.

[0041] Furthermore, by reversing the change to the initial moov(i) atom 126, the newly created MP4 can also be converted back to FMP4 format. In other words, a newly created MP4 file can be converted back to FMP4 format by changing the initial moov(i) atom 126 from the mdat name back to the moov header name. Because the MP4 file includes all the data for the initial moov(i) atom 126 and the fragment headers 128a-n, this name change (from mdat to moov) makes the initial moov(i) atom 126 and the fragment headers 128a-n visible to the multimedia application 120.

[0042] Figure 5 and Figure 6 Modifications to an FMP4 data file 500 for in-situ conversion of audio / video files 124 to MP4 format, based on some disclosed implementations, are shown. (See also:) Figure 5As shown, data file 500 includes initial moov(i) atoms 126, as indicated within the dashed box. Initial moov(i) atoms 126 include a moov header 502 specifying that atom 126 is a moov atom. In some implementations, this initial moov(i) atom 126 is created raw, and the moov header 502 is assigned while the audio / video file 124 is being recorded. The cessation of recording triggers in-place converter 122 to perform an in-place conversion of the disclosed FMP4 data file 500 to MP4 format.

[0043] In some implementations, the in-situ conversion of the FMP4 data file 500 to MP4 format is performed by changing the moov header 502 of the data file 500 from the name "moov" to the name "mdat". This change is as follows: Figure 6 As shown in the figure, the previous moov header 502 of the initial moov(i) atom 126 has been changed to the mdat header 604 by changing "moov" to "mdat". Changing the moov header 502 to the mdat header 604 makes the initial moov(i) atom 126 appear blurry from the perspective of the multimedia application 120. As a result, the FMP4 data file 500 is effectively converted to an MP4 file without having to create or copy the data to a completely new file.

[0044] Figure 7 A flowchart of a workflow 700 for performing in-situ conversion of FMP4 files to MP4 files, according to some of the disclosed implementations, is shown. Initially, a command to begin capturing audio / video data is received, as shown at 702. This can be given by the user activating a recording feature in a multimedia application (such as a video conferencing application). Alternatively, the audio / video data can be captured by accessing streaming audio / video content online, recording audio / video content using the microphone and / or camera of the client computing device 100, accessing audio / video content via the Internet, or any other means.

[0045] The audio / video data is initially stored as an FMP4 file and then converted to an MP4 file, as shown in 702, for the final conversion to MP4 format using the disclosed in-situ conversion technique. During audio / video data capture, initial moov(i) atoms are generated and assigned to the FMP4 file, as shown in 704. Segment headers are generated and assigned to different portions of the FMP4 file, as shown in 706. For example, a first moov header can be assigned to 10 minutes of audio / video data, a second moov header can be assigned to the next 10 minutes of audio / video data, and so on. Additional segment headers are assigned to the captured audio / video data until the audio / video data stops being captured (e.g., recorded), as shown in decision box 708. When capture stops, the created FMP4 file is converted to MP4 format, as shown in 710, using... Figure 8 The operations shown and referenced below are as indicated by the branch connector A.

[0046] As shown in 712, if a system failure of the client computing device occurs before the audio / video data is captured and is closed before in-situ conversion to MP4 format, the FMP4 file is stored as is, as shown in 714. Similarly, as shown in 716, if the multimedia application capturing or accessing the audio / video data is closed before conversion, the FMP4 file is stored as is, as shown in 714. These two paths ensure that at least a portion of the audio / video data is preserved and recoverable for later conversion.

[0047] Figure 8 A flowchart of a workflow 800 for converting an FMP4 file to an MP4 file, according to some disclosed implementations, is shown. As previously described, in-situ conversion of FMP4 can begin when audio / video data capture (e.g., recording) stops. After stopping, the final final moov(f) atom for the FMP4 file of the audio / video data is created, as shown at 802. This final moov(f) atom can be generated in part from the initial moov(i) atom and fragment header of the FMP4 file. In some implementations, the final moov(f) atom is appended to the FMP4 in other ways.

[0048] Once attached, the in-situ conversion is accomplished by changing the header of the initial moov(i) to the mdat name, as indicated at 804. To do this, in some implementations, the in-situ converter searches for the moov(i) header in the FMP4 file and changes it to mdat. Some implementations also change the fragment header to the mdat name. Once at least the moov(i) header has been changed, the FMP4 (a file with the attached final moov(f) atom and the changed initial moov(i) atom) can be stored as an MP4 file, as indicated by 806.

[0049] The examples and implementations disclosed herein can be described in the general context of computer code or machine-usable instructions, including computer-executable instructions (such as program components) that are executed by a computer or other machine (such as a personal data assistant or other handheld device). Typically, program components (including routines, programs, objects, components, data structures, etc.) refer to code that performs a specific task or implements a specific abstract data type. The disclosed examples can be practiced in a variety of system configurations, including personal computers, laptops, smartphones, mobile tablets, handheld devices, consumer electronics, professional computing devices, etc. The disclosed examples can also be practiced in distributed computing environments, where tasks are performed by remote processing devices linked via a communication network.

[0050] Examples of this disclosure may be described in the general context of computer-executable instructions, such as program modules executed by one or more computers or other devices as software, firmware, hardware, or a combination thereof. Computer-executable instructions may be organized into one or more computer-executable components or modules. Typically, program modules include, but are not limited to, routines, programs, objects, components, and data structures that perform a particular task or implement a particular abstract data type. Aspects of this disclosure may be implemented with any number and organization of such components or modules. For example, aspects of this disclosure are not limited to the specific computer-executable instructions or specific components or modules shown in the figures and described herein. Other examples of this disclosure may include different computer-executable instructions or components that have more or fewer functions than those shown and described herein. In examples involving general-purpose computers, aspects of this disclosure transform a general-purpose computer into a special-purpose computing device configured to perform in-situ conversion of FMP4 files to MP4 format when configured to execute the instructions described herein.

[0051] By way of example and not limitation, computer-readable media include computer storage media devices and communication media. Computer storage media devices include volatile and non-volatile, removable and non-removable memory implemented in any method or technology for storing information such as computer-readable instructions, data structures, program modules, etc. Computer storage media devices are tangible and mutually exclusive with communication media. Computer storage media devices are implemented in hardware and do not include carrier waves and propagating signals. For the purposes of this disclosure, computer storage media devices are not signals themselves. Example computer storage media devices include hard disks, flash drives, solid-state storage, phase-change random access memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, optical disc read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage devices, magnetic cartridges, magnetic tapes, disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information for access by a computing device. In contrast, communication media typically embody computer-readable instructions, data structures, program modules, etc., in modulated data signals (such as carrier waves or other transmission mechanisms), and include any information transmission medium.

[0052] Additional examples

[0053] Some examples involve a method (workflow 700) for performing in-situ conversion of fragmented MP4 (FMP4) files to MP4 files. The method includes: capturing audio and video data of the FMP4 file (702); during capture, generating initial moov atoms for the FMP4 file (704), the initial moov atoms including a header with a moov name; detecting a stop in the capture of the audio and video data (708); and, upon a stop, converting the FMP4 file to an MP4 file by changing the header of the initial moov atoms to an mdat name (710). Using the disclosed in-situ conversion to directly convert FMP4 files to MP4 files significantly reduces the amount of memory and processing resources required for such conversion, while also reducing the time required to capture audio / video data in a format that will survive system failures or crashes.

[0054] Some implementations also include storing the FMP4 file with the mdat name as an MP4 file (806). Therefore, creating an MP4 file using in-place conversion only requires the FMP4 file.

[0055] Some examples also include: assigning at least one segment header to a portion of the audio and video data, and creating a final moov atom for the MP4 file from the initial moov atom and at least one segment header (706). The scattered segment header provides security if the conversion to MP4 is not completed before a crash or system failure, ensuring that at least N-1 segments are saved for the user.

[0056] Some examples also include changing at least one fragment header to an mdat name. Figure 4 (128a-n). Therefore, the conversion only requires the FMP4 file. No additional files must be created, and audio / video data does not need to be copied.

[0057] Some examples also include creating a final moov atom from the initial moov atom and attaching the final moov atom to an FMP4 file, which also includes the initial moov atom (802). Attaching the final moov atom to an FMP4 file provides a way to use the FMP4 file as an MP4 file instead of copying the entire audio / video data to a new file.

[0058] In some examples, the header of the initial moov atom is changed after the final moov atom is attached to the FMP4 file (804). Waiting for the final moov atom to be attached ensures that the MP4 file is written correctly.

[0059] In some examples, the conversion from FMP4 to MP4 files is accomplished without having to create or copy the data to a new file outside of the FMP4 file (124). This saves a significant amount of processing and memory resources for FMP4 to MP4 conversion.

[0060] Some examples also include: assigning at least one segment header to a portion of the audio and video data (706); detecting the closure of a multimedia application that captures the audio and video data (716); storing an FMP4 file with at least one segment header (714); receiving an instruction to open the stored FMP4 file (122); and initiating an FMP4-to-MP4 conversion when the FMP4 file is opened (710). As previously mentioned, the segment header can be used to save the FMP4 file if a crash, system failure, or application closure occurs before the in-place conversion is completed.

[0061] Some examples also include initiating playback of the MP4 file, where the initial moov atom is changed to an mdat name (120); and during said playback, the initial moov atom and one or more fragment headers (120) of the FMP4 file are ignored due to the mdat name. Hiding the initial moov atom and fragment header finalization conversion process, the conversion consumes far fewer processing and memory resources than traditional conversions, despite adding some additional data bytes to the MP4 file.

[0062] In some examples, the MP4 file includes an FMP4 file, where the header of the initial moov atom is changed to the mdat name (124). The obfuscation of the initial moov atom partially allows the MP4 file to be created from the FMP4 without having to create a new file.

[0063] In some examples, the capture of audio and visual data includes recording both audio and visual data (110). The examples discussed in this paper ensure that such recordings, or at least most of them, are always captured before the file is converted to MP4 format, regardless of system failure, application closure, or other interruptions.

[0064] In some examples, the recording is performed in at least one of a video conferencing application, a teleconference application, or a video sharing application (110). Adding the in-place conversion discussed herein to such an application enhances the user experience by saving files in the event of system failure, application shutdown, or other interruptions.

[0065] In some examples, FMP4 files are converted to MP4 files on the client computing device (100), providing client-side applications with protection of user records and saving valuable processing and memory resources.

[0066] Other examples pertain to a system (100) for performing in-situ conversion of fragmented MP4 (FMP4) files to MP4 files. The system includes: a memory (108) embodied in a multimedia application configured to convert FMP4 to MP4 files; and one or more processors programmed (102) to: capture audio and video data of the FMP4 file (702); during said capture, generate initial moov atoms (704) for the FMP4 file, the initial moov atoms including a header (124) with a moov name; detect a stop in said capture of the audio and video data (708); and, upon a stop, convert the FMP4 file to an MP4 file by changing the header of the initial moov atom to an mdat name (710). This system for performing in-situ conversion significantly reduces the amount of memory and processing resources required for such conversion, while also reducing the time required to capture audio / video data in a format that will survive system failure or crash.

[0067] In some examples, changing the header of the initial moov atom to the mdat name is done automatically by the multimedia application without user intervention (122). Users may not care about the conversion of their files, so automatic in-place conversion only enhances the user experience—or perhaps does not hinder it.

[0068] In some examples, one or more processors are programmed to assign at least one segment header to a portion of the audio and video data (706); detect the shutdown of the multimedia application capturing the audio and video data (716); store an FMP4 file with at least one segment header (714); and change a second header used for at least one segment header to an mdat name (804). Therefore, the conversion only requires the FMP4 file. No additional files must be created, and the audio / video data does not need to be copied.

[0069] In some examples, the capture of audio and visual data includes recording both audio and visual data (110). Adding in-situ conversion to multimedia applications enhances the user experience by saving audio / video data in FMP4 files and automatically converting the FMP4 files to player-friendly MP4 files.

[0070] In some examples, capture is performed via recording features of at least one of a video conferencing application, a teleconference application, or a video sharing application (110). Such an application may be equipped with a disclosed in-place converter to save processing and memory resources by performing an in-place conversion of a disclosed FMP4 file to an MP4 file.

[0071] Other examples relate to one or more computer storage devices embodied with computer-executable instructions executable by at least one processor (102) to perform an in-situ conversion (108) of fragmented MP4 (FMP4) files to MP4 files. The storage device is embodied with instructions including: a multimedia application (120) executable by at least one processor (102), configured to capture audio and visual data of the FMP4 file and assign initial moov atoms to the FMP4 file (110); and an in-situ converter executable by at least one processor, configured to convert the FMP4 file to an MP4 file by changing the header of the initial moov atom from a moov name to an mdat name (122). By converting FMP4 to MP4 files, significant memory and processing resources can be saved by eliminating the need to create new files and copy data.

[0072] In some examples, the change of the header of the initial moov atom is performed when the capture of the audio and video data stops (708), so that the in-situ conversion is performed automatically and thus reduces the inevitable errors required for user intervention.

[0073] While aspects of this disclosure have been described with reference to various examples and their associated operations, those skilled in the art will understand that combinations of operations from any number of different examples are also within the scope of aspects of this disclosure.

[0074] The order of execution or performance of the operations illustrated and described in the examples of this disclosure is not required and can be performed in different orders in various examples. For example, a particular operation may be expected to be performed or performed before, simultaneously with, or after another operation within the scope of this disclosure.

[0075] When referring to elements of aspects of this disclosure or examples thereof, the articles “a,” “an,” “the,” and “described” are intended to indicate the presence of one or more of the elements. The terms “comprising,” “including,” and “having” are intended to be inclusive and mean that additional elements may be present in addition to those listed. The term “exemplary” is intended to mean “example.” The phrase “one or more of A, B, and C” means “at least one of A and / or at least one of B and / or at least one of C.”

[0076] The aspects of this disclosure have been described in detail, and it is intended that modifications and variations are possible without departing from the scope of the aspects of this disclosure as defined by the appended claims. Since various changes can be made to the above-described structures, products, and methods without departing from the scope of the aspects of this disclosure, all that is included in the foregoing description and shown in the accompanying drawings should be interpreted as illustrative rather than restrictive.

Claims

1. A method for performing in-situ conversion of fragmented MP4 files to MP4 files, comprising: Record the audio and video data of the fragmented MP4 file; While recording the audio and video data, moov atoms are generated for the fragmented MP4 file, the moov atoms including a header with a moov name; The recording of the audio and video data is stopped upon detection; as well as Converting the fragmented MP4 file into the MP4 file includes changing the moov name in the header of the moov atom to the mdat name.

2. The method according to claim 1, further comprising storing the fragmented MP4 file having the mdat name as the MP4 file.

3. The method of claim 1, wherein the moov atom is a first moov atom, and wherein the method further comprises: Assign segment headers to portions of the audio and video data; as well as A second moov atom is created from the first moov atom and the fragment header for the MP4 file.

4. The method according to claim 3 further includes changing the segment header to the mdat name.

5. The method of claim 1, wherein the moov atom is a first moov atom, and wherein the method further comprises: Create a second moov atom from the first moov atom; as well as The second moov atom is attached to the fragmented MP4 file.

6. The method of claim 5, wherein the header of the first moov atom is changed after the second moov atom is attached to the fragmented MP4 file.

7. The method of claim 1, wherein the conversion from the fragmented MP4 file to the MP4 file is performed without creating or copying the data to a new file outside the fragmented MP4 file.

8. The method according to claim 1, further comprising: Assign segment headers to portions of the audio and video data; The detection function shuts down the multimedia application that records the audio and video data. Store the fragmented MP4 file having the aforementioned fragment header; Receive an instruction to open the stored fragmented MP4 file; as well as When the fragmented MP4 file is opened, the conversion from the fragmented MP4 file to the original MP4 file is initiated.

9. The method according to claim 1, further comprising: Initiate playback of the MP4 file containing the moov atom whose name has been changed to the mdat name.

10. The method of claim 1, wherein the MP4 file comprises the fragmented MP4 file having the header having the moov atom whose name has been changed to the mdat name.

11. The method according to claim 1, further comprising: Converting the MP4 file into the fragmented MP4 file includes changing the moov atom from the mdat name to the moov name.

12. The method of claim 1, wherein the recording is performed in at least one of the following: a video conferencing application, a telephone conferencing application, or a video sharing application.

13. The method of claim 1, wherein the fragmented MP4 file is converted into the MP4 file on a client computing device.

14. A system for performing in-situ conversion of fragmented MP4 files to MP4 files, comprising: A memory embodying a multimedia application configured to convert the fragmented MP4 into the MP4 file; as well as The processor is programmed to: Record the audio and video data for the fragmented MP4 file; While recording the audio and video data, moov atoms are generated for the fragmented MP4 file, the moov atoms including a header with a moov name; The recording of the audio and video data is stopped upon detection; as well as Converting the fragmented MP4 file into the MP4 file includes changing the moov name in the header of the moov atom to the mdat name.

15. The system of claim 14, wherein changing the moov name in the header of the moov atom to the mdat name is performed by the multimedia application without user intervention.

16. The system of claim 14, wherein the processor is programmed to: Assign segment headers to portions of the audio and video data; The detection function shuts down the multimedia application that records the audio and video data. Store the fragmented MP4 file having the aforementioned fragment header; and Change the second header for the segment header to the mdat name.

17. The system of claim 14, wherein the processor is programmed to convert the MP4 file into a fragmented MP4 file, including changing the moov atom from the mdat name to the moov name.

18. The system of claim 14, wherein the recording is performed using at least one of the following recording features: a video conferencing application, a telephone conferencing application, or a video sharing application.

19. One or more computer storage devices embodying computer-executable instructions, said computer-executable instructions being processor-executable to perform in-situ conversion of fragmented MP4 files to MP4 files, including: A multimedia application, executable by the processor, is configured to record audio and video data of the fragmented MP4 file, and to allocate moov atoms to the fragmented MP4 file; as well as An in-situ converter, executable by the processor, is configured to convert the fragmented MP4 file into the MP4 file, including changing the header of the moov atom from the moov name to the mdat name.

20. The computer storage memory device of claim 19, wherein the change in the header of the moov atom is performed when the recording of the audio and video data stops.

Images