Methods of storing media data

Abstract

A computer-implemented method of storing media data. A stream of media data payloads is received. For each media data payload in the stream, the media data payload is stored in a first file store, and the location of the media data payload in the first file store is recorded in an index. In response to the media data payloads stored in the first file store exceeding a threshold, an object storage file comprising a plurality of the media data payloads stored in the first file store is created and stored in an object store. The location of the plurality of media data payloads is updated in the index, and the plurality of media data payloads are deleted from the first file store.

Description

[0001] Methods of storing media data

[0002] Field of the Invention

[0003] The present invention concerns methods of storing media data. The invention also concerns methods of retrieving media data, computing devices and computer program products.

[0004] Background of the Invention

[0005] Media data, includingvideo and audio data forfilm and television, is often received as a stream of media data payloads, such as frames or GOPs (“group of pictures”). The media data payloads are then stored together in a media file in a media data storage system. It can be desirable to retrieve the received media data payloads soon after they are received, and before the stream has finished. This may be the case, for example, where the media data relates to a live sporting event, and it is desired to edit video as it is being streamed in order to provide live or near-live coverage, such as a replay of a goal in a football match.

[0006] To allow this, it is known to store the media file containing the media data payloads in a file system that allows files to be easily modified, such as a conventional SMB file system. Using such a file system, the media data file can be modified to add new media data payloads as they are received, at the same time as existing media data payloads are retrieved from the media file.

[0007] Object storage, also known as blob storage, provides an alternative type of data storage to a file system. Object stores allow large amounts of unstructured data to be stored and retrieved at lower cost than is possible with a file system. Object stores also allow for more durable data storage than RAM storage. However, this is possible because files stored in an object store are written once and not subsequently modified. This allows the implementation of the storage to be optimised in ways that are not possible in a conventional file system, in which files may be modified after being stored, and as a consequence locking of files when they are accessed and / or modified needs to be implemented. Consequently, streamed media data payloads cannot be added to a media file stored in an object store as they are received, meaning that the lower cost storage provided by an object store cannot be used if media data payloads are to be retrieved from a media file while its content is still being streamed.

[0008] It would be advantageous to allow existing media data payloads to be retrieved while new media data payloads are still being received, while also enjoying the lower cost storage provided by object storage.

[0009] The present invention seeks to mitigate some or all of the above- mentioned problems. Alternatively or additionally, the present invention seeks to provide improved methods of storing media data, improved methods of retrieving media data, improved computing devices and improved computer program products.

[0010] Summary of the Invention

[0011] In accordance with a first aspect of the invention there is provided a computer-implemented method of storing media data, comprising the steps of: receiving a stream of media data payloads; for each media data payload in the stream: storing the media data payload in a first file store; and recording in an index the location of the media data payload in the first file store; in response to the media data payloads stored in the first file store exceeding a threshold: creating an object storage file comprising a plurality of the media data payloads stored in the first file store; storing the object storage file in an object store; and updating, in the index, the location of the plurality of media data payloads; and deleting the plurality of media data payloads from the first file store. By initially storing the media data payloads in the first file store, they can easily be returned if requested, even while new media data payloads are being streamed. However, by creating an object storage file comprising a plurality of the media data payloads when a threshold is reached, and storingthat in the object store, the advantages of using an object store can be enjoyed, and those media data payloads can be deleted from the first file store, so that the more expensive storage provided by the first file store can be re-used. Any media data payloads in the object storage file in the object store can still easily be returned, but at the same time any newly streamed media data payloads can also be returned from the first file store.

[0012] The first file store may be a file system. The first file store may be a RAM storage system. RAM provides particularly low latency storage, retrieval and modification of data and files. However, provisioning RAM for all storage would become prohibitive as the duration of streamed media data grows, and the use of the object store as secondary storage avoids this. In addition, object storage provides durability guarantees that are stronger than only storing in RAM.

[0013] The first file store may, for example, be a Redis store.

[0014] The index may be stored in the first file store. Alternatively, the index may be stored elsewhere.

[0015] The object storage file may comprise all of the streamed media data payloads stored in the first file store.

[0016] The object storage file may be created in response to the total size of the media data payloads stored in the first file store exceeding a threshold. Alternatively, the object storage file may be created in response to a predetermined interval of time having passed. Alternatively, the object storage file may be created in response to any other appropriate threshold being exceeded, for example more than a particular number of media data payloads having been received, or the cumulative count of RAM in use exceeding a threshold.

[0017] The object storage file may conform to an agreed standard file format used for media interchange, for example MOV (Apple QuickTime file format), MP4 (more formally MPEG-4 Part 14 (ISO / IEC 14496-14) a standard specified as a part of MPEG-4) or MXF (the Material Exchange Format as defined by a series of standards publish by SMPTE). Alternatively, the object storage file may be a tar file containing separate media data payloads, or a file of a bespoke format containing an index and a plurality of media data payloads, for example. The steps performed in response to the media data payloads stored in the first file store exceeding the threshold may be performed each time the threshold exceeded. In this way, following the first object storage file being stored in the object store and the media data payloads it contains being deleted, once the threshold being exceeded again by new media data payloads being received, a second object storage file containing the new media data payloads is created and stored in the object store.

[0018] Alternatively, the method may further comprise, in response to the media data payloads stored in the first file store exceeding the threshold, the steps of: creating a second object storage file comprising the plurality of the media data payloads in the first object storage file stored in the object store and the plurality of the media data payloads stored in the first file store; storing the second object storage file in the object store; updating, in the index, the location of the plurality of media data payloads in the second object storage file; deleting the first object storage file from the object store; and deleting the plurality of media data payloads from the first file store. In this way, rather than a second object storage file being created that contains only the new media data payloads, and is stored in the object store as well as the first object storage file, the second object storage contains both the original media data payloads and the new media data payloads, so that the second object storage file can replace the first object storage file.

[0019] When the stream of media data payloads stops, a final object storage file may be created that contains any final media data payloads stored in the first file store, even though the threshold is not exceeded. Similarly to above, the final object storage file may contain only the final media data payloads, or may contain all media data payloads from the stream. The media data payloads may comprise: an image; a frame; a Group of Pictures; an audio frame; an audio sample; and an audio clip. The media data payloads may also belon to a combination of video and audio streams.

[0020] In accordance with a second aspect of the invention there is provided a computer-implemented method of retrieving media data, wherein the media data has been stored in accordance with the method of any preceding claim, the method comprising the steps of: receiving a request for a media data payload; determiningfrom the index the location of the media data payload; if the media data payload is located in the first file store, retrieving the media data payload from the first file store; if the media data payload is located in the object store, retrieving the media data payload from the object store; and returning the media data payload.

[0021] In this way, the media data payload can be returned from the first file store or the object store as appropriate.

[0022] The index may be used to find the byte range of the required media data in the object store file, and range header requests can then be used to retrieve only the required media data payload, which avoids it being necessary to read the whole object storage file.

[0023] The method may further comprise, if the media data payload is located in the object store, the step of retrieving the object storage file in which the media data payload is contained from the object store. The media data payload can then be retrieved from the retrieved object storage file, rather than only the media data payload being retrieved directly from the object store. In this case, the method may further comprise the step of storing the object storage file in the first file store. It will often be the case that media data payloads streamed closely to a requested media data payload will also be requested, and these will often be in the same object storage file so can be more quickly retrieved. In accordance with a third aspect of the invention there is provided a computing device for storing media data, arranged to perform any of the methods of storing media data described above.

[0024] The computing device may be further arranged to perform any of the method of retrieving media data described above.

[0025] The computing device may be a web server.

[0026] In accordance with a fourth aspect of the invention there is provided a computer program product arranged, when executed on a computing system comprising one or more processors and memory, to cause the computing system to perform any of the methods of storing media data described above.

[0027] The computer program product may be further arranged to cause the computing system to perform any of the method of retrieving media data described above.

[0028] It will of course be appreciated that features described in relation to one aspect of the present invention may be incorporated into other aspects of the present invention. For example, the method of the invention may incorporate any of the features described with reference to the apparatus of the invention and vice versa.

[0029] Description of the Drawings

[0030] Embodiments of the present invention will now be described byway of example only with reference to the accompanying schematic drawings of which:

[0031] Figure 1 is a schematic diagram of media data storage system in accordance with a first embodiment of the invention;

[0032] Figure 2 is a schematic diagram of the Redis file store of the media data storage system of Figure 1 ;

[0033] Figure 3 is a schematic diagram of the object store of the media data storage system of Figure 1 ; Figure 4 is a flow chart showing the operation of the media data storage system of Figure 1 when storing media data payloads;

[0034] Figure 5 is a flow chart showing the operation of the media data storage system of Figure 1 when retrieving media data payloads;

[0035] Figure 6 is a flow chart showing part of the operation of a media data storage system in accordance with another embodiment of the invention when storing media data payloads;

[0036] Figure 7 is a flow chart showing part of the operation of a media data storage system in accordance with another embodiment of the invention when retrieving media data payloads; and

[0037] Figure 8 is a schematic diagram of a computing device accordingto an embodiment of the invention.

[0038] Detailed Description

[0039] A media data storage system in accordance with a first embodiment of the invention is now described with reference to Figures 1 to 5. Figure 1 is a schematic diagram of media data storage system 100. The media data storage system 100 comprises a media web server 101 , which is in communication with a Redis file store 102 and an object store 103, as described in more detail below.

[0040] The media web server 101 is also in communication with a remote media system 110, from which it receives a stream of media data payloads, in particular frames of video. The remote media system 110 may be, for example, a media recording system recording video of a live sporting event, with the stream of media data payloads being the video recorded by one or more cameras of the live sporting event. The media data payloads are stored by the media web server 101 as described in detail below.

[0041] The media web server 101 is also in communication with a remote media editing system 111 , which requests media data payloads from the media web server 101. These may be, for example, frames of video from the stream provided by the remote media system 110, so that the remote media editing system 111 can edit the video to provide coverage of the live sporting event, while it is in process.

[0042] The Redis file store 102 is shown in more detail in Figure 2. A Redis file store provides storage for files in RAM, giving particularly low latency storage and retrieval of data files, and allowing stored filed to be and modified after they have been stored. The Redis file store 102 is provided by, and using the memory of, the media web server 101 , but in other embodiments the Redis file store 102 may be provided separately from the media web server 101 . In other embodiments an alternative type of file store may be used, for example a different RAM-based file store, such as memcahed, or a file store provided on SSD or other non-volatile storage.

[0043] The Redis file store 102 has stored on it an index file 115, which indicates the location of media data payloads as described in more detail below. The Redis file store 102 also has stored on media data payloads 116a, 116b and 116c, which have been previously received from the remote media system 110, again as described in more detail below. While only three media data payloads are shown, it will be appreciated that in practice it is likely that a much larger number will be stored. In other embodiments, the index file 115 and media data payloads 116a, 116b and 116c may be stored in separate file stores.

[0044] The object store 103 is shown in more detail in Figure 3. The object store 103 allows files to be stored cheaply and efficiently compared to a file store such as the Redis file store 102, but does not allow files to be modified subsequent to beingstored.

[0045] The object store 103 has stored on it a media file 117, which contains media data payloads 118a, 118b, 118c, 118d and 118e which have been previously received from the remote media system 110, again as described in more detail below. Similarly to the Redis file store 102, while only five media data payloads are shown, it will be appreciated that in practice it is likely that a much larger number will be contained within the media file 117. The operation of the media data storage system 100 to store media data in accordance with the first embodiment is now described, with reference to the flowchart of Figure 4.

[0046] As discussed above, the media web server 101 receives media data payload from the stream of media data payloads from the remote media system 110 (step 401 ). The media web server 101 then checked if a threshold has been reached (step 402). In the present embodiment, the threshold is whether the total size of media data payloads stored in the Redis file store 102 is above a threshold. However, as discussed below, in other embodiments other thresholds may be used.

[0047] If the threshold has not been reached, the media data payload is stored in the Redis file store 102 (step 403). The location of the media data payload in the Redis file store 102 is then recorded in the index file 115 in the Redis file store 102 (step 404). The process is then repeated, i.e. the media web server 101 waits to receive another media data payload from the stream of media data payloads (step 401 again).

[0048] If on the other hand the threshold has been reached, the media web server 101 creates a media file containingthe media data payloads stored in the Redis file store 102, as well as the newly received media data payload (step 405). This media file is then stored in the object store 103 (step 406). The locations of the media data payloads contained in the media file are then updated in the indexfile 115 (step 407), i.e. to indicate their locations in the media file in the object store 103, rather than in the Redis file store 102. (The location of the newly received media data payload in the media file in the object store 103 is also added to the index file 115.) The media data payloads are then deleted from the Redis file store 102 (step 408).

[0049] As with when the threshold has not been reached, the process is then repeated, i.e. the media web server 101 waits to receive another media data payload from the stream of media data payloads (step 401 again).

[0050] Thus, as can be seen the media web server 101 receives media data payloads and stores them in the Redis file store 102, and then when sufficient have been received that the threshold has been reached, it effectively moves them into a media file in the object store 103. It then continues to receive further media data payloads and again store them in the Redis file store 102, and again move them into a media file in the object store 103, and so on.

[0051] While in the present embodiment the threshold used is based on the total size of media data payloads stored in the Redis file store 102, in other embodiments other thresholds may be used. For example, the threshold may be based upon the number of media data payload stored in the Redis file store 102, or that a period of time has passed since the last media file was created and stored in the object store 103.

[0052] The operation of the media data storage system 100 to retrieve media data in accordance with the first embodiment is now described, with reference to the flowchart of Figure 5.

[0053] First, the media web server 101 receives a request for a media data payload from the remote media editing system 111 (step 501 ). The media web server 101 then obtains the location of the requested media data payload from the index file 115 (step 502), and determines whether it is located in the Redis file store 102 or the object store 103 (step 503).

[0054] If the requested media data payload is located in the Redis file store 102, the media web server 101 retrieves it from the Redis file store 102 (step 504), and returns it to the remote media editing system 111 (step 507).

[0055] If on the other hand the requested media data payload is located in the object store 103, the media web server 101 retrieves the media file containing it from the object store 103 (step 505). It then retrieves the requested media data payload from the media file (step 506), and returns it to the remote media editing system 111 (step 507). In other embodiments, the media file is partially read, and only the data correspondingto the requested media data payload is retrieved from the media file in the object store 103 and returned to remote media editing system 111.

[0056] Thus, as can be seen in either case the media web server 101 returns the requested media data payload to the remote media editing system 111. From the point of view of the remote media editing system 111 , the process is the same, i.e. it does not know whether the requested media data payload was stored in the Redis file store 102 or the object store 103. By combining the use of the Redis file store 102 and the object store 103, the media web server 101 is able to return media data payloads from a stream while it is still being streamed, but is also able to enjoy the storage advantages provided by using the object store 103.

[0057] A part of the operation of the media data storage system 100 to store media data in accordance with another embodiment is now described, with reference to the flowchart of Figure 6.

[0058] The operation shown in Figure 6 replaces the operation of the media data storage system 100 of the first embodiment when the threshold has been reached, i.e. step 405 to step 408 of Figure 4.

[0059] First, the media web server 101 retrieves all the media files stored so far in the object store 103 from the object store 103 (step 601 ). The media web server 101 then extracts all the media data payloads from the media files (step 602). The media web server 101 then creates a media file containing the media data payloads (step 603), i.e. the media data payloads from the retrieved media files, and also the media data payloads stored in the Redis file store 102.

[0060] The media file is then stored in the object store 103 (step 604), and the locations of the media data payloads it contains are updated in the index file 115 (step 605). The media data payloads stored in the Redis file store 102 are then deleted (step 606) from the Redis file store 102, and the media files retrieved from the object store 103 are deleted from the object store 103 (step 607).

[0061] In this way, a single media file containing streamed media data payloads is stored in the object store 103, which is replaced to add the newly streamed media data payload each time the threshold is reached.

[0062] A part of the operation of the media data storage system 100 to retrieve media data in accordance with another embodiment is now described, with reference to the flowchart of Figure 7. The operation shown in Figure 7 replaces the operation of the media data storage system 100 of the first embodiment when the requested media data payload is stored in the object store 103, i.e. step 505 and step 506 of Figure 5.

[0063] First, the media web server 101 checks if the media file containingthe requested media data payload is stored in the Redis file store 102 (step 701 ), which can occur as discussed below. If not, then as in the first embodiment, the media file is retrieved from the object store 103 (step 702). However, the media file is then stored in the Redis file store 102 (step 703). The requested media data payload is then retrieved from the media file (step 704), and returned to the remote media editing system 111 (step 705).

[0064] If on the other hand the media file is stored in the Redis file store 102, the media file is retrieved from the Redis file store 102 (step 706), rather than from the object store 103. Again, the requested media data payload is then retrieved from the media file (step 704), and returned to the remote media editing system 111 (step 705).

[0065] Obtaining the media data payload from the Redis file store 102 will generally be quicker than obtaining it from the object store 103, due to the better performance characteristics of the Redis file store 102. The media file will be stored in the Redis file store 102 if, on a previous occasion, a media data payload from the media file has been requested, leadingto the media file being retrieved from the object store 103 and stored in the Redis file store 102. The previously requested media data payload may not be the same as the currently requested media data payload, but may be one that is consecutive to or nearby the currently requested media data payload. As it is common for such media data payloads to be requested close to one another, this can provide improved performance in commonly occurring situations.

[0066] Embodiments of the invention include at least some of the methods described above performed on a computing device, such as the computing device 1100 shown in Figure 8. The computing device 1100 comprises a data interface 1101 , through which data can be sent or received, for example over a network. The computing device 1100 further comprises a processor 1102 in communication with the data interface 1101 , and memory 1103 in communication with the processor 1102. In this way, the computing device 1100 can receive data, such as media data, via the data interface 1101 , and the processor 1102 can store the received data in the memory 1103, and process it so as to perform the methods described herein. At least some of the methods described herein may be performed by a computing system comprising one or more such computing devices 1100.

[0067] Each device, module, component, machine or function as described in relation to any of the examples described herein may comprise a processor and / or processing system or may be comprised in apparatus comprising a processor and / or processing system. One or more aspects of the embodiments described herein comprise processes performed by apparatus. In some examples, the apparatus comprises one or more processing systems or processors configured to carry out these processes. In this regard, embodiments may be implemented at least in part by computer software stored in (non-transitory) memory and executable by the processor, or by hardware, or by a combination of tangibly stored software and hardware (and tangibly stored firmware). Embodiments also extend to computer programs, particularly computer programs on or in a carrier, adapted for puttingthe above-described embodiments into practice. The program may be in the form of non-transitory source code, object code, or in any other non-transitory form suitable for use in the implementation of processes accordingto embodiments. The carrier may be any entity or device capable of carrying the program, such as a RAM, a ROM, or an optical memory device, etc.

[0068] While the present invention has been described and illustrated with reference to particular embodiments, it will be appreciated by those of ordinary skill in the art that the invention lends itself to many different variations not specifically illustrated herein.

[0069] Where in the foregoing description, integers or elements are mentioned which have known, obvious or foreseeable equivalents, then such equivalents are herein incorporated as if individually set forth. Reference should be made to the claims for determining the true scope of the present invention, which should be construed so as to encompass any such equivalents. It will also be appreciated by the reader that integers or features of the invention that are described as preferable, advantageous, convenient or the like are optional and do not limit the scope of the independent claims. Moreover, it is to be understood that such optional integers or features, whilst of possible benefit in some embodiments of the invention, may not be desirable, and may therefore be absent, in other embodiments.

Claims

Claims1 . A computer-implemented method of storing media data, comprisin the steps of: receiving a stream of media data payloads; for each media data payload in the stream: storing the media data payload in a first file store; and recording in an index the location of the media data payload in the first file store; in response to the media data payloads stored in the first file store exceeding a threshold: creating an object storage file comprising a plurality of the media data payloads stored in the first file store; storing the object storage file in an object store; and updating, in the index, the location of the plurality of media data payloads; and deleting the plurality of media data payloads from the first file store.

2. The method of claim 1 , wherein the first file store is a RAM storage system.

3. The method of claim 1 or claim 2, wherein the index is stored in the first file store.

4. The method of any preceding claim, wherein the object storage file is created in response to the total size of the media data payloads stored in the first file store exceeding a threshold.

5. The method of any of claims 1 to 3, wherein the object storage file is created in response to a predetermined interval of time having passed.

6. The method of any preceding claim, wherein the steps performed in response to the media data payloads stored in the first file store exceeding the threshold are performed each time the threshold exceeded.

7. The method of any of claims 1 to 5, further comprising, in response to the media data payloads stored in the first file store exceeding the threshold, the steps of: creating a second object storage file comprising the plurality of the media data payloads in the first object storage file stored in the object store and the plurality of the media data payloads stored in the first file store; storing the second object storage file in the object store; updating, in the index, the location of the plurality of media data payloads in the second object storage file; deleting the first object storage file from the object store; and deleting the plurality of media data payloads from the first file store.

8. The method of any preceding claim, wherein the media data payloads comprise: an image; a frame; a Group of Pictures; an audio sample; an audio frame; and an audio clip.

9. A computer-implemented method of retrieving media data, wherein the media data has been stored in accordance with the method of any preceding claim, the method comprising the steps of: receiving a request for a media data payload; determiningfrom the index the location of the media data payload; if the media data payload is located in the first file store, retrieving the media data payload from the first file store; if the media data payload is located in the object store, retrieving the media data payload from the object store; and returningthe media data payload.

10. The method of claim 8, further comprising, if the media data payload is located in the object store, the step of retrieving the object storage file in which the media data payload is contained from the object store.11 . The method of claim 9, further comprising the step of storing the object storage file in the first file store.

12. A computing device for storing media data, arranged to perform the method of any of claims 1 to 8.

13. The computing device of claim 12, further arranged to perform the method of claim 9 or 10.

14. The computing device of claim 12 or 13, wherein the computing device is a web server.

15. A computer program product arranged, when executed on a computing system comprising one or more processors and memory, to cause the computing system to perform the method of any of claims 1 to 8.

16. The computer program product of claim 15, further arranged to cause the computing system to perform the method of claim 9 or 10.

Images