Storing and retrieving media records in object storage

Abstract

The present invention relates to storing and retrieving media records in an object store. A first method for storing media records includes the following steps performed at an ingest component of a system: assigning a record ID to a media record; storing the media data in one or more data objects in the object store, each data object carrying the record ID; storing media metadata in an attribute object carrying the record ID; computing a hash of the metadata; and storing the hash, record ID, record interval, and record source in an index object. The first method further includes concatenating the index object with an existing index object using a maintenance component. A second method is applicable to retrieving media records stored in an object store using the first method. A third method is applicable to performing maintenance on the object store by concatenating a particular group of index objects.

Description

Technical Field

[0001] The present disclosure relates to digital data storage technology. Specifically, the present disclosure proposes a method and apparatus for storing and retrieving media records in an object storage and for performing maintenance on an object storage that has stored media records. Background Art

[0002] In addition to the classic file system, where files are organized in a directory structure (folder tree), non-hierarchical data storage architectures are gaining interest in various industry branches and are increasingly used in applications where large amounts of data are stored in network (cloud) storage. An architecture that is well suited for storing and accessing large amounts of unstructured data over a network connection is object storage, where data is managed as objects without a file hierarchy or a block hierarchy. If object storage is implemented on a storage drive, the lower protocol layers of the storage drive are abstracted from the users of the object storage (e.g., human users, software applications, and connected client computers).

[0003] In video surveillance and similar use cases, not only is storage desirable for secure and reliable archiving, but search tools are also needed to efficiently retrieve stored data items. A useful search tool should combine good precision and recall with acceptable execution time. Many available storage solutions achieve this by indexing or tagging the data before searching. Indexing and tagging are computationally expensive preprocessing steps that can add a lot of overhead to the data, and they are often difficult to parallelize.

[0004] US10210191B2 discloses a method for accessing data stored using object storage technology. The storage operation includes: creating an object in the object storage using a file system; obtaining an I-node identifier (ID) of an index node (I-node) associated with the object; creating an object ID for the object, wherein the object ID is unique from any other object ID in the object storage; merging the I-node ID into the object ID to create a merged object ID including a complete I-node ID, which can be extracted from the merged object ID; and modifying the I-node to include at least the merged object ID. The access operation includes: extracting a complete I-node ID from an object ID associated with an object stored in the object storage using a file system; accessing an I-node associated with the I-node ID and determining an object ID stored therein; determining whether the object ID from the I-node matches the object ID associated with the object from which the I-node ID was extracted; and performing an operation on the object. It is suggested in US10210191B2 that I-node-based access can be more easily upgraded than the direct option of using object IDs directly. Based on the prior art, the present disclosure proposes an alternative to this storage / access arrangement and similar techniques.

[0005] Focusing more on the cognitive aspect, US20210112154A1 proposes a software application for analyzing scenes viewed by a mobile phone's camera. Image processing tasks applied to the scene and assisted by additional information such as voice commands can be performed locally in the phone, while related indexing tasks can be delegated to networked processing resources. Specifically, the purpose of local processing can be to identify groups of pixels ("clumps") that potentially depict recognizable visual objects or features in the camera's field of view. Each pixel block carries a "symbol", and if the block can be visually identified as an object, the "block ID" can also be used as an "object ID". Further, the identified visual objects can be annotated with a "key vector" in a way that "each data object known to the local device is instantiated via data in the cloud", and the "key vector" constitutes a group of attributes stored in a cloud-based registry in a "base cloud-based form". Attributes may include recording time and recording source. Pixel data is stored in the phone's own memory. Summary of the invention

[0006] One object of the present disclosure is to enable methods and apparatus to be used for storing media records in an object store. In such a method and apparatus, it is desirable if the media records can be retrieved once stored or only require a limited amount of additional processing. Another object is to enable methods and apparatus to be used for retrieving related media records from among the media records stored in the object store. The efficiency and performance of such methods and apparatus can be improved if the related media records can be located without making unnecessary calls to the object store and the amount of communication in and out of the object store is limited. Another object is to enable methods and apparatus to be used for performing maintenance on the object store containing the stored media records. The maintenance should facilitate subsequent searches for related media records and / or facilitate the cleanup of obsolete media records. The overall goal is to achieve this using general non-customized technology (e.g., object storage solutions) to the greatest extent possible and leveraging functionality already available in the technology.

[0007] At least some of these objects are achieved by the invention as defined by the independent claims.The dependent claims relate to advantageous embodiments of the invention.

[0008] In a first aspect of the present invention, a method for storing a media record is provided, the media record comprising media data associated with metadata. The method comprises: assigning a record identifier ID to the media record; storing the media data in one or more data objects in an object store, wherein each data object carries the record ID; storing the metadata in a property object in the object store, wherein the property object carries the record ID; computing a hash of the metadata; storing the hash, the record ID, the record interval, and the record source of the media record in an index object in the object store; and concatenating the index object with an existing index object.

[0009] Concatenating the index object with existing index objects will tend to reduce the total number of index objects in the object store. Accordingly, this limits the number of index objects that need to be examined when performing an exhaustive search for media records. As will be explained in detail below, the inclusion of a hash of the metadata in the index object enables searches to be performed with the support of a local cache, which helps to reduce the amount of communication in and out of the object store. Preferably, the concatenation step is performed on a component that is separate from one or more components that perform the preceding steps. For example, the concatenation step can be delegated to a maintenance component, while the preceding steps can be delegated to an ingestion component at the same or different location. The proposed separation of responsibilities means that the ingestion component can be optimized to conveniently and quickly store new media records, while the maintenance component can operate independently. In addition, because the maintenance component is not subject to any mandatory execution time constraints, it can be scheduled to execute on shared processing resources during periods of low computational load.

[0010] In a second aspect of the invention, a method of retrieving a media record comprising media data associated with metadata is provided, wherein the method is implemented in a search component that accesses an object store and a local access cache. It will be appreciated that local access is effectively unlimited in capacity and / or speed and therefore does not significantly increase the overall execution time. Access to an object store may be more limited in the sense that exchanging data with the object store implies cost, delay, or some degree of congestion, as is sometimes the case with remote or networked access.

[0011] The method according to the second aspect includes: receiving a search query; identifying matching media records that match the search query based on one or more index objects in an object store, wherein at least one of the index objects references multiple media records and stores a hash of a record identifier ID and metadata of the media record for each media record; determining whether the hash of the matching media record is consistent with the hash of any metadata in the cache (i.e., any metadata item corresponding to the metadata of a media record). The next step depends on the result of the determination. In the case of consistency, the metadata of the matching media record is obtained from the cache. In the case of inconsistency, the attribute object of the matching media record is retrieved from the object store, and the metadata of the matching media record is obtained from the retrieved attribute object. Regardless of whether the hash is found to be consistent or inconsistent, one or more data objects of the matching media record are retrieved from the object store.

[0012] Because at least some of the index objects each reference multiple media records (e.g., as a result of concatenation), the identification step can be performed by examining a limited number of index objects. Since the index objects store hashes of the metadata of the media records, unnecessary calls to the object store to retrieve the property objects are conveniently avoided.

[0013] In a third aspect of the invention, a method is provided for performing maintenance on an object store storing a plurality of media records, each media record comprising media data associated with metadata, the object store comprising one or more index objects, wherein each index object references at least one media record and stores a record identifier ID and a hash of the metadata of the media record for the media record. The method comprises: locating in the object store two or more index objects that reference media records having corresponding recording intervals contained in a common time period; and concatenating the index objects.

[0014] The concatenation of two or more index objects will tend to reduce the total number of index objects in the object store. Accordingly, this limits the number of index objects that need to be examined when performing an exhaustive search for media records. It will be understood that a "time period" in the sense of the claims is longer than a "recording interval"; while a "recording interval" may represent a suitable granularity for managing records (e.g., minutes, hours), a "time period" may correspond to a granularity suitable for managing the cleanup of batches of obsolete media records (e.g., hours, days, weeks). Therefore, the condition that the index objects to be concatenated should belong to a common time period (e.g., the same recording date) will ensure that no index object references records from two different time periods (e.g., different dates), so that the desired cleanup of all media records starting from a specific date can be achieved without complexity.

[0015] A fourth aspect of the present invention relates to a system comprising an ingestion component and a maintenance component configured to perform a storage method.

[0016] A fifth aspect of the invention relates to a system comprising a search component configured to perform a retrieval method.

[0017] A sixth aspect of the present invention relates to a system comprising a maintenance component configured to perform a maintenance method.

[0018] A seventh aspect of the present invention relates to a system comprising an ingestion component configured to perform a storage method in addition to the concatenation step.

[0019] The invention further relates to a computer program comprising instructions for causing a computer or in particular the above-mentioned acquisition, maintenance and / or search components to perform the above-mentioned method or method steps. The computer program can be stored or distributed on a data carrier. As used herein, a "data carrier" can be a temporary data carrier such as a modulated electromagnetic wave or light wave, or a non-temporary data carrier. Non-temporary data carriers include volatile and non-volatile memories, such as permanent and non-permanent storage media of magnetic, optical or solid-state type. Such memories can be fixedly mounted or portable, still within the scope of a "data carrier".

[0020] The system and computer program according to the present invention generally share the effects and advantages of the first, second and third aspects outlined above, and they can be implemented with corresponding degrees of technical variation.

[0021] Generally, all terms used in the claims are to be interpreted according to their ordinary meaning in the technical field, unless otherwise explicitly defined herein. All references to "a / an / the element, device, component, means, step, etc." should be interpreted openly as referring to at least one instance of an element, device, component, means, step, etc., unless otherwise explicitly stated. The steps of any method disclosed herein do not have to be performed in the exact order described, unless explicitly stated. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Various aspects and embodiments will now be described by way of example with reference to the accompanying drawings, in which:

[0023] Figure 1 A system having a component for storing and retrieving media records in an object store and another component for performing maintenance on the object store according to an embodiment of the present invention is shown;

[0024] Figures 2 to 4 is a flowchart of a method according to an embodiment of the present invention; and

[0025] 5 and 6 illustrate example index objects for storing and retrieving media records in an object store. DETAILED DESCRIPTION

[0026] Aspects of the present disclosure will now be described more fully below with reference to the accompanying drawings, in which certain embodiments of the present invention are shown. However, these aspects may be embodied in a variety of different forms and should not be construed as limiting; rather, these embodiments are provided by way of example so that the present disclosure will be thorough and complete and fully convey the scope of all aspects of the present invention to those skilled in the art. Throughout the description, the same reference numerals refer to the same elements.

[0027] Figure 1A system is shown having components 100, 110, 120, 130, 190 for storing and retrieving media records 150 in an object store 123 and another component for performing maintenance on the object store 123. The media record 150 consists of media data 151 and metadata 152, wherein the metadata 152 includes information of a technical or administrative nature that is not necessarily presented to a user during playback of the media record 150 but can quantitatively and qualitatively affect how the presentation is performed. The metadata 152 can also be used for additional content-related data that cannot be absorbed by the specified encoding format of the media data 151. The media record 150 can be further annotated with a recording interval 153 and a recording source 154. With reference to a reference clock (e.g., network time GMT), the recording interval 153 indicates when the acquisition of the media record 150 starts and ends. The recording interval 153 can be represented, for example, as {start, end}, {start, duration} or {duration, end}, wherein the entries start and end refer to the reference clock. With the understanding that a recording source 154 generates at most one media recording 150 at a given point in time, the combination of a recording interval 153 and a recording source 154 may be used as a search query to identify related media recordings 150 .

[0028] exist Figure 1 In the example shown, media record 150 is a video record, where media data 151 and metadata 152 include video data associated with video metadata. Figure 1 The figure shows a camera 130 suitable for acquiring such a video recording. The camera 130 may be a digital video camera, a digital monitoring camera, or a digital surveillance camera. Metadata 152 may relate to technical properties of the video recording, such as resolution, frames per second, encoding format, etc. The metadata may be relied upon to correctly and efficiently present (play back) the video recording.

[0029] Ingest component 190 is configured to store media records in object store 123 as data objects 160, property objects 170, and index objects 180. In some embodiments, data objects 160, property objects 170, and index objects 180 may be instances of the same object type for which object store 123 is configured. In other embodiments, data objects 160, property objects 170, and index objects 180 correspond to two or more different object types (e.g., differing in their specified payload sizes or associated operations).

[0030] The search component 110 is configured to process a search query and retrieve relevant media records 150 based thereon. To this end, the search component 110 includes a processing circuit 112 and a memory as a cache 111. The processing circuit 112 may be a local circuit or a distributed circuit having one or more processor cores. Access to the cache 111 is local, since no perceptible speed, cost or capacity constraints apply.

[0031] The system further includes a storage component 120 (or server) that includes a memory 121 and a processing circuit 122 and implements an object store 123. The object store 123 is operable to accept data from the ingest component 190. The object store 123 is further operable to accept and process search requests from the search component 110 and transmit search results in response. The search component 110 can also retrieve objects from the object store 123, i.e., transmit them from the storage component 120 to the search component 110. Figure 1 As suggested by the intermediate communication network 140 in, when the search component 110 remotely accesses the object store 123, the number of calls and the amount of communication between these components may result in costs in terms of latency, marginal contribution to network congestion, and / or processing work. In the present disclosure, the object store 123 is an instance (warehouse) of storage technology based on the object storage architecture. Optionally, the object store 123 can comply with standards such as INCITS 458-2011 (Information Technology - SCSI Object-Based Storage Device Commands-2; OSD-2) and INCITS 400-2004 (Information Technology - SCSI Object-Based Storage Device Commands; OSD) issued by the International Committee for Information Technology Standards. Amazon Simple Storage Service TM (Amazon S3) and Microsoft Azure Blob Storage TM is an example of a cloud-based object storage service.

[0032] A maintenance component 124 is further provided for performing maintenance (including index object concatenation) on the object store 123. The maintenance component 124 may correspond to dedicated hardware (not shown) internal or external to the storage component 120, or it may be implemented as software code executed on the processing circuit 122 of the storage component 120 or on the processing circuit 122 external to the storage component 120. The maintenance component 124 may optionally have further responsibilities related to checking the data structures currently stored in the object store 123 and the prevention, discovery and correction of faults in these data structures. It should be understood that the low-level aspects of the storage driver implemented with the object store 123 may be subject to separate maintenance.

[0033] The described system components may exchange data, messages, and / or instructions between them at run time. The components need not be under common control or ownership; for example, object store 123 may be hosted by a different organization (e.g., a cloud service provider) than the organization providing media records 150 (e.g., a security company), which in turn may be different from the party that typically searches for and views media records 150 (e.g., a customer of the security company). Thus, it is not necessary for the components to be located in physical proximity to one another. To illustrate the implementer's flexible choice between local and distributed deployments, Figure 1 Some pairs of components are shown connected via a communication network 140 (or multiple communication networks 140), each of which may be a local area network or a wide area network, while other components have direct connections.

[0034] In some embodiments, the source of media recording 150 (e.g., camera 130, optionally equipped with post-processing components) is co-located with storage component 120 and maintenance component 124. Ingestion component 190 can be co-located with or implemented in storage component 120, or ingestion component 190 can be a separate component. Search component 110 (e.g., API) can be co-located with storage component 120, or search component 120 can be a separate component. In some embodiments, a user (e.g., human user, software application, connected client computer) directly accesses search component 110; in other embodiments, such as Figure 1 As shown in , the user interacts with the search component 110 via a user interface 101 (e.g., a web interface) executed on the user-side component 100. This is advantageous because, on the one hand, the communication volume between the storage component 120 and the search component 110 will generally be greater than the communication volume between the search component 110 and the user-side component 100, and on the other hand, the search component 110 can be arranged closer to the storage component 120.

[0035] Figure 2 is a flow chart of a method 200 for storing a media record 150, and will now continue to refer to Figure 1 The steps of method 200 may be distributed in different ways. In one embodiment, all steps are performed by the same component. In another embodiment, the steps of distributing 210, storing 212, 214, 218, and calculating 216 are performed in a first component (e.g., Figure 1 190), while the concatenation step 220 is performed at a separate second component (e.g., maintenance component 124). This alternative embodiment can speed up storage operations and / or can facilitate a more economical implementation in which a major portion of the processing is postponed to a period when the overall processing load is lower. In yet another embodiment, the initial steps 210, 212, 214, 216, 218 are as follows: Figure 2 , but the concatenation step 220 is replaced by a step of initiating concatenation, and the method 200 is performed entirely by the same component. Initiating concatenation may include issuing a command to the maintenance component 124 or leaving data stored in the object store 124 under conditions that indicate to the maintenance component 124 that concatenation should be performed. The indication may be implicit, for example, the maintenance component 124 may be configured to search for index objects 180 below a threshold size in the object store 123, or to periodically count the total number of index objects 180 below a threshold size. In either case, the creation of a new index object 180 by the ingest component 190 will be noticed by the maintenance component 124, and therefore, may be understood as initiating concatenation. The indication may also be explicit, for example, the index object carries a flag indicating "not yet concatenated."

[0036] In a first step 210 of method 200, a recording identifier (ID) is assigned to media recording 150. Figure 1 5 and 6, the record ID may be any numeric or alphanumeric string, such as a sequence number of suitable length. Preferably, the record ID is unique to the media record 150 as long as the media record 150 is stored in the object store 124. The record ID of a deleted, outdated media record 150 may be reassigned to a new media record 150. Note that the record ID is independent of any object identifier that the object store 123 may assign to each stored object.

[0037] In a second step 212, the media data 151 of the media record 150 is stored in one or more data objects 160 in the object store 123, wherein each data object 160 carries a record ID. The record ID may be included in the object identifier of the data object 160 or in the object metadata item. Functionally, the record ID is carried by the data object 160 in a manner that the object store 123 allows efficient retrieval (e.g., by a dedicated request) of all data objects 160 having a specified record ID. The data object 160 may be used to store only the media data 151, or it may contain data in addition to the media data 151. In some embodiments, the data objects 160 may be stored in the object store 123 gradually, for example, as the media record 150 is generated at predefined time intervals or when the media data cache exceeds a predefined level. In other embodiments, the method 200 is not performed until the media record 150 is completed. In this case, the number of data objects 160 to be used may be determined by the maximum allowed size of objects in the object store 123.

[0038] In a third step 214 of method 200, metadata 152 for a media record 150 is stored in a property object 170. Metadata 152 for one media record 150 is sometimes referred to in this disclosure as a "metadata item." As explained under the second step 212, the property object 170 carries the record ID in a manner that the object store 123 allows efficient retrieval of a property object 170 with a specified record ID. Step 214 may include forming 214.1 a new property object 170 and storing the metadata 152 therein. Alternatively, the metadata 152 may be stored in an existing metadata object 152, which is modified (edited) 214.2. Further alternatively, an existing property object 170 containing metadata for one or more previously stored media records is replaced with a new property object 170 containing the metadata for the previously stored media records as well as the metadata for the media record 150 for which method 200 is being executed. Particularly if the metadata 152 is not expected to change after step 214 , a preferred option is to store the metadata 152 in the newly formed 214 . 1 property objects 170 , whereby each property object 170 in the object store 123 contains metadata 152 for only a single media record 150 .

[0039] In a fourth step 216, a hash 171 of the metadata 152 is calculated. This includes mapping the metadata 152 through a hash function h. The hash function or one-way function should be predefined (e.g., it should be reproducible) so that the hash can be regenerated at the search component 110. The hash function can be an encrypted or non-encrypted hash function, such as SHA-256, SHA3-512, and RSA-1024. Because the hash function is not used for security-related purposes within the method 200, a relatively simple hash function can be used as long as the probability of a collision is acceptably small. Hash collisions may cause the search component 110 to incorrectly rely on cached metadata items, but will not expose the metadata to unauthorized parties.

[0040] In different embodiments, step 216 may include hashing the entire metadata 152 or hashing a subset of the metadata 152 extracted according to predefined deterministic rules. The hash 171 may be based on a combination of the metadata 152 and a cryptographic salt. Regardless of these choices, it is advantageous that the hash 171 of the metadata 152 is independent of the record ID, i.e., the record ID preferably should not form part of the input data of the hash function h. In this way, when the hash indicates that the metadata is consistent with the metadata of the earlier media record 150, the search component 110 will be able to abandon the retrieval of the metadata of the later media record from the object store 123.

[0041] In a fifth step 218 of method 200, hash 171, recording ID, and indication of recording interval 153 and recording source 154 of media recording 150 are stored in index object 180 in object store 123. After fifth step 218, execution of method 200 will generate the following: Figure 1 The data object 160, attribute object 170 and index object 180 are shown. More precisely, the hash 171, record ID and indication of the record interval 153 and record source 154 can be stored in:

[0042] - In the newly formed index object 180-2 of 218.1 (see Figure 6B , where ID9876 is the record ID of the media record 150 to be stored),

[0043] - Modify index object 180-1 in 218.2 (see Figure 5B , which shows that the information of media record ID 9876 is appended to Figure 5A ), or

[0044] - a replacement index object, the information is appended to the replacement index object, and the replacement index object replaces 218.3 an existing previously stored index object in the object store 123.

[0045] For purposes of illustration, it is assumed in this example that the hash 171, the record ID, and an indication of the record interval 153 and the record source 154 are stored in the newly formed index object 180-2. In this case, the newly formed index object 180-2 may optionally carry the record ID (i.e., in addition to containing the record ID). If the index object 180 carries the record ID, it will be possible to conveniently list multiple index objects by issuing a single list operation to the object store 123. Otherwise, if the index object 180 does not carry the record ID, then the same number of retrieval operations (e.g., get) as the index object may be required to read the corresponding record ID.

[0046] Then, in the sixth step 220 of the method 220, the newly formed index object 180-2 is concatenated with the existing index object 180-1. For illustration, Fig. 6A , Figure 6B and Figure 6CA subset of the content of the object store 123 at three consecutive points in time is shown. The content includes an existing index object 180-1, which stores information of media record IDs 9874 and 9875; a newly formed index object 180-2, which stores information of media record ID 9876 to be stored by executing the storage method 200; and an index object 180-3 resulting from the concatenation of the existing and newly formed index objects 180-1 and 180-2. Because the concatenated index object 180-3 contains information of all three media record IDs 9874, ID 9875, and ID 9876, the search component 110 will be able to exhaustively evaluate the search query by examining one less index object. As a result of storing an earlier media record 150, the existing index object 180-1 may have been stored in the object store 123. In a specific embodiment of the method 200, the sixth step 220 is performed at a predefined interval (e.g., every n minutes, every n hours, at the end of each business day) and / or when affordable processing resources are available. The sixth step 220 may be configured such that two index objects are concatenated whenever they reference a media record 150 whose recording interval 153 is contained within a common time period. Thus, after the sixth step 220 has been performed a sufficient number of times, a single index object 180 will reference all media records 150 from that time period, which simplifies subsequent searches. Recall that a "time period" (e.g., hour, day, week) is preferably configured to be longer than a typical "recording interval" (e.g., minute, hour). The concatenation of two index objects IO1, IO2 may include

[0047] - Append the contents of IO1 to IO2 and delete IO1 from object storage 123,

[0048] - append the contents of IO2 to IO1 and remove IO2 from object store 123, or

[0049] - A new index object IO3 is formed which stores the combined contents of IO1 and IO2 and replaces IO1 and IO2 in the object store 123 .

[0050] As initially stated, and as one skilled in the art may appreciate after studying this disclosure, method 200 may be performed using a different order of steps. For example, step 218 may be performed before step 214, and step 216 may be performed before step 214.

[0051] See also Figure 3, a method 300 for retrieving a media record 150 will now be described. The method 300 is adapted to be performed by a search component 110 that has access to an object store 123 and has local access to a cache 111. The method 300 may be performed at a point in time when the cache 111 contains metadata 152 for a plurality of media records 150. The metadata 152 in the cache may be a copy of metadata items of a property object 170 in the object store 123, e.g., as a byproduct of an earlier search or as a result of a dedicated storage of a snapshot of all metadata 152 in the object store 123. In various embodiments, the cache 111 may store free metadata items or property objects 170 containing such metadata items.

[0052] In a first step 310 of method 300, a search query is received. Search component 110 may receive a search query from a user (e.g., a human user, a software application, or a connected client computer). The allowed formats of the search query may vary between different embodiments, particularly as a result of a search request supported by object store 123. For example, the search query may reference a recording interval, an identifier or other attribute of a recording source, and / or a recording ID. Because metadata 152 for media recordings 150 is stored outside of index objects 180 in object store 123, it is not possible to process a search query formulated based on metadata 152 in this example.

[0053] If the object store 123 is capable of processing the search request to identify (e.g., by their object IDs) all index objects 180 (index objects 180 referencing media records 150 acquired by the record source 154 having the attributes specified by the search request and / or acquired in a recording interval 153 that overlaps with the interval specified by the search request), then the search query can be evaluated without retrieving the attribute objects 180 in the object store 123 (i.e., without transmitting the attribute objects 180 to the search component 110). This means that execution flow proceeds directly to step 314 of identifying matching media records 150, where the identification 314 is performed based on the index objects maintained by the object store 123. In an alternative embodiment adapted for use with an object store 123 lacking such search capabilities, the method 300 includes retrieving 312 the index objects 180 (e.g., transmitting the index objects 180 to a memory locally accessible to the search component 110), after which the identification 314 of matching media records 150 is performed based on the retrieved index objects 180. Either way, in step 314 a matching media record 150 is identified such that it matches the search query.

[0054] In view of the foregoing discussion of the storage method 200, it can be appreciated that at least one of the index objects 180 references multiple media records 150, which may be the result of concatenation 220. The index object 180 stores, for each media record 150, a record ID and a hash 171 of the metadata 152 of the media record 150. Optionally, the index object 180 further stores an indication of a recording interval 153 and a recording source 154 for each media record 150.

[0055] In the next step 316 of the retrieval method 300, it is determined whether the hash 171 of the matching media record 150 (i.e., the hash of the metadata 152 of the matching media record 150) is consistent with the hash 171 of any metadata 152 (any metadata item) in the cache 111. When the metadata item was stored in the cache 111, the hash 171 of the metadata item in the cache 111 may have been calculated and used the same hash function h as the hash function used in the storage method 200. If the hashes are consistent (e.g., they have the same value; from the "yes" branch of the block 316), the metadata 152 can be conveniently obtained 320.1 by locally accessing the cached property object 170. If the hashes are not consistent (from the "no" branch of the block 316), the corresponding property object 170 is retrieved 318 from the object store 123, and the metadata 152 is obtained 320.2 from the retrieved property object 170. In systems where search component 110 has non-local access to object store 123, such retrieval 318 may incur costs in terms of latency, marginal contribution to network congestion, and / or processing work.

[0056] Regardless of which option is used to obtain 320 metadata 152, metadata 152 may be shared 310 with the user from whom the search query was received. Optionally, subsequent steps 322, 324 of method 300 may be conditional in that they are only performed if the user approves after having had an opportunity to review metadata 152.

[0057] In the next step 322 of method 300, data objects 160 matching media record 150 are retrieved from object store 123. At this point, media data 151 and metadata 152 may be combined so that media record 150 is restored. In this way, media record 150 becomes available for various purposes such as playback 324. Playback 324 may be performed using a playback application or playback device (not shown) and based on metadata 152.

[0058] In some embodiments, retrieval 318 of property object 170 and / or retrieval 322 of one or more data objects 160 includes issuing a request to object store 123 using the record ID of the matching media record 150. As explained above, since property object 170 and data object 160 carry the record ID, such a request can be performed at a relatively low cost.

[0059] In a further development of the method 300, the step 318 of retrieving the property object 170 matching the media record 150 includes storing the retrieved property object 170 in the cache 111. In this way, the relatively expensive retrieval 318 of the property object 170 not only benefits the current search, but also any potential subsequent searches for matching media records 150 that are the same or have the same metadata 152.

[0060] Note that metadata hashing 216 is an optional step of storage method 200. In embodiments where storage method 200 does not include step 216, subsequent step 218 would include storing only the record ID, record interval 153, and record source 154 in index object 180. Accordingly, retrieval method 300 may be modified such that determination 316 is omitted and execution proceeds directly from step 314 to steps 318 and 320.2, i.e., always retrieving property object 170 from object store 123.

[0061] refer to Figure 4 , further provides a method 400 for performing maintenance on an object store 123 for storing a plurality of media records 150 of the described type. Thus, the maintenance method 400 is suitable for execution when the object store 123 stores one or more index objects 180, wherein each index object 180 references at least one media record 150 and stores, for the media record 150, a record ID and a hash 171 of metadata 152 of the media record 150.

[0062] The maintenance method 400 begins at step 410 of locating two or more index objects 180 that reference media records 150 for which corresponding recording intervals 153 are contained within a common time period. The review "time period" (e.g., hours, days, weeks) is preferably configured to be longer than a typical "recording interval" (e.g., minutes, hours).

[0063] If such an index object 180 is found in the object store 123, the method 400 will include concatenating the index object 180 into a smaller number of index objects 180, preferably a single index object. Figure 6B and Figure 6C The diagram illustrates how the contents of the object store 123 are changed due to concatenation. Reference is made to the above discussion of step 220 of the storage method 200 which may be similarly performed.

[0064] As mentioned above, maintenance method 400 may be performed at a low priority, e.g., as a background process, unless there is a specific reason to accelerate the concatenation process. Embodiments of method 400 may include performing maintenance on low-level aspects of a storage driver on which object store 123 is implemented, or focusing on the inspection of data structures currently stored in object store 123 and the prevention, discovery, and correction of faults in these data structures.

[0065] Aspects of the disclosure have mainly been described above with reference to a few embodiments. However, as readily appreciated by a person skilled in the art, other embodiments than the ones disclosed above are equally possible within the scope of the invention, as defined by the appended patent claims.

Claims

1. A method (300) of retrieving a video record (150) in a search component (110), the video record (150) comprising video data (151) associated with video metadata (152), wherein: The video metadata (152) relates to technical properties of the video record (150) relied upon during playback of the video record and includes one or more of resolution, frames per second and encoding format, and wherein the method is implemented in a search component (110) that locally accesses a cache (111) and accesses an object store (123) according to an object store architecture, the method comprising: receiving (310) a search query; identifying (314) matching video records (150) that match the search query based on one or more index objects (180) in the object store (123), wherein at least one of the one or more index objects (180) references a plurality of video records (150) and stores, for each video record (150), a record identifier, a hash (171) of video metadata (152) of the video record (150), and a record interval (153) and a record source (154) of the video record (150); determining (316) whether a hash (171) of video metadata (152) of the matching video record (150) is consistent with a hash (171) of any video metadata (152) in a property object (170) included in the cache (111), wherein the hash (171) of the video metadata (152) of the matching video record (150) and the hash (171) of each video metadata (152) in the cache (111) are calculated by a predefined hash function; In the event of consistency, obtaining (320.1) the video metadata (152) from the attribute object (170) in the cache (111); In case of inconsistency, retrieving (318) the property object (170) of the matching video record (150) from the object store (123), obtaining (320.2) the video metadata (152) from the retrieved property object (170), and storing the retrieved property object (170) in the cache (111), wherein the property object (170) stores the video metadata (152) and carries the record identifier; and One or more data objects (160) of the matching video record (150) are retrieved (322) from the object store (123).

2. The method according to claim 1, wherein: The search query specifies a recording time and / or a recording source.

3. The method according to claim 1, wherein: The at least one of the one or more index objects (180) referencing a plurality of video records is formed by a concatenation of the one or more index objects.

4. The method according to claim 1, further comprising: The one or more index objects (180) are retrieved (312) from the object store (123).

5. The method according to claim 1, further comprising: At the ingest component (190): assigning (210) the recording identifier to the video recording (150); storing (212) the video data (151) in one or more data objects (160) in the object store (123) according to an object storage architecture, wherein each data object (160) carries the record identifier; storing (214) the video metadata (152) in the property object (170) in the object store (123), wherein the property object (170) carries the record identifier; calculating (216) the hash (171) of the video metadata (152) using the predefined hash function; storing (218) the hash (171), the recording identifier, the recording interval (153), and the recording source (154) of the video recording (150) in an index object (180-2) in the object store (123); and At the maintenance part (124): The index object (180-2) is concatenated (220) with the existing index object (180-1).

6. The method according to claim 5, wherein: The existing index object (180-1) references at least one video record (150) having a recording interval (153) contained in the same time period as the video record (150) to be stored.

7. The method according to claim 5, further comprising forming (218.1) a new index object (180-2), in which the hash (171), the record identifier, the record interval (153) and the record source (154) are stored.

8. The method according to claim 5, wherein: The hash (171) of the video metadata (152) is independent of the recording identifier.

9. The method according to claim 5, wherein: The storing (214) of the video metadata (152) includes forming (214.1) a new property object.

10. A system configured to retrieve a video record (150), the video record (150) comprising video data (151) associated with video metadata (152), wherein: The video metadata (152) relates to technical properties of the video record (150) relied upon during playback of the video record and includes one or more of resolution, frames per second, and encoding format, the system comprising a search component (110) that locally accesses a cache (111) and accesses an object store (123) according to an object store architecture, the search component (110) being configured to perform the following steps: receiving (310) a search query; identifying (314) matching video records (150) that match the search query based on one or more index objects (180) in the object store (123), wherein at least one of the one or more index objects (180) references a plurality of video records (150) and stores, for each video record (150), a record identifier, a hash (171) of video metadata (152) of the video record (150), and a record interval (153) and a record source (154) of the video record (150); determining (316) whether a hash (171) of video metadata (152) of the matching video record (150) is consistent with a hash (171) of any video metadata (152) in a property object (170) included in the cache (111), wherein the hash (171) of the video metadata (152) of the matching video record (150) and the hash (171) of each video metadata (152) in the cache (111) are calculated by a predefined hash function; In the event of consistency, obtaining (320.1) the video metadata (152) from the attribute object (170) in the cache (111); In case of inconsistency, retrieving (318) the property object (170) of the matching video record (150) from the object store (123), obtaining (320.2) the video metadata (152) from the retrieved property object (170), and storing the retrieved property object (170) in the cache (111), wherein the property object (170) stores the video metadata (152) and carries the record identifier; and One or more data objects (160) of the matching video record (150) are retrieved (322) from the object store (123).

11. The system of claim 10, further configured to store the video record (150), the system further comprising an ingestion component (190) and a maintenance component (124), the ingestion component (190) and the maintenance component (124) further configured to perform: At the acquisition component (190): assigning (210) the recording identifier to the video recording (150); The video data (151) is stored (212) in one or more data objects (160) in the object store (123) according to an object storage architecture, wherein: Each data object (160) carries the record identifier; storing (214) the video metadata (152) in the property object (170) in the object store (123), wherein the property object (170) carries the record identifier; calculating (216) the hash (171) of the video metadata (152) using the predefined hash function; storing (218) the hash (171), the recording identifier, the recording interval (153), and the recording source (154) of the video recording (150) in an index object (180-2) in the object store (123); and At the maintenance component (124): The index object (180-2) is concatenated (220) with the existing index object (180-1).

Images