Systems and methods for generating metadata describing unstructured data objects at the storage edge

Abstract

To provide a data storage controller and a method for generating metadata describing unstructured data objects at a storage edge.SOLUTION: A storage control device coupled to a storage device and located remotely from a host device receives media object data from the host device, identifies a type of the media object data, and select, based on the identified type, a computational model from among a plurality of computational models for use by a computational engine of the storage control device. The computational engine uses the selected computational model to generate metadata describing the media object data. The metadata is stored in the storage device so as to be selectively retrievable from the storage device separately from the media object data.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] [CROSS-REFERENCE TO RELATED APPLICATIONS] This disclosure claims the benefit under 35 U.S.C. §119(e) of co-pending, commonly assigned U.S. Provisional Application No. 62 / 712,823, filed July 31, 2018, U.S. Provisional Application No. 62 / 714,563, filed August 3, 2018, U.S. Provisional Application No. 62 / 716,269, filed August 8, 2018, U.S. Provisional Application No. 62 / 726,847, filed September 4, 2018, and U.S. Provisional Application No. 62 / 726,852, filed September 4, 2018. Each of the following commonly assigned U.S. non-provisional patent applications also claims the benefit of, and are being filed concurrently with, the aforementioned U.S. provisional patent applications: 1. U.S. Patent Application No. 16 / 264,473, 2. U.S. Patent Application No. 16 / 264,248, 3. U.S. Patent Application No. 16 / 262,975, and 4. U.S. Patent Application No. 16 / 262,971 Each of the foregoing provisional and non-provisional patent applications is hereby incorporated by reference herein in its entirety.

[0002] [Field of use] The present disclosure relates to generating metadata describing media object data at a storage edge using a computation engine. [Background technology]

[0003] The background discussion provided herein is intended to generally present the context for the present disclosure. The work of the inventors herein, to the extent that that work is described in this Background section, and any aspects of the description not otherwise qualified as prior art at the time of filing, are not admitted, expressly or impliedly, to be prior art to the present disclosure.

[0004] A typical system for generating metadata describing unstructured data, such as media data, can generate the metadata at the location where the data is initially received (the ingest point), such as an ingest server or other centralized network processing location. The ingest point may be physically co-located with a user of the system or may be a cloud storage location. For example, video data may be received from a camera or other video source at a server that is responsible for both transcoding the media and generating the metadata describing the media. These systems typically embed metadata in the media data stream with the received or transcoded media data, resulting in the need to search through the media data to access all of the metadata describing the media data. This can increase the load at the ingest point due to the amount of media data that is processed to embed or include the metadata. A typical system generates the metadata describing the media data at the ingest point or centralized server. This can increase the amount of data transferred from the ingest point at the centralized server to storage, resulting in increased network bandwidth required to perform the storage operation. For example, large amounts of data may need to be transmitted between a processing location and a storage location that are located far away. Summary of the Invention

[0005] The implementations described herein provide apparatus and methods for storing and managing media object data. A storage control device, coupled to a storage device and located at a storage edge remote from a host device, receives media object data from the host device. The storage control device identifies a type of the media object data and, based on the identified type, selects a computation model from among multiple computation models used by a computation engine of the storage control device. The computation engine uses the selected computation model to generate metadata describing the media object data. The metadata is stored on the storage device so that it can be selectively retrieved from the storage device separately from the media object data.

[0006] In some implementations, the storage control device accesses the media object data using a data input interface. Each computational model has associated input data interface parameters that identify a data format for which the computational model is configured. The storage control device uses the input data interface parameters to access the media object data and convert the media object data into the identified data format.

[0007] In some implementations, each computational model has associated hardware interface parameters. The storage control device retrieves the hardware interface parameters and applies the hardware interface parameters to the hardware interface. The storage control device accesses the media object data from a first physical connection using the hardware interface and stores the media object data and metadata on a storage device via a second physical connection using the hardware interface.

[0008] In some implementations, each computational model has associated decoding parameters, and the storage control device retrieves the decoding parameters and uses the decoding parameters to decode the media object data.

[0009] In some implementations, the storage control device retrieves from the storage device media object data for which metadata has already been generated using a first computational model. The storage control device identifies the first computational model and selects a second computational model to generate second metadata that differs from the first metadata. The storage control device can store the second metadata so that it is selectively retrievable from the first metadata and the media object data. The storage control device can combine the first metadata and the second metadata to generate combined metadata and store the combined metadata so that it is selectively retrievable from the media object data.

[0010] In some implementations, the storage control device receives a request from a host device for media object data that meets one or more criteria. In response to the request, in some implementations, the storage control device retrieves only metadata describing the media object data from the storage device, separately from the media object data. The storage control device identifies one or more portions of the media object data that meet the one or more criteria based on the retrieved metadata. The storage control device then retrieves only the identified portions of the media object data from the storage device, separately from the metadata describing the media object data.

[0011] In some implementations, the storage control device simultaneously generates a copy of the media object data and provides the media object data to a storage device for storage, and provides the copy of the media object data to a computation engine for processing and generating metadata.

[0012] In some implementations, the storage control device stores the media object data in a logically separated manner so that they can be selectively retrieved separately from one another. [Brief explanation of the drawings]

[0013] Further features of the present disclosure, its nature and various advantages will become apparent from a consideration of the following detailed description in conjunction with the accompanying drawings, in which like reference characters refer to like parts throughout.

[0014] [Figure 1] FIG. 1 is a block diagram illustrating components and the data flow between them for generating metadata describing unstructured data objects, according to some implementations of the present disclosure.

[0015] [Figure 2] FIG. 10 is another block diagram illustrating components and the data flow between them for generating metadata describing unstructured data objects, according to some implementations of the present disclosure.

[0016] [Figure 3] FIG. 2 is a block diagram illustrating the storage of metadata and media data on a non-volatile storage device, in accordance with some implementations of the present disclosure.

[0017] [Figure 4] FIG. 10 is another block diagram illustrating storage of metadata and media data in a non-volatile storage device, in accordance with some implementations of the present disclosure.

[0018] [Figure 5] FIG. 2 is a block diagram illustrating components of a computation engine, according to some implementations of the present disclosure.

[0019] [Figure 6]FIG. 1 is a block diagram illustrating components of a data structure from which a computational model is extracted, according to some implementations of the present disclosure.

[0020] [Figure 7] 1 is a flowchart depicting a process for generating metadata describing an unstructured data object, according to some implementations of the present disclosure.

[0021] [Figure 8] 10 is a flowchart illustrating another process for generating metadata describing a data object, according to some implementations of the present disclosure.

[0022] [Figure 9] 1 is a flowchart depicting a process for retrieving portions of a data object based on separately retrieved metadata describing the data object, according to some implementations of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0023] The implementations described herein provide apparatus and methods for generating metadata describing unstructured data objects at a storage edge. The storage edge refers to an environment local to one or more storage devices, such as solid-state storage drives or hard disk drives. A storage control device located at the storage edge is used to control the storage and retrieval of data on one or more storage devices. Each storage device may be coupled to and controlled by a respective storage control device. Alternatively, several storage devices may be coupled to and controlled by a single storage control device, such as a storage aggregator as described in the above-incorporated U.S. patent application Ser. No. 16 / 264,248. The storage control device receives unstructured data objects, such as media objects, for which descriptive metadata has not yet been generated. Control circuitry in the storage control device identifies a type of the data object, such as a data type. Based on the type, a computational model is selected from among multiple computational models. A computational engine, for example, a computational engine local to the storage edge or a storage device, generates metadata describing the data object using the selected computational model. The storage control device provides the metadata to the storage device for storage within or in association with the data object so that it can be selectively retrieved separately from the data of the data object. The metadata can be embedded in the data object in a header or other separate portion of the data object that can be retrieved separately from the data object. For example, the data object may include video data, and the metadata can be generated as a text file. The storage control device combines the metadata text and the video data into a single file with logically separated text and video portions. The metadata can then be retrieved separately by accessing only the text portion of the file. Alternatively, the metadata can be interleaved with the data of the data object.For example, each keyframe included in the video data includes some metadata that describes the portion of the video data to which the respective keyframe applies. As a third alternative, the metadata may be stored on a physically separate storage device or in a logically separate portion of the storage device from which the data objects are stored. For example, the data objects may be stored on a first solid-state drive and the metadata may be stored on a second solid-state drive. As another example, the data objects may be stored on a first partition of a solid-state drive and the metadata may be stored on a second partition of the same solid-state drive. Any other suitable physical or logical separation of data may be used.

[0024] This disclosure describes methods and systems for generating metadata describing unstructured data objects at a storage edge and storing the metadata so that it can be selectively searched separately from the data of the data objects. A storage control device includes control circuitry and a computation engine, such as a suitable artificial intelligence (AI) or machine learning (ML) engine. The storage control device is located at the storage edge and receives unstructured data, such as media objects, to be stored on the storage device. The storage control device generates metadata about media data for the unstructured media objects and stores the metadata on the storage device so that it can be selectively searched separately from the unstructured media data.

[0025] As used herein, the term "storage edge" refers to a module or component local to a non-volatile storage device. For example, a controller that controls the operation of one or more storage devices to store or retrieve data in one or more instances of non-volatile memory is located on the storage edge. The storage edge may be, for example, a dedicated storage device or storage network, and is separate from a processor that may be located remotely, for example, in a host computer or data center. Communication between the storage edge and the remote host is via a computer network connection.

[0026] As used herein, the terms “media data object,” “data object,” “media object,” or “object” refer to various types of data that may be published by applications executing on a host system and stored on a storage device. Examples of “media object” or “object” may include, but are not limited to, video, audio recordings, still images, text objects such as text messages and emails, data obtained from various types of sensors such as automotive sensors and Internet of Things (IoT) sensors, database objects, and / or any other suitable objects. Often, media objects are unstructured. As used herein, the term “unstructured object” means that the media content (“content media data”) of an object (e.g., text content, audio content, image content, or video content) is provided in raw form and is not pre-organized according to a fixed-field format. Unstructured objects are not tagged a priori with metadata that defines frame-by-frame content or any aspect of other content portions. Unstructured data is non-transactional, and its format does not easily conform to a relational database schema.

[0027] As used herein, the term "metadata" refers to a schematic representation of the actual data content of a media object stored on a non-volatile storage device. "Metadata" may be an abstraction layer over the actual data content, providing a description or meaning of the data content in a compact form. Metadata may be generated in a variety of ways from media objects, which are often unstructured. Examples of metadata may include labels, tags, types of data, objects / concepts / emotions detected in the data content, spatial / temporal locations of such objects / concepts / emotions within the data content, etc.

[0028] As used herein, the term “artificial intelligence (AI) model” refers to any suitable AI algorithm implemented, for example, in a deep neural network or any recurrent neural network, or any variant thereof. In some implementations, the AI model is suitably any other supervised learning, unsupervised learning, or reinforcement learning algorithm. The AI model is trained using a “training set,” i.e., a body of media objects and corresponding metadata that is known to be accurate. The trained AI model is then applied to generate metadata about other media objects. As used herein, a software or hardware module that receives a pre-trained AI model and uses it to calculate object metadata is referred to as an “AI engine” or “AI interface engine.” In some implementations, several different AI models are applied to unstructured or partially structured media objects.

[0029] According to one aspect of the present disclosure, a storage control device receives unstructured data from a host device over a network, such as a local area network (LAN) or wide area network (WAN), using the Internet or other suitable networking protocol. Alternatively, the unstructured data is received from an imaging device (e.g., a video recorder, an audio recorder, a sensor) locally connected to the computer. In such an implementation, the storage control device is incorporated into the computer's storage device and receives data over a bus from other parts of the computer. The storage control device stores the original data object on the storage device while using a computation engine to create a copy of the data object and generate metadata using the copy. After generating the metadata, the copy of the data object is no longer needed. The metadata is stored in a manner that allows it to be selectively retrieved from the unstructured data, and the copy of the data object can be deleted, marked for deletion, overwritten, or removed from the storage control device.

[0030] Metadata is stored separately from unstructured data, as described below, to facilitate selective retrieval of the metadata without requiring retrieval of the associated unstructured data. For example, the metadata and unstructured data may be stored in one or more logically separated portions of a data file or other data structure. In some implementations, one or more portions containing metadata are structured as self-contained data files that can be accessed and read by a storage control device outside the context of the unstructured data. Furthermore, one or more portions containing metadata may be tagged or identified within the data file or data structure to allow the storage control device to easily locate and search them. Furthermore, the metadata identifies or includes an association with the unstructured data it describes. This allows the storage control device to search for relevant portions of the unstructured data based on review or analysis of the metadata. This and other implementations are further described in the above-incorporated U.S. patent application Ser. No. 16 / 262,971.

[0031] Storing metadata in a manner that allows it to be selectively retrieved from unstructured data may increase the efficiency of indexing and search functions. A search engine or indexer can access or retrieve the metadata independently, without having to parse the metadata from the unstructured data object. This selective retrieval reduces the amount of data that needs to be accessed to retrieve the metadata or relevant portions of the unstructured data object, and reduces the processor load when performing search or indexing functions or retrieval of relevant portions of previously unstructured data, such as relevant portions of media objects.

[0032] Generating metadata at the storage edge rather than the host device reduces the amount of data transmitted to the storage edge, resulting in faster overall data transfer rates from the ingest point or central data processing center. For example, if metadata is generated at an ingest point, data processing center, or cloud server (i.e., at a location remote from the storage edge), the metadata must also be transmitted to the storage edge along with the unstructured data object. This increases the amount of data transmitted and therefore the time required to transmit the data. As another example, unstructured data may be transmitted directly to storage without first generating media, and then some of the unstructured data is transmitted over a computer network back to the data processing center for metadata generation. This increases the amount of data transmitted over the computer network and can impact overall network speed and / or performance. By moving metadata generation to the storage edge, metadata can be easily generated for significantly larger amounts of unstructured data, making it easier to use the metadata to identify relevant portions of stored unstructured content media data. Larger volumes of content media can be filtered for relevant segments, and the relevant segments can be transferred from the storage device rather than transferring the entire vast collection of unstructured content media data from storage over a computer network to a data center facility. This facilitates a significant reduction in the amount of content media data that needs to be transferred and promotes the ability to more quickly identify and review only the relevant segments of content media data.Thus, the time required to make relevant portions of unstructured data available for retrieval from the storage edge is reduced by the difference between the time required to transfer the large amount of content data media required to generate the metadata using the data center's computational engine to the data center facility and the time required to transfer the relatively small amount of data required to meet a specific need, including both the metadata generated at the storage edge and the relevant segments of content data, to the data processing facility. For example, metadata is generated for surveillance video to tag the mood of people captured in the surveillance video or the presence of specific people or activities captured in the surveillance video. Only relevant frames containing the desired people, moods, or activities may need to be sent (e.g., to a data processing center or a human operator) for further analysis. As another example, metadata is generated for audio recordings to tag specific audio sequences, such as at a robotic service desk. Specific audio sequences are searched for in the metadata, and only the relevant portions of the recording containing the relevant sequences, rather than the entire recording, are sent to the facility, where these portions are analyzed and used to improve the robot's responses in automated conversations.

[0033] According to another aspect of the present disclosure, previously received unstructured data, for which metadata was previously generated using a first computational model, is retrieved from a storage device. Second metadata is generated for the retrieved unstructured data using a second computational model different from the first computational model. The second computational model may be an updated version of the first computational model that more accurately generates the same type of metadata as the first computational model. The second metadata is compared to the first metadata, and any metadata in the second metadata that differs from corresponding metadata in the first metadata or that is not present in the first metadata is stored. Alternatively, the second computational model may be configured to generate a different type of metadata than the first computational model. The second metadata may be stored so that it can be selectively searched separately from both the previously stored unstructured data and the previously generated first metadata. Alternatively, the second metadata is merged with the first metadata, and the merged metadata is stored so that it can be selectively searched separately from the previously stored unstructured media.

[0034] 1 is a block diagram illustrating components and the data flow between them for generating metadata describing unstructured data objects according to some implementations of the present disclosure. A storage control device 102 is located at a storage edge 100. The storage control device receives unstructured data objects 104 over a network from a host device such as an audio recorder, a video recorder, an image sensor (e.g., an infrared sensor), an environmental sensor (e.g., a thermometer), a motion sensor (e.g., an accelerometer, radar, lidar), or a server (an “ingestion server”) where data from any such recorder or sensor is initially received.

[0035] In some implementations, the control circuitry 106 can receive, retrieve, or access unstructured data 104 using an input data interface. The input data interface can be configurable to access various types of unstructured data as needed. Configuration of the input data interface can be achieved by applying input data interface parameters to the input data interface. As described below in connection with FIG. 6 , the input data interface parameters can be stored in association with a particular computational model configured to process unstructured data in a particular format. If the unstructured data 104 is received in a format other than the particular format, the control circuitry 106 can convert the unstructured data 104 to the particular format to facilitate generation of metadata describing the unstructured data. For example, video data can be received in a compressed format such as H.262, but the computational model is configured to process H.264 video data. The input data interface parameters can identify a set of video compression formats that can be converted to the required H.264 format and can further provide format conversion tools used by the control circuitry 106 to convert the video data to the H.264 format. As another example, image data may be received in JPEG format, but the computational model is configured to process images in bitmap format. The input data interface parameters identify JPEG as a suitable input format for conversion to bitmap format and provide the conversion tools used by control circuitry 106.

[0036] In some implementations, the control circuitry 106 can receive, retrieve, or access the unstructured data 104 using a hardware interface. The hardware interface is configurable to access data from various physical or logical ports and / or connections of the storage control device 102. Configuring the hardware interface can be achieved by applying hardware interface parameters to the hardware interface. As described below in connection with FIG. 6, the hardware interface parameters can be stored in association with a particular computational model. In response to selecting a particular computational model (e.g., in response to instructions 118, which can be received before receiving the unstructured data), the control circuitry 106 applies the hardware interface parameters to the hardware interface. The hardware interface parameters can specify a particular logical port of a network connection, such as TCP port 20 (FTP), 80 (HTTP), or 115 (SFTP). The hardware interface parameters can specify a physical port or connection, such as an Ethernet port, a serial port, a USB port, or other data port through which the storage control device 102 can receive the unstructured data. The hardware interface listens on a specified port and allows the control circuitry 106 to access data received through the specified port.

[0037] In some implementations, the unstructured data 104 is encrypted for transmission to the storage edge. The control circuitry 106 uses the decryption parameters to decrypt the unstructured data 104 so that it can be processed. As described below in connection with FIG. 6 , the decryption parameters may be stored in association with a particular computation model. In response to selecting a particular computation model (e.g., in response to instructions 118, which may be received prior to receiving the unstructured data), the control circuitry 106 retrieves the decryption parameters. The decryption parameters may include a predefined decryption key or a decryption key generation routine, such as an RSA algorithm. The control circuitry applies the decryption key or generates and applies a decryption key using an RSA algorithm to access the unstructured data 104.

[0038] Upon accessing or receiving unstructured data 104, control circuitry 106 of storage control device 102 creates two identical data streams: a first data stream containing the initially received unstructured data 104, and a second data stream containing an identical copy of unstructured data 104 generated by control circuitry 106. Control circuitry 106 then stores first data stream 108 in non-volatile storage 110. Non-volatile storage 110 may be any non-volatile data storage device, such as a hard drive, a flash memory array (e.g., NOR flash memory or NAND flash memory), or any other suitable type of data storage medium. The above-described decoding and / or format conversion may be performed on the initially received unstructured data 104 or on only the second data stream processed by computation engine 116.

[0039] The control circuitry 106 also determines the data type of the unstructured data, such as video data, image data, audio data, or any other data type. The control circuitry 106 directs a second data stream 114, identical to the data stream 108, to the computation engine 116 of the storage control device 102. The computation engine 116 generates the metadata using a computation model. For example, the computation engine 116 may be a machine learning engine, an artificial intelligence engine, or any other suitable metadata generation engine. The computation model used by the computation engine 116 may be an AI model and may employ matrix multiplication operations, convolution operations, other operations defined by neural networks, or any other suitable data processing method or operation. The storage control device 102 selects the computation model to be used by the computation engine 116 and transmits instructions 118 to the computation engine 116 to use the selected model. The storage control device 102 selects the computation model corresponding to the determined data type or the type of metadata desired to be extracted from the content. For example, some computational models are suitable for identifying faces in an image frame or voices in a segment of a sound recording, while other computational models are designed to identify, for example, the behavior or mood of a person in an image frame. Some computational models identify the location of an object within a frame or data stream. Some computational models are configured to identify multiple different characteristics of a segment of media data.

[0040] In one implementation, the computation engine 116 processes a copy of the unstructured data to segment the unstructured data into frames and generate metadata that describes, for example, content and the location of that content within the unstructured data. The metadata 120 is then stored in the non-volatile storage 110 so that it can be selectively searched separately from the unstructured data, as described below in connection with Figures 3 and 4. This may enable more efficient indexing and searching functions within the metadata to, for example, extract relevant segments of content media data.

[0041] In some implementations, the storage of media data and metadata in non-volatile storage 110 is accomplished using a hardware interface configured using the hardware interface parameters, which may further specify a particular port or connection over which storage control device 102 stores the media object data and metadata in non-volatile storage 110. For example, the hardware interface parameters may specify the physical or logical address of a particular server or data storage device for storing the media object data and metadata. Alternatively or additionally, the hardware interface parameters may specify a particular port, such as an Ethernet port, serial port, USB port, or other data port over which storage control device 102 can access non-volatile storage 110.

[0042] FIG. 2 is another block diagram illustrating components and the data flow between them for generating metadata describing an unstructured data object, according to some implementations of the present disclosure. If metadata has not been generated for an unstructured data object already stored in non-volatile storage 110, or if additional metadata is to be generated for the unstructured data object, storage control device 102 retrieves content media data 200 from non-volatile storage 110. Control circuitry 106 identifies the data type of media data 200, as described above in connection with FIG. 1. Control circuitry 106 provides content media data 202 to computation engine 116 for processing. In some scenarios, the content data is unstructured, while in other scenarios, the content is partially structured and partial metadata has previously been generated for the content. Storage control device 102 selects a computation model and transmits instructions 204 to computation engine 116 to use the selected computation model. As described above in connection with FIG. 1, storage control device 102 selects the computation model based on the determined data type of media data 200. If metadata describing the unstructured data object already exists, the control circuitry 106 determines the first computational model used to generate the metadata and selects a second computational model to be used. For example, metadata may have already been generated using a computational model for identifying faces in surveillance video. The control circuitry 106 may select a computational model for identifying facial expressions, actions, moods, or voices in the same surveillance video as the second computational model. The metadata 206 generated using the second computational model is then stored in non-volatile storage 110 for selective retrieval separately from the unstructured data, as described below in connection with FIGS. 3 and 4. The metadata generated using the first computational model and the metadata generated using the second computational model may be stored separately from one another for selective retrieval.Alternatively or additionally, the metadata generated using the second computational model may be combined with the metadata generated using the first computational model to generate a single set of metadata that is stored in a manner that is selectively searchable separately from the unstructured data.

[0043] 3 is a block diagram illustrating storage of metadata and media data in non-volatile storage 110 according to some implementations of the present disclosure. The metadata and media data are stored in a single data file 300 in non-volatile storage 110. The data file 300 is a file format that can host both text-based metadata and unstructured media data in a metadata portion 302 and a media data portion 304, respectively. A data stream 108 containing unstructured media data is received in non-volatile storage 110, and the data file 300 is created to include the unstructured media data in the media data portion 304, for example. Metadata 120 generated by the computation engine 116 is also received in non-volatile storage 110. The non-volatile storage 110 stores the metadata in the metadata portion 302 of the data file 300. In some implementations, the metadata portion 302 is stored separately from the media data portion 304. In another implementation, the respective metadata portion 302 and media data portion 304 are stored together, for example, interleaved but separable from one another. The data file 300 may be created with both the metadata portion 302 and the media data portion 304. Alternatively, if the unstructured media data 108 is received in the non-volatile storage 110 before the metadata 120, the data file 300 may be created with only the media data portion 304, and upon receipt of the metadata 120, the metadata portion 302 may be appended to the end of, appended to the beginning of, concatenated with, or interleaved with the media portion 304. Storing the metadata 120 separable from the media data 108 may enable the metadata 120 to be selectively searchable separately from the media data 108, resulting in increased efficiency in accessing the metadata 120 or in accessing particular portions of the media data 108 based on the metadata 120, as described above.

[0044] FIG. 4 is another block diagram illustrating alternative storage of metadata and media data in non-volatile storage 110, according to some implementations of the present disclosure. Non-volatile storage 110 may include physically separate storage areas (e.g., several hard disk drives or solid-state drives) or logically separate storage areas (e.g., several partitions on a single hard disk or solid-state drive). Non-volatile storage 110 may designate a first storage area 400 for metadata storage and a second storage area 402 for data object storage. A data stream 108 is received at non-volatile storage 110. Non-volatile storage 110 determines that the received data stream 108 includes unstructured media data and stores the unstructured media data in data object storage 402. Non-volatile storage 110 also receives metadata 120 generated by computation engine 116. Non-volatile storage 110 determines that the received data is text-based metadata and stores the metadata 120 in metadata storage 400. As described above, storing the metadata 120 separably from the media data 108 allows the metadata 120 to be searchable separately from the media data 108, thereby increasing efficiency in accessing the metadata 120 or in accessing particular portions of the media data 108 based on the metadata 120.

[0045] In some implementations, the computational models are stored in respective data structures in the memory of the storage control device 102. The computational engine 116 or the control circuitry 106 extracts the selected computational model from its respective data structure for use by the computational engine 116. FIG. 5 is a block diagram illustrating components of a computational engine according to some implementations of the present disclosure. The computational engine 116 includes a data structure runtime module 500. The data structure runtime module 500 provides an environment in which the contents of the data structure can be executed by or applied to the computational engine 116. Furthermore, the data structure runtime module 500 allocates the virtual system resources required for a particular data structure. That is, the data structure runtime module 500 provides an application layer in which the contents of the data structure can be executed using the hardware of the computational engine 116, such as the CPU 506 and hardware accelerator 512, as described below in connection with FIG. 6. The computational engine 116 also includes an orchestration agent 502. The data structure runtime module 500 is responsible for executing data structures (such as data structure 514n, as described below in connection with FIG. 6), with the orchestration agent 502 determining which data structure to execute. The orchestration agent 502 receives commands, such as instructions 118, to select a particular data structure or receives an indication of the determined data type of the unstructured data object to be processed and determines an appropriate data structure. The orchestration agent 502 transmits a request 516 to the memory of the storage control device 102 to retrieve a data structure (e.g., data structure 514b) from among multiple data structures (514a, 514b, 514c, ... 514n). The memory of the storage control device 102 transmits a response 318 containing the selected data structure 514b, which is then loaded into the data structure runtime module 500 by the computation engine 116.

[0046] The data structure runtime module 500 extracts a computational model from the data structure 514b. The data structure runtime module 500 loads the extracted computational model 520 into the volatile memory 510 of the computation engine 116. The data structure runtime module 500 can also use the extracted computational model 520 to extract other data and parameters from the selected data structure needed to process the media data. In one implementation, the computation engine 116 includes an operating system 504 running on a CPU 506, through which the data structure runtime module 500 and orchestration agent 502 perform the operations described above.

[0047] In some implementations, the computation engine 116 further includes a hardware accelerator 512. The computation model includes thousands of operations that cannot be efficiently performed on the CPU 506. The computation engine 116 uses the hardware accelerator 512 to execute the extracted computation model.

[0048] FIG. 6 is a block diagram illustrating components of a data structure from which a computational model is extracted, according to some implementations of the present disclosure. In some implementations, in addition to the compiled computational model 600, a data structure, such as data structure 514n, includes several other items and parameters that can be used by the computation engine 116 when processing the unstructured data object. For example, an input data interface 602 provides parameters for the input format of the unstructured data. For example, a computational model contained within a particular data structure may be configured to process images in a bitmap-type format. If the media object data provides images in a compressed format, such as JPEG, the computation engine 116 must convert the compressed image to a bitmap. The media object data may be encrypted. For example, the media data may be subject to security concerns or copyright. A data encryption / decryption module 604 provides the encryption or decryption algorithms and keys necessary to access the unstructured data, for example, by decrypting the data. A data preprocessing module 606 provides mechanisms for preparing the unstructured data for processing, such as transforming the unstructured data required by the input data interface 602, as described above. Additionally, the data pre-processing module 606 can provide a mechanism for extracting individual video frames or audio samples from the media object data. The data post-processing module 608, in some implementations, provides parameters for formatting the metadata generated by the computational model 600. For example, the data post-processing module provides specific data structures and / or data fields into which metadata associated with the media object data is placed. The data structures or data fields are specific to both the type of data being processed and the configuration of the compiled computational model 600. The hardware interface 610 provides parameters for accessing the data using specific components, such as a network interface or serial data connection.

[0049] FIG. 7 is a flowchart illustrating a process 700 for generating metadata describing an unstructured data object according to some implementations of the present disclosure. As a non-limiting example, the unstructured data object is a content media data object. At 702, the storage control device 102 receives a media object from a host device using the control circuitry 106. For example, the control circuitry 106 may include a network interface or other data input connection through which data is received from a remote source. Remote sources include any host device or server where a media object is generated and / or transmitted to the storage edge, such as a video recorder, an audio recorder, an ingest server, and a cloud server. Performing metadata generation at the storage edge for data received at or already present at the storage edge, rather than data stored in a data storage center or distributed storage system (i.e., cloud storage), significantly reduces the amount of data that needs to be transferred across the network. In some implementations, the storage control device 102 may receive a request from the host device to store media object data in non-volatile storage 110. At 704, the control circuitry 106 creates a copy of the media object data, resulting in two data streams containing the same content data. At 706, the control circuitry 106 provides one data stream to the non-volatile storage 110 for storage. At 708, the control circuitry 106 identifies a data type of the media object data. For example, the media object data may be a video stream. The control circuitry 106 may process, inspect, or examine packet headers to determine the type of data received from the host device. The control circuitry 106 may further determine a data format, such as MPEG-2, MPEG-4, or other video data format. As another example, the media object data may be an audio stream.The control circuitry 106 may process, inspect, or examine the packet headers to determine the type of data being received and may further determine the data format, such as MP3, WAV, or other suitable audio data format. As a third example, the media object data may be sensor data captured from an image sensor, such as an infrared sensor. The control circuitry 106 may identify the sensor type from packet header data, such as the sensor's signature.

[0050] At 710, the control circuitry 106 selects a computational model from among multiple computational models for use by the computation engine 116 based on the identified data type. For example, the control circuitry 106 selects a computational model to perform image processing on successive frames of video data to identify objects depicted in each frame. In some implementations, the selection of a computational model by the control circuitry 106 is further based on the type of metadata desired to be generated. For example, several computational models may be available for processing the video data, each generating a different set of metadata. One computational model may identify faces in the video data, while another computational model may identify actions in the video data. The control circuitry 106 may receive input from a user or from a program running on the storage edge to generate a particular type of metadata. Thus, the control circuitry 106 selects a computational model configured to process the video data and generate the desired metadata. Further concepts are described in the above-incorporated U.S. patent application Ser. No. 16 / 262,971.

[0051] At 712, the computation engine 116 processes the second data stream using the selected computational model to generate metadata describing the media object data. At 714, the control circuitry 106 provides the metadata to the non-volatile storage 110 for storage so that it can be selectively retrieved from the non-volatile storage 110 separately from the media object data, using the methods previously described in connection with Figures 3 and 4. In this manner, the metadata can be retrieved separately from the media object data, such that in the above process, only the metadata generated by the first computational model can be retrieved independently of the media object data.

[0052] FIG. 8 is a flowchart illustrating another process 800 for generating metadata describing a data object according to some implementations of the present disclosure. In some implementations, media object data that has already been processed using a first computational model to generate first metadata is again processed using a second computational model to generate additional or supplemental metadata. At 802, the control circuitry 106 retrieves first metadata describing the media object data previously generated using the first computational model. For example, the control circuitry 806 can use an SQL “select” command to query non-volatile storage 110 or metadata storage 400, and in response to the command, non-volatile storage 110 provides the first metadata. Alternatively, the control circuitry 106 can access a file allocation table in non-volatile storage 110 to identify a memory location or memory address range where the first metadata is stored and retrieve the first metadata from the identified location or address range.

[0053] At 804, the control circuitry 106 identifies a first computational model used to generate the first metadata. For example, in some implementations, the control circuitry 106 can analyze a type of previously generated metadata or a particular metadata field included in the first metadata for one or more computational models, or an index of available computational models to identify a second computational model for processing the identified data type that generates the type of metadata or the particular metadata field included in the first metadata. For example, if the first metadata includes a field describing faces and their positions over time, the control circuitry 106 determines that the first metadata was generated using a computational model configured to identify faces in video content.

[0054] At 806, the control circuit 106 determines the type of metadata to be generated. The control circuit 106 can determine whether an updated version of the first computational model exists. For example, the control circuit 106 can identify the date and time the first computational model was compiled or when the data structure containing the first computational model was first made available. If an updated version of the first computational model exists, the control circuit 106 can determine that metadata of the same type as the first metadata should be generated. Alternatively, the control circuit 106 can automatically select the type of metadata to supplement the first metadata or can receive input from a user identifying the particular type of metadata to be generated. For example, the control circuit 106 can identify the first computational model as an image processing model for identifying objects, such as faces, depicted in video frames. The control circuit 106 can then determine that metadata identifying additional characteristics associated with faces, such as mood, should be generated. Alternatively, the control circuit 106 can determine that metadata describing spoken words in the video data should be generated.

[0055] At 808, the control circuitry 106 selects a computational model based on the determined type of metadata to be generated. For example, if the control circuitry 106 determines that the type of metadata to be generated is the same as the first metadata type and indicates that an updated version of the first computational model exists, the control circuitry 106 selects the updated computational model. Alternatively, if a different type of metadata is to be generated, the control circuitry 106 selects a computational model configured to generate that type of metadata. For example, if metadata describing mood is to be generated, the control circuitry 106 selects a computational model configured to process frames in which faces are identified to identify moods. If metadata describing spoken language is to be generated, the control circuitry 106 may select an audio processing computational model to process audio data for the entire video or for segments of the video containing faces to identify segments containing spoken language within the video, and then use the same or a different computational model to transcribe the spoken language and / or analyze the audio content. Alternatively or additionally, the control circuitry 106 may select a text processing computational model to extract and / or process text, such as closed captions, within or associated with content media.

[0056] At 810, control circuitry 106 retrieves at least a portion of the media object data from non-volatile storage 110 based on the configuration of the selected computational model. For example, a first computational model is configured to identify faces in video content, and the selected computational model is configured to identify moods based on the faces in the video data. Rather than retrieving the entire video content from non-volatile storage 110, control circuitry 106 uses first metadata generated by the first computational model to identify portions of the video content that include faces for processing by computation engine 116 using the selected computational model. Control circuitry 106 then retrieves only the identified portions of the video content.

[0057] At 812, the computation engine 116 generates second metadata describing content within the media object data using the second computational model. At 814, the control circuitry 106 provides the second metadata to the non-volatile storage 110 for storage so that it can be selectively retrieved from the storage device separately from the media object data, using the methods previously described in connection with Figures 3 and 4. In this manner, the metadata can be retrieved separately from the media object data, such that in the process described above, only the metadata generated by the first computational model can be retrieved independently of the media object data.

[0058] 9 is a flowchart illustrating a process 900 for retrieving a portion of a data object based on separately retrieved metadata describing the data object, according to some implementations of the present disclosure. At 902, the storage control device 102 receives a request from a host device for a media object that meets one or more criteria. For example, the media object is surveillance video, and the host device requests video data that includes human faces. At 904, the storage control device 102 retrieves only the metadata describing the media object data from the non-volatile storage 110. For example, the storage control device 102 accesses a logically separate portion of the media object in which the metadata describing the surveillance video is stored and extracts, copies, or retrieves the metadata contained in that logically separate portion without retrieving any of the surveillance video data.

[0059] At 906, the storage control device 102, using the control circuitry 106, analyzes the metadata to identify portions of the media object data that meet one or more criteria. For example, the control circuitry 106 analyzes the metadata describing the surveillance video to identify portions of the surveillance video that include human faces. At 908, the storage control device 102 retrieves only the identified portions of the media object data separately from the metadata describing the media object data. For example, the storage control device 102 retrieves only the portions of the surveillance video that include human faces identified based on the metadata. The storage control device 102 retrieves the portions of the surveillance video separately from the metadata describing the video.

[0060] 1-9 are performed by various electronic components of one or more electronic circuits, such as, but not limited to, an integrated circuit, an application specific integrated circuit (ASIC), and / or a DSP. Various components described throughout this disclosure, such as, but not limited to, a control circuit and a hardware accelerator (e.g., control circuit 106 and hardware accelerator 312), and / or the like, include a set of electronic circuit components and are configured to communicatively operate on one or more electronic circuits. Each electronic circuit is configured to include, but is not limited to, any of logic gates, memory cells, amplifiers, and / or filters, etc. The various implementations and components disclosed herein are configured to be operated and / or implemented at least in part by processor-executable instructions stored on one or more transient or non-transitory processor-readable media.

[0061] While various implementations of the subject matter of this disclosure have been shown and described herein, such implementations are provided by way of example only. Numerous variations, changes, and substitutions related to the implementations described herein may be applied without departing from the disclosure. It should be noted that various alternatives to the implementations of the disclosure described herein may be employed in practicing the disclosure. It is intended that the following claims define the scope of the disclosure and that methods and structures within the scope of these claims and their equivalents be covered thereby.

[0062] Although the figures depict acts in a particular order, this should not be construed as requiring that such acts be performed in the particular order or sequence shown, or that all of the acts shown must be performed to achieve desirable results.

[0063] Although the subject matter herein has been described in terms of particular embodiments, other embodiments may be implemented and are within the scope of the following claims. For example, the actions recited in the claims may be performed in a different order and still achieve desirable results. As an example, the processes depicted in Figures 7, 8, and 9 do not necessarily require the particular order or sequence shown to achieve desirable results. In certain implementations, multitasking and parallel processing may be advantageous. Other variations are included within the scope of the following claims.

[0064] [Other possible items] [Item 1] 1. A method for storing and managing media object data, comprising: receiving the media object data at a storage control device, the storage control device being located remotely from the host device and coupled to the storage device; identifying, at the storage control device, a type of the media object data using control circuitry of the storage control device; selecting, at the storage control device, a computation model from among a plurality of computation models for use by a computation engine of the storage control device based on the identified type; generating, by the computation engine, metadata describing the media object data using the selected computation model; providing the generated metadata to the storage device for storage so as to be selectively retrievable from the storage device separately from the media object data; A method for providing [Item 2] Each computational model of the plurality of computational models is stored in a respective data structure among a plurality of data structures, the storage control device is located at a storage edge, and the method includes: extracting, at the storage control device, the selected computational model from the respective data structure; loading the computational model into a memory of the storage control device; Item 1, further comprising: [Item 3] accessing the media object data at the storage edge using an input data interface, wherein each data structure of the plurality of data structures includes input data interface parameters that identify a data format for which the computational model included in the respective data structure is configured; converting said media object data into said identified data format; 3. The method according to item 2, further comprising: [Item 4] retrieving hardware interface parameters from a data structure of the plurality of data structures by the storage control device; applying the hardware interface parameters to a hardware interface; accessing the media object data at the storage edge from a particular physical connection using the hardware interface; 3. The method according to item 2, further comprising: [Item 5] the media object data is encrypted, and the method comprises: retrieving decryption parameters from a data structure of the plurality of data structures by the storage control device; decrypting the media object data at the storage edge using the decoding parameters; Item 3. The method of item 2, further comprising: [Item 6] The media object data is retrieved from the storage device, the method comprising: identifying the computational model used to generate the metadata, wherein the computational model is a first computational model and the metadata is first metadata; selecting a second computation model different from the first computation model to generate second metadata different from the first metadata; storing the second metadata so that it can be selectively retrieved from the first metadata and the media object data; Item 1, further comprising: [Item 7] combining the first metadata and the second metadata to generate combined metadata; providing the combined metadata to the storage device for storage such that the combined metadata can be selectively retrieved from the storage device separately from the media object data; Item 7. The method according to item 6, further comprising: [Item 8] Retrieving media object data is determining the type of metadata to be generated; identifying at least one portion of the media object data to be processed based on the first metadata and the type of metadata to be generated; retrieving, at the storage edge, only the identified portion of the media object data from the storage device; Item 7. The method according to item 6, further comprising: [Item 9] generating a copy of the media object data; the storage control device simultaneously providing the media object data to the storage device for storage and providing the copy of the media object data to the computation engine for processing to generate metadata; Item 1. The method according to item 1, further comprising: [Item 10] storing the media object data in a first storage location and the metadata in a second storage location, the second storage location being distinct from the first storage location by one of a physical separation and a logical separation; Item 1. The method according to item 1, further comprising: [Item 11] 1. A data storage controller located remotely from a host device and coupled to one or more storage devices, comprising: a control circuit local to the storage device, receiving media object data for storage on said storage device; Identify the type of media object data above a control circuit configured as follows: a computational engine associated with the control circuitry, an orchestration agent configured to select a computation model from among a plurality of computation models used by the computation engine based on the identified type and to generate metadata describing the media object data. and generating metadata describing the media object data using the selected computational model; and providing the generated metadata to the storage device for storage so as to be selectively retrievable from the storage device separately from the media object data; Calculation engine and A data storage controller comprising: [Item 12] the data storage controller is disposed at a storage edge and is further configured to store each computational model of the plurality of computational models in a respective data structure among a plurality of data structures; The calculation engine is extracting the selected computational models from the respective data structures; and loading the selected computation model into volatile memory of the computation engine for execution by the computation engine in processing the media object data. further configured as follows: Item 12. The data storage controller of item 11. [Item 13] each data structure of the plurality of data structures includes an input data interface parameter that identifies a format in which the computational model included in the respective data structure is structured; and the computation engine: accessing the media object data at the storage edge using the input data interface parameters; and Converting the media object data into the identified data format. Item 13. The data storage controller of item 12, further configured as follows: [Item 14] The control circuit includes: retrieving hardware interface parameters from a data structure of the plurality of data structures stored on the storage edge; applying the hardware interface parameters to the hardware interface; and Accessing the media object data from the storage device at the storage edge using the hardware interface over a specific physical connection without consuming network bandwidth. Item 13. The data storage controller of item 12, further configured as follows: [Item 15] The control circuit includes: retrieving decryption parameters from a data structure of the plurality of data structures stored on the storage edge; and At the storage edge, decrypt the media object data using the decoding parameters. Item 13. The data storage controller of item 12, further configured as follows: [Item 16] The media object data is retrieved from the storage device, and the control circuitry: identifying the computational model used to generate the metadata, wherein the computational model is a first computational model and the metadata is first metadata; selecting a second computation model different from the first computation model to generate second metadata different from the first metadata; storing said second metadata so as to be selectively retrievable from said first metadata and said media object data; Item 12. The data storage controller of item 11, further configured to: [Item 17] The control circuit includes: combining the first metadata and the second metadata to generate combined metadata; and providing the combined metadata to the storage device for storage such that the combined metadata can be selectively retrieved from the storage device separately from the media object data; Item 17. The data storage controller of item 16, further configured as follows: [Item 18] The control circuitry converts the media object data into determining the type of metadata to be generated; identifying at least one portion of the media object data to be processed based on the first metadata and the type of metadata to be generated; retrieving, at the storage edge, only the identified portion of the media object data from the storage device; Item 17. The data storage controller of item 16, configured to search by [Item 19] The control circuit includes: generating a copy of the media object data; and simultaneously providing the media object data to the storage device for storage and providing the copy of the media object data to the computation engine for processing to generate metadata. Item 12. The data storage controller of item 11, further configured as follows: [Item 20] Item 12. The data storage controller of item 11, wherein the control circuit is further configured to store the media object data in a first storage location and the metadata in a second storage location, the second storage location being distinguished from the first storage location by one of physical separation and logical separation.

Claims

1. 1. A data storage controller located at a storage edge remote from a host device and coupled to one or more storage devices, comprising: a control circuit local to a storage device of the one or more storage devices, receiving media object data from the host device for storage on the storage device; and Identifying the type of the media object data a control circuit configured as follows: a computational engine associated with the control circuitry, selecting a computational model from among a plurality of computational models used by the computational engine based at least on the identified type, wherein each computational model of the plurality of computational models is configured to generate metadata independently of each other computational model of the plurality of computational models. generating metadata describing the media object data using the selected computational model; and providing the generated metadata to the storage device for storing the generated metadata separately from the media object data so that the metadata can be selectively retrieved from the storage device; A calculation engine configured as follows: Equipped with If metadata describing the media object data already exists, the control circuit determines a first computational model that was used to generate the metadata and selects a second computational model to be used.

2. The control circuit retrieving input data interface parameters associated with the selected computational model, the input data interface parameters identifying a data format that the selected computational model can process; converting the media object data into the identified data format; The data storage controller of claim 1 , further configured to:

3. The control circuit retrieving hardware interface parameters associated with the selected computational model; applying the hardware interface parameters to a hardware interface; accessing the media object data at the storage edge via a first physical connection using the hardware interface; and storing the media object data and metadata to the storage device via a second physical connection using the hardware interface; 3. The data storage controller of claim 1, further configured to:

4. The control circuit retrieving decoding parameters associated with the selected computational model; and At the storage edge, decrypting the media object data using the decoding parameters.

4. The data storage controller of claim 1, further configured to:

5. The media object data is retrieved from the storage device, and the control circuitry: identifying the computational model used to generate the metadata, wherein the computational model is a first computational model and the metadata is first metadata; selecting a second computation model different from the first computation model to generate second metadata different from the first metadata; storing the second metadata so as to be selectively retrievable from the first metadata and the media object data; The data storage controller of claim 1 , further configured to:

6. The control circuit combining the first metadata and the second metadata to generate combined metadata; and providing the combined metadata to the storage device for storing the combined metadata separately from the media object data so that the combined metadata can be selectively retrieved from the storage device; The data storage controller of claim 5 further configured to:

7. The data storage controller of claim 1 , wherein the control circuitry is further configured to receive a request from the host device for media object data that meets one or more criteria.

8. The control circuit in response to said request for media object data meeting one or more criteria; retrieving only the metadata describing the media object data from the storage device separately from the media object data; Identifying one or more portions of the media object data that satisfy the one or more criteria based on the retrieved metadata; and Retrieving only the identified one or more portions of the media object data from the storage device separately from the metadata describing the media object data.

8. The data storage controller of claim 7, further configured to:

9. The control circuit generating a copy of the media object data; and simultaneously providing the media object data to the storage device for storage and providing the copy of the media object data to the computation engine for processing to generate metadata.

9. The data storage controller of claim 1, further configured to:

10. 10. The data storage controller of claim 1, wherein the control circuitry is further configured to store the media object data and the metadata in a logically separated manner so that they can be selectively retrieved separately from one another.

11. A data storage controller as described in any one of claims 1 to 10, wherein the metadata generated using the first computational model and the metadata generated using the second computational model are stored separately from each other so as to be selectively searchable.

12. 1. A method for storing and managing media object data, comprising: a control circuit of a storage control device receiving media object data from a host device, the storage control device being located at a storage edge remote from the host device and coupled to a storage device; the control circuitry identifying a type of the media object data; a computing engine of the storage control device selecting, based on the identified type, a computing model from a plurality of computing models for use by the computing engine, each computing model of the plurality of computing models configured to generate metadata independently of each other computing model of the plurality of computing models; generating, by the computation engine, metadata describing the media object data using the selected computation model; providing the generated metadata to the storage device by the computation engine for storage such that the generated metadata can be selectively retrieved from the storage device separately from the media object data; Equipped with The method further comprises, if metadata describing the media object data already exists, the control circuit determining a first computational model that was used to generate the metadata and selecting a second computational model to be used.

13. the control circuitry retrieving input data interface parameters associated with the selected computational model, the input data interface parameters identifying a data format that the selected computational model can process; the control circuit converting the media object data into the identified data format; The method of claim 12 further comprising:

14. the control circuitry retrieving hardware interface parameters associated with the selected computational model; the control circuitry applying the hardware interface parameters to a hardware interface; the control circuitry accessing the media object data from a first physical connection at the storage edge using the hardware interface; storing the media object data and metadata to the storage device via a second physical connection using the hardware interface; 14. The method of claim 12 or 13, further comprising:

15. the media object data is encrypted, the method comprising: the control circuitry retrieving decoding parameters associated with the selected computational model; the control circuitry decrypting, at the storage edge, the media object data using the decoding parameters; 15. The method of any one of claims 12 to 14, further comprising:

16. The media object data is retrieved from the storage device, the method comprising: the control circuit identifying the computational model used to generate the metadata, the computational model being a first computational model and the metadata being first metadata; the control circuit selecting a second computational model different from the first computational model to generate second metadata different from the first metadata; storing the second metadata so that the second metadata can be selectively retrieved from the first metadata and the media object data; 16. The method of any one of claims 12 to 15, further comprising:

17. the control circuitry combining the first metadata and the second metadata to generate combined metadata; the control circuitry providing the combined metadata to the storage device for storage such that the combined metadata can be selectively retrieved from the storage device separately from the media object data; 17. The method of claim 16 further comprising:

18. 18. The method of any one of claims 12 to 17, further comprising the control circuitry receiving from the host device a request for media object data that meets one or more criteria.

19. in response to said request for media object data that meets said one or more criteria; the control circuitry retrieving only the metadata describing the media object data from the storage device separately from the media object data; the control circuitry identifying one or more portions of the media object data that meet the one or more criteria based on the retrieved metadata; the control circuitry retrieving only the identified one or more portions of the media object data from the storage device separately from the metadata describing the media object data; 20. The method of claim 18 further comprising:

20. the control circuitry generating a copy of the media object data; the control circuitry simultaneously providing the media object data to the storage device for storage and providing the copy of the media object data to the computation engine for processing to generate metadata; 20. The method of any one of claims 12 to 19, further comprising:

21. the control circuit storing the media object data and the metadata in a logically separated manner so that they can be selectively retrieved separately from one another.

21. The method of any one of claims 12 to 20, further comprising:

22. A method described in any one of claims 12 to 21, wherein the metadata generated using the first computational model and the metadata generated using the second computational model are stored separately from each other so as to be selectively searchable.

Images