4316 results about "Data segment" patented technology

An improved system for accessing data within a distributed data storage network (“DDSN”) is disclosed. In a system implementing the disclosed invention, traffic is routed to individual slice servers within the DDSN in accordance with objective criteria as well as user-defined policies. In accordance with one aspect of the disclosed invention, when a data segment is written to a DDSN, the segment is divided into multiple data slices, which are simultaneously transmitted to different slice servers. In accordance with another aspect of the disclosed invention, when a data segment is read from a DDSN, a list of slice servers, each containing a data slice that could be used to reconstruct the requested data segment, is assembled, and sorted in accordance with a preference rating assigned to each of the slice servers. Sufficient data slices to reconstruct the data segment are then read in accordance with the preference ranking of the slice servers.

An apparatus for collecting data from a plurality of diverse data sources, the diverse data sources generating input data selected from the group including text, audio, and graphics, the diverse data sources selected from the group including real-time and recorded, human and mechanically-generated audio, single-speaker and multispeaker, the apparatus comprising: means for dividing the input data into one or more data segments, the dividing means acting separately on the input data from each of the plurality of diverse data sources, each of the data segments being associated with at least one respective data buffer such that each of the respective data buffers would have the same number of segments given the same data; means for selective processing of the data segments within each of the respective data buffers; and means for distributing at least one of the respective data buffers such that the collected data associated therewith may be used for further processing.

A rebuilder application operates on a dispersed data storage grid and rebuilds stored data segments that have been compromised in some manner. The rebuilder application actively scans for compromised data segments, and is also notified during partially failed writes to the dispersed data storage network, and during reads from the dispersed data storage network when a data slice is detected that is compromised. Records are created for compromised data segments, and put into a rebuild list, which the rebuilder application processes.

A system and method are disclosed for providing efficient data storage. A plurality of data segments is received in a data stream. The system determines whether a data segment has been stored previously in a low latency memory. In the event that the data segment is determined to have been stored previously, an identifier for the previously stored data segment is returned.

Methods, devices, and computer programs are presented for creating a unified data stream from multiple data streams acquired from multiple devices. One method includes an operation for receiving activity data streams from the devices, each activity data stream being associated with physical activity data of a user. Further, the method includes an operation for assembling the unified activity data stream for a period of time. The unified activity data stream includes data segments from the data streams of at least two devices, and the data segments are organized time-wise over the period of time.

An apparatus, system, and method are disclosed for managing data in a solid-state storage device. A solid-state storage and solid-state controller are included. The solid-state storage controller includes a write data pipeline and a read data pipeline The write data pipeline includes a packetizer and an ECC generator. The packetizer receives a data segment and creates one or more data packets sized for the solid-state storage. The ECC generator generates one or more error-correcting codes (“ECC”) for the data packets received from the packetizer. The read data pipeline includes an ECC correction module, a depacketizer, and an alignment module. The ECC correction module reads a data packet from solid-state storage, determines if a data error exists using corresponding ECC and corrects errors. The depacketizer checks and removes one or more packet headers. The alignment module removes unwanted data, and re-formats the data as data segments of an object.

An encryption technique is disclosed for encrypting a data segment comprising a plurality of data blocks, wherein the security and throughput of the encryption is enhanced by using blockwise independent bit vectors for reversible combination with the data blocks prior to key encryption. Preferably, the blockwise independent bit vectors are derived from a data tag associated with the data segment. Several embodiments are disclosed for generating these blockwise independent bit vectors. In a preferred embodiment, the data tag comprises a logical block address (LBA) for the data segment. Also disclosed herein is a corresponding decryption technique as well as a corresponding symmetrical encryption/decryption technique.

A content sharing method of an intermediate node based on a content oriented network (CON) is provided. The method includes receiving, from a content requesting terminal, a proxy sharing request message with respect to a content including a content name of the content, the content including a plurality of data segments, generating data segment request messages with respect to each of the plurality of data segments corresponding to the content, in response to the reception of the proxy sharing request message, transmitting the data segment request messages to at least one terminal having the plurality of data segments, receiving the plurality of data segments from the at least one terminal, and transmitting the content to the content requesting terminal, using an aggregated data transmission scheme based on the received plurality of data segments.

A system and method for inspecting a data stream for data segments matching one or more patterns each having a predetermined allowable error, which includes filtering a data stream for a plurality of patterns of symbol combinations with a plurality of parallel filter mechanisms, detecting a plurality of potential pattern piece matches, identifying a plurality of potentially matching patterns, reducing the identified plurality of potentially matching patterns to a set of potentially matching patterns with a reduction stage, providing associated data and the reduced set of potentially matching patterns, each having an associated allowable error, to a verification stage, and verifying presence of a pattern match in the data stream from the plurality of patterns of symbol combinations and associated allowable errors with the verification stage.

Various embodiments of a computer system and methods are disclosed. In one embodiment, a computer system includes hosts coupled to a backup medium. The backup medium stores data segments and objects, each object referencing one or more segments. A second host conveys a write request to a first host identifying a data entity comprising segments referenced by a first object. The first host restores a copy of the data entity and creates a second object referencing the segments that were referenced by the first object in response to receiving the write request. The first host adds segment references to the second object in response to receiving a change that adds data to the copy or deletes a portion of a segment from the copy. The first host removes a segment reference from the second object if a change deletes all of a segment from the copy.

An apparatus, system, and method are disclosed for validating that correct data is read from a storage device. A read request receiver module receives a read storage request to read a data segment of a file or object stored on a data storage device. The storage request includes one or more source parameters for the data segment. The source parameters include one or more virtual addresses that identify the data segment. A hash generation module generates one or more hash values from the virtual addresses. A read data module reads the requested data segment and returns one or more data packets and corresponding stored hash values stored with the data packets. The stored hash values were generated from a data segment written to the data storage device that contains data of the data packets. A hash check module verifies that the generated hash values match the respective stored hash values.

Delta compression after identity deduplication is disclosed. A first data segment is determined to be identical to a first previous data segment. A second data segment, not determined to be identical to a second previous data segment, is then determined to be similar to a third previous data segment.

An apparatus, system, and method are disclosed for converting a storage request to an append data storage command. A storage request receiver module receives a storage request from a requesting device. The storage request is to store a data segment onto a data storage device. The storage request includes source parameters for the data segment. The source parameters include a virtual address. A translation module translates the storage request to storage commands. At least one storage command includes an append data storage command that directs the data storage device to store data of the data segment and the one or more source parameters with the data, including a virtual address, at one or more append points. A mapping module maps source parameters of the data segment to locations where the data storage device appended the data packets of the data segment and source parameters.

An apparatus, system, and method are disclosed for efficiently mapping virtual and physical addresses. A forward mapping module uses a forward map to identify physical addresses of data of a data segment from a virtual address. The data segment is identified in a storage request. The virtual addresses include discrete addresses within a virtual address space where the virtual addresses sparsely populate the virtual address space. A reverse mapping module uses a reverse map to determine a virtual address of a data segment from a physical address. The reverse map maps the data storage device into erase regions such that a portion of the reverse map spans an erase region of the data storage device erased together during a storage space recovery operation. A storage space recovery module uses the reverse map to identify valid data in an erase region prior to an operation to recover the erase region.

A content-addressable and searchable storage system for managing and exploring massive amounts of feature-rich data such as images, audio or scientific data, is shown. The system comprises a segmentation and feature extraction unit for segmenting data corresponding to an object into a plurality of data segments and generating a feature vector for each data segment; a sketch construction component for converting a feature vector into a compact bit-vector corresponding to the object; a similarity index comprising a plurality of compact bit-vectors corresponding to a plurality of objects; and an index insertion component for inserting a compact bit-vector corresponding to an object into the similarity index. The system may further comprise an indexing unit for identifying a candidate set of objects from said similarity index based upon a compact bit-vector corresponding to a query object. Still further, the system may additionally comprise a similarity ranking component for ranking objects in said candidate set by estimating their distances to the query object.

A system and method for enabling the use of real-world objects (16), data segments or information segments as direct links to network based information, knowledge, services and data sources. The system comprises a communications device (10) with an input mechanism for capturing data from a real world object (16), connecting the device (10) to a network server (12) in order to search for a related online source for the object, transferring the information to the device (10), or providing the service to the device or the user. Alternatively, the present invention enables connecting the real world object (16) data to an online link, or initiate a predefined action, either automatically or manually.

A DTV signal includes frames, and the frames include data segments. Data segments are replaced by codewords which represent digital data. The DTV signals are broadcast and received by receivers. The receivers recover the digital data from the codewords in the DTV signals, for example by using matched filtering. By using long codewords, the digital data can be broadcast over a longer range and recovered by simpler circuitry in comparison to the television programming contained in the DTV signal. In one implementation, the DTV signal is an American Television Standards Committee (ATSC) DTV signal.

A system and method for data backup. A computer system includes clients coupled to backup server and single instance store via a network. Prior to requesting an initial data backup, a new client segments and fingerprints the data to be backed up. The new client creates a sorted list of hash values and removes duplicate values in the list. In response to receiving the sorted list, a backup server may further reduce the sorted list to a unique list by removing hash values corresponding to data segments already residing on the single instance store. In response to receiving the unique list, the new client may transmit corresponding data segments to the backup server via the network. The transmitted data segments are added to a single instance store.

A data field of transmitted digital television signals includes a first set of A/53-compliant data segments that convey payload information and further includes a second set of A/53-compliant data segments that contain parity bytes for transverse Reed-Solomon forward-error-correction coding of the data contained within the first set of A/53-compliant data segments. A digital television receiver uses the parity bytes in the second set of A/53-compliant data segments to implement transverse Reed-Solomon forward-error-correction decoding that corrects byte errors in the data contained in the first set of A/53-compliant data segments.