11314results about "Redundant data error correction" patented technology

The invention discloses a data validation, mirroring and error/erasure correction method for the dispersal and protection of one and two-dimensional data at the micro level for computer, communication and storage systems. Each of 256 possible 8-bit data bytes are mirrored with a unique 8-bit ECC byte. The ECC enables 8-bit burst and 4-bit random error detection plus 2-bit random error correction for each encoded data byte. With the data byte and ECC byte configured into a 4 bit×4 bit codeword array and dispersed in either row, column or both dimensions the method can perform dual 4-bit row and column erasure recovery. It is shown that for each codeword there are 12 possible combinations of row and column elements called couplets capable of mirroring the data byte. These byte level micro-mirrors outperform conventional mirroring in that each byte and its ECC mirror can self-detect and self-correct random errors and can recover all dual erasure combinations over four elements. Encoding at the byte quanta level maximizes application flexibility. Also disclosed are fast encode, decode and reconstruction methods via boolean logic, processor instructions and software table look-up with the intent to run at line and application speeds. The new error control method can augment ARQ algorithms and bring resiliency to system fabrics including routers and links previously limited to the recovery of transient errors. Image storage and storage over arrays of static devices can benefit from the two-dimensional capabilities. Applications with critical data integrity requirements can utilize the method for end-to-end protection and validation. An extra ECC byte per codeword extends both the resiliency and dimensionality.

Provided are a method, system, and program for constructing data including reconstructing data organized in a data organization type, such as a Redundant Array of Independent Disks (RAID) organization, for example, which permits data reconstruction In one embodiment, blocks of data are transferred from a stripe of data stored across storage units, such as disk drives in a RAID array, to a logic engine of a storage processor, bypassing the cache memory of the storage processor. A store queue performs a logic function, such as Exclusive-OR, on each block of data as it is transferred from the disk drives, to reconstruct a block of data from the stripe. The constructed block of data may be subsequently transferred to a disk drive of the RAID array to replace a lost block of data in the stripe of data across the RAID array or to replace an old block of parity data.

A semi-static distribution technique distributes parity across disks of an array. According to the technique, parity is distributed (assigned) across the disks of the array in a manner that maintains a fixed pattern of parity blocks among the stripes of the disks. When one or more disks are added to the array, the semi-static technique redistributes parity in a way that does not require recalculation of parity or moving of any data blocks. Notably, the parity information is not actually moved; the technique merely involves a change in the assignment (or reservation) for some of the parity blocks of each pre-existing disk to the newly added disk.

The present invention is a method for integrating a file system with a RAID array that exports precise information about the arrangement of data blocks in the RAID subsystem. The file system examines this information and uses it to optimize the location of blocks as they are written to the RAID system. Thus, the system uses explicit knowledge of the underlying RAID disk layout to schedule disk allocation. The present invention uses separate current-write location (CWL) pointers for each disk in the disk array where the pointers simply advance through the disks as writes occur. The algorithm used has two primary goals. The first goal is to keep the CWL pointers as close together as possible, thereby improving RAID efficiency by writing to multiple blocks in the stripe simultaneously. The second goal is to allocate adjacent blocks in a file on the same disk, thereby improving read back performance. The present invention satisfies the first goal by always writing on the disk with the lowest CWL pointer. For the second goal, a new disk is chosen only when the algorithm starts allocating space for a new file, or when it has allocated N blocks on the same disk for a single file. A sufficient number of blocks is defined as all the buffers in a chunk of N sequential buffers in a file. The result is that CWL pointers are never more than N blocks apart on different disks, and large files have N consecutive blocks on the same disk.

Methods, devices and computer readable code for reading data from one or more flash memory cells, and for recovering from read errors are disclosed. In some embodiments, in the event of an error correction failure by an error detection and correction module, the flash memory cells are re-read at least once using one or more modified reference voltages, for example, until a successful error correction may be carried out. In some embodiments, after successful error correction a subsequent read request is handled without re-writing data (for example, reliable values of the read data) to the flash memory cells in the interim. In some embodiments, reference voltages associated with a reading where errors are corrected may be stored in memory, and retrieved when responding to a subsequent read request. In some embodiments, the modified reference voltages are predetermined reference voltages. Alternatively or additionally, these modified reference voltages may be determined as needed, for example, using randomly generated values or in accordance with information provided by the error detection and correction module. Methods, devices and computer readable code for reading data for situations where there is no error correction failure are also provided.

A rebuilder application operates on a dispersed data storage grid and rebuilds stored data segments that have been compromised in some manner. The rebuilder application actively scans for compromised data segments, and is also notified during partially failed writes to the dispersed data storage network, and during reads from the dispersed data storage network when a data slice is detected that is compromised. Records are created for compromised data segments, and put into a rebuild list, which the rebuilder application processes.

A serial flash integrated circuit is provided with an integrated error correction coding ("ECC") system that is used with an integrated volatile page memory for fast automatic data correction. The ECC code has the capability of correcting any one or two bit errors that might occur on a page of the flash memory array. One bit corrections are done automatically in hardware during reads or transfer to the page memory, while two-bit corrections are handled in external software, firmware or hardware. The ECC system uses a syndrome generator for generating both write and read syndromes, and an error trapper to identify the location of single bit errors using very little additional chip space. The flash memory array may be refreshed from the page memory to correct any detected errors. Data status is made available to the application prior to the data. The use of the ECC is optional.

In order to maintain the integrity of data stored in a flash memory that are susceptible to being disturbed by operations in adjacent regions of the memory, disturb events cause the data to be read, corrected and re-written before becoming so corrupted that valid data cannot be recovered. The sometimes conflicting needs to maintain data integrity and system performance are balanced by deferring execution of some of the corrective action when the memory system has other high priority operations to perform. In a memory system utilizing very large units of erase, the corrective process is executed in a manner that is consistent with efficiently rewriting an amount of data much less than the capacity of a unit of erase.

The present invention relates to an apparatus and method for detecting malware. The malware detection apparatus and method of the present invention determines whether a file is malware or not by analyzing the header of an executable file. Since the malware detection apparatus and method can quickly detect presence of malware, it can shorten detection time considerably. The malware detection apparatus and method can also detect even unknown malware as well as known malware to thereby estimate and determine presence of malware. Therefore, it is possible to cope with malware in advance, protect a system with a program, and increase security level remarkably.

A cluster receives a request to store an object using replication or erasure coding. The cluster writes the object using erasure coding. A manifest is written that includes an indication of erasure coding and a unique identifier for each segment. The cluster returns a unique identifier of the manifest. The cluster receives a request from a client that includes a unique identifier. The cluster determines whether the object has been stored using replication or erasure coding. If using erasure coding, the method reads a manifest. The method identifies segments within the cluster using unique segment identifiers of the manifest. Using these unique segment identifiers, the method reconstructs the object. A persistent storage area of another disk is scanned to find a unique identifier of a failed disk. If using erasure coding, a missing segment previously stored on the disk is identified. The method locates other segments. Missing segments are regenerated.

A plurality of storage nodes in a single chassis is provided. The plurality of storage nodes in the single chassis is configured to communicate together as a storage cluster. Each of the plurality of storage nodes includes nonvolatile solid-state memory for user data storage. The plurality of storage nodes is configured to distribute the user data and metadata associated with the user data throughout the plurality of storage nodes such that the plurality of storage nodes maintain the ability to read the user data, using erasure coding, despite a loss of two of the plurality of storage nodes. The chassis includes power distribution, a high speed communication bus and the ability to install one or more storage nodes which may use the power distribution and communication bus. A method for accessing user data in a plurality of storage nodes having nonvolatile solid-state memory is also provided.

A method for storing data in a fault-tolerant storage subsystem having an array of failure independent data storage units, by dividing the storage area on the storage units into a logical mirror area and a logical stripe area, such that when storing data in the mirror area, duplicating the data by keeping a duplicate copy of the data on a pair of storage units, and when storing data in the stripe area, storing data as stripes of blocks, including data blocks and associated error-correction blocks.

A system and method for using cameras to download data to cell phones or other devices as an alternative to CDMA/GPRS, BlueTooth, Infrared or cable connections. The data is encoded as a sequence of images such as 2D bar codes, which can be displayed in any flat panel display, acquired by a camera, and decoded by software embedded in the device. The decoded data is written to a file. The system and method meet the following challenges: (1) To encode arbitrary data as a sequence of images. (2) To process captured images under various lighting variations and perspective distortions while maintaining real time performance. (3) To decode the processed images robustly even when partial data is lost.

Embodiments of the solid-state storage system provided herein are configured to perform improved mechanisms for testing of error recovery of solid state storage devices. In some embodiments, the system is configured to introduce or inject errors into data storage commands or operations performed in the non-volatile memory. Injected errors include corruption of data stored in the non-volatile memory, deliberate failure to execute storage operations, and errors injected into communication protocols used between various elements of the device. In some embodiments, injected errors can include direct errors that trigger an immediate execution of error recovery mechanisms and delayed errors that trigger execution of error recovery mechanisms at a later time. Error recovery mechanisms can be tested in an efficient, reliable, and deterministic manner to help ensure effective operation of storage devices. The integrity of non-volatile memory can also be tested.

Systems and methods are disclosed for storing the firmware and other data of a flash memory controller, such as using a RAID configuration across multiple flash memory devices or portions of a single memory device. In various embodiments, the firmware and other data used by a controller, and error correction information, such as parity information for RAID configuration, may be stored across multiple flash memory devices, multiple planes of a multi-plane flash memory device, or across multiple blocks or pages of a single flash memory device. The controller may detect the failure of a memory device or a portion thereof, and reconstruct the firmware and/or other data from the other memory devices or portions thereof.

A NAND controller for interfacing between a host device and a flash memory device (e.g., a NAND flash memory device) fabricated on a flash die is disclosed. In some embodiments, the presently disclosed NAND controller includes electronic circuitry fabricated on a controller die, the controller die being distinct from the flash die, a first interface (e.g. a host-type interface, for example, a NAND interface) for interfacing between the electronic circuitry and the flash memory device, and a second interface (e.g. a flash-type interface) for interfacing between the controller and the host device, wherein the second interface is a NAND interface. According to some embodiments, the first interface is an inter-die interface. According to some embodiments, the first interface is a NAND interface. Systems including the presently disclosed NAND controller are also disclosed. Methods for assembling the aforementioned systems, and for reading and writing data using NAND controllers are also disclosed.

A cluster recovery process is implemented across a set of distributed archives, where each individual archive is a storage cluster of preferably symmetric nodes. Each node of a cluster typically executes an instance of an application that provides object-based storage of fixed content data and associated metadata. According to the storage method, an association or “link” between a first cluster and a second cluster is first established to facilitate replication. The first cluster is sometimes referred to as a “primary” whereas the “second” cluster is sometimes referred to as a “replica.” Once the link is made, the first cluster's fixed content data and metadata are then replicated from the first cluster to the second cluster, preferably in a continuous manner. Upon a failure of the first cluster, however, a failover operation occurs, and clients of the first cluster are redirected to the second cluster. Upon repair or replacement of the first cluster (a “restore”), the repaired or replaced first cluster resumes authority for servicing the clients of the first cluster. This restore operation preferably occurs in two stages: a “fast recovery” stage that involves preferably “bulk” transfer of the first cluster metadata, following by a “fail back” stage that involves the transfer of the fixed content data. Upon receipt of the metadata from the second cluster, the repaired or replaced first cluster resumes authority for the clients irrespective of whether the fail back stage has completed or even begun.

A method for storing data in a memory (28) that includes analog memory cells (32) includes identifying one or more defective memory cells in a group of the analog memory cells. An Error Correction Code (ECC) is selected responsively to a characteristic of the identified defective memory cells. The data is encoded using the selected ECC and the encoded data is stored in the group of the analog memory cells. In an alternative method, an identification of one or more defective memory cells among the analog memory cells is generated. Analog values are read from the analog memory cells in which the encoded data were stored, including at least one of the defective memory cells. The analog values are processed using an ECC decoding process responsively to the identification of the at least one of the defective memory cells, so as to reconstruct the data.