1212 results about "Data fragment" patented technology

More efficient querying of a multi-tenant database using dynamic tuning of database indices. A layer of meta-data associates data items with tenants, e.g., via tags, and the meta-data is used to optimize searches by channeling processing resources during a query to only those pieces of data bearing the relevant tenant's unique tag.

A method for creating and inserting an electronic mail signature fragment into an electronic mail message during the creation of the electronic mail message is disclosed. A computer-readable medium on which is stored a program module for creating and inserting an electronic mail signature fragment into an electronic mail message during the creation of the electronic mail message is also disclosed.

A synthetic speech system includes a phoneme segment storage section for storing multiple phoneme segment data pieces; a synthesis section for generating voice data from text by reading phoneme segment data pieces representing the pronunciation of an inputted text from the phoneme segment storage section and connecting the phoneme segment data pieces to each other; a computing section for computing a score indicating the unnaturalness of the voice data representing the synthetic speech of the text; a paraphrase storage section for storing multiple paraphrases of the multiple first phrases; a replacement section for searching the text and replacing with appropriate paraphrases; and a judgment section for outputting generated voice data on condition that the computed score is smaller than a reference value and for inputting the text after the replacement to the synthesis section to cause the synthesis section to further generate voice data for the text.

A system and method of making unstructured data available to structured data analysis tools. The system includes middleware software that can be used in combination with structured data tools to perform analysis on both structured and unstructured data. Data can be read from a wide variety of unstructured sources. The data may then be transformed with commercial data transformation products that may, for example, extract individual pieces of data and determine relationships between the extracted data. The transformed data and relationships may then be passed through an extraction/transform/load (ETL) layer and placed in a structured schema. The structured schema may then be made available to commercial or proprietary structured data analysis tools.

Method of dividing data to be stored in storage device into data fragments; recording the data by using configurations of divided data fragments; judging whether identical data fragments exist in data fragments; when it is judged that identical data fragments exist, storing one of the identical data fragments in storage area of the storage device, and generating and recording data-fragment attribute information indicating an attribute unique to the data fragment stored; upon receipt of request to read data stored in the storage area of the storage device, acquiring the configurations of the data fragments forming the read-target data, reading the corresponding data fragments from the storage area of the storage device, and restoring the data; acquiring and coupling the recorded data fragments to generate concatenation target data targeted for judgment on whether chunk concatenation is possible or not, and detecting whether the concatenation target data has a repeated data pattern

An infrastructure for archiving data among a client, a broker, and a plurality of archives, wherein the client comprises: a backup agent configured to fragment and erasure encode the data to create a set of erasure encoded data fragments; a communications agent configured to communicate the erasure encoded data fragments to the broker, issue a challenge for a challenge/response protocol to the broker, and to request data from the archives; and a restore agent configured to combine the data fragments obtained from the broker upon a data restore request.

A system and method of making unstructured data available to structured data analysis tools. The system includes middleware software that can be used in combination with structured data tools to perform analysis on both structured and unstructured data. Data can be read from a wide variety of unstructured sources. The data may then be transformed with commercial data transformation products that may, for example, extract individual pieces of data and determine relationships between the extracted data. The transformed data and relationships may then be passed through an extraction/transform/load (ETL) layer and placed in a structured schema. The structured schema may then be made available to commercial or proprietary structured data analysis tools.

A flash system has multiple channels of flash memory chips that can be accessed in parallel. Host data is assigned to one of the channels by a multi-channel controller processor and accumulated in a multi-channel page buffer. When a page boundary in the page buffer is reached, the page buffer is written to a target physical block if full, or combined with old data fragments in an Aggregating Flash Block (AFB) when the logical-sector addresses (LSA's) match. Thus small fragments are aggregated using the AFB, reducing erases and wear of flash blocks. The page buffer is copied to the AFB when a STOP command occurs. Each channel has one or more AFB's, which are tracked by an AFB tracking table.

A system and method of making unstructured data available to structured data analysis tools. The system includes middleware software that can be used in combination with structured data tools to perform analysis on both structured and unstructured data. Data can be read from a wide variety of unstructured sources. The data may then be transformed with commercial data transformation products that may, for example, extract individual pieces of data and determine relationships between the extracted data. The transformed data and relationships may then be passed through an extraction/transform/load (ETL) layer and placed in a structured schema. The structured schema may then be made available to commercial or proprietary structured data analysis tools.

A method and apparatus for detecting policy violations in information content containing data in a character-based language is described. In one embodiment, the method includes identifying a policy for protecting source data having a tabular format. The source data contains one or more data fragments in the character-based language. The method further includes receiving information content having at least a portion in the character-based language, and determining whether any part of the information content, including the portion in the character-based language, violates the policy.

A method for distributively storing a data object on a data storage system includes determining a layout map ID and associating the layout map ID with the data object. The layout map ID is used in determining a layout map, and the layout map is used in determining a layout given a current configuration of the data storage system. The data object is fragmented into data fragments and parity fragments, and the data fragments and parity fragments are distributively stored on data storage system in accordance with the layout.

Methods systems and computer readable media for manipulating and displaying sparse data contained within very large datasets. Particular applications includes those to proteomics and metabolomics. Navigation of the large data sets from a global perspective may be practically implemented, so that a user is not confined to a narrow slice of data at a time, thus providing context of the whole dataset while viewing data within the dataset.

A system for structuring digitally stored information and methods for searching the resulting structure are described. The structuring system comprises a database comprising a number of cells arranged in rows and columns for holding pieces of data representing the information. Only one piece of data may be stored in each cell and the pieces of data contained in cells in one row of the database constitutes a certain information. This information may be files on a computer hard disk, pages on the www etc. An index for each unique piece of data occurring in the database, where each index table providing information concerning all locations of that unique piece of data in the database, a main index listing once all the unique pieces of data in the database together with a corresponding index table identifier, the index table identifier providing a link to the corresponding index table for that particular piece of data, are also necessary elements in the structuring system, The structure of the database/index system in itself provides fast and simple access to the information contained therein. The information structuring system and the methods for structuring and searching digitally stored information may e.g. be used in a search engine for the Internet, in a mobile phone, WAP or as a search system for retrieving files and controlling the information stored in a LAN or computer hard disk.

A method for recovering user data from a host device stored on a data storage medium where a said data may become corrupted during a read operation comprises: performing a read operation to read at least one encoded data fragment of said plurality of encoded data fragments from said data storage medium; storing status data indicating whether said at least one data fragment was correctly read; writing said read at least one data fragment to a buffer memory; attempting to decode said at least one encoded data fragment in said buffer memory wherein if said attempted decoding is unsuccessful then said method further comprises the steps of: re-reading incorrectly read encoded data fragments from said data storage medium; and attempting to decode said encoded data fragments located in said buffer memory.

The invention discloses a data storage method of a cloud storage system, which comprises the steps of: carrying out coding fragmentation on a file on the basis of a buffer zone erasure code technology to obtain data fragments and check fragments; processing the data fragments and the check fragments to obtain a check identification; storing the check identification in an index server; requesting storage resource information to the index server according to the quantity of the data fragments and the check fragments; sending the data fragments and the check fragments to a plurality of storage nodes according to the storage resource information; judging whether needing to carry out integrity check on the data fragments and the check fragments in the storage nodes; if needing to carry out integrity check on the data fragments and the check fragments in the storage nodes, sending the data integrity check information to the storage nodes, receiving an identification information set returned by the storage nodes, and judging whether data storage errors or data losses exist according to the identification information set, if yes, coding for recovering the data, and sending the recovered data to the corresponding storage nodes.

The invention discloses a distributed storage oriented cloud storage security architecture and a data access method thereof. The distributed cloud storage oriented security architecture comprises a cloud storage server, a distributed storage manager, a wafer breaker and storage devices, wherein the distributed storage manager and the cloud storage server are respectively connected with a client-side; the distributed storage manager and the cloud storage server are connected by a fiber channel (FC) or a routing switch; the wafer breaker is imbedded in the distributed storage manager; the distributed storage manager is connected with the storage devices by the FC or Iscsi; and the storage devices are connected by a network attached storage (NAS) or a storage area network (SAN). The storage devices are located at different places and are redundant mutually, thus improving the fault-tolerant ability of the devices and the optimal storage use ratio; the wafer breaker is used for carrying out fragment on stored data, so that the data are fragmented into data fragments which can not be identified by other authentication systems, and the fragmented data have relative confidentiality and security in the process of network transmission and data storage; and the storage devices and the server are the devices with higher cost performance, thus reducing the cost.

The invention discloses a data storage method, data storage equipment and a distributed storage system. The method, the equipment and the system belong to the technical field of computers. The data storage method comprises the steps that access nodes of the distributed storage system cut a data block which is written in by a user into a plurality of data fragments with preset sizes; a plurality of check fragments corresponding to the data fragments which are obtained by cutting are calculated through a redundancy check algorithm; the data fragments and the check fragments are stored into the data nodes of the distributed storage system, and each data fragment and each check fragment only store one copy. According to the data storage method, the data storage equipment and the distributed storage system, the proportion of redundant data can be reduced on the premise of ensuring data reliability, so as to save the storage space.

A method begins with a processing module of a dispersed storage network (DSN) identifying an encoded data slice of a set of encoded data slices that requires rebuilding and identifying storage units of the DSN that store the set of encoded data slices. The method continues with the processing module determining a rebuilding metric regarding the identified encoded data slice and selecting a sub-set of the storage units for retrieving a decode threshold number of encoded data slices of the set of encoded data slices based on the rebuilding metric. When the decode threshold number of encoded data slices have been retrieved, the method continues with the processing module decoding the decode threshold number of encoded data slices to produce a reconstructed data segment and generating a rebuilt encoded data slice from the reconstructed data segment.

A method of generating IP identification numbers for IP datagrams. A plurality of IP identification number generators are maintained. A plurality of receiving stations are associated with the plurality of IP identification number generators such that each receiving station has an IP identification number generator associated therewith. An IP identification number is generated for a datagram sent to a receiving station based on an output of the associated IP identification number generator. Also, a method of reducing a likelihood of misassembly of received data fragments from fragmented IP datagrams. The data fragments have an IP identification number. All received data fragments of the datagram are discarded upon detection of receipt of an overlapping data fragment having the IP identification number. Timeouts for reassembling datagrams are reduced. First, timeouts for reassembling the datagrams are reduced to less than a standard timeout. Second, timeouts are reduced upon detection of a gap in the received data fragments. Third, timeouts are reduced upon detection of a datagram having the same protocol identification number and source address, but having a different IP identification number. Furthermore, a method of detecting a likelihood of misassembly of data fragments from fragmented IP datagrams. Communication errors between a sending station and a receiving station are detected. A rate is determined at which an IP identification number generator associated with the receiving station wraps around. NFS re-transmissions are determined. Based on one or more of these, a likelihood of misassembly at the receiving station is determined to be high.

The invention discloses a data storage method for improving the storage consistency of a distributed memory system in an embodiment. The method comprises: a major storage node performs EC coding on a data fragment to be stored to obtain a target EC stripe; (m+k) target storage nodes for storing (m+k) target EC blocks of the target EC stripe are determined in a storage node group; a preparation message is sent to the target storage nodes; a response sent by each target storage node is received; and after the response messages of all the target storage nodes are received, an execution message is sent to the target storage nodes to instruct the target storage nodes to write in the target EC blocs in a preparation log. The embodiment of the invention further provides a data recovery method and a relevant device.

A data migration system supports a low-latency and reduced overhead data storage protocol for data storage sharing in a non-collision fashion which does not require inter-communication and permanent arbitration between data storage controllers to decide on the data placement/routing. The multiple data fragments of data sets are prevented from routing to the same storage devices by a multi-step selection protocol which selects (in a first phase of the selection routine) a healthy highest ranked drive enclosure, and further selects (in a second phase of the selection routine) a healthy highest-ranked data storage controller residing in the selected drive enclosure, for routing data fragments to different storage pools assigned to the selected data storage devices for exclusive “writing” and data modification. The selection protocol also contemplates various failure scenarios in a data placement collision free manner.