419 results about "Data placement" patented technology

An apparatus and method defines a markup language for accessing data in a database. The markup language is preferably defined in extensible markup language (XML) by creating suitable document type definitions (DTDs), which define the grammar for accessing data in the database using the markup language. A bridge interprets the data request from the client in markup language format, a suitable database query for the database is formulated, and the data is then placed within a document for delivery in markup language format to the user. As new data types are added to the database, corresponding document type definitions (DTDs) may be dynamically generated, allowing a user to access new kinds of data in a database with a software tool that has a user-friendly graphical user interface without having to manually update the software tool for each new data type that is added to the database.

A hybrid media storage architecture has a log-structured file system configured to control a plurality of different storage media organized as hybrid storage media that cooperate to provide a total storage space of a storage system. The log-structured file system is configured to perform initial placement and migration of data, as well as fine-grain write allocation of the data, among storage space locations of the hybrid storage media to thereby improve the performance characteristics of the media. By defining and implementing heuristics and policies directed to, e.g., types of data, the file system may initially place data on any of the different media and thereafter migrate data between the media at fine granularity and without the need for manual enforcement.

A Small Computer System Interface (SCSI) transport for fabric backplane enterprise servers provides for local and remote communication of storage system information between storage sub-system elements of an ES system and other elements of an ES system via a storage interface. The transport includes encapsulation of information for communication via a reliable transport implemented in part across a cellifying switch fabric. The transport may optionally include communication via Ethernet frames over any of a local network or the Internet. Remote Direct Memory Access (RDMA) and Direct Data Placement (DDP) protocols are used to communicate the information (commands, responses, and data) between SCSI initiator and target end-points. A Fibre Channel Module (FCM) may be operated as a SCSI target providing a storage interface to any of a Processor Memory Module (PMM), a System Control Module (SCM), and an OffLoad Module (OLM) operated as a SCSI initiator.

A map-reduce compatible distributed file system that consists of successive component layers that each provide the basis on which the next layer is built provides transactional read-write-update semantics with file chunk replication and huge file-create rates. A primitive storage layer (storage pools) knits together raw block stores and provides a storage mechanism for containers and transaction logs. Storage pools are manipulated by individual file servers. Containers provide the fundamental basis for data replication, relocation, and transactional updates. A container location database allows containers to be found among all file servers, as well as defining precedence among replicas of containers to organize transactional updates of container contents. Volumes facilitate control of data placement, creation of snapshots and mirrors, and retention of a variety of control and policy information. Key-value stores relate keys to data for such purposes as directories, container location maps, and offset maps in compressed files.

A user interface for parsing unstructured data using pattern recognition. The patterns used in parsing data are formed from regular expressions. The parsed data may be displayed in a first format and unmatched strings in the unstructured text may be displayed in a second format. A format may comprise a desired color, font or any other user interface parameter. In addition, the data that parses according to a pattern may be placed in a column associated with the pattern in a tabular user interface, for example a spreadsheet like Excel™. Associating a pattern with a position to display successful matches in allows for breaking unstructured text into pieces associated with a particular field or column. Modification of the patterns allows for more and more of the unstructured text to match the patterns and when the data has been parsed to the desired level, the data may be imported into a database.

Embodiments of the present invention enable dynamic repartitioning of data storage in response to one or more triggers. In embodiments, a trigger may be a user-initiated action, a system-generated action, and/or an inference based on storage usage parameters. Applications of the present invention are its use in embodiments of a storage management system comprising a file system manager and a volume manager, where the placement of data into a partition (data storage region) may be specified by matching one or more disk region placement data attributes assigned to data with corresponding disk region attributes. In embodiments, dynamic repartitioning comprises adjustment of the location of the boundary between adjacent disk partitions and, if necessary, rebalancing of the data stored within the partitions by identifying mismatched data and relocating it to the partition with which it is associated.

A method and system for evaluating a path query are disclosed. The path query corresponds to a query tree including a plurality of query nodes. At least one query node corresponds to at least one predicate and is at a level. The predicate(s) are evaluated for previous query node(s). The method and system include scanning data nodes of a document and determining if the data nodes match the query nodes. The method and system also include placing data related to the data node in match stacks corresponding to matched query nodes. The data for the query node(s) include attribute(s) corresponding to the predicate(s). The method and system further include propagating a matching of the at least one query node backward to a matching of the at least one previous query node.

Embodiments of the present invention provide memory systems having a plurality of memory devices sharing an interface for the transmission of read data. A controller can identify consecutive read requests sent to different memory devices. To avoid data contention on the interface, for example, the controller can be configured to delay the time until read data corresponding to the second read request is placed on the interface.

The present invention provides a method and apparatus for searching multiple strings within a packet data using deterministic finite automata. The apparatus includes means for updating memory tables stored in a layered memory architecture comprising a BRAM, an SRAM and a DRAM; a mechanism to strategically store the relevant data structure in the three memories based on the characteristics of data, size/capacity of the data structure, and frequency of access. The apparatus intelligently and efficiently places the associated data in different memories based on the observed fact that density of most rule-sets is around 10% for common data in typical network intrusion prevention systems. The methodology and layered memory architecture enable the apparatus implementing the present invention to achieve data processing line rates over 2 Gbps.

A system and method for placement of sharing physical buffer lists in RDMA communication. According to one embodiment, a network adapter system for use in a computer system includes a host processor and host memory and is capable for use in network communication in accordance with a direct data placement (DDP) protocol. The DDP protocol specifies tagged and untagged data movement into a connection-specific application buffer in a contiguous region of virtual memory space of a corresponding endpoint computer application executing on said host processor. The DDP protocol specifies the permissibility of memory regions in host memory and specifies the permissibility of at least one memory window within a memory region. The memory regions and memory windows have independently definable application access rights, the network adapter system includes adapter memory and a plurality of physical buffer lists in the adapter memory. Each physical buffer list specifies physical address locations of host memory corresponding to one of said memory regions. A plurality of steering tag records are in the adapter memory, each steering tag record corresponding to a steering tag. Each steering tag record specifies memory locations and access permissions for one of a memory region and a memory window. Each physical buffer list is capable of having a one to many correspondence with steering tag records such that many memory windows may share a single physical buffer list. According to another embodiment, each steering tag record includes a pointer to a corresponding physical buffer list.

A system and method for placement of RDMA payload into application memory of a processor system. Under one embodiment, a network adapter system is capable of use in network communication in accordance with a direct data placement (DDP) protocol, e.g., RDMA. The network adapter system includes adapter memory and a plurality of placement records in the adapter memory. Each placement record specifies per-connection placement data including at least network address information and port identifications of source and destination network entities for a corresponding DDP protocol connection. Placement record identification logic uniquely identifies a placement record from network address information and port identification information contained in a DDP message received by the network adapter system. Untagged message payload placement logic directly places the payload of the received untagged DDP message into physical address locations of host memory corresponding to one of said connection-specific application buffers. Tagged message payload placement logic directly places the payload of the tagged DDP message into physical address locations of host memory corresponding to the identifier in the received DDP message. According to one embodiment, the placement records are organized as an array of hash buckets with each element of the array containing a placement record and each placement record containing a specification of a next placement record in the same bucket. The placement record identification logic includes hashing logic to create a hash index pointing to a bucket in the array by hashing a 4-tuple consisting of a source address, a destination address, a source port, and a destination port contained in the received DDP message.

An RNIC implementation that performs direct data placement to memory where all segments of a particular connection are aligned, or moves data through reassembly buffers where all segments of a particular connection are non-aligned. The type of connection that cuts-through without accessing the reassembly buffers is referred to as a “Fast” connection because it is highly likely to be aligned, while the other type is referred to as a “Slow” connection. When a consumer establishes a connection, it specifies a connection type. The connection type can change from Fast to Slow and back. The invention reduces memory bandwidth, latency, error recovery using TCP retransmit and provides for a “graceful recovery” from an empty receive queue. The implementation also may conduct CRC validation for a majority of inbound DDP segments in the Fast connection before sending a TCP acknowledgement (Ack) confirming segment reception.

System and method for automated page-based management in storage systems. The system includes host computers, file servers and a storage system having automated page-based management means. The storage system interface receives instructions to change the condition for decision for migration regarding particular parts or the whole volume. The host computer can control execution of the migration performed by the storage system by specifying areas or volumes with the condition via the interface. Highly optimized, appropriate data placement and data relocation in computer system can be achieved when the application, host computer or management computer can recognize or predict the usage of the data or files. The storage system having automated page-based management may include compression/decompression and a control method for the compression and decompression process.

An RNIC implementation that performs direct data placement to memory where all segments of a particular connection are aligned, or moves data through reassembly buffers where all segments of a particular connection are non-aligned. The type of connection that cuts-through without accessing the reassembly buffers is referred to as a “Fast” connection because it is highly likely to be aligned, while the other type is referred to as a “Slow” connection. When a consumer establishes a connection, it specifies a connection type. The connection type can change from Fast to Slow and back. The invention reduces memory bandwidth, latency, error recovery using TCP retransmit and provides for a “graceful recovery” from an empty receive queue. The implementation also may conduct CRC validation for a majority of inbound DDP segments in the Fast connection before sending a TCP acknowledgement (Ack) confirming segment reception.

A method and system for handling or processing out-of-order TCP segments in a wireless system may comprise at least one of: placing a first TCP segment received by a wireless network processor in a host buffer and having a mapping between a TCP sequence number and a corresponding buffer address. It may be determined whether a second TCP segment received by the wireless network processor is one of an in-order TCP segment or an out-of-order TCP segment. If the second received TCP segment is an out-of-order TCP segment, then control information associated with at least the second TCP segment may be stored locally on the wireless network processor. The out-of-order TCP segment may be placed in a portion of the host buffer.

A technique places content, such as data, of one or more data containers on volumes of a striped volume set (SVS). The placement of data across the volumes of the SVS allows specification of a deterministic pattern of fixed length. That is, the pattern determines a placement of data of a data container that is striped among the volumes of the SVS. The placement pattern is such that the stripes are distributed exactly or nearly equally among the volumes and that, within any local span of a small multiple of the number of volumes, the stripes are distributed nearly equally among the volumes. The placement pattern is also substantially similar for a plurality of SVSs having different numbers of volumes.

A method is used in managing data in a data storage system. A tiered storage pool is identified wherein the storage pool includes multiple tiers having multiple storage units. A wear indicator for each of the multiple storage units is monitored. A first storage unit having a wear indicator greater than a second storage unit is identified. High activity data on the first storage unit is also identified. The identified high activity data is migrated from the first storage unit to the second storage unit.

A data migration system supports a low-latency and reduced overhead data storage protocol for data storage sharing in a non-collision fashion which does not require inter-communication and permanent arbitration between data storage controllers to decide on the data placement/routing. The multiple data fragments of data sets are prevented from routing to the same storage devices by a multi-step selection protocol which selects (in a first phase of the selection routine) a healthy highest ranked drive enclosure, and further selects (in a second phase of the selection routine) a healthy highest-ranked data storage controller residing in the selected drive enclosure, for routing data fragments to different storage pools assigned to the selected data storage devices for exclusive “writing” and data modification. The selection protocol also contemplates various failure scenarios in a data placement collision free manner.

A system and method for reducing the overhead associated with direct data placement is provided. Processing time overhead is reduced by implementing packet-processing logic in hardware. Storage space overhead is reduced by combining results of hardware-based packet-processing logic with ULP software support; parameters relevant to direct data placement are extracted during packet-processing and provided to a control structure instantiation. Subsequently, payload data received at a network adapter is directly placed in memory in accordance with parameters previously stored in a control structure. Additionally, packet-processing in hardware reduces interrupt overhead by issuing system interrupts in conjunction with packet boundaries. In this manner, wire-speed direct data placement is approached, zero copy is achieved, and per byte overhead is reduced with respect to the amount of data transferred over an individual network connection. Movement of ULP data between application-layer program memories is thereby accelerated without a fully offloaded TCP protocol stack implementation.

A growing amount of data is produced daily resulting in a growing demand for storage solutions. While cloud storage providers offer a virtually infinite storage capacity, data owners seek geographical and provider diversity in data placement, in order to avoid vendor lock-in and to increase availability and durability. Moreover, depending on the customer data access pattern, a certain cloud provider may be cheaper than another. In this respect is provided a method and a system that facilitates allocation of data objects in a distributed data storage environment. The system continuously adapts the placement of data based on its access patterns and subject to optimization objectives, such as storage costs. The system efficiently considers repositioning of only selected objects that may significantly lower the storage cost.

A method is used in managing data placement in storage systems. A portion of an allocated storage space is identified in a data storage system for managing data placement. The allocated storage space includes first and second set of slices. The first set of slices associated with the portion of the allocated storage space in the data storage system is identified. Data of the first set of slices is transferred to the second set of slices of the allocated storage space in the data storage system. The first set of slice is removed from the allocated storage space in the data storage system.

A method, apparatus and program storage device for providing intelligent rebuild order selection is a storage array is disclosed. The interrelationship of the data on the drives to be rebuilt and the remaining drives in the system is examined. A relative score for a rebuild record to be rebuilt is based upon the manner in which the RAID data is placed on the other drives. The rebuild record with the highest relative risk is identified as the rebuild record to be rebuilt next.