65results about How to "Improve I/O efficiency" patented technology

A system for host-based virtualization optimizations in storage environments employing off-host virtualization may include a host, one or more physical storage devices, and an off-host virtualizer such as a virtualizing switch. The off-host virtualizer may be configured to aggregate storage within the one or more physical storage devices into a virtual storage device such as a logical volume, and to provide control data for the virtual storage device to the host. The host may be configured to use the control data to perform a function in response to an I/O request from a storage consumer directed at the virtual storage device, and to use a result of the function to coordinate one or more I/O operations corresponding to the I/O request.

The invention discloses a data page caching method for a file system of a solid-state hard disc, which comprises the following implementation steps of: (1) establishing a buffer link list used for caching data pages in a high-speed cache; (2) caching the data pages read in the solid-state hard disc in the buffer link list for access, classifying the data pages in the buffer link list into cold clean pages, hot clean pages, cold dirty pages and hot dirty pages in real time according to the access states and write access states of the data pages; (3) firstly searching a data page as a page to be replaced in the buffer link list according to the priority of the cold clean pages, the hot clean pages, the cold dirty pages and the hot dirty pages, and replacing the page to be replaced with a new data page read from the solid-state hard disc when a free space does not exist in the buffer link list. In the invention, the characteristics of the solid-state hard disc can be sufficiently utilized, the performance bottlenecks of the external storage can be effectively relieved, and the storage processing performance of the system can be improved; moreover, the data page caching method has the advantages of good I/O (Input/Output) performance, low replacement cost for cached pages, low expense and high hit rate.

The invention discloses a hash table concurrent access performance optimization method under a multi-core environment. The method comprises the steps that aiming at the concurrent connection treatment of high concurrent access of a hash table, a half-synchronous half-asynchronous network connection treatment mechanism is adopted; aiming at the concurrent data treatment of high concurrent access of the hash table, the hash table is divided into multiple independent sub hash tables according to the number of CPU cores, working threads correspond to the independent sub hash tables in a one-to-one mode, and each working thread is only in charge of the data of the corresponding sub hash table; a main thread selects the corresponding working threads to process by adopting a consistent hash strategy according to the Key of each datum; each sub hash table maintains one LRU queue, when a memory space is insufficient, cold data are deleted, and new elements are inserted. The method improves the concurrent connection treatment capacity and the concurrent access capacity of the hash table, and the synchronization overhead problem and the cache consistency overhead problem are solved when the shared hash table is visited by multiple threads.

A high speed interface type device can reduce power consumption and a circuit area, and transmit/receive a 4 bit data in one clock period. The high speed interface type device includes a DRAM unit for generating first clock and clock bar signals which do not have a phase difference from a main clock signal, and second clock and clock bar signals having 90° phase difference from the first clock and clock bar signals in a write operation, storing an inputted 4 bit data in one period of the main clock signal according to the first clock to second clock bar signals, synchronizing the stored data with data strobe signals according to the first clock to second clock bar signals in a read operation, and outputting a 4 bit data in one period of the main clock signal, and a controller for transmitting a command, address signal and data signal synchronized with the main clock signal to the DRAM unit in the write operation, and receiving data signals from the DRAM unit in the read operation.

The invention discloses a method for supporting the parallel input and output (I/O) of trace files in parallel simulation, in particular to a method for supporting the distributed parallel I/O of the Trace files in order to improve the I/O efficiency and the simulation precision of the Trace files in the parallel simulation. The technical scheme comprises the following steps: a host machine for running parallel simulators is constructed and comprises a main control node and simulation nodes; a main configuration program is executed on the main control node for global configuration; then, scheduling software A on the main control node schedules a simulator A on a simulation node to execute a simulation job; afterwards, Trace distribution software on the main control node generates and sends Trace mobile scripts to all destination simulation nodes, and each destination simulation node executes a Trace mobile script and moves a Trace file from a corresponding source node to the own node; and finally, scheduling software B on the main control node schedules a simulator B on a simulation node to execute a simulation job. The method can greatly improve the I/O efficiency of the Trace files and the simulation precision.

A web buffering method in a virtual machine environment is characterized by comprising the steps that the http connection of a virtual machine is intercepted through a virtual machine management program to obtain an http response data package of the http connection; a buffering identifier in the http response data package is obtained; whether the buffering identifier can be buffered or not is judged, if yes, the link address corresponding to the http response data package is obtained, data content of the http response data package is copied, and the extracted data content and the link address are correspondingly stored in a web buffering memory set by the virtual machine management program; the buffering identifier in the http response data package is set to be not capable of being buffered and is transmitted to a virtual machine corresponding to the http connection. In addition, the invention provides a web buffering device in the virtual machine environment. The web buffering method and device in the virtual machine environment can improve I/O efficiency.

The invention discloses data processing equipment and a data processing method thereof, and the data processing equipment and the data processing method are used in a relational database. The data processing method used in the relational database comprises the steps that: at least one table is selected from the relational database; each of the selected at least one table is divided into more than two sub-tables according to columns, wherein at least one of the sub-tables comprises at least two columns; and the sub-tables are written into tables of a corresponding row store database.

The invention relates to a system for realizing metadata cache and analysis based on NoSQL. The system comprises a metadata analyzing client side, a metadata analyzing server side, a NoSQL database server and a communication module. The invention further relates to a method adopting the system for realizing metadata cache and analysis based on the NoSQL. The method comprises a metadata caching method based on the NoSQL and realized through the adoption of the system and a metadata processing method based on the NoSQL and realized through the adoption of the metadata caching method. Through the adoption of the system for realizing metadata cache and analysis based on the NoSQL and the method for realizing metadata cache and analysis based on the NoSQL, according to actual requirement of metadata analysis system under a big data environment, a NoSQL data base is adopted to cache metadata and relevant inquiry methods are provided, the data accessing performance is improved, the response speed of metadata analysis is accelerated, the user experience is improved and the application range is wider.

An index structure, tendency B trees, to alleviate the high cost of string partial search in large data sets is presented. A tendency B tree is a two layered data structure, including a logical layer and a physical layer. In the logical layer, a tendency tree provides a hierarchical structure to group similar tendency features together to facilitate fast partial search for a given query. The physical layer is a B-tree like structure. In addition, the balanced topology of B trees provides consistent I/O complexity. The tendency B tree is dynamically compressed during the construction process to reduce storage and enhance search efficiency. Experiments on both dictionary search and DNA sequence search using tendency B trees show that consistent, fast search times can be achieved in large data sets, requiring lower space usage and linear construction time.

The invention discloses an RAID model. The RAID model comprises an RAID core verification module and an RAID core scheduling module; the RAID core verification module adopts an Erasure code verification algorithm of RS code on the basis of a Vandermonde matrix, the Erasure code verification algorithm supports N data blocks to generate M verification data, and the RAID core scheduling module is used for conducting unified scheduling on all data reading and writing operations of the RAID model; the RAID core scheduling module selects N correct data blocks through a scheduling algorithm, and corresponding M verification data is calculated through the RAID core verification module; an RAID core I/O module is used for executing data reading and writing operations of the RAID; the RAID core scheduling module creates an object container containing all data. According to the RAID model, the RAID core scheduling module conducts unified scheduling on all the data reading and writing operations of the RAID, a user does not need to conduct caching for length adaptation, the verification number can be customized, no state exists for attitude to data, overall reconstruction is not needed, and the reading and writing efficiency is high.

The embodiment of the invention discloses a method, a system and equipment for controlling data writing. The method comprises the following steps of obtaining disc distribution information of dirty data pages of a plurality of database instances; according to the disc distribution information of the dirty data pages, generating data writing strategies for the database instances; and distributing the data writing strategies to the database instances, so as to enable each database instance to write the dirty data pages in a buffer into a disc array according to the distributed data writing strategies. The method, the system and the equipment have the advantages that according to the disc distribution information of the database instances, the control equipment is used for uniformly controlling and coordinating the writing strategies of the dirty data pages of the database instances; and as the control equipment can control the database instances to averagely write the dirty data pages into discs in the disc array at each time according to the writing strategies, the I/O (input/output) efficiency of the disc array is improved.

The invention relates to a link switching method and system. The switching method comprises: a link state is detected in real time at a Qlogic layer; when occurrence of a fault is detected at the link, a netlink mechanism is triggered and fault information is sent to a socket that is established in advance in the Qlogic layer; the socket is detected in real time by a daemon thread that is established in advance and the fault information is received; according to a link switching instruction contained by the fault information, a new link is selected at a multipath tool layer; after the new link selection is completed, a first notification message for switching the link with the fault into the new link is sent; on the basis of the first notification message, an interrupted IO block is pointed to the new link at a multiplepath inner nuclear layer and a second notification message for link switching is sent to an SCSI layer; and according to the second notification message, the link switching is executed at the SCSI layer. On the basis of the scheme, the IO efficiency can be effectively improved.

The invention discloses a method for controlling the parallelism degree of a program capable of sensing the band width of a storage device. By means of the method, the parallelism degree of the application program is dynamically adjusted according to the comprehensive performance and real-time loading conditions of the storage device of an operating platform of the application program in a self-adaptive mode, namely the number of courses or threads, so that the application program keeps and obtains the parallelism degree with the optimal I/O efficiency. According to the method, the information of the I/O efficiency of the application program is monitored and recorded in real time and is used as feedback information to control and adjust the parallelism degree of the application program. The parallelism degree of the program is probed step by step and increased gradually until the parallelism degree reaches the inflection point of the actual I/O efficiency, and then fine adjustment is carried out to obtain the optimal parallelism degree for different application program platforms; further according to the real-time loading conditions of the different application program platforms, periodic, dynamic and self-adaptive operation combining active adjustment and passive adjustment is carried out to obtain the optimal parallelism degree of the application program.

The invention provides a data updating system and method based on a resource mutual exclusion scheduling model, which can be applied to the field of data processing and finance, and the system comprises a resource cache module which obtains market information push data, and stores the market information data into a preset cache queue according to the receiving time of the market information push data and the data category of each piece of market information data in the market information push data, a coordination thread module which obtains the resource type of each product, and constructs characteristic data corresponding to each product according to the resource type, acquires market data in the cache queue, and constructs a product set according to resource types and characteristic data of the market data and performs mutual exclusion scheduling according to resource types of the product set, distributes market information data to different execution queues, and calculates resource updating data of each product through working threads corresponding to the execution queues, and a synthesis calculation module which obtains resource update data calculated by each working thread, and updates corresponding resources in each product through the resource update data.