214 results about "Zero-copy" patented technology

A number of improvements in network adapters that offload protocol processing from the host processor are provided. Specifically, mechanisms for handling memory management and optimization within a system utilizing an offload network adapter are provided. The memory management mechanism permits both buffered sending and receiving of data as well as zero-copy sending and receiving of data. In addition, the memory management mechanism permits grouping of DMA buffers that can be shared among specified connections based on any number of attributes. The memory management mechanism further permits partial send and receive buffer operation, delaying of DMA requests so that they may be communicated to the host system in bulk, and expedited transfer of data to the host system.

The invention proposes a network device driver architecture with functionality distributed between kernel space and user space. The overall network device driver comprises a kernel-space device driver (10) and user-space device driver functionality (20). The kernel-space device driver (10) is adapted for enabling access to the user-space device driver functionality (20) via a kernel-space-user-space interface (15). The user-space device driver functionality (20) is adapted for enabling direct access between user space and the NIC (30) via a user-space-NIC interface (25), and also adapted for interconnecting the kernel-space-user-space interface (15) and the user-space-NIC interface (25) to provide integrated kernel-space access and user-space access to the NIC (30). The user-space device driver functionality (20) provides direct, zero-copy user-space access to the NIC, whereas information to be transferred between kernel space and the NIC will be “tunneled” through user space by combined use of the kernel-space device driver (10), the user-space device driver functionality (20) and the two associated interfaces (15,25).

The apparatus comprises: a zero-copy initializing module used for allocating a segment of inner core space in the inner core, and dividing the inner cored space into multi data blocks, and identifying each block; a network card driving module used for sending the received data packet to the data block in the inner core to save; recording the identifier of the data block in skb of the data packet, and sending the skb to the protocol stack; a protocol stack used for receiving and analyze the skb of the data packet, and getting the identifier of the data block; a identifier buffer connected to the protocol stack and is directly mapped to the user progress module, and is used for saving the identifier of data block obtained by the protocol stack; a user progress module used for getting the identifier from the identifier buffer, and getting the data packet from the data block in the inner core space.

Methods and apparatus for transferring data from an application server are provided. By offloading network and file system stacks to a common stack accessible by multiple operating systems in a virtual computing system, embodiments of the present invention may achieve data transfer support for web and application servers without the data needing to be copied to or reside in the address space of the server operating systems.

One embodiment of the present invention provides a system for sending data to a remote host using a socket. During operation the system receives a request from an application to write data to the socket, wherein the data is stored in a source memory buffer in user memory. Next, the system initiates a DMA (Direct Memory Access) transfer to transfer the data from the source memory buffer to a target memory buffer in a TCP (Transmission Control Protocol) Offload Engine. The system then returns control to the application without waiting for the TCP Offload Engine to send the data to the remote host.

One embodiment of the present invention sets forth a technique for reducing the copying of data between memory allocated to a primary processor and a coprocessor is disclosed. The system memory is aliased as device memory to allow the coprocessor and the primary processor to share the same portion of memory. Either device may write and/or read the shared portion of memory to transfer data between the devices rather than copying data from a portion of memory that is only accessible by one device to a different portion of memory that is only accessible by the other device. Removal of the need for explicit primary processor memory to coprocessor memory and coprocessor memory to primary processor memory copies improves the performance of the application and reduces physical memory requirements for the application since one portion of memory is shared rather than allocating separate private portions of memory.

A system and method for reducing the overhead associated with direct data placement is provided. Processing time overhead is reduced by implementing packet-processing logic in hardware. Storage space overhead is reduced by combining results of hardware-based packet-processing logic with ULP software support; parameters relevant to direct data placement are extracted during packet-processing and provided to a control structure instantiation. Subsequently, payload data received at a network adapter is directly placed in memory in accordance with parameters previously stored in a control structure. Additionally, packet-processing in hardware reduces interrupt overhead by issuing system interrupts in conjunction with packet boundaries. In this manner, wire-speed direct data placement is approached, zero copy is achieved, and per byte overhead is reduced with respect to the amount of data transferred over an individual network connection. Movement of ULP data between application-layer program memories is thereby accelerated without a fully offloaded TCP protocol stack implementation.

A system and method enable a storage operating system to partition data into fixed sized data blocks that can be written to disk without having to copy the contents of memory buffers (mbufs). The storage operating system receives data from a network and stores the data in chains of mbufs having various lengths. However, the operating system implements a file system that manipulates data in fixed sized data blocks. Therefore, a set of buffer pointers is generated by the file system to define a fixed sized block of data stored in the mbufs. The set of buffer pointers address various portions of data stored in one or more mbufs, and the union of the data portions form a single fixed sized data block. A buffer header stores the set of pointers associated with a given data block, and the buffer header is passed among different layers in the storage operating system. Thus, received data is partitioned into one or more fixed sized data blocks each defined by a set of buffer pointers stored in a corresponding buffer header. Because the buffer pointers address data directly in one or more mbufs, the file system does not need to copy data out of the mbufs when partitioning the data into fixed sized data blocks.

The invention discloses a method and a system which can send stream media data with zero copy, which is based on the network protocol stack of Linux. The method of the invention comprises the steps as follows: when a stream media server receives the data request of user equipment, the system calling of data to be transmitted is carried out; the stream media data is read into a user data cache from the hard disk space; the stream media data memorized in the user data cache is packaged as real-time transmission protocol data packages; the transmission of stream media data package with separation of head and load is adopted for the real-time transmission protocol data. The method and the system of the invention sufficiently use the DMA function and SG (Scatter/Gather) function of a network card and realizes a type of stream media data transmission with zero-copy; compared with the existing network protocol stack of Linux kernel, the method and the system of the invention realize the transmission type of stream media data packages with separated head and load, and reduces the once data copy operation required during the RTP packaging process of the stream media data.

A computer-implemented system and method for a lock-less, zero data copy messaging mechanism in a multi-core processor for use on a modem in a telecommunications network are described herein. The method includes, for each of a plurality of processing cores, acquiring a kernel to user-space (K-U) mapped buffer and corresponding buffer descriptor, inserting a data packet into the buffer; and inserting the buffer descriptor into a circular buffer. The method further includes creating a frame descriptor containing the K-U mapped buffer pointer, inserting the frame descriptor onto a frame queue specified by a dynamic PCD rule mapping IP addresses to frame queues, and creating a buffer descriptor from the frame descriptor.

Mechanisms to implement memory management to enable protocol-aware asynchronous, zero-copy transmits. A transport protocol engine exposes interfaces via which memory buffers from a memory pool in operating system (OS) kernel space may be allocated to applications running in an OS user layer. The memory buffers may be used to store data that is to be transferred to a network destination using a zero-copy transmit mechanism, wherein the data is directly transmitted from the memory buffers to the network via a network interface controller. The transport protocol engine also exposes a buffer reuse API to the user layer to enable applications to obtain buffer availability information maintained by the protocol engine. In view of the buffer availability information, the application may adjust its data transfer rate.

A capturing method for high speed network data packet based on the zero-copy technology, relating to field of network flow capacity monitoring and analysis. The method comprises a computer at least with one network card; the computer is arranged with operation system and data packet capturing program; the data packet capturing program captures the data packet via the network card. The method comprises following steps: 1) initialization of the network card, 2) initialization of the data packet capturing program, 3) starting data capture, being characterized in that: steps of the starting data capture are: a, the network card of a hardware layer passes the received data packet to the network card driven program of the core layer via DMA way; b, the data packet capturing program of a user layer obtains the data packet received by the network card driven program via mapping way. The capturing method for high speed network data packet based on zero-copy technology of this invention implements high speed network data packet capture based on the hardware platform and zero-copy technology. The invention has advantages of low system expense, high efficiency of capture and low reclaiming cost.

The invention discloses a flash memory storage system and a reading, writing and deleting method thereof. The flash memory storage system comprises a cache, a main control module, a cache metadata record sheet, a read mapping table and a write mapping table. The write mapping table is used for being stored in the cache and writing in the correspondence of a logical storage block and a physical storage block. The read mapping table is used for being stored in the cache and reading out the correspondence of the logical storage block and the physical storage block. The cache metadata record sheet is used for storing correspondences of metadata sheet addresses, the physical storage blocks and rear end flash memory addresses. By means of the flash memory storage system, the unnecessary write-in or read-out of a rear end flash memory can be reduced, the zero copy on a read-write data channel is achieved, the unnecessary intermediate copy process is omitted, and therefore the read-write efficiency is improved; the size of the read-write access of a front end application can be matched with the size of the rear end flash memory.

The invention discloses a management method for network data transmission of zero copy buffer queue, which belongs to the field of management method of data buffer queue of network data zero copy. The technical proposal of the invention includes the following operating steps: A. a continuous-memory allocated in a user space is taken as a DMA buffer for messages; B. a control handle of the buffer is established inside a kernel-mode driver; C. the control handle is mapped by an application program through an interface repository so as to realize operation of the buffer; and D. when messages are transmitted, the physical address of a full buffer is written into a network card by the driver, a buffer filled with messages taken out from the full buffer is applied, then the buffer is positioned inside an empty buffer queue after the messages are transmitted. The invention is applicable to the flexible management of a message buffer by a zero copy driver.

A system and method for zero copy block protocol write operations obviates the need to copy the contents of memory buffers (mbufs) at a storage system. A storage operating system of the storage system receives data from a network and stores the data in chains of mbufs having various lengths. An iSCSI driver processes (interprets) the received mbufs and passes the appropriate data and interpreted command to a SCSI target module of the storage operating system. The SCSI target module utilizes the mbufs and the data contained therein to perform appropriate write operations in response to write requests received from clients.

A buffered message queue architecture for managing messages in a database management system is disclosed. A “buffered message queue” refers to a message queue implemented in a volatile memory, such as a RAM. The volatile memory may be a shared volatile memory that is accessible by a plurality of processes. The buffered message queue architecture supports a publish and subscribe communication mechanism, where the message producers and message consumers may be decoupled from and independent of each other. The buffered message queue architecture provides all the functionality of a persistent publish-subscriber messaging system, without ever having to store the messages in persistent storage. The buffered message queue architecture provides better performance and scalability since no persistent operations are needed and no UNDO/REDO logs need to be maintained. Messages published to the buffered message queue are delivered to all eligible subscribers at least once, even in the event of failures, as long as the application is “repeatable.” The buffered message queue architecture also includes management mechanisms for performing buffered message queue cleanup and also for providing unlimited size buffered message queues when limited amounts of shared memory are available. The architecture also includes “zero copy” buffered message queues and provides for transaction-based enqueue of messages.

A buffered message queue architecture for managing messages in a database management system is disclosed. A “buffered message queue” refers to a message queue implemented in a volatile memory, such as a RAM. The volatile memory may be a shared volatile memory that is accessible by a plurality of processes. The buffered message queue architecture supports a publish and subscribe communication mechanism, where the message producers and message consumers may be decoupled from and independent of each other. The buffered message queue architecture provides all the functionality of a persistent publish-subscriber messaging system, without ever having to store the messages in persistent storage. The buffered message queue architecture provides better performance and scalability since no persistent operations are needed and no UNDO/REDO logs need to be maintained. Messages published to the buffered message queue are delivered to all eligible subscribers at least once, even in the event of failures, as long as the application is “repeatable.” The buffered message queue architecture also includes management mechanisms for performing buffered message queue cleanup and also for providing unlimited size buffered message queues when limited amounts of shared memory are available. The architecture also includes “zero copy” buffered message queues and provides for transaction-based enqueue of messages.