356 results about "Remote direct memory access" patented technology

Multipathing for session-based remote direct memory access (SRDMA) may be used for congestion management. A given SRDMA session group may be associated with multiple SRDMA sessions, each having its own unique 5-tuple. A queue pair (QP) associated with the SRDMA session group may provide a packet for transmission using the SRDMA session group. The SRDMA session group may enable the packet to be transmitted using any of the associated SRDMA sessions. Congestion levels for each of the SRDMA sessions may be monitored and weighted. Therefore, when a packet is received, an SRDMA session may be selected based, at least, on the weight to enable routing of packets to reduce latency and improve overall system efficiency.

The invention provides an RDMA (Remote Direct Memory Access) message aggregation processing method and a network card device, which belong to the technical field of data communication, and comprise the following steps: S1, an RDMA message analysis step which comprises the links of hardware-level message capture, BTH header analysis and message classification and metadata generation; s2, an RDMA (Remote Direct Memory Access) caching step, wherein the step comprises a message descriptor recording link and a caching strategy link; s3, an RDMA message aggregation step, wherein the step comprises a context table aggregation step, an aggregation process step, a new aggregation process step and an existing context aggregation process step; s4, a zero-copy aggregation step, wherein the step comprises the links of metadata transmission, DMA delay processing and host memory writing; in order to solve the problem that stateless aggregation processing is not tried to be carried out on RDMA messages when an existing network card processes the RDMA messages, a plurality of received RDMA messages are aggregated into a larger RDMA message, and the larger RDMA message is sent to a downstream module for processing, so that the number of messages processed by a rear-stage module is reduced, and the bandwidth of the network card for processing the RDMA messages is improved.

The embodiment of the invention provides a network card equipment switching method and device based on a bonding mode, and relates to the technical field of network card equipment. The method comprises the following steps: in response to a switching operation of network card equipment based on a bonding mode, generating a switching operation message, and transmitting the switching operation message to a preset callback interface; calling a switching operation message from a preset callback interface based on the remote direct memory access driving module; and based on the remote direct memory access driving module, switching the current data information of the first network card device to the second network card device through the shared memory according to the switching operation message, and executing the current data information in the second network card device. By adopting the technical scheme, when one network card device has only one port, a communication mode of remote direct memory access with other devices can be carried out, so that the stability of data transmission is improved.

The invention provides a stream computing resource scheduling method and system based on a dynamic time window, and relates to the field of stream computing resource scheduling, and the method comprises the following steps: constructing a master-slave event time monitoring network to obtain a task feature vector, and establishing an adaptive time window scheduling mechanism to dynamically adjust the window size; and constructing a multi-level task priority queue, carrying out state transition by adopting a remote direct memory access mechanism of incremental transmission, and finally coordinating computing nodes through a distributed task arrangement protocol to complete resource switching. According to the method, computing resource competition can be effectively reduced, the flow computing task processing efficiency is improved, dynamic optimal configuration of resources is realized, and the system throughput is enhanced.

The invention is suitable for the technical field of network scheduling, and provides a network scheduling method and system for hybrid service transmission based on RDMA, and the method comprises the steps: receiving a to-be-transmitted service demand instruction, and determining a corresponding service type according to a service quality parameter in the service demand instruction; matching a corresponding virtual transmission channel for the service demand instruction according to the service type, wherein the virtual transmission channel is used for isolating hardware-level resources of different types of traffic; selecting an optimal transmission path for the flow of each virtual transmission channel according to a preset path evaluation model; analyzing the congestion risk of the optimal transmission path by adopting a prediction model; and dynamically adjusting a flow transmission path according to a congestion risk analysis result, and ensuring the transmission quality of the highest priority service through a hardware-level scheduling strategy. According to the invention, the optimal balance between the service quality and the resource efficiency can be realized in a complex mixed service environment.

The invention provides a DPU (Data Processing Unit)-based RDMA (Remote Direct Memory Access) virtualized memory address conversion and data transmission method and a DPU-based RDMA virtualized memory address conversion and data transmission system. The method comprises the following steps that: a user process specifies a client virtual address and a corresponding memory The virtual machine kernel generates a first memory conversion table, and the first memory conversion table records conversion from the client virtual address to the client physical address. When the drive program of the remote direct memory access device obtains the first memory conversion table, a preset instruction is triggered to enable the execution control right to be switched to the virtual machine supervision program from the virtual machine kernel. The customer physical address is converted into the host virtual address in the switching process. The virtual machine supervision program obtains mapping from the host virtual address to the host physical address through the host system information, and a second memory conversion table containing the conversion relation from the client virtual address to the host physical address is obtained. The execution control right is switched back to the kernel of the virtual machine, and the second memory conversion table is configured to the remote direct memory access device DPU. According to the method, address conversion can be simplified, hardware dependence is reduced, and RDMA performance is improved.

The invention provides an RDMA data transmission method, network equipment and a system, and relates to the technical field of communication, in particular to the technical fields of chip processors, computing power clusters, ensemble communication operation, large models and the like. The method is applied to a network device comprising an xPU and an RNIC, the xPU comprises a first engine, and the RNIC comprises a second engine communicating with the first engine, a WQE Buffer and an RDMA engine communicating with the second engine. The method comprises the steps that a first engine assembles a WE based on a hardware unloading asynchronous copy instruction set and transmits the WE to a second engine; the second engine stores the WE in a WQE Buffer and transmits the WE to the RDMA engine; the RDMA engine executes data processing based on the WE, wherein the data processing comprises data transmission, memory access and queue management; the first engine receives feedback of data processing transmitted via the second engine. The RDMA data transmission task is directly unloaded to the special engine through the hardware asynchronous instruction set, decoupling of the RDMA data transmission task and computing resources of the system is achieved, and an efficient and low-delay RDMA data transmission solution which does not depend on CPU / GPU resources is provided for data transmission intensive scenes such as an MoE model.

A computer-implemented method for replicating a log to a remote computer system is disclosed. The method involves identifying a log comprising a data portion and a metadata portion for replication. The data portion is sent to the remote computer system using a Remote Direct Memory Access (RDMA) write operation, while the metadata portion is sent using a first RDMA send operation after the data portion has been sent. The method further includes identifying a second RDMA send operation received from the remote computer system, which indicates the completion of the first RDMA send operation. Based on identifying the second RDMA send operation, the method determines the completion of log replication to the remote computer system. This method enables efficient and reliable replication of logs in a computer system.

The invention relates to the technical field of computers and provides a data transmission method based on remote direct memory access, computer equipment and a medium. The method comprises the following steps: when a sending side sends a plurality of request messages associated with a remote direct memory access transmission request, respectively writing a queue index value of a first queue at a receiving side into respective extension heads of the request messages; on the receiving side, based on the queue index value of the first queue in the extension header of the received request message, the message content is written into the position corresponding to the queue index value in the first queue, and the effective identifier of the corresponding position is set to be effective; and on the receiving side, when the effective identifier of the first position in the first queue is effective, reading contents in the positions with the effective identifiers being effective in the first queue one by one according to the sequence of the first queue, so as to generate a plurality of response messages. Therefore, the problem of out-of-order receiving is solved, and resources are saved.

The invention provides a remote direct memory access system, equipment and a cluster. The system comprises the following steps: a remote direct memory access device sends a first address conversion request to a host under the condition that a first physical address corresponding to a first virtual address does not exist; the host queries a first physical address corresponding to the first virtual address in a memory of the host according to the first address translation request, and returns a translation failure response to the remote direct memory access device under the condition that the first physical address corresponding to the first virtual address cannot be queried; the remote direct memory access device sends a page request to the host based on the conversion failure response; the host migrates data corresponding to the first virtual address from a hard disk of the host based on the page request and locks the data in a memory of the host, and returns a page request response; and the remote direct memory access device stores the corresponding relationship between the first virtual address and the first physical address into an address translation list based on the page request response.

In an aspect, systems and methods of connected nodes, including electronic control units (ECUs) in a vehicle, integrate remote direct memory access (RDMA) capabilities with time-sensitive networking (TSN) traffic shaper configurations in a manner that guarantees lossless, bounded-latency critical traffic (CT) streams, while also allowing for different classes of traffic to flow through the connected nodes. In another aspect, mechanisms are introduced to enable an automotive software framework (e.g., classic AUTOSAR) to support RDMA communications.

The invention provides an RDMA (Remote Direct Memory Access) network transmission packet loss recovery method and system. The method comprises the following steps: a sending end determines that a data stream to be transmitted is a larger or smaller data stream based on a multiple of a bandwidth delay product of an RDMA network; performing redundant coding and code marking on the smaller data stream, generating a data stream carrying a code mark and a redundant data packet, and transmitting the data stream to a receiving end; dividing a larger data stream into a non-tail BDP part and a tail BDP part based on the multiple of the bandwidth delay product, performing redundant coding and coding marking on the tail BDP part, and generating the tail BDP part carrying a coding mark and a redundant data packet; and transmitting a data stream formed by the non-tail BDP part and the tail BDP part carrying the coding mark and the redundant data packet to a receiving end, so that the receiving end carries out packet loss and coding mark detection on the received data stream, and if the packet loss and the coding mark exist, the lost data packet is recovered based on the carried redundant data packet. And if there is packet loss and there is no coding mark, requesting the sending end to select to retransmit and recover the lost data packet.

The present specification provides a method and an apparatus for creating a shared remote direct memory access link, which are applied to a local terminal. The method includes: receiving a first creating request for a first shareable remote direct memory access RDMA link initiated by a first application process on the local terminal, where an associated memory address indicated by the first creating request for the first shareable RDMA link is a shared memory address of the first application process and a second application process on the local terminal; creating the first shareable RDMA link between the local terminal and an opposite terminal according to the first creating request, so that the shared memory address is registered to an RDMA device on the local terminal as the associated memory address of the first shareable RDMA link.

The invention discloses a system, a server, a method, a medium and a program product for sharing a memory resource pool, and relates to the field of artificial intelligence, a plurality of switching components of a switch are connected with processors of a plurality of computer devices and the memory resource pool, and the plurality of computer devices share memory resources of the memory resource pool through the switch, different memory capacities are allocated according to the requirements of the computer equipment, dynamic allocation and efficient sharing of memory resources are achieved, the memory utilization rate of a whole machine system is improved, the memory access delay of the computer equipment is reduced, the memory sharing consistency of multiple pieces of computer equipment is ensured, and the requirements of multiple computing scenes are met. Therefore, the technical problems that in the prior art, a server cooperative computing method based on the remote direct memory access (RDMA) technology is large in transmission delay, the requirements of various computing scenes cannot be met, the consistency of shared memories of multiple computer devices cannot be guaranteed, and the resource utilization rate is low are solved.

The invention relates to the technical field of remote direct memory access, in particular to an RDMA shared receiving queue optimization method and device, equipment, a medium and a product. A shared receiving queue is divided into multiple containers, and the multiple containers are connected in series through a logic linked list; when the RDMA data of the queue pair arrives, if the container is not bound, distributing a container for the queue pair; monitoring the release state of the shared receiving queue entry; and when the receiving side software monitors that the target container is in the completely idle state, re-linking the target container into the logic linked list for distribution independent of the states of other containers in the plurality of containers. According to the method disclosed by the invention, dynamic reuse of resources can be realized through resource monitoring taking the container as granularity and an out-of-order recovery mechanism independent of other container states. After all the SRQEs in any container are released, the SRQEs can be immediately put into redistribution, so that the problem of waste of tail idle resources caused by occupation of head resources in a traditional linear queue can be thoroughly avoided.

The invention belongs to the field of computer systems, and provides a low-latency RDMA (Remote Direct Memory Access) acceleration system based on memory registration, which comprises the following steps of: registering a memory region applied by an application program by utilizing a memory pin technology in a kernel; receiving a remote direct memory access (RDMA) work request initiated by an application program, and packaging information comprising a source virtual address and a target virtual address in the request to form a work queue element; the assembly line is used for processing data receiving from a remote system and sending of local data, and hardware logic used for table look-up, QoS detection and message processing is embedded in the assembly line; the server manages the working queue, the data transmission state and the abnormal information processing in a unified manner, and is used for adjusting the transmission parameters of the RDMA acceleration system in real time according to the received context signal; a standardized software calling interface is provided, and cooperative control between software and hardware is achieved.

The embodiment of the invention relates to a data processing method and device based on a key value database, equipment, a storage medium and a key value database system.The method comprises the steps that after it is determined that to-be-cached key value data exists in a video memory of a graphic processing unit, a data storage request for the to-be-cached key value data is sent to a key value database server side, the data storage request carries storage address information and storage length information of value data of to-be-cached key value data in a video memory; receiving a data reading request which is sent by a key value database server and carries storage address information, and controlling the first network card supporting the remote direct memory access protocol to read value data of the key value data to be cached from the video memory according to the storage address information, sending the value data to the key value database server side, and writing the value data through the second network card supporting the remote direct memory access protocol. Visibly, according to the embodiment of the invention, the key value data in the GPU video memory can be stored without a memory copying process, and the processing efficiency of the key value data is improved.

The invention discloses a distributed file system metadata index optimization method based on remote direct memory access, which comprises directory metadata index based on path name hash: defining a plurality of layers of directories in a directory tree as key directories, and regarding the path as a character string. After calculation of the hash function, determining the storage position of the data in the memory of the metadata server of the distributed file system; and optimizing the memory position of the file metadata based on the popularity, namely placing the file metadata which is frequently accessed at a position adjacent to the key directory metadata to which the file metadata belongs. A metadata management module in a metadata server of the distributed file system regularly checks the popularity of the file and carries out cold and hot migration according to the situation; and client cache optimization oriented to key directory metadata: a metadata cache of a distributed file system client caches the metadata and directory entries of the key directory. Optimally assuming that the metadata and directory entries of the key directory in the cache are not invalid, and updating the cache according to the result of the next-level metadata search. According to the method, the remote reading performance of the metadata index of the distributed file system is optimized by fully utilizing the characteristic of remote direct memory access, the network transmission frequency is reduced, and the load of the metadata server is effectively reduced.

The invention provides a message sending and processing method and system based on RDMA (Remote Direct Memory Access) and electronic equipment. The message sending method comprises the steps that a first message is sent to a receiving end, the first message carries indication information and identification information of a message to which the first message belongs, and the indication information is used for indicating whether disordered receiving between messages is allowed or not. The sending end carries the identification information of the message and the indication information used for indicating whether to allow the out-of-order receiving between the messages in the first message, so that for a receiving end, under the condition that the indication information carried in the first message indicates that the out-of-order receiving between the messages is allowed, the receiving end only needs to preserve the order of the messages in the message, and the receiving end does not need to perform the out-of-order receiving between the messages. According to the technical scheme of the invention, the method does not need to carry out order preservation between messages, that is, a receiving end can trigger retransmission only under the condition that packet loss or disorder occurs to the messages in the messages without triggering retransmission under the condition that the messages are out of order, so that the retransmission range can be reduced (that is, the retransmission range is limited in the messages), and the transmission efficiency of the network is improved.

The invention discloses an RDMA Chain-based LSM Tree key value type database access method. The method comprises the following steps: a database client sends a request containing a version number, a key and an operation type to a database server main node; the server main node analyzes the request and processes the request according to the operation type; during write operation, a new serial number is generated, and data is inserted into a skip list of a memory table MemTable; when reading operation is carried out and the version number is smaller than 0, whether a key exists or not is checked through a Bloom filter, if yes, query is carried out at a secondary node through the RDMA technology, and if not, an SSTable file in a disk is directly accessed; and when the reading operation is performed and the version number is greater than or equal to 0, preferentially searching the data of the corresponding version in the MemTable, and if the data of the corresponding version fails, querying the SSTable. Through the RDMA technology, CPU occupation and network round-trip time are reduced, and the response speed and concurrent processing capacity of the system are improved.

An integrated circuit formed on (i) a single semiconductor die or (ii) a plurality semiconductor dies that are integrated into a single package. The integrated circuit may include a communication interface including a serializer / deserializer (SerDes) interface; a fabric adapter communicatively coupled to the communication interface; a plurality of inference engine clusters, each inference engine cluster including a respective memory element and / or memory interface; and a data interconnect communicatively coupling each respective memory element and / or memory interfaces of the plurality of inference engine clusters to the fabric adapter. The fabric adapter may be configured to facilitate remote direct memory access (RDMA) read and write services and / or datagram communication over a cell-based switch fabric to and from the respective memory elements and / or memory interfaces of the plurality of inference engine clusters via the data interconnect.

The invention provides a DPU-based remote direct memory access queue pair state machine implementation method and device, and the method comprises the steps: creating a flag bit, and setting the value of the flag bit according to each state of a queue pair state machine, so as to achieve state switching; creating a queue pair, and entering a reset state; configuring attributes of a queue pair, and switching to an initialization state; setting parameters of a receiving end, and switching to a ready receiving state; establishing communication connection, setting parameters of a sending end, switching to a ready-to-send state, and executing remote direct memory access operation so as to perform data transmission; if all the data in the sending queue is sent, switching to a sending completion state; if the sending queue has an error, switching from a sending preparation state or a sending completion state to a sending error state; if the queue pair has an error, switching from the current state to the queue pair error; and switching to a reset state after processing an error. According to the method, the flexibility and expansibility of the system are improved, and the processing efficiency of error and abnormal conditions is improved.

The invention provides a congestion control fairness compensation method for a wide-area RDMA (Remote Direct Memory Access) network. The system is composed of an active notification point PNP and a temporary response point TRP which are deployed on an external switch ESW connecting a DC and a WAN. When near-source congestion occurs, the switch marks congestion for the data packet, and the active notification point clears the congestion mark and actively sends a congestion notification packet CNP through the corresponding sending end; when a receiving end receives the data packet containing the congestion mark, responding to the congestion notification packet; the temporary reaction point checks the congestion notification packet, determines the severity of the congestion, and temporarily slows the corresponding cross-DC traffic before the sender begins to decelerate. Therefore, the congestion feedback signal is fed back in advance by using the programmable switch, the congestion feedback fairness of the new-source is ensured, the congestion feedback signal is responded in advance by using the programmable switch, the congestion response fairness of the new-source is ensured, and finally, the fairness in the wide-area RDMA network is maintained, and the overall network performance is improved.

An integrated circuit formed on (i) a single semiconductor die or (ii) a plurality semiconductor dies that are integrated into a single package. The integrated circuit may include a communication interface including a serializer / deserializer (SerDes) interface; a fabric adapter communicatively coupled to the communication interface; a plurality of inference engine clusters, each inference engine cluster including a respective memory element and / or memory interface; and a data interconnect communicatively coupling each respective memory element and / or memory interfaces of the plurality of inference engine clusters to the fabric adapter. The fabric adapter may be configured to facilitate remote direct memory access (RDMA) read and write services and / or datagram communication over a cell-based switch fabric to and from the respective memory elements and / or memory interfaces of the plurality of inference engine clusters via the data interconnect.

The invention discloses a bandwidth determination method and device for a host channel adapter, and relates to the field of computers.The bandwidth determination method comprises the steps that a preset interface of an operating system is adopted to drive and directly access a hardware register of the host channel adapter through the host channel adapter, and data invisibility of an operating system layer in a remote direct memory access framework is penetrated; original values of a receiving byte counter and a sending byte counter maintained by firmware in a hardware register are accurately captured, and the byte increment change is dynamically calculated in combination with a preset time interval, so that the aim of millisecond-level real-time monitoring on the network traffic is fulfilled on the premise of not modifying a kernel and not intercepting a data packet; therefore, the technical effects of high-precision bandwidth measurement, ultra-low performance overhead, full-flow penetration capability and cross-platform expandability are realized, and the fault diagnosis and performance tuning efficiency of the distributed system is remarkably improved.

The invention provides an RDMA (Remote Direct Memory Access) out-of-order receiving and selective retransmission method, which comprises the following steps of: recording an arrival state of a data packet by adopting a Bitmap algorithm and / or a segmentation marking algorithm of a sliding window, dynamically generating ACK (Acknowledgement Character) or NAK (Negative Attached Keying) feedback information according to the algorithm, and only retransmitting a lost data packet according to the received NAK feedback information. The RDMA out-of-order receiving and selective retransmission method supports Bitmap statistics of a sliding window or a segment marking method of saving a memory, is compatible with an existing protocol and realizes an ACK mechanism of selective retransmission, and avoids the problem of redundant transmission in a traditional mechanism. The invention further discloses computer equipment, a data transmission system and a computer readable storage medium.

The invention relates to a data synchronization method and device, equipment, a readable storage medium and a program product, the data synchronization method and device are applied to a master cluster in a distributed storage system, the distributed storage system comprises the master cluster and a slave cluster, and the method comprises the steps that firstly, target data needing to be processed of the distributed system are obtained, and then the target data are sent to the slave cluster; storing the target data into a first memory space of the master cluster, and finally, synchronously writing the target data stored in the first memory space into the same position of a second memory space of the slave cluster based on a remote direct memory access technology; wherein the first memory space and the second memory space are mirror image memory spaces. By adopting the method, the data transmission delay and the CPU overhead can be reduced, and the data synchronization efficiency is higher.

The invention discloses a data stream processing method and device, electronic equipment and a storage medium, and relates to the technical field of distributed storage. According to the data stream processing method and device, the data stream transmission mode is pre-judged to be intra-node transmission or cross-node transmission based on a hardware topology mapping table, and read-write caches are dynamically allocated according to the pre-judged data stream transmission mode; data received by the front-end network card is directly written into a controller memory buffer area of the target equipment through point-to-point direct memory access for data streams transmitted in the nodes, and is read and forwarded from the target equipment by the rear-end network card; the sending cache and the receiving cache of the data stream transmitted across the nodes are locked to the non-uniform memory access node to which the target network card belongs, and the data are sent to the target node through the remote direct memory access register memory, so that the problem that the data stream processing is not optimized in combination with hardware topology characteristics can be solved; the technical effects of reducing data replication overhead, avoiding redundant access across hardware nodes, improving data stream transmission efficiency and realizing end-to-end zero replication transmission are achieved.

The invention provides a distributed database system migration method and device supporting online migration of active transactions. The method and device are suitable for a distributed database to complete dynamic migration of fragmented or tenant-level data and transactions under the condition that services are not interrupted. The method comprises the following steps: generating a consistency snapshot of a source node and transmitting the consistency snapshot to a target node; capturing an incremental transaction log after snapshot in real time; a log is efficiently synchronized to a target node through a remote direct memory access (RDMA) mechanism; the logs are played back in parallel at the target node to construct a complete data state; identifying active transactions in the source node and migrating their context; the target node continues to execute the transaction until submission is completed; and finally updating system routing information to complete migration takeover. According to the method, migration delay is effectively reduced, online transaction migration is realized, and the method has the advantages of high availability, low overhead, high adaptability and the like, and is suitable for database scenes such as elastic capacity expansion and contraction, load balancing and the like.

The invention discloses a communication verification method, platform and device and a readable storage medium, and the method comprises the steps: obtaining an excitation signal and event level information corresponding to event excitation, and transmitting the excitation signal to a to-be-tested device supporting remote direct memory access; simulating a remote direct memory access protocol stack to process the event-level information to obtain a reference data frame; acquiring a response signal output by the device to be tested after processing the excitation signal, and converting the response signal to obtain a real data frame; and performing communication verification on the to-be-tested equipment by using the real data frame and the reference data frame. In the application, the event level information and the excitation signal are excited corresponding to the same event, and the response signal output after the excitation signal is processed by the to-be-tested device is converted to obtain the real data frame. And processing the event-level information by simulating the behavior of the to-be-tested equipment to obtain a reference data frame. Therefore, based on the real data frame and the reference data frame, communication verification can be carried out on the to-be-tested device.