291 results about "Remote direct memory access" patented technology

The invention provides an RDMA (Remote Direct Memory Access) message aggregation processing method and a network card device, which belong to the technical field of data communication, and comprise the following steps: S1, an RDMA message analysis step which comprises the links of hardware-level message capture, BTH header analysis and message classification and metadata generation; s2, an RDMA (Remote Direct Memory Access) caching step, wherein the step comprises a message descriptor recording link and a caching strategy link; s3, an RDMA message aggregation step, wherein the step comprises a context table aggregation step, an aggregation process step, a new aggregation process step and an existing context aggregation process step; s4, a zero-copy aggregation step, wherein the step comprises the links of metadata transmission, DMA delay processing and host memory writing; in order to solve the problem that stateless aggregation processing is not tried to be carried out on RDMA messages when an existing network card processes the RDMA messages, a plurality of received RDMA messages are aggregated into a larger RDMA message, and the larger RDMA message is sent to a downstream module for processing, so that the number of messages processed by a rear-stage module is reduced, and the bandwidth of the network card for processing the RDMA messages is improved.

The invention provides a stream computing resource scheduling method and system based on a dynamic time window, and relates to the field of stream computing resource scheduling, and the method comprises the following steps: constructing a master-slave event time monitoring network to obtain a task feature vector, and establishing an adaptive time window scheduling mechanism to dynamically adjust the window size; and constructing a multi-level task priority queue, carrying out state transition by adopting a remote direct memory access mechanism of incremental transmission, and finally coordinating computing nodes through a distributed task arrangement protocol to complete resource switching. According to the method, computing resource competition can be effectively reduced, the flow computing task processing efficiency is improved, dynamic optimal configuration of resources is realized, and the system throughput is enhanced.

The invention is suitable for the technical field of network scheduling, and provides a network scheduling method and system for hybrid service transmission based on RDMA, and the method comprises the steps: receiving a to-be-transmitted service demand instruction, and determining a corresponding service type according to a service quality parameter in the service demand instruction; matching a corresponding virtual transmission channel for the service demand instruction according to the service type, wherein the virtual transmission channel is used for isolating hardware-level resources of different types of traffic; selecting an optimal transmission path for the flow of each virtual transmission channel according to a preset path evaluation model; analyzing the congestion risk of the optimal transmission path by adopting a prediction model; and dynamically adjusting a flow transmission path according to a congestion risk analysis result, and ensuring the transmission quality of the highest priority service through a hardware-level scheduling strategy. According to the invention, the optimal balance between the service quality and the resource efficiency can be realized in a complex mixed service environment.

The invention relates to the technical field of computers and provides a data transmission method based on remote direct memory access, computer equipment and a medium. The method comprises the following steps: when a sending side sends a plurality of request messages associated with a remote direct memory access transmission request, respectively writing a queue index value of a first queue at a receiving side into respective extension heads of the request messages; on the receiving side, based on the queue index value of the first queue in the extension header of the received request message, the message content is written into the position corresponding to the queue index value in the first queue, and the effective identifier of the corresponding position is set to be effective; and on the receiving side, when the effective identifier of the first position in the first queue is effective, reading contents in the positions with the effective identifiers being effective in the first queue one by one according to the sequence of the first queue, so as to generate a plurality of response messages. Therefore, the problem of out-of-order receiving is solved, and resources are saved.

In an aspect, systems and methods of connected nodes, including electronic control units (ECUs) in a vehicle, integrate remote direct memory access (RDMA) capabilities with time-sensitive networking (TSN) traffic shaper configurations in a manner that guarantees lossless, bounded-latency critical traffic (CT) streams, while also allowing for different classes of traffic to flow through the connected nodes. In another aspect, mechanisms are introduced to enable an automotive software framework (e.g., classic AUTOSAR) to support RDMA communications.

The invention discloses a system, a server, a method, a medium and a program product for sharing a memory resource pool, and relates to the field of artificial intelligence, a plurality of switching components of a switch are connected with processors of a plurality of computer devices and the memory resource pool, and the plurality of computer devices share memory resources of the memory resource pool through the switch, different memory capacities are allocated according to the requirements of the computer equipment, dynamic allocation and efficient sharing of memory resources are achieved, the memory utilization rate of a whole machine system is improved, the memory access delay of the computer equipment is reduced, the memory sharing consistency of multiple pieces of computer equipment is ensured, and the requirements of multiple computing scenes are met. Therefore, the technical problems that in the prior art, a server cooperative computing method based on the remote direct memory access (RDMA) technology is large in transmission delay, the requirements of various computing scenes cannot be met, the consistency of shared memories of multiple computer devices cannot be guaranteed, and the resource utilization rate is low are solved.

The invention relates to the technical field of remote direct memory access, in particular to an RDMA shared receiving queue optimization method and device, equipment, a medium and a product. A shared receiving queue is divided into multiple containers, and the multiple containers are connected in series through a logic linked list; when the RDMA data of the queue pair arrives, if the container is not bound, distributing a container for the queue pair; monitoring the release state of the shared receiving queue entry; and when the receiving side software monitors that the target container is in the completely idle state, re-linking the target container into the logic linked list for distribution independent of the states of other containers in the plurality of containers. According to the method disclosed by the invention, dynamic reuse of resources can be realized through resource monitoring taking the container as granularity and an out-of-order recovery mechanism independent of other container states. After all the SRQEs in any container are released, the SRQEs can be immediately put into redistribution, so that the problem of waste of tail idle resources caused by occupation of head resources in a traditional linear queue can be thoroughly avoided.

The invention belongs to the field of computer systems, and provides a low-latency RDMA (Remote Direct Memory Access) acceleration system based on memory registration, which comprises the following steps of: registering a memory region applied by an application program by utilizing a memory pin technology in a kernel; receiving a remote direct memory access (RDMA) work request initiated by an application program, and packaging information comprising a source virtual address and a target virtual address in the request to form a work queue element; the assembly line is used for processing data receiving from a remote system and sending of local data, and hardware logic used for table look-up, QoS detection and message processing is embedded in the assembly line; the server manages the working queue, the data transmission state and the abnormal information processing in a unified manner, and is used for adjusting the transmission parameters of the RDMA acceleration system in real time according to the received context signal; a standardized software calling interface is provided, and cooperative control between software and hardware is achieved.

The invention discloses a distributed file system metadata index optimization method based on remote direct memory access, which comprises directory metadata index based on path name hash: defining a plurality of layers of directories in a directory tree as key directories, and regarding the path as a character string. After calculation of the hash function, determining the storage position of the data in the memory of the metadata server of the distributed file system; and optimizing the memory position of the file metadata based on the popularity, namely placing the file metadata which is frequently accessed at a position adjacent to the key directory metadata to which the file metadata belongs. A metadata management module in a metadata server of the distributed file system regularly checks the popularity of the file and carries out cold and hot migration according to the situation; and client cache optimization oriented to key directory metadata: a metadata cache of a distributed file system client caches the metadata and directory entries of the key directory. Optimally assuming that the metadata and directory entries of the key directory in the cache are not invalid, and updating the cache according to the result of the next-level metadata search. According to the method, the remote reading performance of the metadata index of the distributed file system is optimized by fully utilizing the characteristic of remote direct memory access, the network transmission frequency is reduced, and the load of the metadata server is effectively reduced.

The invention discloses an RDMA Chain-based LSM Tree key value type database access method. The method comprises the following steps: a database client sends a request containing a version number, a key and an operation type to a database server main node; the server main node analyzes the request and processes the request according to the operation type; during write operation, a new serial number is generated, and data is inserted into a skip list of a memory table MemTable; when reading operation is carried out and the version number is smaller than 0, whether a key exists or not is checked through a Bloom filter, if yes, query is carried out at a secondary node through the RDMA technology, and if not, an SSTable file in a disk is directly accessed; and when the reading operation is performed and the version number is greater than or equal to 0, preferentially searching the data of the corresponding version in the MemTable, and if the data of the corresponding version fails, querying the SSTable. Through the RDMA technology, CPU occupation and network round-trip time are reduced, and the response speed and concurrent processing capacity of the system are improved.

An integrated circuit formed on (i) a single semiconductor die or (ii) a plurality semiconductor dies that are integrated into a single package. The integrated circuit may include a communication interface including a serializer / deserializer (SerDes) interface; a fabric adapter communicatively coupled to the communication interface; a plurality of inference engine clusters, each inference engine cluster including a respective memory element and / or memory interface; and a data interconnect communicatively coupling each respective memory element and / or memory interfaces of the plurality of inference engine clusters to the fabric adapter. The fabric adapter may be configured to facilitate remote direct memory access (RDMA) read and write services and / or datagram communication over a cell-based switch fabric to and from the respective memory elements and / or memory interfaces of the plurality of inference engine clusters via the data interconnect.

The invention provides a DPU-based remote direct memory access queue pair state machine implementation method and device, and the method comprises the steps: creating a flag bit, and setting the value of the flag bit according to each state of a queue pair state machine, so as to achieve state switching; creating a queue pair, and entering a reset state; configuring attributes of a queue pair, and switching to an initialization state; setting parameters of a receiving end, and switching to a ready receiving state; establishing communication connection, setting parameters of a sending end, switching to a ready-to-send state, and executing remote direct memory access operation so as to perform data transmission; if all the data in the sending queue is sent, switching to a sending completion state; if the sending queue has an error, switching from a sending preparation state or a sending completion state to a sending error state; if the queue pair has an error, switching from the current state to the queue pair error; and switching to a reset state after processing an error. According to the method, the flexibility and expansibility of the system are improved, and the processing efficiency of error and abnormal conditions is improved.

The invention provides a congestion control fairness compensation method for a wide-area RDMA (Remote Direct Memory Access) network. The system is composed of an active notification point PNP and a temporary response point TRP which are deployed on an external switch ESW connecting a DC and a WAN. When near-source congestion occurs, the switch marks congestion for the data packet, and the active notification point clears the congestion mark and actively sends a congestion notification packet CNP through the corresponding sending end; when a receiving end receives the data packet containing the congestion mark, responding to the congestion notification packet; the temporary reaction point checks the congestion notification packet, determines the severity of the congestion, and temporarily slows the corresponding cross-DC traffic before the sender begins to decelerate. Therefore, the congestion feedback signal is fed back in advance by using the programmable switch, the congestion feedback fairness of the new-source is ensured, the congestion feedback signal is responded in advance by using the programmable switch, the congestion response fairness of the new-source is ensured, and finally, the fairness in the wide-area RDMA network is maintained, and the overall network performance is improved.

An integrated circuit formed on (i) a single semiconductor die or (ii) a plurality semiconductor dies that are integrated into a single package. The integrated circuit may include a communication interface including a serializer / deserializer (SerDes) interface; a fabric adapter communicatively coupled to the communication interface; a plurality of inference engine clusters, each inference engine cluster including a respective memory element and / or memory interface; and a data interconnect communicatively coupling each respective memory element and / or memory interfaces of the plurality of inference engine clusters to the fabric adapter. The fabric adapter may be configured to facilitate remote direct memory access (RDMA) read and write services and / or datagram communication over a cell-based switch fabric to and from the respective memory elements and / or memory interfaces of the plurality of inference engine clusters via the data interconnect.

The invention discloses a bandwidth determination method and device for a host channel adapter, and relates to the field of computers.The bandwidth determination method comprises the steps that a preset interface of an operating system is adopted to drive and directly access a hardware register of the host channel adapter through the host channel adapter, and data invisibility of an operating system layer in a remote direct memory access framework is penetrated; original values of a receiving byte counter and a sending byte counter maintained by firmware in a hardware register are accurately captured, and the byte increment change is dynamically calculated in combination with a preset time interval, so that the aim of millisecond-level real-time monitoring on the network traffic is fulfilled on the premise of not modifying a kernel and not intercepting a data packet; therefore, the technical effects of high-precision bandwidth measurement, ultra-low performance overhead, full-flow penetration capability and cross-platform expandability are realized, and the fault diagnosis and performance tuning efficiency of the distributed system is remarkably improved.

The invention provides an RDMA (Remote Direct Memory Access) out-of-order receiving and selective retransmission method, which comprises the following steps of: recording an arrival state of a data packet by adopting a Bitmap algorithm and / or a segmentation marking algorithm of a sliding window, dynamically generating ACK (Acknowledgement Character) or NAK (Negative Attached Keying) feedback information according to the algorithm, and only retransmitting a lost data packet according to the received NAK feedback information. The RDMA out-of-order receiving and selective retransmission method supports Bitmap statistics of a sliding window or a segment marking method of saving a memory, is compatible with an existing protocol and realizes an ACK mechanism of selective retransmission, and avoids the problem of redundant transmission in a traditional mechanism. The invention further discloses computer equipment, a data transmission system and a computer readable storage medium.

The invention relates to a data synchronization method and device, equipment, a readable storage medium and a program product, the data synchronization method and device are applied to a master cluster in a distributed storage system, the distributed storage system comprises the master cluster and a slave cluster, and the method comprises the steps that firstly, target data needing to be processed of the distributed system are obtained, and then the target data are sent to the slave cluster; storing the target data into a first memory space of the master cluster, and finally, synchronously writing the target data stored in the first memory space into the same position of a second memory space of the slave cluster based on a remote direct memory access technology; wherein the first memory space and the second memory space are mirror image memory spaces. By adopting the method, the data transmission delay and the CPU overhead can be reduced, and the data synchronization efficiency is higher.

The invention discloses a data stream processing method and device, electronic equipment and a storage medium, and relates to the technical field of distributed storage. According to the data stream processing method and device, the data stream transmission mode is pre-judged to be intra-node transmission or cross-node transmission based on a hardware topology mapping table, and read-write caches are dynamically allocated according to the pre-judged data stream transmission mode; data received by the front-end network card is directly written into a controller memory buffer area of the target equipment through point-to-point direct memory access for data streams transmitted in the nodes, and is read and forwarded from the target equipment by the rear-end network card; the sending cache and the receiving cache of the data stream transmitted across the nodes are locked to the non-uniform memory access node to which the target network card belongs, and the data are sent to the target node through the remote direct memory access register memory, so that the problem that the data stream processing is not optimized in combination with hardware topology characteristics can be solved; the technical effects of reducing data replication overhead, avoiding redundant access across hardware nodes, improving data stream transmission efficiency and realizing end-to-end zero replication transmission are achieved.

The invention provides a distributed database system migration method and device supporting online migration of active transactions. The method and device are suitable for a distributed database to complete dynamic migration of fragmented or tenant-level data and transactions under the condition that services are not interrupted. The method comprises the following steps: generating a consistency snapshot of a source node and transmitting the consistency snapshot to a target node; capturing an incremental transaction log after snapshot in real time; a log is efficiently synchronized to a target node through a remote direct memory access (RDMA) mechanism; the logs are played back in parallel at the target node to construct a complete data state; identifying active transactions in the source node and migrating their context; the target node continues to execute the transaction until submission is completed; and finally updating system routing information to complete migration takeover. According to the method, migration delay is effectively reduced, online transaction migration is realized, and the method has the advantages of high availability, low overhead, high adaptability and the like, and is suitable for database scenes such as elastic capacity expansion and contraction, load balancing and the like.

The invention discloses a communication verification method, platform and device and a readable storage medium, and the method comprises the steps: obtaining an excitation signal and event level information corresponding to event excitation, and transmitting the excitation signal to a to-be-tested device supporting remote direct memory access; simulating a remote direct memory access protocol stack to process the event-level information to obtain a reference data frame; acquiring a response signal output by the device to be tested after processing the excitation signal, and converting the response signal to obtain a real data frame; and performing communication verification on the to-be-tested equipment by using the real data frame and the reference data frame. In the application, the event level information and the excitation signal are excited corresponding to the same event, and the response signal output after the excitation signal is processed by the to-be-tested device is converted to obtain the real data frame. And processing the event-level information by simulating the behavior of the to-be-tested equipment to obtain a reference data frame. Therefore, based on the real data frame and the reference data frame, communication verification can be carried out on the to-be-tested device.

The present application provides a packet transmission method. A terminal device or a network device can determine a service type of a remote direct memory access (RDMA) packet, and, on the basis of the service type of said first packet, map the first packet to a first radio bearer. The radio bearer to which the first packet is correspondingly mapped is determined on the basis of the service type of the first packet, so that the service type of the first packet matches the mode of the radio bearer, thereby avoiding issues in packet transmission performance caused by mapping deviations, such as reduced reliability and increased latency. Therefore, the transmission performance of RDMA packets is ensured.

The application discloses a network interface card, a storage device, a message receiving method and a message sending method, relates to the storage field, and is used for improving the reliability of a storage device using an RDMA network and improving the storage access performance of a single QP connection. The network interface card comprises a parser and at least one shared connector. The parser is used for distributing a received first remote direct memory access (RDMA) message to a first shared connector in the at least one shared connector. The first shared connector is used for establishing a QP connection with a peer network device, and determining a first queue pair (QP) corresponding to a first controller according to the first RDMA message to realize communication with the first controller. The first shared connector is connected with a plurality of controllers including the first controller, and one QP corresponding to each controller in the plurality of controllers corresponds to the first shared connector.

The invention discloses a selective retransmission function test method, device and system for remote direct memory access, a medium and a product, and relates to the technical field of network tests.The method comprises the steps that when message features of a received test message are successfully matched with a packet loss rule, redirection processing is carried out on the test message so as to achieve controllable discarding, and the message features of the test message are successfully matched with the packet loss rule; according to the method and the device, the selective retransmission function is triggered, and the release rule corresponding to the test message takes effect to ensure that the subsequent corresponding retransmission message can be normally forwarded, so that only the first transmission of the original message is intercepted, the retransmission message is released, and the purpose of controllable packet loss is achieved; and finally, the selective retransmission function is evaluated according to the redirected message information in the function test process and the message receiving information of the message receiving end, so that the technical problems that the test packet loss is uncontrollable and the accurate test requirement is difficult to meet in the related technology can be solved, and the test efficiency is improved. The technical effects of accurate and controllable verification and performance evaluation of the selective retransmission function are achieved.

Devices, systems, methods, and processes for incast congestion management are described herein. Typically, in a Packet Sequence Number (PSN) based Remote Direct Memory Access (RDMA) network, Priority Flow Control (PFC) is asserted upstream when an incast congestion event occurs, which can victimize unrelated flows. Thus, instead of asserting PFC, a switch in the PSN based RDMA network detects an incast congestion event and directly notifies one or more Reliable Connection (RC) Queue Pairs (QPs) of various sending devices, associated with the incast congestion event using Receiver Not Ready (RNR) negative acknowledgements (NACKs). These RNR NACKs are associated with unique pause time-periods. The associated RC QPs receive the RNR NACKs and pause packet transmission. The associated RC QPs resume packet transmission upon expiration of corresponding pause time-periods. Thus, the packet transmission from the contributing RC QPs is spaced out, avoiding all packets reaching a switch output port at the same time.

The invention relates to the technical field of supercomputing system operation and maintenance, and discloses a method for resetting an intelligent operating system of a supercomputing system node, which comprises the following steps of: 101, acquiring data of each computing node of a supercomputing system in real time, transmitting the data to a monitoring center through an RDMA (Remote Direct Memory Access) high-speed network for retention to form historical operating data, the historical operation data comprises historical index time sequence data and historical fault label data; 102, a three-dimensional array is generated in the historical operation data through a sliding window method, the three-dimensional array comprises sample information, time information and feature information, and the information in the three-dimensional array is spliced in sequence; and step 103, constructing a fault prediction model, inputting the three-dimensional array into the fault prediction model, and outputting a fault probability by the fault prediction model. According to the method, the fault prediction model is constructed, the node data of the supercomputing system is collected and processed in real time, the fault occurrence probability can be predicted, and an intelligent decision basis is provided for resetting of the supercomputing node operating system.

The embodiment of the invention provides a method, a device, equipment and a medium for connection establishment. According to the method, a connection establishment request is first received at a first device from a second device. The connection establishment request comprises first data used for establishing the remote direct memory access connection. Then, based on the thread configuration information for the first device, a thread in the first device associated with the first data is determined. The thread configuration information indicates an association between the thread and the data. A thread associated with the first data is used to determine whether to establish a remote direct memory access connection. And if it is determined that the remote direct memory access connection is established, sending a connection establishment response from the first device to the second device. The connection establishment response includes second data for establishing the remote direct memory access connection. The second data is associated with the first data. By using the method, the connection establishment efficiency can be effectively improved.

A communication method and apparatus based on remote direct memory access (RDMA), enabling timely transmission of an RDMA management packet to an RDMA node, thus enhancing remote memory read performance. The method comprises: a terminal sends a first message and receives a second message. The first message is used for requesting to establish a protocol data unit (PDU) session, and the PDU session is used for transmitting an RDMA packet. The second message is used for indicating acceptance of the establishment of the PDU session. The first message comprises first information, the first information is used for indicating the types of the RDMA packet transmitted by means of the PDU session, the types of the RDMA packet comprise a first-type packet and a second-type packet, and the transmission priority of the first-type packet is higher than the transmission priority of the second-type packet. The first-type packet is a management packet, the second-type packet is a data packet, or the first-type packet is a first-type management packet, the second-type packet comprises a second-type management packet and / or a data packet, and the first-type management packet is used for subnet management.

Disclosed are apparatuses, systems, and techniques that improve efficiency and decrease latency of remote direct memory access (RDMA) operations. The techniques include but are not limited to unified RDMA operations that are recognizable by various communicating devices, such as network controllers and target memory devices, as requests to establish, set, and / or update arrival indicators in the target memory devices responsive to arrival of one or more portions of the data being communicated.

The invention provides an RDMA-based big data transmission and storage system and method and a semiconductor detection device.The big data transmission and storage system comprises a hardware layer and a software layer, the hardware layer comprises an NVMe SSD storage array, an NVMe RAID acceleration card based on a GPU architecture and an RDMA network card, and the software layer comprises an NVMe SSD storage array, an NVMe RAID acceleration card based on the GPU architecture and an RDMA network card; the software layer comprises a multi-thread transmission module, a self-adaptive memory pool module, a transmission control module, a data verification and retransmission module, a scheduling and optimizing module and a disk falling and storage management module. According to the method, multi-channel transmission parallelization is achieved through data direct connection, zero copy transmission, RAID acceleration and multi-thread parallel RDMA transmission in a hardware link, data is directly connected and transmitted from a memory to the NVMe SSD storage array, data receiving and disk falling thread decoupling directly access the NVMe SSD storage array through asynchronous writing, network overload or storage blockage is avoided, and the data transmission efficiency is improved. The data transmission efficiency and reliability are high, and popularization and application in scenes needing large-scale image transmission processing such as semiconductor detection are facilitated.

According to the communication method and device based on the remote direct memory access, the management message of the remote direct memory access RDMA can be transmitted to the RDMA node in time, and the remote memory reading performance is improved. The method comprises the steps that a terminal sends a first message and receives a second message; wherein the first message is used for requesting to establish a protocol data unit (PDU) session, and the PDU session is used for transmitting the RDMA message. The second message is used for indicating to accept establishment of the PDU session. The first message comprises first information, the first information is used for indicating the type of an RDMA message transmitted by the PDU session, the type of the RDMA message comprises a first type of message and a second type of message, and the transmission priority of the first type of message is higher than that of the second type of message. Wherein the first type of message is a management message, the second type of message is a data message, or the first type of message is a first type of management message, the second type of message comprises a second type of management message and / or a data message, and the first type of management message is used for subnet management.