Data transmission method, device, equipment and medium

Abstract

The invention discloses a data transmission method and device, equipment and a medium, and relates to the field of RDMA (Remote Direct Memory Access). The data transmission method comprises the steps of generating to-be-backed-up queue data based on original to-be-transmitted data sent by a current processor and data requested to be read by a target data receiving end; and backing up the to-be-backed-up queue data in a network card memory, and sending the to-be-backed-up queue data to a target data receiving end. According to the technical scheme provided by the embodiment of the invention, the data transmission operation complexity can be reduced while reliable data transmission is ensured, so that ultra-low-delay data transmission is realized.

Description

Technical Field

[0001] The present invention relates to the field of RDMA technology, and in particular to a data transmission method, apparatus, device and medium. Background Art

[0002] With the rapid development of the internet and big data, data transmission performance between servers within data centers has become a significant bottleneck to overall system efficiency. Traditional data transmission solutions rely on the operating system kernel, drivers, and multi-level cache processing, often requiring multiple data copies and frequent DMA (Direct Memory Access) operations, resulting in long transmission delays and high system resource utilization. To meet the needs of low latency and high throughput, RDMA (Remote Direct Memory Access) technology has emerged. It enables high-speed data transmission between remote hosts through direct memory access, reducing intermediate link delays.

[0003] Traditional RDMA-based transmission implementations typically include the following steps: 1. The software stores the data to be sent in DRAM (Dynamic Random Access Memory). 2. The software generates a WQE (Work Queue Element) and stores it in DRAM. 3. The software notifies the RDMA module to begin data transmission by updating the send pointer. 4. After receiving the pointer update, the hardware reads the WQE and the data to be sent from DRAM and encapsulates the data according to predefined packetization rules. 5. The hardware transmits the data packet to the target device via Ethernet. 6. The aforementioned hardware, acting as the receiving end, decapsulates the data packet and verifies the data integrity.

[0004] The traditional RDMA transmission implementation process has complex software data transmission processes and involves multiple DMA operations, which increases the complexity of data sending operations and the difficulty of communication system scheduling. The multiple links will lead to a significant increase in transmission delay, especially in the scenario of high-frequency transmission of small packets, where the delay problem is particularly prominent. Summary of the Invention

[0005] The present invention provides a data transmission method, apparatus, device and medium to solve the problems in the traditional RDMA transmission implementation process of high complexity of data sending operation and the need to further reduce transmission delay.

[0006] According to one aspect of the present invention, there is provided a data transmission method, comprising:

[0007] Generate queue data to be backed up based on the original data to be transmitted sent by the current processor and the data requested to be read by the target data receiving end;

[0008] The queue data to be backed up is backed up in the network card storage, and the queue data to be backed up is sent to the target data receiving end.

[0009] According to another aspect of the present invention, there is provided a data transmission device, comprising:

[0010] A module for generating data of a queue to be backed up is used to generate data of a queue to be backed up based on the original data to be transmitted sent by the current processor and the data requested to be read by the target data receiving end;

[0011] The data backup and sending module is used to back up the queue data to be backed up in the network card memory and send the queue data to be backed up to the target data receiving end.

[0012] According to another aspect of the present invention, a communication unit is provided, comprising an RDMA data transmission system for executing the data transmission method according to any embodiment of the present invention.

[0013] According to another aspect of the present invention, an electronic device is provided, which includes the communication unit described in the embodiment of the present invention.

[0014] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable an RDMA data transmission system to implement the data transmission method according to any embodiment of the present invention when executed.

[0015] The technical solution of an embodiment of the present invention generates backup queue data based on the original data to be transmitted sent by the current processor and the data requested to be read by the target data receiving end, then backs up the backup queue data in the network card memory and sends the backup queue data to the target data receiving end. In this solution, the original data to be transmitted is directly pushed by the processor, rather than actively capturing data stored in the memory by the processor, ensuring low latency during the initial transmission of the data packet, which can significantly reduce the overall access latency. The backup of the backup queue data can also ensure reliable data transmission, solving the problems of high complexity in data transmission operations and the need to further reduce transmission latency in the traditional RDMA transmission implementation process. While ensuring reliable data transmission, it can reduce the complexity of data transmission operations to achieve ultra-low latency data transmission.

[0016] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0018] Figure 1 A flowchart of a data transmission method provided in Example 1 of the present invention;

[0019] Figure 2 A flowchart of a data transmission method provided in Embodiment 2 of the present invention;

[0020] Figure 3 A communication logic diagram of an ultra-low latency RDMA data transmission system provided in Example 3 of the present invention;

[0021] Figure 4 A schematic structural diagram of a data transmission device provided in a fourth embodiment of the present invention;

[0022] Figure 5 A schematic structural diagram of an electronic device that can be used to implement an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0023] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0024] It should be noted that the terms "original", "target", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or that are inherent to these processes, methods, products or devices.

[0025] Example 1

[0026] Figure 1This is a flowchart of a data transmission method provided in the first embodiment of the present invention. This embodiment is applicable to low-latency data transmission in the RDMA scenario. The method can be executed by a data transmission device, which can be implemented in the form of hardware and / or software. The data transmission device can be configured in an electronic device, which can include but is not limited to a smart network card (such as an RDMA network card) or a computer-specific acceleration component. Figure 1 As shown, the method includes:

[0027] Step 110 : Generate queue data to be backed up based on the original data to be transmitted sent by the current processor and the data requested to be read by the target data receiving end.

[0028] The current processor may be any device capable of executing instructions, processing data, and coordinating software and hardware. For example, the current processor may include, but is not limited to, a central processing unit (CPU) or a graphics processing unit (GPU). The original data to be transmitted may be data to be transmitted directly sent by the current processor. The original data to be transmitted may include a read request type and a write request type. The target data receiving end may be the data receiving end of the data sent by the current processor, and the data packet format received and sent by the target data receiving end is an RDMA packet format. The queue data to be backed up may be data to be transmitted consisting of the original data to be transmitted sent by the current processor and the data requested to be read by the target data receiving end, and exists in the form of a queue.

[0029] In an embodiment of the present invention, when there is a need for data interaction between the current processor and the target data receiving end, the current processor directly sends the original data to be transmitted. When the target data receiving end sends a data read request to the current processor, the current processor returns the data requested to be read to the target data receiving end based on the RDMA mechanism. The original data to be transmitted and the data requested to be read by the target data receiving end are further sorted to obtain the queue data to be backed up.

[0030] Optionally, the original data to be transmitted and the data requested to be read by the target data receiving end can be sorted based on a first-come, first-served policy, or the original data to be transmitted and the data requested to be read by the target data receiving end can be sorted according to a priority queue policy.

[0031] Exemplarily, if the target data receiving end does not send a data read request to the current processor, no data will be returned to the target data receiving end, that is, the data requested to be read by the target data receiving end is empty.

[0032] Step 120: Back up the queue data to be backed up in the network card memory, and send the queue data to be backed up to the target data receiving end.

[0033] The network card memory may be the system memory of the current processor.

[0034] In an embodiment of the present invention, the queue data to be backed up can be stored in the network card memory to realize the backup of the queue data to be backed up, and at the same time as the queue data to be backed up is backed up to the network card memory, the queue data to be backed up in the RDMA packet structure (the structure of the RDMA data packet) is sent to the target data receiving end.

[0035] The technical solution of an embodiment of the present invention generates backup queue data based on the original data to be transmitted sent by the current processor and the data requested to be read by the target data receiving end, then backs up the backup queue data in the network card memory and sends the backup queue data to the target data receiving end. In this solution, the original data to be transmitted is directly pushed by the processor, rather than actively capturing data stored in the memory by the processor, ensuring low latency during the initial transmission of the data packet, which can significantly reduce the overall access latency. The backup of the backup queue data can also ensure reliable data transmission, solving the problems of high complexity in data transmission operations and the need to further reduce transmission latency in the traditional RDMA transmission implementation process. While ensuring reliable data transmission, it can reduce the complexity of data transmission operations to achieve ultra-low latency data transmission.

[0036] Example 2

[0037] Figure 2 This is a flowchart of a data transmission method provided by the second embodiment of the present invention. This embodiment is specific based on the above embodiment and provides a specific optional implementation method for generating queue data to be backed up based on the original data to be transmitted sent by the current processor and the data requested to be read by the target data receiving end. Figure 2 As shown, the method includes:

[0038] Step 210: parse the original data to be transmitted sent by the current processor to obtain the target data to be transmitted.

[0039] The target data to be transmitted may be the payload content in the original data to be transmitted.

[0040] In an embodiment of the present invention, the original data to be transmitted sent by the current processor can be received through the bus interface, the original data to be transmitted sent by the current processor can be further parsed, the payload content in the original data to be transmitted can be identified, and the identified payload content can be used as the target data to be transmitted.

[0041] Step 220: convert the target data to be transmitted according to the custom packet format to generate pre-processed data to be transmitted, and create queue data to be backed up based on the pre-processed data to be transmitted and the data requested to be read by the target data receiving end.

[0042] The custom packet format may be a predefined packet structure. The preprocessed data to be transmitted may be the result of converting the target data to be transmitted into a data packet format according to the custom packet format. Optionally, when the original data to be transmitted is a write request type, the preprocessed data to be transmitted includes the write data plus a control flag. When the original data to be transmitted is a read request type, the preprocessed data to be transmitted may include only the control flag.

[0043] In an embodiment of the present invention, the target data to be transmitted can be converted into a data packet format based on a custom packet format to obtain preprocessed data to be transmitted, and then a serial number for describing the data sorting order is added to the preprocessed data to be transmitted and the data requested to be read by the target data receiving end to obtain the queue data to be backed up.

[0044] Step 230: back up the queue data to be backed up in the network card memory, and send the queue data to be backed up to the target data receiving end.

[0045] In an optional embodiment of the present invention, while sending the queue data to be backed up to the target data receiving end, the method may also include: sending the queue data not to be backed up to the target data receiving end; wherein the queue data not to be backed up includes a data reception response and / or target retransmission data; arbitrating the current data to be encapsulated from the queue data to be backed up and the queue data not to be backed up; encapsulating the current data to be encapsulated according to the RDMA packet structure to obtain the target encapsulated data, and sending the target encapsulated data to the target data receiving end.

[0046] Among them, the non-queue data to be backed up may be data other than the queue data to be backed up that needs to be sent to the target data receiving end, and exists in the form of a queue. The data reception response may be a response signal obtained after the target data receiving end sends a request (such as a read request and / or a write request) to the current processor. The target retransmitted data may be data that the target data receiving end fails to receive and is backed up in the network card memory. The current data to be encapsulated may be the arbitrated data that currently needs to be sent to the target data receiving end and has not been encapsulated. The target encapsulated data may be a data packet encapsulated by the current data to be encapsulated according to the RDMA packet structure.

[0047] In an embodiment of the present invention, when sending queue data to be backed up to a target data receiving end, if there is also queue data not to be backed up to be sent to the target data receiving end at the same time, the queue data to be backed up and the queue data not to be backed up can be sorted according to a preset data sorting rule, and the data at the top of the sorting order is used as the current data to be encapsulated, and then the current data to be encapsulated is encapsulated according to the RDMA packet structure to obtain the target encapsulated data, and then the target encapsulated data is sent to the target data receiving end via Ethernet.

[0048] In an optional embodiment of the present invention, before sending non-backup queue data to the target data receiving end, it may also include: parsing the to-be-received RDMA data sent by the target data receiving end, and generating a data reception response when the to-be-received RDMA data passes the verification; and / or, when there is a data reception anomaly at the target data receiving end, determining the target retransmission data in the network card memory.

[0049] The RDMA data to be received may be data in an RDMA packet structure sent by a target data receiving end to the current processor.

[0050] In an embodiment of the present invention, if the target data receiving end sends RDMA data to be received to the current processor, the RDMA data to be received sent by the target data receiving end is decapsulated based on the RDMA packet structure, and the data validity check is performed on the decapsulated data. When the decapsulated data passes the data validity check, it indicates that the RDMA data to be received has passed the verification, and a data reception response for the RDMA data to be received is generated. If there is a data reception anomaly at the target data receiving end, it indicates that the historical backup queue data (the queue data previously backed up in the network card memory with the backup queue data mentioned above, the data structure of the historical backup queue data can be specifically referred to the queue data to be backed up) sent by the current processor to the target data receiving end has not been successfully received, then the target retransmission data that has not been successfully received by the target data receiving end is queried from the network card memory.

[0051] In an optional embodiment of the present invention, after parsing the RDMA data to be received sent by the target data receiving end, the method may also include: when the RDMA data to be received passes the verification, determining the packet header information of the target queue data successfully received by the target data receiving end; and deleting data from the network card memory based on the packet header information of the target queue data.

[0052] The target queue data may be data sent by the current processor to the target data receiving end and successfully received by the target data receiving end, and exists in the form of a queue.

[0053] In an embodiment of the present invention, if the RDMA data to be received passes the verification, it is possible to further determine the target queue data sent by the current processor that is successfully received by the target data receiving end, and then obtain the packet header information of the target queue data, so as to locate the space for storing the target queue data from the network card memory based on the packet header information of the target queue data, and clear the space.

[0054] In an optional embodiment of the present invention, determining the target retransmission data in the network card memory may include: obtaining abnormal reception operation associated data of the target data receiving end; generating a target retransmission request based on the abnormal reception operation associated data, and reading the target retransmission data from the network card memory based on the target retransmission request.

[0055] The abnormal receive operation-related data may be data generated by the target data receiving end describing the abnormal receive operation when the current processor first sends a message that is not successfully received by the target data receiving end. The abnormal receive operation-related data may include, but is not limited to, a timestamp, a data source identifier, an error type, and context information. The target retransmission request may be used to read the target retransmission data.

[0056] In an embodiment of the present invention, the abnormal reception operation associated data generated by the target data receiving end can be first obtained, and then the abnormal reception operation associated data can be parsed to obtain the necessary parameters for data retransmission, and based on the determined necessary parameters, a target retransmission request can be generated, and then based on the target retransmission request, the target retransmission data can be read from the network card memory.

[0057] In an optional embodiment of the present invention, generating a target retransmission request based on abnormal reception operation associated data may include: obtaining a target fallback frame number; and generating a target retransmission request based on the abnormal reception operation associated data and the target fallback frame number.

[0058] The target fallback frame number may be a preset number of frames for data retransmission. The network card memory may include DRAM.

[0059] In an embodiment of the present invention, the target fallback frame number set by the developer can be obtained, and the abnormal reception operation associated data can be parsed to determine the storage address of the target retransmission data, and then a target retransmission request can be generated based on the storage address of the target retransmission data and the target fallback frame number.

[0060] The technical solution of the embodiment of the present invention is to parse the original data to be transmitted sent by the current processor to obtain the target data to be transmitted, and then convert the target data to be transmitted according to the custom packet format to generate pre-processed data to be transmitted, and create the queue data to be backed up based on the pre-processed data to be transmitted and the data requested to be read by the target data receiving end, so as to back up the queue data to be backed up in the network card memory and send the queue data to the target data receiving end. In this solution, the original data to be transmitted directly pushed by the processor is used instead of actively grabbing the data stored in the memory by the processor to ensure low latency when the data packet is first transmitted, which can significantly reduce the overall access latency, and the backup of the queue data to be backed up can ensure reliable data transmission. Since the queue data to be backed up exists in the form of a queue, the current processor can realize synchronous data interaction with multiple target data receiving ends, solving the problem of high complexity of data sending operations and the need to further reduce transmission delay in the traditional RDMA transmission implementation process. It can reduce the complexity of data sending operations while ensuring reliable data transmission to achieve ultra-low latency data transmission.

[0061] Example 3

[0062] Figure 3 A communication logic diagram of an ultra-low latency RDMA data transmission system provided in Example 3 of the present invention. Figure 3 The RDMA data transmission system in can be understood as a new type of RDMA module (i.e. a new type of RDMA network card). Figure 3 As shown in the figure, the RDMA data transmission system includes a bus interface module, a data receiving module, and a data sending module. These three modules are distinguished by different background colors. This system is a hardware-level system. The RDMA data transmission system supports multiple queues, each of which can be connected to a different remote RDMA device (target data receiving end).

[0063] The bus interface module is used to implement high-speed data interaction between the RDMA data transmission system and the Noc (Network on Chip) bus, supporting direct memory access operations.

[0064] The data transmission module converts the raw data to be transmitted, received from the bus interface module, into a custom packet format. It arbitrates data from multiple sources (pre-processed data to be transmitted and data requested by the target data receiver). It also adds sequence numbers and control flags to the arbitrated data, generating data in the backup queue. It then allocates a fixed amount of DRAM space for each backup packet in the backup queue, based on the maximum packet length. This simplifies backup cache management, achieving a good balance between complexity and DRAM space efficiency. Packet information is recorded per queue for the backup queue data to ensure reliable transmission. If the target data receiver reports that a packet has been correctly received, the stored information for that packet is discarded. If it is not correctly received, packet retransmission control is triggered. By monitoring ACK / NAK feedback in real time, a Go Back N (Go Back N) retransmission mechanism is quickly activated, enabling hardware-level retransmission of abnormally transmitted packets. Specifically, when an ACK signal is detected, indicating that the target data receiver has successfully received the data sent by the current processor, the processor further transmits the data read from the target data receiver. The module also determines the packet information corresponding to the data successfully received by the target data receiver for subsequent data deletion. When the nak signal is detected, it indicates that the target data receiving end has not successfully received the data sent by the current processor, and the packet retransmission control is triggered.

[0065] The data receiving module is used to receive the RDMA data to be received transmitted by the target data receiving end, decapsulate the packet header information and payload, further perform RX TM processing (i.e., verify the data integrity, sequence and check code) to complete the data legitimacy verification, and finally convert it into a read or write request for the bus interface module.

[0066] When users transmit data, the current processor does not need to write the data to external DRAM first. Instead, it directly writes the original data to be transmitted to the data transmission module through the bus interface module. The data transmission module converts the data format of the original data to be transmitted (that is, converts it into a custom packet format that conforms to the internal transmission protocol), arbitrates multiple data sources, manages packet backup after arbitration, and records packet information by queue (data packet header information is recorded in the queue to confirm the correct transmission or use it when retransmission is required).

[0067] To ensure reliable transmission, the data transmission module uses a dual-channel transmission design for initial data transmission: one channel uses the MAC (Media Access Control) layer and PHY (Physical Media Access Layer) for high-speed transmission, while the other channel uses DRAM as a packet backup to facilitate hardware-level fast retransmission in the event of transmission errors. After decapsulating the incoming RDMA data, the data receiving module verifies the data's legitimacy through a verification mechanism and outputs the valid data to the bus interface module. In the event of transmission anomalies, the data will be promptly fed back to the data transmission module to trigger a retransmission mechanism.

[0068] After receiving the data packet sent by the current processor, the target data receiver performs a preliminary physical layer verification and then sends the pending RDMA data to the data receiving module. The data receiving module first decapsulates the RDMA packet, parses the packet header information, and then performs RX TM processing. Verified data packets are immediately converted into bus read / write request outputs. If an anomaly is detected, a nak signal is fed back, triggering the packet retransmission control to initiate the fast retransmission process. The packet retransmission control in this solution uses a hardware-level design. When a data packet error or loss is detected, the N-frame fallback protocol is immediately initiated to quickly retransmit the data packet in the backup buffer (DRMA).

[0069] In a specific example, an ultra-low-latency RDMA data transmission communication system may be formed by an RDMA data transmission system, a Noc bus, and a target data receiving end.

[0070] In terms of data transmission, users write the original data to be transmitted directly into the RDMA data transmission system through the high-speed Noc bus interface. After the data is written, the original data to be transmitted is added with serial number and control flag information and packaged. Further arbitration is performed based on first-come-first-served and priority queue strategies. Data packets from different sources are uniformly scheduled and distributed to high-speed output and backup channels. The high-speed channel achieves fast physical layer transmission through Mac, PHY and external Ethernet, while the packet backup channel stores data packets in the form of queues in the large-capacity DRAM cache. This ensures that in the event of data loss or errors, the data packets can be quickly retransmitted through the hardware retransmission mechanism, thereby achieving reliable transmission.

[0071] Regarding data reception, after the target data receiving end transmits a data packet (to be received RDMA data) via Ethernet, it first parses the packet header information and extracts the data payload for RX TM processing. For qualified packets, the module converts them into read and write requests compliant with the Noc bus protocol and outputs them to the current processor via the bus interface module. If a data anomaly is detected, an immediate nak signal is fed back, triggering the retransmission control to initiate a hardware-level fallback N-frame retransmission mechanism, rapidly retrieving and retransmitting the packet from the backup cache.

[0072] This solution greatly simplifies the user data transmission process: users can directly write the original data to be transmitted into the RDMA data transmission system through the bus interface module, eliminating the tedious operations of multiple DRAM storage, WQE generation and pointer updates in traditional RDMA solutions, reducing the number of data copies and processing links; significantly reducing transmission delay: due to the reduction in transmission steps, the delay in data transmission is significantly reduced (to 1 / 4 of the original, that is, the delay is reduced from 1700ns of traditional RDMA to 400ns). It is particularly suitable for small packet high-frequency transmission scenarios; reducing user operation load: due to the optimization of the transmission process, the number of steps involved by the user is reduced, thereby reducing the user's workload. The bandwidth demand for DRAM is halved: the traditional RDAM solution software writes the original data to be transmitted to DRAM, and then the hardware reads the data from DRAM; this solution only has packet backup write bandwidth for DRAM, and there is no need to read data from DRAM (because the probability of retransmission is relatively low, the DRAM access bandwidth can be ignored).

[0073] This solution achieves a leap from the traditional multi-step data transfer process to a single-step direct data write. Users can bypass external DRAM storage and the complex WQE generation process by directly transferring data to the RDMA data transfer system through the bus interface module, significantly reducing data transmission latency. Direct write technology not only simplifies the operation process but also reduces the number of data copies, providing a new technical path for high-speed data transmission.

[0074] The RDMA data transmission system utilizes a standardized modular design across all functional pathways and interfaces, facilitating integration with existing and future transmission protocol systems and enabling system-level expansion. This design not only reduces system development costs but also lays a solid foundation for subsequent product upgrades and functional expansion. The overall solution offers broad application prospects and high market competitiveness.

[0075] Example 4

[0076] Figure 4 This is a structural diagram of a data transmission device provided by the fourth embodiment of the present invention. Figure 4 As shown, the device includes:

[0077] The to-be-backed-up queue data generating module 310 is configured to generate to-be-backed-up queue data based on the original to-be-transmitted data sent by the current processor and the data requested to be read by the target data receiving end;

[0078] The data backup and sending module 320 is used to back up the queue data to be backed up in the network card memory and send the queue data to be backed up to the target data receiving end.

[0079] The technical solution of an embodiment of the present invention generates backup queue data based on the original data to be transmitted sent by the current processor and the data requested to be read by the target data receiving end, then backs up the backup queue data in the network card memory and sends the backup queue data to the target data receiving end. In this solution, the original data to be transmitted is directly pushed by the processor, rather than actively capturing data stored in the memory by the processor, ensuring low latency during the initial transmission of the data packet, which can significantly reduce the overall access latency. The backup of the backup queue data can also ensure reliable data transmission, solving the problems of high complexity in data transmission operations and the need to further reduce transmission latency in the traditional RDMA transmission implementation process. While ensuring reliable data transmission, it can reduce the complexity of data transmission operations to achieve ultra-low latency data transmission.

[0080] Optionally, the backup queue data generation module 310 is used to parse the original data to be transmitted sent by the current processor to obtain the target data to be transmitted; convert the target data to be transmitted according to a custom packet format to generate preprocessed data to be transmitted, and create the backup queue data based on the preprocessed data to be transmitted and the data requested to be read by the target data receiving end.

[0081] Optionally, the data transmission device also includes a current data to be encapsulated arbitration module, which is used to send non-to-be-backed up queue data to the target data receiving end; wherein, the non-to-be-backed up queue data includes data reception response and / or target retransmission data; arbitrate the current data to be encapsulated from the to-be-backed up queue data and the non-to-be-backed up queue data; encapsulate the current data to be encapsulated according to the remote direct memory access RDMA packet structure to obtain target encapsulated data, and send the target encapsulated data to the target data receiving end.

[0082] Optionally, the data transmission device also includes a non-backup queue data acquisition module, which is used to parse the RDMA data to be received sent by the target data receiving end, and generate the data reception response when the RDMA data to be received passes the verification; and / or, when there is a data reception anomaly at the target data receiving end, determine the target retransmission data in the network card memory.

[0083] Optionally, the data transmission device also includes a data deletion module, which is used to determine the packet header information of the target queue data successfully received by the target data receiving end when the RDMA data to be received passes the verification; and delete data from the network card memory according to the packet header information of the target queue data.

[0084] Optionally, a non-backup queue data acquisition module is used to obtain abnormal reception operation associated data of the target data receiving end; based on the abnormal reception operation associated data, generate a target retransmission request, and based on the target retransmission request, read the target retransmission data from the network card memory.

[0085] Optionally, a non-backup queue data acquisition module is used to obtain the target fallback frame number; based on the abnormal reception operation associated data and the target fallback frame number, the target retransmission request is generated; wherein the network card memory includes a dynamic random access memory DRAM.

[0086] The data transmission device provided in the embodiment of the present invention can execute the data transmission method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0087] Example 5

[0088] Figure 5 The components, their connections and relationships, and their functions shown herein are merely examples and are not intended to limit the implementation of the invention described and / or claimed herein.

[0089] like Figure 5 As shown, electronic device 10 includes at least one processor 11 and memory, such as ROM 12 and RAM 13, communicatively connected to at least one processor 11. The memory stores computer programs executable by the at least one processor, and processor 11 can perform various appropriate actions and processes based on the computer programs stored in ROM 12 or loaded from storage unit 18 into RAM 13. RAM 13 can also store various programs and data required for the operation of electronic device 10. Processor 11, ROM 12, and RAM 13 are interconnected via bus 14. An I / O interface 15 is also connected to bus 14. ROM 12 is a read-only memory, RAM 13 is a random access memory, and I / O interface 15 is an input / output interface.

[0090] Multiple components within electronic device 10 are connected to I / O interface 15, including an input unit 16, such as a keyboard and mouse; an output unit 17, such as various types of displays and speakers; a storage unit 18, such as a magnetic disk and optical disk; and a communication unit 19, such as a network card, a modem, or a wireless communication transceiver. Communication unit 19 allows electronic device 10 to exchange information / data with other devices via computer networks such as the Internet and / or various telecommunications networks. It should be noted that communication unit 19 specifically includes the RDMA data transmission system described in the aforementioned embodiments. The RDMA data transmission system is used to execute the data transmission method described in any of the embodiments of the present invention.

[0091] The processor 11 may be any general-purpose and / or specialized processing component with processing and computing capabilities. Examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various processors for running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, and the like.

[0092] In some embodiments, the data transmission method may be implemented as computer instructions tangibly embodied in a computer-readable storage medium, where the computer instructions are configured to cause an RDMA data transmission system to implement the data transmission method in any embodiment when executed.

[0093] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0094] Computer programs can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer program can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0095] In the context of the present invention, computer-readable storage medium can be a tangible medium that can contain or store a computer program for use with an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. Computer-readable storage medium can include but is not limited to electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, computer-readable storage medium can be a machine-readable signal medium. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, RAM, ROM, an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device or any suitable combination of the foregoing.

[0096] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0097] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0098] A computing system may include clients and servers. The clients and servers are generally remote from each other and typically interact via a communication network. This client-server relationship arises through computer programs running on the respective computers, creating a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host. This server is a hosting product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosts and VPS servers.

[0099] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present invention can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved. This is not limited herein.

[0100] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A data transmission method, characterized in that: include: Generate queue data to be backed up based on the original data to be transmitted sent by the current processor and the data requested to be read by the target data receiving end; The queue data to be backed up is backed up in a network card memory, and the queue data to be backed up is sent to the target data receiving end.

2. The method according to claim 1, characterized in that Based on the original data to be transmitted sent by the current processor and the data requested to be read by the target data receiving end, the queue data to be backed up is generated, including: Parsing the original data to be transmitted sent by the current processor to obtain target data to be transmitted; The target data to be transmitted is converted according to a custom packet format to generate pre-processed data to be transmitted, and the queue data to be backed up is created based on the pre-processed data to be transmitted and the data requested to be read by the target data receiving end.

3. The method according to claim 1, characterized in that While sending the queue data to be backed up to the target data receiving end, the method further includes: Sending non-to-be-backed-up queue data to the target data receiving end; wherein the non-to-be-backed-up queue data includes data reception response and / or target retransmission data; arbitrating the current data to be encapsulated from the queue data to be backed up and the queue data not to be backed up; According to the Remote Direct Memory Access (RDMA) packet structure, the current data to be encapsulated is encapsulated to obtain target encapsulated data, and the target encapsulated data is sent to the target data receiving end.

4. The method according to claim 3, characterized in that Before sending the non-to-be-backed-up queue data to the target data receiving end, the method further includes: parsing the to-be-received RDMA data sent by the target data receiving end, and generating the data reception response when the to-be-received RDMA data passes verification; and / or, When a data reception exception occurs at the target data receiving end, target retransmission data in the network card memory is determined.

5. The method according to claim 4, characterized in that After parsing the RDMA data to be received sent by the target data receiving end, the method further includes: When the to-be-received RDMA data passes the verification, determining the packet header information of the target queue data successfully received by the target data receiving end; Data is deleted from the network card memory according to the packet header information of the target queue data.

6. The method according to claim 4, characterized in that Determining target retransmission data in the network card memory includes: Acquiring abnormal receiving operation associated data of the target data receiving end; A target retransmission request is generated based on the abnormal reception operation associated data, and the target retransmission data is read from the network card memory based on the target retransmission request.

7. The method according to claim 6, characterized in that Generating a target retransmission request based on the abnormal reception operation associated data includes: Get the target fallback frame number; generating the target retransmission request based on the abnormal reception operation associated data and the target backoff frame number; Wherein, the network card memory includes a dynamic random access memory DRAM.

8. A data transmission device, characterized in that: include: A module for generating data of a queue to be backed up is used to generate data of a queue to be backed up based on the original data to be transmitted sent by the current processor and the data requested to be read by the target data receiving end; The data backup and sending module is used to back up the queue data to be backed up in the network card memory and send the queue data to be backed up to the target data receiving end.

9. A communication unit, characterized in that: The communication unit includes an RDMA data transmission system for executing the data transmission method according to any one of claims 1 to 7.

10. An electronic device, characterized in that: The electronic device includes the communication unit according to claim 9.

11. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the RDMA data transmission system to implement the data transmission method according to any one of claims 1 to 7 when executed.

Images