Data transmission method, system and device, electronic equipment and medium

Abstract

The invention provides a data transmission method, system and device, electronic equipment and a medium, and relates to the field of artificial intelligence, and the method comprises the steps: determining corresponding states of a first cache region at a host side and a second cache region at a card side; and sending to-be-processed data or a processing instruction or a transmission instruction to the card side on the basis of a state corresponding to a first cache region on the host side and a state corresponding to a second cache region on the card side under the condition that an interruption condition is met, so that the card side performs data processing on the to-be-processed data or first data in the second cache region, or the card side sends the second data to the host side. According to the method provided by the invention, the control logic can be realized by the host side, the load of the card side is reduced, the data transmission with the card side is established by the host side on the basis of the state of the cache region under the condition that the interruption condition is met, the interruption times are reduced, the process of copying from the kernel to the user space is avoided, the data transmission efficiency is improved, and the user experience is improved. And the CPU resource utilization rate of the host side is further improved.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence, and in particular to a data transmission method and system, device, electronic device, and medium. Background Art

[0002] The rapid rise and application of artificial intelligence, especially large language models, has placed extremely high demands on computing power. During the training phase, clusters consisting of large numbers of high-performance computing cards are required. These models also require considerable computing power during the inference phase (i.e., text generation) to quickly generate high-quality responses, especially when handling complex long text generation tasks or multi-turn dialogue scenarios. Summary of the Invention

[0003] The present disclosure provides a data transmission method and system, device, electronic device and medium to solve the problems in related technologies such as frequent interruptions, excessive card-side load, and low data transmission efficiency caused by the need to copy from kernel space to user space.

[0004] The first aspect embodiment of the present disclosure proposes a data transmission method, including: determining the status corresponding to the first cache area on the host side and the status corresponding to the second cache area on the card side; based on the status corresponding to the first cache area on the host side and the status corresponding to the second cache area on the card side, when an interrupt condition is met, sending to-be-processed data or a processing instruction or a transmission instruction to the card side, so that the card side processes the to-be-processed data or the first data in the second cache area, or sends the second data from the card side to the host side.

[0005] In some embodiments of the present disclosure, determining the state corresponding to the first cache area on the host side and the state corresponding to the second cache area on the card side includes: reading the first cache area and the second cache area; when the free capacity of the first cache area is greater than the first preset capacity, determining that the first cache area is in the first state, otherwise, determining that the first cache area is in the second state; and, when the free capacity of the second cache area is greater than the second preset capacity, determining that the second cache area is in the first state, otherwise, determining that the second cache area is in the second state.

[0006] In some embodiments of the present disclosure, based on the status corresponding to the first cache area on the host side and the status corresponding to the second cache area on the card side, when the interrupt condition is met, to-be-processed data or a processing indication or a transmission indication is sent to the card side, including any of the following: when the status corresponding to the second cache area is the first state, based on the user-mode polling PMD, the to-be-processed data is sent to the card side through direct memory access DMA, so that the card side processes the to-be-processed data; when the status corresponding to the second cache area is the second state, a processing indication is sent to the card side, so that the card side processes the first data and deletes the processed data from the second cache area; when the status corresponding to the first cache area is the first state, based on the PMD, a transmission indication is sent to the card side, so that the card side sends the second data to the host side through direct memory access; when the status corresponding to the first cache area is the second state, based on the PMD, the to-be-processed data is sent to the card side through DMA, so that the card side processes the to-be-processed data.

[0007] In some embodiments of the present disclosure, the method further includes: polling each transmission channel in the transmission channel group to determine the status of each transmission channel, wherein the status of the transmission channel includes a completion status and a failure status, the completion status indicates that the card side has completed data processing of the first data or the data to be processed, and the failure status indicates that the card side has not completed data processing of the first data or the data to be processed.

[0008] In some embodiments of the present disclosure, based on user-mode polling PMD, the data to be processed is sent to the card side through direct memory access DMA, including any of the following: when the status of the first transmission channel in the transmission channel group is a completed state, the resources of the first transmission channel are released, and the current data to be processed is sent to the card side through the first transmission channel; when the status of the first transmission channel in the transmission channel group is a failed state, the last data to be processed is resent through the first transmission channel.

[0009] In some embodiments of the present disclosure, based on PMD, a transmission indication is sent to the card side, including: when the status of the second transmission channel in the transmission channel group is a completion state, releasing the resources of the second transmission channel and sending a transmission indication to the card side; receiving the second data sent by the card side through the second transmission channel.

[0010] In the above embodiment, the data transmission method reduces the load on the card side by implementing the control logic on the host side. The host side establishes data transmission with the card side based on the status of the cache area when the interrupt conditions are met, thereby reducing the number of interrupts and avoiding the process of copying from the kernel to the user space, thereby improving data transmission efficiency and further improving the CPU resource utilization on the host side.

[0011] The second aspect embodiment of the present disclosure proposes a data transmission system, including: a host side and a card side, the host side including: a first cache area, a control unit; the card side including: a second cache area; a transmission channel group is provided between the host side and the card side, and the transmission channel group includes multiple transmission channels; the control unit is used to determine the state corresponding to the first cache area on the host side and the state corresponding to the second cache area on the card side; based on the state corresponding to the first cache area on the host side and the state corresponding to the second cache area on the card side, when an interrupt condition is met, the host side sends to-be-processed data or a processing instruction or a transmission instruction to the card side, so that the card side processes the to-be-processed data or the first data in the second cache area, or sends the second data to the host side.

[0012] In some embodiments of the present disclosure, the system also includes: a status unit, which is used to maintain the status of each transmission channel in the transmission channel group, wherein the status of the transmission channel includes a completion status and a failure status, the completion status indicates that the card side has completed data processing of the first data, and the failure status indicates that the card side has not completed data processing of the first data.

[0013] In some embodiments of the present disclosure, the system also includes: an interrupt unit, which is used to send pending data or transmission instructions or processing instructions from the host side to the card side when the interrupt condition is met, wherein, when the interrupt condition is met, sending pending data or transmission instructions or processing instructions from the host side to the card side includes any one of the following: when the state corresponding to the first cache area is the first state, based on the user state polling PMD, sending a transmission indication to the card side; when the state corresponding to the second cache area is the second state, sending a processing indication to the card side; when the state corresponding to the second cache area is the first state or the state corresponding to the first cache area is the second state, based on the PMD, sending pending data to the card side through direct memory access DMA; wherein, the first state is that the free capacity of the first cache area is greater than the first preset capacity, and the second state is that the free capacity of the second cache area is greater than the second preset capacity.

[0014] In the above embodiment, the data transmission system improves the efficiency of hardware DMA utilization by implementing a set of IPC protocol stack encapsulation on the host side; uses PMD polling and interrupts to implement data channel management, reduces interrupt events in the system, improves the CPU resource utilization on the host side, and reduces the load on the card side.

[0015] The third aspect embodiment of the present disclosure proposes a data transmission device, including: a determination module, used to determine the status corresponding to the first cache area on the host side and the status corresponding to the second cache area on the card side; a transmission module, used to send to-be-processed data or processing instructions or transmission instructions to the card side based on the status corresponding to the first cache area on the host side and the status corresponding to the second cache area on the card side, when an interrupt condition is met, so that the card side processes the to-be-processed data or the first data in the second cache area, or sends the second data from the card side to the host side.

[0016] An embodiment of the fourth aspect of the present disclosure provides an electronic device, comprising: a processor and a memory for storing a computer program that can be run on the processor, wherein the processor, when used to run the computer program, executes any one of the methods described in the first aspect of the present disclosure.

[0017] The fifth aspect embodiment of the present disclosure provides a non-transitory computer-readable storage medium storing computer instructions, where the computer instructions are used to enable a computer to execute any one of the methods described in the first aspect of the present disclosure.

[0018] In summary, according to the data transmission method proposed in the present invention, the control logic is implemented by the host side, which reduces the load on the card side. The host side establishes data transmission with the card side based on the status of the cache area when the interrupt conditions are met, thereby reducing the number of interruptions and avoiding the process of copying from the kernel to the user space, thereby improving data transmission efficiency and further improving the CPU resource utilization on the host side.

[0019] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute an improper limitation of the present disclosure.

[0021] Figure 1 A flowchart of a data transmission method provided in an embodiment of the present disclosure;

[0022] Figure 2 A flowchart of a data transmission method provided in an embodiment of the present disclosure;

[0023] Figure 3 A schematic diagram of a data transmission process according to an embodiment of the present disclosure;

[0024] Figure 4 An architectural diagram of a data transmission system provided in an embodiment of the present disclosure;

[0025] Figure 5A Schematic diagram of system implementation;

[0026] Figure 5B This is a diagram of the data path architecture;

[0027] Figure 5C It is an abstract diagram of the data channel software;

[0028] Figure 6 A schematic structural diagram of a data transmission device provided in an embodiment of the present disclosure;

[0029] Figure 7 A schematic diagram of the structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0030] The following describes in detail embodiments of the present disclosure, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present disclosure, and should not be construed as limiting the present disclosure.

[0031] Traditional computing services use the PCIE bus as the server's high-speed bus for device interconnection. In the PCIE topology, the CPU sits at the top of the tree, and computing accelerator cards primarily join this topology via the PCIE bus. This structure offers advantages in ease of management and expansion. For example, motherboard manufacturers can flexibly add PCIE slots as needed, and different devices can be managed hierarchically through the chipset. However, the tree topology also has limitations. Because the communication path between devices must pass through the CPU or chipset, it can increase the CPU burden when there are many devices or high data traffic. Furthermore, computing accelerator cards, due to the large amount of data offloaded in their business scenarios, require frequent information exchange between the server and the accelerator card. This introduces numerous hardware interrupts and creates the risk of terminal storms, reducing data transmission performance between the host and the device.

[0032] However, the data transmission channel between the AI ​​accelerator card and the host is currently controlled on the card side. The host CPU is notified to process the data packet by sending an interrupt. The card-side data packet then reaches the host's kernel space and is copied from the kernel space to the user space. This type of solution has drawbacks such as frequent interruptions in data exchange during card load and multiple copies of the transmission link, making it unsuitable for current AI acceleration applications.

[0033] In order to solve the problems existing in the related technology, the present disclosure proposes a data transmission method and system, which addresses the drawbacks of the data transmission channel between the above-mentioned host and accelerator card, solves the problem of frequent interruptions, reduces the board load, and reduces data copying on the communication link.

[0034] The data transmission method and system, device, electronic device and medium provided in this application are described in detail below with reference to the accompanying drawings.

[0035] Figure 1 The data transmission method of the present disclosure is executed by the host side, such as Figure 1 As shown, the following steps are included:

[0036] Step 101: Determine a state corresponding to a first buffer area on a host side and a state corresponding to a second buffer area on a card side.

[0037] In some embodiments, the first cache area and the second cache area are read; when the free capacity of the first cache area is greater than the first preset capacity, the first cache area is determined to be in the first state, otherwise, the first cache area is determined to be in the second state; and, when the free capacity of the second cache area is greater than the second preset capacity, the second cache area is determined to be in the first state, otherwise, the second cache area is determined to be in the second state.

[0038] In some embodiments, the host side can read the first buffer area on the host side and the second buffer area on the card side, so as to judge the status of the first buffer area and the second buffer area to control whether to interrupt.

[0039] In some embodiments, the first preset capacity and the second preset capacity may be pre-set free capacity values ​​of the cache area. The values ​​of the first preset capacity and the second preset capacity may be the same or different, which is not limited in the present disclosure.

[0040] In some embodiments, when the free capacity of the first cache area is greater than the first preset capacity, it may be that the first cache area is full, that is, the first cache area is determined to be in the first state; otherwise, the first cache area is determined to be in the second state.

[0041] In some embodiments, when the free capacity of the second cache area is greater than the second preset capacity, it may be that the second cache area is full, that is, the second cache area is determined to be in the first state; otherwise, the second cache area is determined to be in the second state.

[0042] Step 102 : Based on the status corresponding to the first buffer area on the host side and the status corresponding to the second buffer area on the card side, if an interruption condition is met, sending data to be processed or a processing instruction or a transmission instruction to the card side.

[0043] In some embodiments, the host side sends data to be processed or a processing instruction or a transmission instruction to the card side, so that the card side processes the data to be processed or the first data in the second buffer area, or sends second data to the host side.

[0044] In some embodiments, the interrupt condition may be a pre-set condition for triggering the transmission of control signaling. When the interrupt condition is met, the control unit on the host side triggers the transmission of control signaling, i.e., sends data to be processed or processing instructions or transmission instructions to the card side.

[0045] In some embodiments, the processing instruction is used to instruct the card side to perform data processing on the first data stored in the second buffer area to release space resources in the second buffer area.

[0046] In some embodiments, the transmission instruction is used to instruct the card side to send the second data to the host side, so that the host side stores the second data.

[0047] In some embodiments, the host side sends the data to be processed to the card side, and the card side processes the data to be processed.

[0048] In some embodiments, the data to be processed may be data required for model training or data required for model inference.

[0049] In some embodiments, data processing may be performing model training using data required for model training, or performing model inference using data required for model inference.

[0050] In some embodiments, when the interruption condition is met, sending the data to be processed or the processing instruction or the transmission instruction to the card side includes the following schemes:

[0051] Option 1:

[0052] In some embodiments, when the state corresponding to the second buffer area is the first state, based on the user state polling PMD, the data to be processed is sent to the card side via direct memory access DMA, so that the card side processes the data to be processed.

[0053] In some embodiments, the state corresponding to the second buffer area is the first state, which means that there is space in the buffer area on the card side, and the data to be processed can continue to be sent to the card side.

[0054] In some embodiments, the user-mode polling PMD may poll the data transmission channel between the host side and the card side to determine whether data in the data transmission channel has been processed.

[0055] In some embodiments, direct memory access (DMA) enables the host side to exchange data directly with the memory without intervention by the CPU, thereby reducing CPU burden and improving system performance.

[0056] For example, Figure 5CAs shown in the diagram, a ringbuffer is maintained on both the host and EP sides, enabling full-duplex communication between the two accelerator cards. Data is sent as packets and received as streams. The host-side PMD polls the ringbuffer's read and write pointers to implement pipelined data transmission. If the PMD finds the EP's space is not full, it sends data to the EP via DMA.

[0057] Option 2:

[0058] In some embodiments, when the state corresponding to the second cache area is the second state, a processing instruction is sent to the card side so that the card side processes the first data and deletes the processed data from the second cache area.

[0059] In some embodiments, if the state corresponding to the second buffer area is the second state, it means that there is no space in the buffer area on the card side, and the card side needs to be notified to process the data in the second buffer area to release space in the buffer area.

[0060] For example, Figure 5C As shown in the diagram, the host side reads that the cache on the EP side is full, and tells the EP side to fetch data through an interrupt. The host can continue to send data only after the EP side fetches the data, otherwise the data transmission will be blocked.

[0061] Option 3:

[0062] In some embodiments, when the state corresponding to the first buffer area is the first state, a transmission instruction is sent to the card side based on the PMD, so that the card side sends the second data to the host side through direct memory access.

[0063] In some embodiments, the state corresponding to the first buffer area is the first state, which indicates that there is space in the buffer area of ​​the host side and an instruction needs to be sent to the card side to instruct the card side to send data to the host side.

[0064] In some embodiments, the control unit on the host side reads that there is still free capacity in its buffer area, and then sends a transmission instruction to the card side through an interrupt to inform the card side that data needs to be sent to the host side.

[0065] For example, Figure 5C As shown in the diagram, if the host reads its own cache and it is not full, it needs to inform the EP side to send data to it through an interrupt.

[0066] Option 4:

[0067] In some embodiments, when the state corresponding to the first buffer area is the second state, the data to be processed is sent to the card side via DMA based on PMD, so that the card side processes the data to be processed.

[0068] In some embodiments, when the state corresponding to the first cache area is the second state, it means that the cache area on the host side is full and has no free capacity, so the host side needs to send the data in the cache area to the card side for data processing by the card side.

[0069] In some embodiments, the host side sends the data to be processed in the first buffer area to the card side via DMA. That is, the host side sends the data to be processed to the card side by directly accessing the user space, bypassing the copy process from the operating system kernel, thereby improving the data packet transmission and processing speed.

[0070] For example, Figure 5C As shown in the schematic diagram, if the host reads that its cache space is full, it needs to send the data in the cache space to the EP side, which processes the data.

[0071] In the above embodiment, by implementing control logic on the host side to send an interrupt to the card side, the card side executes the relevant processing process, which reduces the number of interrupts, improves the overall stability of the system, and reduces the load on the card side; at the same time, based on the status of the cache area on the host side and the card side, data transmission between the card side and the card side can be achieved through direct memory access, which reduces the kernel transfer copy process and improves data transmission efficiency.

[0072] Figure 2 A flow chart of a data transmission method provided in an embodiment of the present disclosure, based on Figure 1 The embodiment shown, as Figure 2 As shown, the following steps are included:

[0073] Step 201: poll each transmission channel in the transmission channel group to determine the status of each transmission channel.

[0074] In some embodiments, the status of the transmission channel includes a completion status and a failure status. The completion status indicates that the card side has completed data processing of the first data or the data to be processed, and the failure status indicates that the card side has not completed data processing of the first data or the data to be processed.

[0075] In some embodiments, the host side can poll the status of each transmission channel in the transmission channel group through the PMD, wherein the status of the transmission channel can be that after the host side transmits data to the card side once, the card side writes the processed status into the channel status space when the data processing is completed, or writes the failed processing status into the channel status space when the data processing fails. The host side polls the status in the channel status space to determine whether the transmission channel can be released.

[0076] In some embodiments, the card side may also determine the status of each transmission channel to determine the transmission channel for data transmission when the host side releases the resources of the transmission channel.

[0077] In some embodiments, the maintenance space for the status of the transmission channel may be a BAR, which stores the status of each transmission channel. When the host side polls the BAR, it can determine whether the status of each transmission channel is a completed state or a failed state.

[0078] For example, Figure 5B As shown in the data path architecture diagram, a BAR is maintained between the host side and the EP side. The BAR stores the status of each transmission channel. After the card side completes data processing, it writes the status of the channel corresponding to the transmission data. The host side can poll the status of the transmission channel in the BAR to determine whether the transmission channel resources can be released for the next data transmission process.

[0079] In some embodiments, when the state of the first transmission channel in the transmission channel group is the completion state, the resources of the first transmission channel are released, and the current to-be-processed data is sent to the card side through the first transmission channel.

[0080] In some embodiments, the first transmission channel is any transmission channel in the transmission channel group.

[0081] In some embodiments, if the status of the first transmission channel is a completed status, it means that the card side has completed processing the data received through the first transmission channel, and the host side can release the resources of the first transmission channel to use the first transmission channel for the next data transmission.

[0082] In some embodiments, when the host side needs to send data to be processed to the card side, based on at least one first transmission channel obtained through polling, the host side selects any one of the at least one first transmission channel to send the data to be processed to the card side.

[0083] In some embodiments, combined Figure 1 In step 102, when the state corresponding to the second buffer area is the first state, it is determined through polling that the first transmission channel is in the completion state, and the data to be processed is sent to the card side through the first transmission channel.

[0084] In some embodiments, combined Figure 1 In step 102, when the state corresponding to the first buffer area is the second state, it is determined through polling that the first transmission channel is in the completion state, and the data to be processed is sent to the card side through the first transmission channel.

[0085] In some embodiments, when the state of the first transmission channel in the transmission channel group is a failure state, the last to-be-processed data is resent through the first transmission channel.

[0086] In some embodiments, if the status of the first transmission channel is a failure status, it means that the card side has not completed processing the data received through the first transmission channel. The host side needs to resend the data transmitted in the first transmission channel to the card side to avoid invalid data processing.

[0087] In the above embodiment, by maintaining a state space of a transmission channel group between the host side and the card side, the host side can determine the state of each transmission channel in the transmission channel group through user-mode polling, thereby reducing the delay in data packet processing and improving the efficiency of data transmission.

[0088] Figure 3 A schematic diagram of a data transmission process provided by an embodiment of the present disclosure, based on Figure 1-Figure 2 The embodiment shown, Figure 3 right Figure 1 Step 102 in the embodiment is further described as follows: Figure 3 As shown, the following steps are included:

[0089] Step 301: When the status of the second transmission channel in the transmission channel group is the completion status, release the resources of the second transmission channel and send a transmission instruction to the card side.

[0090] In some embodiments, based on user-mode polling, if it is determined that the second transmission channel is in a completed state, the resources of the second transmission channel may be released.

[0091] In some embodiments, when the state corresponding to the first cache area is the first state, the host side determines that its cache space is not full and needs to inform the card side to provide data to the host side, and then sends a transmission instruction to the card side.

[0092] In some embodiments, the transmission indication is used to instruct the card side to send data to the host side.

[0093] Step 302: Receive second data sent by the card side through the second transmission channel.

[0094] In some embodiments, both the host side and the card side can determine the status of the transmission channel. After the host side releases resources of the second transmission channel, the card side can send the second data to the host side through the second transmission channel.

[0095] In some embodiments, after the card side receives the transmission instruction, since the resources of the second transmission channel have been released, the second data can be sent to the host side using the second transmission channel.

[0096] In the above embodiment, the status of the transmission channel is maintained by the card side, so that the host side polls the status of the transmission channel group, so that the host side can grasp the status of the transmission channel in real time. When data transmission is required, the resources of the transmission channel in the completed state are released for the current data transmission, thereby reducing the delay in data processing and improving the data transmission efficiency.

[0097] In summary, according to the data transmission method proposed in the present invention, by implementing control logic on the host side to send interrupts to the card side, the card side executes the relevant processing process, which reduces the number of interrupts, improves the overall stability of the system, and reduces the load on the card side; at the same time, based on the status of the cache area on the host side and the card side, data transmission between the card side and the card side can be achieved through direct memory access, which reduces the kernel transfer copy process and improves data transmission efficiency.

[0098] Figure 4 This is an architecture diagram of a data transmission system proposed in an embodiment of the present disclosure, such as Figure 4 As shown, the data transmission system 400 includes a host side and a card side.

[0099] The host side includes: a first buffer area 401, a control unit 402;

[0100] The card side includes: a second buffer area 403;

[0101] There is a transmission channel group 404 between the host side and the card side, and the transmission channel group includes multiple transmission channels;

[0102] The control unit is used to determine the status corresponding to the first cache area on the host side and the status corresponding to the second cache area on the card side; based on the status corresponding to the first cache area on the host side and the status corresponding to the second cache area on the card side, when the interrupt condition is met, the host side sends the data to be processed or the processing instruction or the transmission instruction to the card side, so that the card side processes the data to be processed or the first data in the second cache area, or sends the second data to the host side.

[0103] In some embodiments, the data transmission system further includes: a status unit 405 and an interrupt unit 406 .

[0104] In some embodiments, the status unit is used to maintain the status of each transmission channel in the transmission channel group, wherein the status of the transmission channel includes a completion status and a failure status, the completion status indicates that the card side has completed data processing of the first data, and the failure status indicates that the card side has not completed data processing of the first data.

[0105] In some embodiments, the interrupt unit is used to send pending data or transmission instructions or processing instructions from the host side to the card side when the interrupt condition is met, wherein, when the interrupt condition is met, sending pending data or transmission instructions or processing instructions from the host side to the card side includes any one of the following: when the state corresponding to the first cache area is the first state, based on the user state polling PMD, sending a transmission instruction to the card side; when the state corresponding to the second cache area is the second state, sending a processing instruction to the card side; when the state corresponding to the second cache area is the first state or the state corresponding to the first cache area is the second state, based on the PMD, sending pending data to the card side through direct memory access DMA; wherein, the first state is that the free capacity of the first cache area is greater than the first preset capacity, and the second state is that the space capacity of the second cache area is greater than the second preset capacity.

[0106] In the above embodiment, the data transmission system is used to perform Figure 1-Figure 3 The data transmission method shown is used to realize data transmission between the host side and the card side, and to achieve high-speed unloading of AI model parameter data, which will not be repeated here.

[0107] The following is a specific implementation of a data transmission method based on DPDK (Data Plane Development Kit) user mode driver and interrupt:

[0108] like Figure 5A Figure 2 shows a schematic diagram of the system implementation. To address the interrupt storm problem in PCIE high-speed interconnects, this solution implements a DPDK user-mode driver, combined with the MHU (Message Processing Unit) interrupt channel, to achieve high-speed transmission between the host and the device. This solution is divided into two parts: the host (host side) and the EP (card side). The host uses the DPDK PMD (Poll Mode Drivers) architecture to achieve high-speed data transmission and reception, while the EP uses a Linux driver to achieve data transmission and reception. The PMD framework directly manages hardware resources in user mode, using efficient polling, zero-copy technology, and cache reuse to reduce packet processing latency and improve processing efficiency.

[0109] This solution is mainly used in AI accelerator card usage scenarios and general SOC underlying software design, providing efficient data transmission between heterogeneous systems and accelerating the inference training of mainstream AI models.

[0110] like Figure 5B The figure shows the data path architecture of this solution. The PMD module on the host side of this solution is based on the DMA (Direct Memory Access) capability of PCIE. It is encapsulated into a highly multiplexed logical channel and provided to the upper-layer application software. It also transmits control signaling through MHU interrupts. The model training data sent by the AI ​​runtime is encapsulated into fixed packets and unloaded to the EP side for training using DMA.

[0111] This solution abstracts a set of ringbuffer concepts on the host and EP sides to achieve full-duplex communication on both ends of the accelerator card. The data transmission channel groups packets in packet mode and receives packets in stream mode. On the host side, PMD polls the read and write pointers of the ringbuffer to implement pipeline data transmission, such as Figure 5C Shown is the data channel software abstract diagram.

[0112] This solution uses PMD polling and MHU interrupts to implement data channel management, reducing interrupt events in the system, improving host-side CPU resource utilization, and reducing board load; at the same time, it implements an IPC protocol stack encapsulation on the host side, improving the efficiency of hardware DMA usage.

[0113] Figure 6 Schematic diagram of a data transmission device 600 provided in an embodiment of the present disclosure. Figure 6 As shown, the device includes:

[0114] The determination module 610 is configured to determine a state corresponding to the first buffer area on the host side and a state corresponding to the second buffer area on the card side.

[0115] The transmission module 620 is used to send the pending data or processing instructions or transmission instructions to the card side based on the status corresponding to the first cache area on the host side and the status corresponding to the second cache area on the card side when the interrupt condition is met, so that the card side can process the pending data or the first data in the second cache area, or send the second data from the card side to the host side.

[0116] In some embodiments, the determination module is also used to read the first cache area and the second cache area; when the free capacity of the first cache area is greater than the first preset capacity, the first cache area is determined to be in the first state, otherwise, the first cache area is determined to be in the second state; and, when the free capacity of the second cache area is greater than the second preset capacity, the second cache area is determined to be in the first state, otherwise, the second cache area is determined to be in the second state.

[0117] In some embodiments, the transmission module is also used to, when the state corresponding to the second cache area is the first state, based on the user-mode polling PMD, send the data to be processed to the card side through direct memory access DMA, so that the card side processes the data to be processed; when the state corresponding to the second cache area is the second state, send a processing indication to the card side, so that the card side processes the first data and deletes the processed data from the second cache area; when the state corresponding to the first cache area is the first state, based on the PMD, send a transmission indication to the card side, so that the card side sends the second data to the host side through direct memory access; when the state corresponding to the first cache area is the second state, based on the PMD, send the data to be processed to the card side through DMA, so that the card side processes the data to be processed.

[0118] In some embodiments, the determination module is also used to poll each transmission channel in the transmission channel group to determine the status of each transmission channel, wherein the status of the transmission channel includes a completion status and a failure status, the completion status indicates that the card side has completed data processing of the first data or the data to be processed, and the failure status indicates that the card side has not completed data processing of the first data or the data to be processed.

[0119] In some embodiments, the transmission module is also used to release the resources of the first transmission channel and send the current pending data to the card side through the first transmission channel when the status of the first transmission channel in the transmission channel group is a completed state; and to resend the last pending data through the first transmission channel when the status of the first transmission channel in the transmission channel group is a failed state.

[0120] In some embodiments, the transmission module is further used to release the resources of the second transmission channel and send a transmission instruction to the card side when the status of the second transmission channel in the transmission channel group is a completed state; and receive the second data sent by the card side through the second transmission channel.

[0121] In summary, according to the data transmission device proposed in the present invention, the operating load of the board is reduced by unloading the control logic to the host side; device data is obtained through user-state polling, interrupt events are reduced, the occurrence of interrupt storms is avoided, and the stability of the system is improved; at the same time, through user-state driving, the operating system kernel is bypassed and data packets are processed directly in user space, while avoiding the complexity and risks of kernel driving, and improving the data packet sending and receiving speed and processing speed.

[0122] In the embodiments provided above, the methods and devices provided in the embodiments of the present application are introduced. In order to implement the various functions of the methods provided in the embodiments of the present application, the electronic device may include a hardware structure and a software module, and implement the aforementioned functions in the form of a hardware structure, a software module, or a hardware structure plus a software module. One of the aforementioned functions may be executed in the form of a hardware structure, a software module, or a hardware structure plus a software module.

[0123] Figure 7 FIG2 is a block diagram of an electronic device 700 for implementing the above-mentioned data transmission method according to an exemplary embodiment. For example, the electronic device 700 may be a mobile phone, a computer, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0124] Reference Figure 7 , electronic device 700 may include one or more of the following components: a processing component 702 , a memory 704 , a power component 706 , a multimedia component 708 , an audio component 710 , an input / output (I / O) interface 712 , a sensor component 714 , and a communication component 716 .

[0125] The processing component 702 generally controls the overall operation of the electronic device 700, such as operations associated with display, phone calls, data communications, camera operation, and recording operations. The processing component 702 may include one or more processors 720 to execute instructions to perform all or part of the steps of the above-described method. In addition, the processing component 702 may include one or more modules to facilitate interaction between the processing component 702 and other components. For example, the processing component 702 may include a multimedia module to facilitate interaction between the multimedia component 708 and the processing component 702.

[0126] The memory 704 is configured to store various types of data to support operations on the electronic device 700. Examples of such data include instructions for any application or method operating on the electronic device 700, contact data, phone book data, messages, pictures, videos, etc. The memory 704 can be implemented by any type of volatile or non-volatile storage device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0127] The power supply component 706 provides power to the various components of the electronic device 700. The power supply component 706 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 700.

[0128] The multimedia component 708 includes a screen that provides an output interface between the electronic device 700 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, slides, and gestures on the touch panel. The touch sensor can not only sense the boundaries of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 708 includes a front camera and / or a rear camera. When the electronic device 700 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have focal length and optical zoom capabilities.

[0129] The audio component 710 is configured to output and / or input audio signals. For example, the audio component 710 includes a microphone (MIC), which is configured to receive external audio signals when the electronic device 700 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signal can be further stored in the memory 704 or transmitted via the communication component 716. In some embodiments, the audio component 710 also includes a speaker for outputting audio signals.

[0130] I / O interface 712 provides an interface between processing component 702 and peripheral interface modules, such as a keyboard, click wheel, buttons, etc. These buttons may include but are not limited to: a home button, volume buttons, a start button, and a lock button.

[0131] The sensor assembly 714 includes one or more sensors for providing various aspects of status assessment for the electronic device 700. For example, the sensor assembly 714 can detect the open / closed state of the electronic device 700, the relative positioning of components, such as the display and keypad of the electronic device 700. The sensor assembly 714 can also detect changes in the position of the electronic device 700 or a component of the electronic device 700, the presence or absence of user contact with the electronic device 700, the orientation or acceleration / deceleration of the electronic device 700, and temperature changes of the electronic device 700. The sensor assembly 714 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 714 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 714 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0132] The communication component 716 is configured to facilitate wired or wireless communication between the electronic device 700 and other devices. The electronic device 700 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, 4G LTE, 5G NR (NewRadio) or a combination thereof. In an exemplary embodiment, the communication component 716 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 716 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0133] In an exemplary embodiment, the electronic device 700 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.

[0134] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 704 including instructions, and the instructions can be executed by the processor 720 of the electronic device 700 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0135] The embodiments of the present disclosure further provide a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the data transmission method described in the above embodiments of the present disclosure.

[0136] An embodiment of the present disclosure further provides a computer program product, including a computer program. The computer program is used by a processor to execute the data transmission method described in the above embodiment of the present disclosure.

[0137] It should be noted that the terms "first," "second," and the like in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the numbers used in this manner are interchangeable where appropriate so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure as detailed in the appended claims.

[0138] Throughout this specification, references to terms such as "one embodiment," "some embodiments," "illustrative embodiments," "examples," "specific examples," or "some examples" indicate that a specific feature, structure, material, or characteristic described in conjunction with the embodiment or example is included in at least one embodiment or example of the present invention. In this specification, illustrative uses of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0139] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may be performed out of the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present invention pertain.

[0140] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processing module, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (non-exhaustive list) of computer-readable media include the following: an electrical connection having one or more wires (control method), a portable computer disk cartridge (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and a portable compact disc read-only memory (CDROM). Furthermore, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or otherwise processing it in a suitable manner if necessary, and then storing it in a computer memory.

[0141] It should be understood that various parts of the embodiments of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0142] Those skilled in the art will understand that all or part of the steps in the above-mentioned embodiment method can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0143] Furthermore, the functional units in the various embodiments of the present invention may be integrated into a single processing module, each unit may exist physically separately, or two or more units may be integrated into a single module. The aforementioned integrated modules may be implemented in either hardware or software functional modules. If the integrated modules are implemented as software functional modules and sold or used as standalone products, they may also be stored in a computer-readable storage medium. The aforementioned storage medium may be a read-only memory, a magnetic disk, or an optical disk, etc.

[0144] Although the embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are exemplary and are not to be construed as limitations on the present invention. A person skilled in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A data transmission method, characterized in that: The method is executed by the host side, and the method includes: Determine a state corresponding to the first buffer area on the host side and a state corresponding to the second buffer area on the card side; Based on the status corresponding to the first cache area on the host side and the status corresponding to the second cache area on the card side, when the interrupt condition is met, the data to be processed or the processing instruction or the transmission instruction is sent to the card side, so that the card side processes the data to be processed or the first data in the second cache area, or sends the second data to the host side.

2. The method according to claim 1, characterized in that The determining of a state corresponding to the first buffer area on the host side and a state corresponding to the second buffer area on the card side includes: Reading the first buffer area and the second buffer area; When the free capacity of the first cache area is greater than the first preset capacity, the first cache area is determined to be in the first state; otherwise, the first cache area is determined to be in the second state; and, when the free capacity of the second cache area is greater than the second preset capacity, the second cache area is determined to be in the first state; otherwise, the second cache area is determined to be in the second state.

3. The method according to claim 1, characterized in that The sending of to-be-processed data, a processing instruction, or a transmission instruction to the card side based on a status corresponding to the first buffer area on the host side and a status corresponding to the second buffer area on the card side when an interrupt condition is satisfied includes any one of the following: When the state corresponding to the second buffer area is the first state, based on user-mode polling PMD, sending the data to be processed to the card side through direct memory access DMA, so that the card side processes the data to be processed; When the state corresponding to the second buffer area is the second state, sending the processing instruction to the card side, so that the card side performs the data processing on the first data and deletes the processed data from the second buffer area; When the state corresponding to the first buffer area is the first state, based on the PMD, sending the transmission instruction to the card side, so that the card side sends the second data to the host side through the direct memory access; When the state corresponding to the first buffer area is the second state, the data to be processed is sent to the card side through the DMA based on the PMD, so that the card side processes the data to be processed.

4. The method according to claim 3, characterized in that The method further comprises: Poll each transmission channel in the transmission channel group to determine the status of each transmission channel, wherein the status of the transmission channel includes a completion status and a failure status, the completion status indicates that the card side has completed data processing of the first data or the data to be processed, and the failure status indicates that the card side has not completed data processing of the first data or the data to be processed.

5. The method according to claim 4, characterized in that The sending of the to-be-processed data to the card side through direct memory access (DMA) based on user-mode polling PMD includes any one of the following: When the state of the first transmission channel in the transmission channel group is the completion state, releasing resources of the first transmission channel, and sending current to-be-processed data to the card side through the first transmission channel; When the state of the first transmission channel in the transmission channel group is the failure state, the last data to be processed is resent through the first transmission channel.

6. The method according to claim 4, characterized in that The sending the transmission indication to the card side based on the PMD includes: When the state of the second transmission channel in the transmission channel group is the completion state, releasing resources of the second transmission channel and sending the transmission instruction to the card side; The second data sent by the card side is received through the second transmission channel.

7. A data transmission system, characterized in that: Including host side and card side, The host side includes: a first buffer area, a control unit; The card side includes: a second buffer area; There is a transmission channel group between the host side and the card side, and the transmission channel group includes a plurality of transmission channels; The control unit is used to determine the status corresponding to the first cache area on the host side and the status corresponding to the second cache area on the card side; based on the status corresponding to the first cache area on the host side and the status corresponding to the second cache area on the card side, when an interrupt condition is met, the host side sends to-be-processed data or a processing instruction or a transmission instruction to the card side, so that the card side processes the to-be-processed data or the first data in the second cache area, or sends the second data to the host side.

8. The data transmission system according to claim 7, characterized in that: The system further comprises: a status unit, The status unit is used to maintain the status of each transmission channel in the transmission channel group, wherein the status of the transmission channel includes a completion status and a failure status, the completion status indicates that the card side has completed data processing of the first data, and the failure status indicates that the card side has not completed data processing of the first data.

9. The data transmission system according to claim 8, characterized in that The system further comprises: an interruption unit, The interrupt unit is used to send the data to be processed or the transmission instruction or the processing instruction from the host side to the card side when the interrupt condition is met, Wherein, when the interrupt condition is met, sending the to-be-processed data or the transmission instruction or the processing instruction by the host side to the card side includes any one of the following: When the state corresponding to the first buffer area is the first state, sending the transmission indication to the card side based on the user mode polling PMD; When the state corresponding to the second buffer area is the second state, sending the processing instruction to the card side; When the state corresponding to the second buffer area is the first state or the state corresponding to the first buffer area is the second state, sending the data to be processed to the card side through direct memory access (DMA) based on the PMD; The first state is that the free capacity of the first cache area is greater than a first preset capacity, and the second state is that the free capacity of the second cache area is greater than a second preset capacity.

10. A data transmission device, characterized in that: include: a determination module, configured to determine a state corresponding to the first buffer area on the host side and a state corresponding to the second buffer area on the card side; A transmission module is used to send to-be-processed data or a processing instruction or a transmission instruction to the card side based on the status corresponding to the first cache area on the host side and the status corresponding to the second cache area on the card side, when an interrupt condition is met, so that the card side can process the to-be-processed data or the first data in the second cache area, or send the second data from the card side to the host side.

11. An electronic device, characterized in that: include: A processor and a memory for storing a computer program that can be run on the processor, wherein the processor performs the method according to any one of claims 1 to 6 when running the computer program.

12. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 6.

13. A program product, characterized in that The method comprises computer instructions for causing a computer to execute the interactive method according to any one of claims 1 to 6.

Images