Data transmission method and device, computer equipment and storage medium
By introducing programmable logic devices to transmit data and status information in the DPU, the processor judges and instructs retransmission, solving the problem of data retransmission bandwidth waste caused by PCIe interface exceptions, and achieving efficient bandwidth utilization.
Patent Information
- Application Number
- CN202510337287.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-20
- Publication Date
- 2025-07-04
AI Technical Summary
During data transmission, existing DPUs are data retransmission due to abnormal PCIe interface, resulting in bandwidth waste.
By introducing programmable logic devices into the DPU, data and status information are transmitted to the processor. The processor determines whether data needs to be retransmitted based on the status information, and sends retransmission instructions to the programmable logic device to achieve accurate data retransmission.
Reduces requirements for programmable logic devices, improves bandwidth utilization, and ensures that the processor obtains the correct data.
Smart Images
Figure CN120256199A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the technical field of data processing units, and particularly to a data transmission method, apparatus, computer device, and storage medium. Background Art
[0002] A DPU (Data Processing Unit) is a hardware accelerator dedicated to data processing; currently, a DPU can adopt an FPGA+CPU configuration. This architecture of the DPU combines the advantages of an FPGA (Field Programmable Gate Array) and a CPU (Central Processing Unit), and has excellent software and hardware programmability.
[0003] The DPU mainly performs data reading and writing with a server based on DMA (Direct Memory Access) technology. Based on DMA descriptors, the FPGA can directly transfer data between the memory of the server and the memory of the DPU without the participation of the CPU, and the transmission efficiency is high.
[0004] When the DPU reads data from the server memory, due to reasons such as an abnormal PCIe (Peripheral Component Interconnect express, a high-speed serial computer expansion bus standard) interface between the two, data retransmission may be required. Since the FPGA cache space is limited, to ensure the correctness of data transmission, generally the FPGA needs to retransmit all data, which seriously wastes bandwidth. Summary of the Invention
[0005] In view of this, the present disclosure provides a data transmission method, apparatus, computer device, and storage medium to solve the problem that data retransmission by a data processing unit affects bandwidth.
[0006] In a first aspect, the present disclosure provides a data transmission method, which is applied to a processor of a data processing unit, and the data processing unit further includes a programmable logic device;
[0007] The method includes:
[0008] Obtain a target data block transmitted by the programmable logic device and first status information of the target data block; the target data block includes a plurality of sub-data;
[0009] Judge whether there is sub-data that needs to be retransmitted in the target data block according to the first status information;
[0010] In the case that there is first sub-data that needs to be retransmitted in the target data block, send a retransmission instruction for at least retransmitting the first sub-data to the programmable logic device; the retransmission instruction is used to instruct the programmable logic device to reread the first sub-data and transmit it to the processor;
[0011] Update the target data block according to the intermediate data retransmitted by the programmable logic device; the intermediate data at least includes the first sub-data reread by the programmable logic device.
[0012] In a second aspect, the present disclosure provides a data transmission device, which is applied to a processor of a data processing unit, and the data processing unit further includes a programmable logic device; the device includes:
[0013] An acquisition module, configured to acquire the target data block transmitted by the programmable logic device and the first status information of the target data block; the target data block includes a plurality of sub-data;
[0014] A judgment module, configured to judge whether there is sub-data that needs to be retransmitted in the target data block according to the first status information;
[0015] A processing module, configured to, in the case that there is first sub-data that needs to be retransmitted in the target data block, send a retransmission instruction for at least retransmitting the first sub-data to the programmable logic device; the retransmission instruction is used to instruct the programmable logic device to reread the first sub-data and transmit it to the processor; update the target data block according to the intermediate data retransmitted by the programmable logic device; the intermediate data at least includes the first sub-data reread by the programmable logic device.
[0016] In a third aspect, the present disclosure provides a computer device, including: a memory and a processor, which are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to execute the data transmission method according to the first aspect or any corresponding implementation manner thereof.
[0017] In a fourth aspect, the present disclosure provides a computer-readable storage medium, on which computer instructions are stored, and the computer instructions are used to cause a computer to execute the data transmission method according to the first aspect or any corresponding implementation manner thereof.
[0018] In a fifth aspect, the present disclosure provides a computer program product, including computer instructions, and the computer instructions are used to cause a computer to execute the data transmission method according to the first aspect or any corresponding implementation manner thereof.
[0019] In the present disclosure, in addition to transmitting the read data to the processor, the programmable logic device of the data processing unit also transmits corresponding status information to the processor, enabling the processor to determine whether data retransmission is required based on the status information; when data retransmission is required, a corresponding retransmission instruction is sent to the programmable logic device to cause the programmable logic device to reread the data, ultimately effectively ensuring that the processor can obtain the correct data. The present disclosure does not require the programmable logic device to execute complex processing logics, nor does it require the programmable logic device to cache a large amount of data, greatly reducing the requirements for the programmable logic device; the processor specifies which data to retransmit, which can effectively improve the bandwidth utilization rate. BRIEF DESCRIPTION OF THE DRAWINGS
[0020] In order to more clearly illustrate the specific embodiments of the present disclosure or the technical solutions in the related art, the following will briefly introduce the drawings required for use in the description of the specific embodiments or the related art. Obviously, the drawings in the following description are some embodiments of the present disclosure. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.
[0021] Figure 1 is a schematic diagram of an architecture of a data processing unit;
[0022] Figure 2 is a schematic diagram of a process for an FPGA to read data from a server memory;
[0023] Figure 3 is a schematic diagram of a data block IO descriptor;
[0024] Figure 4 is a schematic diagram of a process of a data transmission method according to an embodiment of the present disclosure;
[0025] Figure 5 is a schematic diagram of a process of another data transmission method according to an embodiment of the present disclosure;
[0026] Figure 6 is a schematic diagram of a process of transmitting data according to an embodiment of the present disclosure;
[0027] Figure 7 is a schematic diagram when a programmable logic device continuously reads multiple data blocks according to an embodiment of the present disclosure;
[0028] Figure 8 is a block diagram of the structure of a data transmission device according to an embodiment of the present disclosure;
[0029] Figure 9 is a schematic diagram of the hardware structure of a computer device according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0030] To make the objectives, technical solutions, and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present disclosure. Apparently, the described embodiments are some, but not all, of the embodiments of the present disclosure. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present disclosure without creative efforts shall fall within the protection scope of the present disclosure.
[0031] Figure 1 shows a schematic architecture diagram of a data processing unit. As Figure 1 shown, the server includes a CPU 101 and a memory 102, and corresponding data Data is stored in the memory; the data processing unit (i.e., DPU) includes a CPU 111, a memory 112, and an FPGA 113, and the FPGA 113 is connected to the CPU 111 through a PCIe bus. Moreover, the FPGA 113 accesses the server through a PCIe interface; for example, the data processing unit can be a smart board (such as a smart network card, etc.) and is inserted on the server through a PCIe interface.
[0032] Figure 2 shows a schematic flowchart of the FPGA reading data in the server memory. As Figure 2 shown, when the FPGA 113 reads data on the server side based on the DMA method, it will initiate a request to read the memory 102 and then wait for the data to return; after receiving the data, it determines whether the returned data is normal; if data retransmission is required (such as data return failure, etc.), it will re-initiate a request to read the memory 102; if the data is normal, it will directly write the data into the memory 112 of the data processing unit, and then the CPU 111 processes the data, and finally stores the data in the corresponding storage cluster through wired or wireless means, etc.
[0033] When the FPGA 113 can read data normally, that is, when the data returns normally, the processing path is: 201, 204, 205, 206, 207.
[0034] If data retransmission is required due to reasons such as abnormal PCIe interface, etc., the processing path corresponding to the first retransmission situation is: 201, 202, 203, 201, 204, 205, 206, 207.
[0035] Similarly, the processing path corresponding to the second retransmission situation is: 201, 202, 203, 201, 202, 203, 201, 204, 205, 206, 207.
[0036] Among them, when using the DMA method, data is generally transferred in groups. The FPGA will continuously initiate multiple read requests to read data from the memory 102. Since there is a certain delay from the initiation of the read request to the return of the data, the FPGA needs to be able to cache a large amount of intermediate data. In addition, when the FPGA determines that a certain data needs to be retransmitted, at this time, it may have already or will soon correctly read the data of subsequent requests. To ensure the correctness of data transmission, generally, the FPGA needs to retransmit all data, which seriously wastes bandwidth.
[0037] Specifically, the FPGA reads data based on the input / output (IO) descriptors of data blocks. Figure 3 A schematic diagram of the data block IO descriptor is shown, as Figure 3 shown. It includes a head pointer Hdr and a tail pointer Tail. And if the data block includes N + 1 data, it also includes descriptors corresponding to the data Data0 to DataN. Each descriptor includes information such as the storage address and length of the corresponding data, so that each data in the memory 102 can be read separately based on each descriptor.
[0038] When the FPGA reads data, it will initiate read requests for the data Data0 to DataN, and can also initiate read requests for other data blocks in sequence. For example, if the data Data2 needs to be retransmitted, the FPGA needs to reread the subsequent data such as Data3 and Data4, as well as the data of other subsequent data blocks, wasting bandwidth.
[0039] In the data transmission method provided by the embodiments of the present disclosure, the programmable logic device of the data processing unit transmits the read data and the corresponding status information to the processor, so that the processor can determine whether to retransmit the data based on the status information; when it is necessary to retransmit the data, a corresponding retransmission instruction is sent to the programmable logic device, and the programmable logic device rereads the data to ensure that the processor can obtain the correct data. This method does not require the programmable logic device to execute complex processing logic, nor does it require the programmable logic device to cache a large amount of data, greatly reducing the requirements for the programmable logic device; the processor specifies which data to retransmit, which can effectively improve the bandwidth utilization rate.
[0040] According to the embodiments of the present disclosure, an embodiment of a data transmission method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.
[0041] In this embodiment, a data transmission method is provided, which is applied to a processor of a data processing unit, and the data processing unit further includes a programmable logic device; for example, the processor is a CPU, and the programmable logic device is an FPGA, a CPLD (Complex Programmable Logic Device), etc.; for example, the processor is Figure 1 the CPU 111 shown, and the programmable logic device is the FPGA 113. Figure 4 is a flowchart of the data transmission method according to an embodiment of the present disclosure, as Figure 4 shown, and the process includes the following steps.
[0042] Step S401, obtain a target data block transmitted by the programmable logic device and first status information of the target data block; the target data block includes a plurality of sub-data.
[0043] In this embodiment, when the DPU (i.e., the data processing unit) needs to transmit data, the programmable logic obtains the corresponding data block (Block) from the server, and this data block generally includes multiple groups of data. For ease of description, the currently required data block to be transmitted is referred to as the target data block, and each data in the target data block is referred to as sub-data.
[0044] Among them, the programmable logic device can obtain the corresponding target data block from the server side based on the IO descriptor of the target data block, and the form of the IO descriptor can be as Figure 3 shown. It can be understood that Figure 3 Data0 to DataN in it represent descriptors of each sub-data, which include information such as the address and length of the corresponding sub-data in the server memory, so that the programmable logic device can directly read the data in the server memory without the participation of the server's CPU.
[0045] Moreover, when the programmable logic device obtains the target data block, it can also determine the status information that can represent the transmission status of the target data block, that is, the first status information. The transmission status of the target data block can include, for example: whether the transmission is abnormal, and information such as the type of abnormality when the transmission is abnormal.
[0046] When the programmable logic device transmits the target data block, it transmits the first status information of the target data block together, so that the processor of the DPU can obtain the target data block and the first status information of the target data block.
[0047] Step S402, determine whether there are sub-data that need to be retransmitted in the target data block according to the first status information.
[0048] In this embodiment, the first status information may indicate whether there is an abnormality in the transmission of the target data block. Therefore, based on this first status information, it can be determined whether there is data with a transmission failure in the target data block that needs to be retransmitted, that is, it can be determined whether there is sub-data in the target data block that needs to be retransmitted.
[0049] Step S403, when there is first sub-data that needs to be retransmitted in the target data block, send a retransmission instruction for at least retransmitting the first sub-data to the programmable logic device; the retransmission instruction is used to instruct the programmable logic device to reread the first sub-data and transmit it to the processor.
[0050] In this embodiment, if the processor determines that there is sub-data that needs to be retransmitted in the target data block, it can send a corresponding retransmission instruction to the programmable logic device so that the programmable logic device can perform data retransmission. For the convenience of description, the sub-data that needs to be retransmitted is referred to as the first sub-data.
[0051] Specifically, the processor can determine which specific sub-data or which sub-data in the target data block need to be retransmitted, so as to formulate which sub-data the programmable logic device needs to retransmit; or, when the processor determines that there is data loss in the target data block, it can also not pay attention to which specific sub-data is lost and directly instruct the programmable logic device to retransmit all sub-data, that is, retransmit the entire target data block, and the target data block includes the first sub-data.
[0052] After receiving the retransmission instruction, the programmable logic device can reread the corresponding first sub-data from the server memory, and then send the first sub-data to the processor so that the processor can reprocess the retransmitted data.
[0053] Step S404, update the target data block according to the intermediate data retransmitted by the programmable logic device; the intermediate data at least includes the first sub-data reread by the programmable logic device.
[0054] In this embodiment, after the processor obtains the data retransmitted by the programmable logic device, it can perform data processing; among them, for the convenience of distinction and description, the data retransmitted by the programmable logic device is referred to as intermediate data, and the intermediate data includes the above-mentioned first sub-data that needs to be retransmitted. For example, the intermediate data is the reread target data block, or the intermediate data only includes the reread first sub-data.
[0055] The processor can update the target data block according to the newly obtained intermediate data, so as to obtain the updated target data block, and then perform subsequent processing based on the updated target data block.
[0056] It can be understood that in the case where there is no sub-data that needs to be retransmitted in the updated target data block, the processor can perform subsequent processing on the updated target data block. For example, the updated target data block can be stored in a storage cluster, etc. In the case where there is still sub-data that needs to be retransmitted in the updated target data block, the processor resends an instruction for retransmitting the corresponding data to the programmable logic device until there is no need to retransmit data.
[0057] As described above, the traditional method requires the FPGA to determine by itself whether data needs to be retransmitted. It not only requires the FPGA to perform processing but also requires sufficient cache space to cache the data that has been read; in this embodiment, the FPGA only needs to transmit the read data and the corresponding status information, without the need for the FPGA to perform processing and caching. Instead, the complex judgment and processing work are handed over to the processor. Utilizing the ability of the processor to handle complex tasks, data retransmission can be achieved.
[0058] In the data transmission method provided in this embodiment, in addition to transmitting the read data to the processor, the programmable logic device of the data processing unit also transmits the corresponding status information to the processor, enabling the processor to determine whether data needs to be retransmitted based on the status information; when data needs to be retransmitted, a corresponding retransmission instruction is sent to the programmable logic device to enable the programmable logic device to reread the data, ultimately effectively ensuring that the processor can obtain the correct data. This method does not require the programmable logic device to execute complex processing logic, nor does it require the programmable logic device to cache a large amount of data, greatly reducing the requirements for the programmable logic device; the processor specifies which data needs to be retransmitted, which can effectively improve the bandwidth utilization rate.
[0059] In this embodiment, a data transmission method is provided, which is applied to the processor of a data processing unit, and the data processing unit further includes a programmable logic device; for example, the processor is Figure 1 the CPU 111 shown, and the programmable logic device is the FPGA 113. And the data processing unit further includes a first memory; for example, the first memory is Figure 1 the memory 112 shown.
[0060] Figure 5 is a flowchart of the data transmission method according to an embodiment of the present disclosure. As Figure 5 shown, the process includes the following steps.
[0061] Step S501, the programmable logic device initiates a read request for the second memory of the server, reads the target data block to be processed, and determines the corresponding first status information.
[0062] In this embodiment, when the data processing unit needs to process the target data block, as Figure 1As shown in the figure, the FPGA 113 (i.e., programmable logic device) can obtain the IO descriptor of the target data block. This IO descriptor is a type of DMA descriptor, which includes descriptors of each sub-data in the target data block. Based on this IO descriptor, the FPGA 113 can initiate a request to read the memory 102 (i.e., the second memory), thereby directly reading the target data block in the server memory 102.
[0063] Moreover, the FPGA 113 can determine the status information of the target data block, i.e., the first status information. For example, the FPGA 113 can determine whether each sub-data is successfully received. Or, after the FPGA 113 initiates a data read request, in case of a data timeout exception when the data cannot be normally returned, the CPU 101 of the server can send the corresponding error status information to the FPGA 113, and the FPGA generates the first status information based on this error status information.
[0064] Step S502, the programmable logic device writes the target data block and the first status information into the first memory based on the direct memory access method.
[0065] In this embodiment, when the FPGA 113 transfers data to the CPU 111 of the DPU (i.e., the processor of the data processing unit), it also uses the DMA method, that is, the FPGA 113 directly writes the target data block and the first status information into the memory 113 of the DPU (i.e., the first memory), and this process does not require the participation of the CPU 111.
[0066] Step S503, obtain the target data block transmitted by the programmable logic device and the first status information of the target data block; the target data block includes multiple sub-data.
[0067] Specifically, the above step S503 "obtain the target data block transmitted by the programmable logic device and the first status information of the target data block" may include step S5031.
[0068] Step S5031, obtain the target data block to be processed and the first status information of the target data block from the first memory; the target data block and the first status information are read from the second memory of the server by the programmable logic device based on the direct memory access method and written into the first memory.
[0069] In this embodiment, the CPU 111 of the DPU can obtain the target data block to be processed and the first status information of the target data block from the memory 112. Among them, the CPU 111 of the DPU can wait to receive the data and status information, and after determining that the complete target data block has been received, then execute the subsequent step S504.
[0070] Step S504: Determine whether there is sub-data in the target data block that needs to be retransmitted according to the first status information.
[0071] For details, please refer to Figure 4 Step S402 of the illustrated embodiment, which will not be elaborated here.
[0072] In some alternative embodiments, the first status information includes the transmission error type of the target data block. And the above step S504, "Determine whether there is sub-data in the target data block that needs to be retransmitted according to the first status information", may include steps A1 to A2.
[0073] Step A1: Determine whether the transmission error type of the target data block belongs to the type that needs to be retransmitted.
[0074] Step A2: When it is determined that the transmission error type of the target data block belongs to the type that needs to be retransmitted, determine that there is sub-data in the target data block that needs to be retransmitted.
[0075] In this embodiment, the first status information transmitted by the FPGA 113 can include not only whether the data is transmitted incorrectly, but also the specific transmission error type, such as data timeout, data verification failure, etc. The CPU 111 can determine whether it belongs to the type that needs to be retransmitted based on the transmission error type of the target data block, and then determine whether to retransmit the data.
[0076] For example, when a data timeout exception occurs, the data can be retransmitted; if the lost data is redundant data, there is no need to retransmit the redundant data.
[0077] Step S505: When there is first sub-data in the target data block that needs to be retransmitted, send a retransmission instruction for at least retransmitting the first sub-data to the programmable logic device; the retransmission instruction is used to instruct the programmable logic device to reread the first sub-data and transmit it to the processor.
[0078] For details, please refer to Figure 4 Step S403 of the illustrated embodiment, which will not be elaborated here.
[0079] In some alternative embodiments, the programmable logic device and the processor are also connected through a PCIe bus. To avoid affecting the DMA process of the programmable logic device, a retry channel for transmitting the retransmission instruction is set up.
[0080] Specifically, the data transmitted by the programmable logic device to the processor is based on a data channel; that is, a data channel is provided between the programmable logic device and the processor, and the programmable logic device transmits data such as the target data block and the first status information to the processor based on this data channel. Specifically, the programmable logic device writes the corresponding data into the first memory based on this data channel.
[0081] Moreover, a retry channel is provided between the programmable logic device and the processor. The above step S505, "sending a retransmission instruction for at least retransmitting the first sub-data to the programmable logic device", may include: based on the retry channel, sending a retransmission instruction for at least retransmitting the first sub-data to the programmable logic device; the retry channel is different from the data channel.
[0082] In this embodiment, both the retry channel and the data channel are channels in the PCIe bus, and they are different. Generally, their transceiver directions are different. That is, the data channel is a channel from the programmable logic device to the processor, while the retry channel is a channel from the processor to the programmable logic device, ensuring that when the processor sends a retransmission instruction, it does not affect the DMA process of the programmable logic device.
[0083] Figure 6 Shows a schematic diagram of a data transmission process in this embodiment. As Figure 6 shown, when the FPGA 113 reads data on the server side based on the DMA method, it will initiate a request to read the memory 102 and then wait for the data to return; after receiving the data, it will directly write the data and the corresponding status information into the memory 112 of the data processing unit. Then, the CPU 111 of the data processing unit determines the integrity of the data to determine whether retransmission is required; when retransmission is required, it sends a retransmission instruction to the FPGA 113 through the retry channel, causing the FPGA 113 to re-initiate a request to read the memory 102 and re-transmit the re-read data, so that the CPU 111 can finally store the complete target data block in the corresponding storage cluster.
[0084] It can be understood that when the FPGA 113 can read data normally, that is, when the data returns normally, the processing path is: 601, 602, 603, 604, 607.
[0085] If retransmission is required due to reasons such as a PCIe interface exception, the processing path corresponding to a single retransmission is: 601, 602, 603, 604, 605, 606, 601, 602, 603, 604, 607.
[0086] Step S506, updating the target data block according to the intermediate data retransmitted by the programmable logic device; the intermediate data includes at least the first sub-data re-read by the programmable logic device.
[0087] In this embodiment, the intermediate data is also read from the second memory of the server by the programmable logic device based on the direct memory access method and written into the first memory.
[0088] For example, after the FPGA 113 obtains the retransmission instruction, it reads the data in the memory 102 again and rewrites the read intermediate data into the memory 112 of the data processing unit, so that the processor 111 can update the local target data block based on the intermediate data.
[0089] Optionally, when the programmable logic device retransmits the intermediate data, the status information of the intermediate data, that is, the second status information, is transmitted along the same path. Specifically, after the above step S506 "sending a retransmission instruction for at least retransmitting the first sub-data to the programmable logic device", the method further includes:
[0090] Obtaining the second status information retransmitted by the programmable logic device; the second status information is used to represent the status of the intermediate data and is used to determine whether there is sub-data that needs to be retransmitted in the intermediate data; in the case where there is no sub-data that needs to be retransmitted in the intermediate data, the step of updating the target data block according to the intermediate data retransmitted by the programmable logic device is executed, that is, step S506 is executed.
[0091] In this embodiment, after the processor obtains the intermediate data retransmitted by the programmable logic device, it continues to determine whether data needs to be retransmitted based on the second status information of the intermediate data. The determination process is similar to the determination principle of the above step S504 and will not be elaborated here.
[0092] If it is determined based on the second status information that there is still data that needs to be retransmitted in the target data block, a retransmission instruction needs to be generated again, and so on, until the processor obtains the complete target data block.
[0093] In some optional embodiments, the above step S505 "sending a retransmission instruction for at least retransmitting the first sub-data to the programmable logic device" may include step B1.
[0094] Step B1, sending a first retransmission instruction for retransmitting the target data block to the programmable logic device. Correspondingly, the intermediate data retransmitted by the programmable logic device includes the target data block reread by the programmable logic device.
[0095] In this embodiment, when there is sub-data that needs to be retransmitted in a certain target data block, the processor can directly send a first retransmission instruction for retransmitting the entire target data block to the programmable logic device, that is, retransmitting data in units of data blocks. This does not require the programmable logic device to retransmit other data blocks, and if there is no abnormal sub-data (that is, no sub-data that needs to be retransmitted) in the retransmitted target data block, the processor can directly use the retransmitted target data block as the updated target data block and process it.
[0096] When a programmable logic device transfers data based on the DMA mode, it reads data in the second memory of the server sequentially based on multiple IO descriptors. When a timeout occurs during the reading of a certain sub-data, data corresponding to subsequent other IO descriptors may have been read at this time.
[0097] Figure 7 The figure shows a schematic diagram when a programmable logic device continuously reads multiple data blocks. As Figure 7 shown, the programmable logic device reads the corresponding data blocks from the memory of the server sequentially based on the IO descriptors of each data block; among them, the second IO descriptor B includes five sub-data, and the descriptors of each sub-data are D0 to D4 respectively. In addition, the programmable logic device also sends the status information of each sub-data to the processor, Figure 7 using the status bit flag_err to represent this status information. If flag_err = 0, it means the transmission is normal. If flag_err = 1, it means the transmission is abnormal, and at this time, the data needs to be retransmitted.
[0098] Taking Figure 7 as an example, if the status information of descriptor D2 indicates that it needs to be retransmitted (flag_err = 1), due to reasons such as delay, at this time, the programmable logic device may have obtained subsequent sub-data, and even obtained the data blocks corresponding to IO descriptor C and IO descriptor D, resulting in the programmable logic device needing to reread this sub-data (the sub-data corresponding to descriptor D2) and subsequent data.
[0099] In this embodiment, if the processor determines based on this status information that the sub-data corresponding to descriptor D2 needs to be retransmitted, it can generate a first retransmission instruction for only retransmitting the data block corresponding to IO descriptor B (that is, at this time, there are sub-data that need to be retransmitted in the data block corresponding to IO descriptor B). For example, the first retransmission instruction includes the entire IO descriptor B; the programmable logic device does not need to retransmit the data blocks corresponding to subsequent IO descriptors C, IO descriptor D, etc., thereby saving bandwidth. And, when retransmitting data in data blocks, the processor does not need to perform data recombination, that is, the processor can directly process the target data block after retransmission.
[0100] In some alternative embodiments, the first status information is used to represent the status of each sub-data in the target data block. The above step S505 "sending a retransmission instruction for at least retransmitting the first sub-data to the programmable logic device" may include step C1.
[0101] Step C1, the programmable logic device sends a second retransmission instruction for only retransmitting the first sub-data.
[0102] Moreover, step S506 "updating the target data block according to the intermediate data retransmitted by the programmable logic device" described above includes: obtaining the first sub-data retransmitted by the programmable logic device; reorganizing the target data block according to the retransmitted first sub-data to obtain the updated target data block.
[0103] In this embodiment, the processor can determine whether each sub-data needs to be retransmitted according to the first status information. Thus, when there is sub-data that needs to be retransmitted in the target data block, the first sub-data that needs to be retransmitted can be specifically determined, and the programmable logic device is instructed to only reread and transmit the first sub-data, without retransmitting the entire target data block.
[0104] After the processor obtains the retransmitted first sub-data, the retransmitted first sub-data is added to the previous target data block to replace the missing or incorrect data in the target data block, so as to reorganize and obtain a target data block without sub-data that needs to be retransmitted, realizing the update of the target data block.
[0105] Still taking Figure 7 as an example, the processor can send a second retransmission instruction to the programmable logic device. The second retransmission instruction only includes the descriptor D2, to instruct the programmable logic device to only reread the sub-data corresponding to the descriptor D2, which can greatly save the bandwidth between the DPU and the server and improve the bandwidth utilization rate. After the processor obtains the retransmitted first sub-data (the sub-data corresponding to the descriptor D2), combined with the sub-data that has been obtained before, a new target data block is reorganized, that is, the data block corresponding to the IO descriptor B, and then subsequent processing is performed on the target data block. At this time, the data already transmitted by the programmable logic device is stored in the first memory of the data processing unit, and the processor can read it at any time.
[0106] It can be understood that the processor can judge what kind of retransmission instruction to generate based on its own situation. For example, when the utilization rate of the processor itself is relatively low, data reorganization does not affect its own performance, then a second retransmission instruction can be generated to ensure the bandwidth utilization rate; if the utilization rate of the processor is relatively high, a first retransmission instruction can be generated to reduce the impact of the processor's processing performance due to the need for data reorganization.
[0107] In the data transmission method provided in this embodiment, the programmable logic device and the processor of the data processing unit cooperate. The programmable logic device only transmits data and status information, and the processor decides whether retransmission is needed, which greatly reduces the requirements for the programmable logic device and can effectively improve the bandwidth utilization rate. The programmable logic device directly writes the data into the memory of the data processing unit, which can ensure the efficiency of data reading and writing; the processor can instruct the programmable logic device to retransmit the entire target data block or only the abnormal sub-data therein, which can balance the bandwidth utilization rate and the processing performance of the processor.
[0108] In this embodiment, a data transmission device is further provided. This device is used to implement the above embodiments and preferred implementation manners, and those that have been described will not be elaborated again. As used hereinafter, the term "module" may be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, implementation in hardware, or a combination of software and hardware is also possible and contemplated.
[0109] This embodiment provides a data transmission device, which is applied to a processor of a data processing unit, and the data processing unit further includes a programmable logic device; as Figure 8 shown, this device includes:
[0110] An acquisition module 801, configured to acquire a target data block transmitted by the programmable logic device and first status information of the target data block; the target data block includes a plurality of sub-data;
[0111] A judgment module 802, configured to judge whether there is sub-data that needs to be retransmitted in the target data block according to the first status information;
[0112] A processing module 803, configured to, when there is first sub-data that needs to be retransmitted in the target data block, send a retransmission instruction for at least retransmitting the first sub-data to the programmable logic device; the retransmission instruction is used to instruct the programmable logic device to reread the first sub-data and transmit it to the processor; update the target data block according to intermediate data retransmitted by the programmable logic device; the intermediate data at least includes the first sub-data reread by the programmable logic device.
[0113] In some optional implementation manners, the data processing unit further includes a first memory;
[0114] The acquisition module 801 acquiring the target data block transmitted by the programmable logic device and the first status information of the target data block includes:
[0115] Acquiring the target data block to be processed and the first status information of the target data block from the first memory; the target data block and the first status information are read by the programmable logic device from a second memory of a server based on direct memory access and written into the first memory;
[0116] Moreover, the intermediate data is also reread by the programmable logic device from the second memory of the server based on direct memory access and written into the first memory.
[0117] In some alternative embodiments, the processing module 803 sends a retransmission instruction for at least retransmitting the first sub-data to the programmable logic device, including:
[0118] Sending a first retransmission instruction for retransmitting the target data block to the programmable logic device;
[0119] The intermediate data retransmitted by the programmable logic device includes the target data block reread by the programmable logic device.
[0120] In some alternative embodiments, the processing module 803 sends a retransmission instruction for at least retransmitting the first sub-data to the programmable logic device, including:
[0121] Sending a second retransmission instruction for only retransmitting the first sub-data to the programmable logic device;
[0122] The processing module 803 updates the target data block according to the intermediate data retransmitted by the programmable logic device, including:
[0123] Obtaining the first sub-data retransmitted by the programmable logic device; reorganizing the target data block according to the retransmitted first sub-data to obtain an updated target data block.
[0124] In some alternative embodiments, after the processing module 803 sends a retransmission instruction for at least retransmitting the first sub-data to the programmable logic device, it is further configured to:
[0125] Obtaining second status information of the programmable logic device retransmission; the second status information is used to represent the status of the intermediate data and is used to determine whether there is sub-data that needs to be retransmitted in the intermediate data;
[0126] In the case where there is no sub-data that needs to be retransmitted in the intermediate data, execute the step of updating the target data block according to the intermediate data retransmitted by the programmable logic device.
[0127] In some alternative embodiments, the first status information includes the transmission error type of the target data block;
[0128] The determination module 802 determines whether there is sub-data that needs to be retransmitted in the target data block according to the first status information, including:
[0129] Determining whether the transmission error type of the target data block belongs to a type that needs to be retransmitted;
[0130] In the case where it is determined that the transmission error type of the target data block belongs to a type that needs to be retransmitted, it is determined that there is sub-data that needs to be retransmitted in the target data block.
[0131] In some alternative embodiments, the data transmitted by the programmable logic device to the processor is transmitted based on a data channel;
[0132] The processing module 803 sends a retransmission instruction to the programmable logic device for at least retransmitting the first sub-data, including:
[0133] Based on a retry channel, sending a retransmission instruction to the programmable logic device for at least retransmitting the first sub-data; the retry channel is different from the data channel.
[0134] The further functional descriptions of the above-mentioned various modules and units are the same as those in the corresponding foregoing embodiments, and will not be elaborated herein.
[0135] The data transmission device in this embodiment is presented in the form of functional units. Here, the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, including a processor and a memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.
[0136] Embodiments of the present disclosure further provide a computer device having the above-mentioned Figure 8 shown data transmission device.
[0137] Please refer to Figure 9 , Figure 9 which is a schematic structural diagram of a computer device provided by an alternative embodiment of the present disclosure. As Figure 9 shown, the computer device includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including a high-speed interface and a low-speed interface. Each component communicates with each other using different buses and can be installed on a common motherboard or installed in other ways as needed. The processor can process instructions executed within the computer device, including instructions stored in the memory or on the memory for displaying graphical information of a GUI on an external input / output device (such as a display device coupled to the interface). In some alternative embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories. Similarly, multiple computer devices can be connected, and each device provides some necessary operations (for example, as a server array, a set of blade servers, or a multi-processor system). Figure 9 In
[0138] The processor 10 may be a central processing unit, a network processor, or a combination thereof. Among them, the processor 10 may further include a hardware chip. The above-mentioned hardware chip may be an application-specific integrated circuit, a programmable logic device, or a combination thereof. The above-mentioned programmable logic device may be a complex programmable logic device, a field-programmable gate array, a generic array logic, or any combination thereof.
[0139] Among them, the memory 20 stores instructions executable by at least one processor 10, so that the at least one processor 10 executes the method shown in the above embodiments.
[0140] The memory 20 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system and application programs required for at least one function; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include a high-speed random access memory, and may further include a non-transitory memory, such as at least one disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some alternative embodiments, the memory 20 may optionally include a memory remotely disposed relative to the processor 10, and these remote memories may be connected to the computer device through a network. Examples of the above-mentioned network include but are not limited to the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.
[0141] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk, or a solid-state drive; the memory 20 may further include a combination of the above types of memories.
[0142] The computer device further includes a communication interface 30 for the computer device to communicate with other devices or communication networks.
[0143] The embodiments of the present disclosure also provide a computer-readable storage medium. The methods according to the embodiments of the present disclosure may be implemented in hardware, firmware, or be implemented as computer code that can be recorded on a storage medium, or be implemented as computer code originally stored in a remote storage medium or a non-temporary machine-readable storage medium and downloaded through a network and will be stored in a local storage medium, so that the methods described herein can be stored in such software processes on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium may be a magnetic disk, an optical disk, a read-only memory, a random access memory, a flash memory, a hard disk, or a solid-state drive, etc.; further, the storage medium may further include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code, and when the software or computer code is accessed and executed by the computer, the processor, or the hardware, the methods shown in the above embodiments are implemented.
[0144] Part of the present disclosure can be applied as a computer program product, such as computer program instructions. When executed by a computer, through the operation of the computer, the methods and / or technical solutions according to the present disclosure can be invoked or provided. Those skilled in the art should understand that the forms of existence of computer program instructions in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways for computer program instructions to be executed by a computer include, but are not limited to: the computer directly executes the instructions, or the computer compiles the instructions and then executes the corresponding compiled program, or the computer reads and executes the instructions, or the computer reads and installs the instructions and then executes the corresponding installed program. Herein, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to the computer.
[0145] Although the embodiments of the present disclosure are described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the present disclosure, and such modifications and variations should all be covered within the protection scope of the present disclosure.
Claims
1. A data transmission method, characterized in that, A processor applied to a data processing unit, and the data processing unit further includes a programmable logic device; The method includes: Obtaining a target data block transmitted by the programmable logic device and first status information of the target data block; the target data block includes a plurality of sub-data; Judging whether there is sub-data that needs to be retransmitted in the target data block according to the first status information; When there is first sub-data that needs to be retransmitted in the target data block, sending a retransmission instruction for at least retransmitting the first sub-data to the programmable logic device; the retransmission instruction is used to instruct the programmable logic device to reread the first sub-data and transmit it to the processor; Updating the target data block according to the intermediate data retransmitted by the programmable logic device; the intermediate data at least includes the first sub-data reread by the programmable logic device.
2. The method according to claim 1, characterized in that, The data processing unit further includes a first memory; The obtaining the target data block transmitted by the programmable logic device and the first status information of the target data block includes: Obtaining the target data block to be processed and the first status information of the target data block from the first memory; the target data block and the first status information are read by the programmable logic device from a second memory of the server based on direct memory access and written into the first memory; And, the intermediate data is also reread by the programmable logic device from the second memory of the server based on direct memory access and written into the first memory.
3. The method according to claim 1, wherein The sending the retransmission instruction for at least retransmitting the first sub-data to the programmable logic device includes: Sending a first retransmission instruction for retransmitting the target data block to the programmable logic device; The intermediate data retransmitted by the programmable logic device includes the target data block reread by the programmable logic device.
4. The method according to claim 1, characterized in that, The sending the retransmission instruction for at least retransmitting the first sub-data to the programmable logic device includes: Sending a second retransmission instruction for only retransmitting the first sub-data to the programmable logic device; The updating the target data block according to the intermediate data retransmitted by the programmable logic device includes: Obtaining the first sub-data retransmitted by the programmable logic device; reorganizing the target data block according to the retransmitted first sub-data to obtain an updated target data block.
5. The method according to any one of claims 1 to 4, characterized in that, After the sending the retransmission instruction for at least retransmitting the first sub-data to the programmable logic device, the method further includes: Obtaining second status information retransmitted by the programmable logic device; the second status information is used to represent the status of the intermediate data and is used to judge whether there is sub-data that needs to be retransmitted in the intermediate data; When there is no sub-data that needs to be retransmitted in the intermediate data, performing the step of updating the target data block according to the intermediate data retransmitted by the programmable logic device.
6. The method according to claim 1, wherein The first status information includes the transmission error type of the target data block; The judging whether there is sub-data that needs to be retransmitted in the target data block according to the first status information includes: Determine whether the transmission error type of the target data block belongs to the type that requires retransmission; In the case where it is determined that the transmission error type of the target data block belongs to the type that requires retransmission, determine that there is sub-data in the target data block that needs to be retransmitted.
7. The method according to claim 1, characterized in that The data transmitted by the programmable logic device to the processor is based on a data channel; Sending the retransmission instruction for at least retransmitting the first sub-data to the programmable logic device includes: Based on a retry channel, send a retransmission instruction for at least retransmitting the first sub-data to the programmable logic device; the retry channel is different from the data channel.
8. A data transmission device, characterized in that, Applied to a processor of a data processing unit, and the data processing unit further includes a programmable logic device; the device includes: An acquisition module, configured to acquire a target data block transmitted by the programmable logic device and first status information of the target data block; the target data block includes a plurality of sub-data; A judgment module, configured to judge whether there is sub-data in the target data block that needs to be retransmitted according to the first status information; A processing module, configured to, in the case where there is a first sub-data in the target data block that needs to be retransmitted, send a retransmission instruction for at least retransmitting the first sub-data to the programmable logic device; the retransmission instruction is used to instruct the programmable logic device to reread the first sub-data and transmit it to the processor; update the target data block according to the intermediate data retransmitted by the programmable logic device; the intermediate data at least includes the first sub-data reread by the programmable logic device.
9. A computer device, characterized in that, Includes: A memory and a processor, the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to execute the data transmission method according to any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that, Computer instructions are stored on the computer-readable storage medium, and the computer instructions are used to cause a computer to execute the data transmission method according to any one of claims 1 to 7.