Data transmission method, device, medium and computer program product

Abstract

The embodiment of the present invention provides a data transmission method, device, medium and computer program product, including: creating a data transmission request object when a data transmission request is detected; determining the number of virtual queues to be allocated based on the data transmission request object; determining the target virtual queue based on the number of virtual queues to be allocated, and the data transmission volume corresponding to each target virtual queue; configuring the data transmission information of each target virtual queue based on the data transmission volume, and transmitting the data transmission information in parallel through the target virtual queue. The embodiment of the present invention transmits data in parallel through multiple Virtqueue queues, effectively improving the data transmission bandwidth between the host and the hardware accelerator, and making the application acceleration effect obvious.

Description

Technical Field

[0001] The present invention relates to the technical field of data transmission, and in particular to a data transmission method, device, medium and computer program product. Background Art

[0002] In cloud data centers, hardware accelerators are usually configured to provide additional computing power to accelerate the execution of specific applications, such as encryption and decryption acceleration cards, video codec acceleration cards, etc. Currently, the interfaces for these hardware accelerators to communicate with the host are defined by the accelerator hardware manufacturers themselves, and they need to install their own matching device drivers and other software before they can be used. Therefore, every time a new hardware accelerator is introduced into a cloud data center, the operation and maintenance personnel need to install and maintain a different device driver, which creates a certain burden on the operation and maintenance of the data center.

[0003] In the related art, in order to solve the above problems, the network device virtio-net performs data transmission by setting multiple single-queue service modes. When data transmission is required, even if there are multiple idle Virtqueue queues, only one Virtqueue queue is selected for transmission, and the remaining idle Virtqueue queues are used to serve subsequent data transmission requests, resulting in low data transmission bandwidth, which is not suitable for hardware accelerator device types that are sensitive to data transmission bandwidth. Summary of the invention

[0004] The purpose of the embodiments of the present invention is to provide a data transmission method, device, medium and computer program product. The specific technical solutions are as follows:

[0005] In a first aspect of the present invention, a data transmission method is provided, the method comprising:

[0006] In case of detecting receiving a data transfer request, creating a data transfer request object;

[0007] Determining the number of virtual queues to be allocated based on the data transmission request object;

[0008] Determine the target virtual queue based on the number of virtual queues to be allocated, and the data transmission volume corresponding to each of the target virtual queues;

[0009] The data transmission information of each of the target virtual queues is configured based on the data transmission amount, and the data transmission information is transmitted in parallel through the target virtual queues.

[0010] Optionally, determining the number of virtual queues to be allocated based on the data transmission request object includes:

[0011] Get the number of descriptors supported by each device virtual queue in the virtual queue list of the target device;

[0012] Determine a minimum number of descriptors based on the number of descriptors supported by all of the device virtual queues;

[0013] The number of virtual queues to be allocated is generated based on the data transmission length corresponding to the data transmission request object, the minimum number of descriptors, and the preset page size.

[0014] Optionally, the determining of the target virtual queue based on the number of virtual queues to be allocated, and the data transmission volume corresponding to each of the target virtual queues include:

[0015] Traversing the virtual queue linked list of the target device based on the number of virtual queues to be allocated and a preset traversal rule, determining a target virtual queue, and recording a queue number corresponding to the target virtual queue based on a preset bitmap data structure, and marking a state corresponding to the target virtual queue as a non-idle state;

[0016] The preset traversal rule includes that the target virtual queue is in a non-abnormal state and an idle state, the non-abnormal state is determined based on the current actual transmission state of the target virtual queue, and the idle state is pre-configured based on the target device;

[0017] The data transmission volume corresponding to each target virtual queue is determined based on the number corresponding to the target virtual queue.

[0018] Optionally, determining the data transmission volume corresponding to each target virtual queue based on the number corresponding to the target virtual queue includes:

[0019] Obtaining an average data transmission volume corresponding to each of the target virtual queues based on the number of target virtual queues and the data transmission length;

[0020] If the data transmission length exceeds the sum of the average data transmission amount and the number of virtual queues to be allocated, it is determined that the data transmission amount is exceeded;

[0021] The excess data transmission amount is allocated to a first target virtual queue among the target virtual queues, wherein the first target virtual queue has the highest priority in the activation sequence corresponding to all the target virtual queues.

[0022] Optionally, traversing the virtual queue linked list of the target device based on the number of virtual queues to be allocated and a preset traversal rule to determine the target virtual queue includes:

[0023] If the number of target virtual queues satisfying the preset traversal rules obtained by traversing the virtual queue linked list of the target device is zero, the data transmission status corresponding to the data transmission request object is set to a data transmission failure status, a data transmission request end notification is sent to the target device, and the data transmission request is stopped.

[0024] Optionally, after the step of transmitting the data transmission information in parallel through the target virtual queue, the method includes:

[0025] When it is detected that any one of the target virtual queues is in an abnormal state, setting the data transmission state corresponding to the data transmission request object to a data transmission failure state, and setting the bit position corresponding to the current target virtual queue to a first identification value;

[0026] Controlling the remaining target virtual queues to stop transmission operations based on the data transmission failure state and the first identification value;

[0027] When it is detected that the bit position corresponding to any one of the target virtual queues remaining in the preset bitmap data structure variable is the first identification value and the preset bitmap data structure variable is the preset value, the data transmission request object is in a data transmission failure state.

[0028] Optionally, the transmitting the transmission failure status to all the target virtual queues comprises:

[0029] The data transmission status of the data transmission request object is a transmission failure status;

[0030] The transmission failure status is transmitted to all the target virtual queues.

[0031] Optionally, after the step of setting the data transmission state corresponding to the data transmission request object to a data transmission failure state when any one of the target virtual queues among all the target virtual queues is detected to be in an abnormal state, and setting the bit position corresponding to the current target virtual queue to a first identification value, the method includes:

[0032] When all the target virtual queues receive the transmission failure status, a data transmission request end notification is sent to the target device.

[0033] Optionally, after the step of transmitting the data transmission information in parallel through the target virtual queue, the method includes:

[0034] When it is detected that any one of the target virtual queues completes the transmission of the data transmission information, and the preset bitmap data structure variable is set to a preset value when the bit position corresponding to the current target virtual queue in the preset bitmap data structure variable is set to a first identification value, the data transmission request object is in a successful data transmission state.

[0035] Optionally, the data transmission request object includes a preset bitmap data structure variable, and the preset bitmap data structure variable is used to record the number of each device virtual queue in the virtual queue linked list corresponding to the target device; and the parallel transmission of the data transmission information through the target virtual queue includes:

[0036] Determining whether the data transmission request object is in an abnormal state;

[0037] If not, determining whether the data transmission state corresponding to the target virtual queue is in an abnormal state;

[0038] If not, determining whether the data transmission information corresponding to the target virtual queue has been transmitted;

[0039] If not, a descriptor of the target virtual queue is configured, and the data transmission information is transmitted in parallel through the target virtual queue.

[0040] Optionally, after the step of determining whether the data transmission request object is in an abnormal state, the method includes:

[0041] If so, the data transmission termination operation is performed;

[0042] The data transmission termination operation includes setting the target virtual queue to an idle state, and / or deleting the number corresponding to the target virtual queue in the preset bitmap data structure variable.

[0043] Optionally, each of the target virtual queues corresponds to a descriptor chain, the descriptor chain includes a head descriptor and an intermediate descriptor, the head descriptor in the descriptor chain is used to point to first data transmission information of the target virtual queue, the first data transmission information includes a data transmission operation, a data transmission address, a data transmission length and a data transmission status, the intermediate descriptor is used to point to second data transmission information of the target virtual queue, the second data transmission information includes a data transmission address and a data transmission length; the descriptor configuring the target virtual queue includes:

[0044] determining the number of target descriptors for each of the target virtual queues;

[0045] The first data transmission information corresponding to the header descriptor and the second data transmission information corresponding to the intermediate descriptor are configured.

[0046] Optionally, after the step of creating a data transmission request object when receiving a data transmission request is detected, the method includes:

[0047] Create a work queue object, where the work queue object is used to schedule data transmission information corresponding to each target virtual queue to different CPUs.

[0048] Optionally, configuring the data transmission information of each target virtual queue based on the data transmission amount, and transmitting the data transmission information in parallel through the target virtual queue includes:

[0049] The data transmission information of each target virtual queue is configured based on the data transmission amount, the target virtual queue is added to the work queue object, and the data transmission information is transmitted through the work queue object.

[0050] Optionally, before the step of creating a data transmission request object when receiving a data transmission request is detected, the method includes:

[0051] Get the initial number of virtual queues supported by the hardware accelerator and the number of descriptors corresponding to the virtual queues.

[0052] In a second aspect of the present invention, a data transmission device is provided, the device comprising:

[0053] A creation module, used for creating a data transmission request object when a data transmission request is detected;

[0054] A first determining module, configured to determine the number of virtual queues to be allocated based on the data transmission request object;

[0055] A second determining module, configured to determine a target virtual queue based on the number of virtual queues to be allocated, and a data transmission volume corresponding to each of the target virtual queues;

[0056] A transmission module is used to configure data transmission information of each of the target virtual queues based on the data transmission amount, and transmit the data transmission information in parallel through the target virtual queues.

[0057] In a third aspect of the present invention, there is also provided an electronic device, comprising: a transceiver, a memory, a processor, and a program stored in the memory and executable on the processor;

[0058] The processor is used to read the program in the memory to implement the data transmission method as described in any one of the first aspects.

[0059] In a fourth aspect of the implementation of the present invention, there is further provided a computer-readable storage medium, wherein instructions are stored in the computer-readable storage medium, and when the computer-readable storage medium is executed, the computer implements the data transmission method as described in any one of the first aspects.

[0060] In a fourth aspect of the implementation of the present invention, a computer program product is also provided, including a computer program / instruction, which, when executed by a processor, implements any data transmission method as described in the first aspect.

[0061] The data transmission method provided by the embodiment of the present invention comprises: creating a data transmission request object when a data transmission request is detected; determining the number of virtual queues to be allocated based on the data transmission request object; determining the target virtual queue based on the number of virtual queues to be allocated, and the data transmission volume corresponding to each target virtual queue; configuring the data transmission information of each target virtual queue based on the data transmission volume, and transmitting the data transmission information in parallel through the target virtual queue. The embodiment of the present invention can determine the required number of virtual queues to be allocated by detecting the received data transmission request and creating the corresponding request object, and then select the appropriate target virtual queue. This process not only ensures the accurate allocation of the data transmission volume, but also realizes the parallel transmission of multiple virtual queues by configuring special data transmission information for each target virtual queue. This efficient data transmission mechanism significantly improves the data transmission bandwidth between the host and the hardware accelerator, thereby greatly enhancing the acceleration effect of the application and ensuring the efficiency of data processing and overall performance optimization. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below.

[0063] Figure 1 The steps of the data transmission method provided by the embodiment of the present invention are as follows: Figure 1 ;

[0064] Figure 2 The steps of the data transmission method provided by the embodiment of the present invention are as follows: Figure 2 ;

[0065] Figure 3 The steps of the data transmission method provided by the embodiment of the present invention are as follows: Figure 3 ;

[0066] Figure 4is a device block diagram of a data transmission device provided by an embodiment of the present invention;

[0067] Figure 5 It is a schematic diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0068] To make the purpose, technical scheme and advantages of the embodiments of the present invention clearer, the embodiments of the present invention will be described in detail below in conjunction with the accompanying drawings. However, it will be appreciated by those skilled in the art that in the embodiments of the present invention, many technical details are proposed in order to enable the reader to better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical scheme claimed in the present application can be implemented. The division of the following embodiments is for the convenience of description, and should not constitute any limitation on the specific implementation of the present invention. The various embodiments can be combined and referenced with each other without contradiction.

[0069] Reference Figure 1 , showing the steps of the data transmission method provided by the embodiment of the present invention Figure 1 , the method may include:

[0070] Step 101, when a data transmission request is detected, a data transmission request object is created;

[0071] It should be noted that in the embodiment of the present application, the VirtIO specification defines a data transmission mechanism between the host and the device, called a virtual queue Virtqueue. The virtual queue consists of three parts. The first part is the descriptor array descriptor, which is used to describe the location and length of the data to be transmitted in the host memory. The second part is the available descriptor array available, which stores the subscripts of the available descriptors in the descriptor array, and is updated by the host side to inform the device side which descriptors are available; the third part is the used descriptor array used, which stores the subscripts of the used descriptors in the descriptor array, and is updated by the device side to inform the host side which descriptors have been used by the device and can be recycled and reused by the host side.

[0072] The data transfer request sent by the user usually contains only basic information, such as whether the request is read or write, host address, device address and length, etc. In order to support multiple Virtqueue queues to serve a single data transfer request, some preparation is required. Therefore, when receiving a data transfer request, a data transfer request object is created.

[0073] Specifically, a data transmission request object is created upon receiving, for example, the data transmission request object is called data_work, which contains the above basic information and the data transmission status to indicate whether the data transmission is successful or failed; as well as the number of Virtqueues responsible for the data transmission and the corresponding Virtqueue number recorded by the bitmap data structure variable queue_mask, for example, if the Virtqueue queue numbered 2 is used, the second bit of the bitmap is set to 1. The reason for using a bitmap data structure instead of an array to record the Virtqueue number is that it is easy to operate and takes up less resources; as well as a notification object used to send a notification to the user when the data transmission is completed.

[0074] Step 102, determining the number of virtual queues to be allocated based on the data transmission request object;

[0075] Further, the determining the number of virtual queues to be allocated based on the data transmission request object includes:

[0076] Get the number of descriptors supported by each device virtual queue in the virtual queue list of the target device;

[0077] Determine a minimum number of descriptors based on the number of descriptors supported by all of the device virtual queues;

[0078] The number of virtual queues to be allocated is generated based on the data transmission length corresponding to the data transmission request object, the minimum number of descriptors, and the preset page size.

[0079] It should be noted that, to determine the required number of virtual queues to be allocated, N, in the embodiment of the present application, a strategy of one descriptor pointing to a page size space is adopted to perform data transmission.

[0080] Specifically, based on the data transmission length L and the minimum value Qmin of the number of descriptors supported by all virtual queues, the number of virtual queues to be allocated required for this data transmission is determined, which can be expressed by the formula:

[0081] N = ((L / PageSize) / Qmin) + 1 (Formula 1)

[0082] Among them, PageSize is the page size, usually 4096 bytes.

[0083] Step 103, determining a target virtual queue based on the number of virtual queues to be allocated, and a data transmission volume corresponding to each of the target virtual queues;

[0084] Further, the determining of the target virtual queue based on the number of virtual queues to be allocated, and the data transmission volume corresponding to each of the target virtual queues include:

[0085] Traversing the virtual queue linked list of the target device based on the number of virtual queues to be allocated and a preset traversal rule, determining a target virtual queue, and recording a queue number corresponding to the target virtual queue based on a preset bitmap data structure, and marking a state corresponding to the target virtual queue as a non-idle state;

[0086] The preset traversal rule includes that the target virtual queue is in a non-abnormal state and an idle state, the non-abnormal state is determined based on the current actual transmission state of the target virtual queue, and the idle state is pre-configured based on the target device;

[0087] The data transmission volume corresponding to each target virtual queue is determined based on the number corresponding to the target virtual queue.

[0088] Further, determining the data transmission volume corresponding to each target virtual queue based on the number corresponding to the target virtual queue includes:

[0089] Obtaining an average data transmission volume corresponding to each of the target virtual queues based on the number of target virtual queues and the data transmission length;

[0090] If the data transmission length exceeds the sum of the average data transmission amount and the number of virtual queues to be allocated, it is determined that the data transmission amount is exceeded;

[0091] The excess data transmission amount is allocated to a first target virtual queue among the target virtual queues, wherein the first target virtual queue has the highest priority in the activation sequence corresponding to all the target virtual queues.

[0092] Further, traversing the virtual queue linked list of the target device based on the number of virtual queues to be allocated and a preset traversal rule to determine the target virtual queue includes:

[0093] If the number of target virtual queues satisfying the preset traversal rules obtained by traversing the virtual queue linked list of the target device is zero, the data transmission status corresponding to the data transmission request object is set to a data transmission failure status, a data transmission request end notification is sent to the target device, and the data transmission request is stopped.

[0094] It should be noted that after determining the number of virtual queues to be allocated, the number M of Virtqueue queues that are currently idle and available and whose data transmission status is normal can be determined. Specifically, the current queue linked list of the device is traversed, and the Virtqueue queue whose queue status is idle and whose data transmission status is normal is selected. The queue number is recorded through the bitmap data structure queue_mask mentioned above, and the status of this idle queue is marked as busy. The traversal ends after the required N idle queues are found or the linked list is traversed.

[0095] If the number of queues M found at last is 0, it indicates that the current device has no available Virtqueue queue, and the state of the data transmission request object data_work is set to busy, and the user is informed through the notification object to end the data transmission request.

[0096] It should be noted that, in the embodiment of the present application, if the number of queues found at last is not 0, the data transmission volume of each of the M queues is determined and converted into the number of pages:

[0097] First, calculate the average data transmission volume of a single queue:

[0098] unit_pages = ((L / PageSize) / M) (Formula 2)

[0099] The value obtained in formula 2 may not be divisible, so it is necessary to calculate the remaining data transmission amount:

[0100] left_pages = (L / PageSize) – (unit_pages * M) (Formula 3)

[0101] The data transmission volume of the first queue among the M queues is unit_pages+left_pages, and the data transmission volume of other queues is left_pages.

[0102] It should be noted that, in the embodiment of the present application, the reason why the first queue is selected to bear a larger amount of data transmission is because the first queue is executed first.

[0103] Step 104: configure data transmission information of each of the target virtual queues based on the data transmission volume, and transmit the data transmission information in parallel through the target virtual queues.

[0104] It should be noted that based on the data transmission volume determined in step 105, the data transmission information of each queue is configured in sequence, including the host-side address, the device-side address, the data transmission volume and the corresponding data transmission request object data_work, and then the configured Virtqueue queue is placed in the work queue data_workqueue to start data transmission.

[0105] It should be noted that the parallel transmission in the embodiment of the present application means that multiple virtual queues can perform data transmission in the same time period, but the start time of the data transmission is not exactly the same, and there may be a sequence.

[0106] The data transmission method provided by the embodiment of the present invention comprises: creating a data transmission request object when a data transmission request is detected; determining the number of virtual queues to be allocated based on the data transmission request object; determining the target virtual queue based on the number of virtual queues to be allocated; determining the data transmission volume corresponding to the target virtual queue; configuring the data transmission information of each target virtual queue based on the data transmission volume, and transmitting the data transmission information in parallel through the target virtual queue. The embodiment of the present invention can determine the required number of virtual queues to be allocated by detecting the received data transmission request and creating the corresponding request object, and then select the appropriate target virtual queue. This process not only ensures the accurate allocation of the data transmission volume, but also realizes the parallel transmission of multiple virtual queues by configuring special data transmission information for each target virtual queue. This efficient data transmission mechanism significantly improves the data transmission bandwidth between the host and the hardware accelerator, thereby greatly enhancing the acceleration effect of the application and ensuring the efficiency of data processing and overall performance optimization.

[0107] In addition, during the Virtqueue interrupt processing phase, when a Virtqueue transmission interrupt is received, the data_header content of the object pointed to by the first descriptor of the Virtqueue queue is read, and the data transmission completion status in the data_header is checked to determine success or failure. If it fails, the data transmission status of the current queue is set to abnormal, and then the Virtqueue queue data transmission operation is started.

[0108] Reference Figure 2 , showing the steps of the data transmission method provided by the embodiment of the present invention Figure 2 , the method may include:

[0109] Step 201, when a data transmission request is detected, a data transmission request object is created;

[0110] Step 202, determining the number of virtual queues to be allocated based on the data transmission request object;

[0111] Step 203, determining the target virtual queues based on the number of virtual queues to be allocated, and the data transmission volume corresponding to each of the target virtual queues;

[0112] Step 204, configuring data transmission information of each of the target virtual queues based on the data transmission volume, and transmitting the data transmission information in parallel through the target virtual queues;

[0113] Step 205, when it is detected that any one of the target virtual queues is in an abnormal state, setting the data transmission state corresponding to the data transmission request object to a data transmission failure state, and setting the bit position corresponding to the current target virtual queue to a first identification value;

[0114] Step 206, controlling the remaining target virtual queues to stop transmission operations based on the data transmission failure status and the first identification value;

[0115] Step 207, when it is detected that the preset bitmap data structure variable is a preset value when the bit position corresponding to any one of the target virtual queues remaining in the preset bitmap data structure variable is a first identification value, the data transmission request object is in a data transmission failure state.

[0116] It should be noted that, in an embodiment of the present application, when the bit position corresponding to any one of the remaining target virtual queues in the preset bitmap data structure variable is a first identification value, the preset bitmap data structure variable is a preset value, and the first identification value can be 0. Therefore, when it is detected that the bit corresponding to any one of the remaining target virtual queues in the preset bitmap data structure variable is 0, and the sum of the bits corresponding to all target virtual queues is 0, it can be determined that the data transmission request object is in a data transmission failure state.

[0117] Further, after the step of setting the data transmission state corresponding to the data transmission request object to a data transmission failure state and setting the bit position corresponding to the current target virtual queue to a first identification value when any one of the target virtual queues among all the target virtual queues is detected to be in an abnormal state, the method includes:

[0118] When all the target virtual queues receive the transmission failure status, a data transmission request end notification is sent to the target device.

[0119] It should be noted that in the embodiment of the present application, the data transmission information of each queue is configured in sequence, including the host-side address, device-side address, data transmission volume and the corresponding data transmission request object data_work, and then the configured Virtqueue queue is placed in the work queue data_workqueue to start data transmission.

[0120] By detecting the received data transmission request and creating the corresponding request object, the embodiment of the present invention can determine the required number of virtual queues to be allocated, and then select the appropriate target virtual queue. This process not only ensures the accurate allocation of data transmission volume, but also realizes parallel transmission of multiple virtual queues by configuring special data transmission information for each target virtual queue. This efficient data transmission mechanism significantly improves the data transmission bandwidth between the host and the hardware accelerator, thereby greatly enhancing the acceleration effect of the application and ensuring the high efficiency of data processing and overall performance optimization.

[0121] Reference Figure 3 , showing the steps of the data transmission method provided by the embodiment of the present invention Figure 3 , the method may include:

[0122] Step 301, obtaining the initial number of virtual queues supported by the hardware accelerator and the number of descriptors corresponding to the virtual queues;

[0123] It should be noted that in the embodiment of the present application, first, the Virtqueue queue is initialized. When the hardware accelerator driver is loaded, the maximum number of Virtqueue queues supported by the hardware accelerator is read, and then all Virtqueue queues are initialized.

[0124] Specifically, initialization mainly includes:

[0125] Determine the size of the descriptor array contained in the Virtqueue queue, that is, the number of descriptors Q. The number of descriptors supported by different Virtqueues may be different.

[0126] Initialize Virtqueue queue and data transmission related information, including queue index, queue idle state, queue data transmission state, remaining data volume, sent data volume, device address, etc. Use default settings, such as setting the queue idle state to idle and the data transmission state to normal;

[0127] Register the callback function when Virtqueue is interrupted. When a Virtqueue data transmission is completed, the host receives an interrupt from Virtqueue and calls this callback function. In addition, in order to execute the data transmission of each Virtqueue queue in parallel, a workqueue object is created based on the workqueue mechanism provided by the Linux kernel, such as data_workqueue, which is responsible for scheduling the data transmission tasks corresponding to each Virtqueue queue to different CPUs for execution.

[0128] Step 302, when a data transmission request is detected, a data transmission request object is created;

[0129] Further, after the step of creating a data transmission request object when receiving a data transmission request is detected, the method includes:

[0130] Create a work queue object, where the work queue object is used to schedule data transmission information corresponding to each target virtual queue to different CPUs.

[0131] Step 303, determining the number of virtual queues to be allocated based on the data transmission request object;

[0132] Step 304, determining a target virtual queue based on the number of virtual queues to be allocated, and a data transmission volume corresponding to each of the target virtual queues;

[0133] Step 305, configuring data transmission information of each of the target virtual queues based on the data transmission volume, and transmitting the data transmission information in parallel through the target virtual queues;

[0134] Further, configuring the data transmission information of each of the target virtual queues based on the data transmission amount, and transmitting the data transmission information in parallel through the target virtual queues includes:

[0135] The data transmission information of each target virtual queue is configured based on the data transmission amount, the target virtual queue is added to the work queue object, and the data transmission information is transmitted through the work queue object.

[0136] Further, the data transmission request object includes a preset bitmap data structure variable, and the preset bitmap data structure variable is used to record the number of each device virtual queue in the virtual queue linked list corresponding to the target device; and the parallel transmission of the data transmission information through the target virtual queue includes:

[0137] Determining whether the data transmission request object is in an abnormal state;

[0138] If not, determining whether the data transmission state corresponding to the target virtual queue is in an abnormal state;

[0139] If not, determining whether the data transmission information corresponding to the target virtual queue has been transmitted;

[0140] If not, a descriptor of the target virtual queue is configured, and the data transmission information is transmitted in parallel through the target virtual queue.

[0141] Further, after the step of determining whether the data transmission request object is in an abnormal state, the method includes:

[0142] If so, the data transmission termination operation is performed;

[0143] The data transmission termination operation includes setting the target virtual queue to an idle state, and / or deleting the number corresponding to the target virtual queue in the preset bitmap data structure variable.

[0144] Further, each of the target virtual queues corresponds to a descriptor chain, the descriptor chain includes a head descriptor and an intermediate descriptor, the head descriptor in the descriptor chain is used to point to the first data transmission information of the target virtual queue, the first data transmission information includes a data transmission operation, a data transmission address, a data transmission length and a data transmission status, the intermediate descriptor is used to point to the second data transmission information of the target virtual queue, the second data transmission information includes a data transmission address and a data transmission length; the descriptor configuring the target virtual queue includes:

[0145] determining the number of target descriptors for each of the target virtual queues;

[0146] The first data transmission information corresponding to the header descriptor and the second data transmission information corresponding to the intermediate descriptor are configured.

[0147] It should be noted that, in the embodiment of the present application, the Virtqueue queue data transmission stage includes:

[0148] a) First, check whether the state of the data transmission request object data_work corresponding to the current Virtqueue queue is abnormal. If it is abnormal, perform the data transmission termination operation: 1. Set the idle state Idle of the current queue to idle; 2. Delete the number corresponding to this Virtqueue queue from the bitmap data structure queue_mask that records the queue number of the data transmission request object data_work; 3. Check whether queue_mask has deleted all Virtqueue queue numbers. If so, send a message to the user through the notification object contained in data_work to end this data transmission request.

[0149] b) If the status of data_work is normal, then check the data transmission status Status of the current Virtqueue. If Status shows that the data transmission is abnormal, then mark the status of the data transmission request object data_work corresponding to this queue as abnormal. Then perform the data transmission termination operation mentioned above.

[0150] c) If the data transmission status Status of the current Virtqueue is normal, then check whether the data transmission task of the current Virtqueue queue is completed. If completed, perform the data transmission termination operation mentioned above.

[0151] Configure the Virtqueue queue descriptor to execute the data transmission task. In the embodiment of the present application, a data transmission task is described by a descriptor chain, wherein the first descriptor in the chain points to an object data_header that describes the relevant information of this data transmission. The specific information includes: ① whether this transmission is a read operation or a write operation; ② the device end address; ③ the length of the data transmitted; ④ the data transmission completion status, success or failure, which is updated by the device end. The remaining descriptors in the chain point to the location and length of the data to be transmitted in the host end memory.

[0152] Specifically, the specific process of configuring the Virtqueue queue descriptor and executing the data transmission task is:

[0153] Determine the number of descriptors X required for the current queue to point to the data to be transmitted in this transmission: X is the smaller value between the number of descriptors Q supported by the current queue minus 1 and the number of pages corresponding to the amount of data remaining to be transmitted;

[0154] Fill in the data_header related information, including read / write type, device address, length, and set the first descriptor to point to data_header;

[0155] Set the remaining X descriptors to point to the address and length of the data to be transferred.

[0156] Then, after the descriptor is set, the queue transmission is started, and the data transmission information of this queue is updated, including the amount of data sent and the amount of data remaining to be transmitted.

[0157] It should be noted that in the embodiment of the present application, the above assumption is that the current queue is assigned to transmit 10M=10*1024KB of data, and the number of descriptors supported by this queue is Q=1024, so the maximum amount of data transmitted by this queue at one time = (1024-1)*4KB=4092KB; the reason for subtracting 1 is that the first descriptor is used for data_header, not for data transmission. 10M is divided into 3 transmissions, the first time uses 1023 descriptors, the second time also uses 1023 descriptors, but the last time only has (10240KB-2*4092KB)=2056KB, so the last time only needs 2056 / 4=514 descriptors.

[0158] The X descriptor mentioned here is the descriptor used for data. The descriptor finally used is X+1, where 1 is the descriptor used for the header.

[0159] Step 306, when it is detected that any one of the target virtual queues completes the transmission of the data transmission information, and when the bit position corresponding to the current target virtual queue in the preset bitmap data structure variable is set to the first identification value and the preset bitmap data structure variable is set to the preset value, the data transmission request object is in a data transmission success state.

[0160] It is noted that in the embodiment of the present application, the current queue list of the device is traversed, and a Virtqueue queue whose queue status is idle and whose data transmission status is normal is selected. The queue number is recorded through the bitmap data structure queue_mask mentioned above, and the status of this idle queue is marked as busy. The traversal ends until the required N idle queues are found or the list traversal is completed.

[0161] If the number of queues M found at last is 0, it indicates that the current device has no available Virtqueue queue, and the state of the data transmission request object data_work is set to busy, and the user is informed through the notification object to end the data transmission request.

[0162] By detecting the received data transmission request and creating the corresponding request object, the embodiment of the present invention can determine the required number of virtual queues to be allocated, and then select the appropriate target virtual queue. This process not only ensures the accurate allocation of data transmission volume, but also realizes parallel transmission of multiple virtual queues by configuring special data transmission information for each target virtual queue. This efficient data transmission mechanism significantly improves the data transmission bandwidth between the host and the hardware accelerator, thereby greatly enhancing the acceleration effect of the application and ensuring the high efficiency of data processing and overall performance optimization.

[0163] In addition, in order to facilitate those skilled in the art to understand the technical solution of the present application, it is assumed that the maximum number of Virtqueues supported by a hardware accelerator is 4, and the number of descriptors contained in each Virtqueue after initialization is 10, 12, 12, and 14 respectively, and the registered interrupt processing function is virtqueue_done. Now a data transmission request of 96KB of data is received from the host side to the device side, which specifically includes the following four parts:

[0164] The first part, i.e., preparation before data transmission, first creates a data transmission request object data_work, calculates the required number of Virtqueue queues N=(96KB / 4KB) / 10 + 1 = 3; finds a Virtqueue queue whose Virtqueue queue status is idle and whose data transmission status is normal, assumes that all queues are available, traverses the queues of the device in turn, finds Virtqueue queues numbered 1, 2, and 3, the number of all available Virtqueue queues M=3, and the bitmap data structure variable queue_mask=0x7 used to record the Virtqueue number, its lowest 3 bits are all set to 1; determines the data transmission volume of each queue, unit_pages = (96KB / 4KB) / 3 = 8; left_pages= (96KB / 4KB) – 3*8 = 0; the transmission volume of all three Virtqueue queues is 24KB. Finally, configure the data transmission information of each Virtqueue, and execute Virtqueue queue data transmission.

[0165] The second part is that the Virtqueue queue performs data transmission. Taking Virtqueue queue 1 as an example, first check whether the status of the data transmission request object corresponding to Virtqueue queue 0 is normal, that is, the status of data_work created in the previous step. At the beginning of transmission, there should be no abnormality; then check whether the data transmission status Status of the Virtqueue queue is abnormal. At the beginning of transmission, there should be no abnormality; then check whether the data transmission task of the Virtqueue queue is completed. At the beginning of transmission, there is still 24KB of data to be transmitted; then determine the number of descriptors X = 6 required for the current queue to point to the data to be transmitted, because the number of pages corresponding to the amount of data to be transmitted 6 (24KB / 4KB) is less than the number of descriptors of this queue minus one (10-1). Fill in the relevant information of data_header, such as the request is a write operation, the amount of data to be transmitted is 24KB, etc. Finally, initiate data transmission and update the data transmission information of this Virtqueue queue at the same time, such as the amount of data sent is updated from 0 to 24KB, and the amount of remaining data is updated from 24KB to 0.

[0166] The third part is Virtqueue interrupt processing. Taking Virtqueue queue 1 as an example, after receiving the interrupt, read the content of the object data_header pointed to by the first descriptor, and check the data transfer completion status in data_header to determine success or failure. Assuming the status is successful, the Virtqueue queue data transfer operation is then executed.

[0167] The fourth part is that the Virtqueue queue performs data transmission. Taking Virtqueue queue 1 as an example, at this time, the status of data_work and the data transmission status of Virtqueue queue 1 are checked again, and both are passed. Then check whether the data transmission task of the Virtqueue queue is completed, and find that the remaining data volume is 0, indicating that the task has been completed. Therefore, the data transmission termination operation is performed: 1. Set the idle state Idle of Virtqueue queue 1 to idle; 2. Delete the number corresponding to this Virtqueue queue from the bitmap data structure queue_mask that records the queue number of the data transmission request object data_work, that is, set bit 1 of queue_maks to 0; 3. Check whether queue_mask has deleted all Virtqueue queue numbers, that is, whether queue_mask is 0. If so, send a message to the user through the notification object contained in data_work to end this data transmission request.

[0168] Reference Figure 4 , shows a schematic diagram of the structure of a data transmission device provided by an embodiment of the present invention, the device comprising:

[0169] A creation module 401, configured to create a data transmission request object when a data transmission request is detected;

[0170] A first determining module 402, configured to determine the number of virtual queues to be allocated based on the data transmission request object;

[0171] A second determination module 403 is used to determine the target virtual queues based on the number of virtual queues to be allocated, and the data transmission volume corresponding to each of the target virtual queues;

[0172] The transmission module 404 is used to configure the data transmission information of each of the target virtual queues based on the data transmission volume, and transmit the data transmission information in parallel through the target virtual queues.

[0173] The data transmission device provided by the embodiment of the present invention creates a data transmission request object when a data transmission request is detected; determines the number of virtual queues to be allocated based on the data transmission request object; determines the target virtual queue based on the number of virtual queues to be allocated; determines the data transmission volume corresponding to the target virtual queue; configures the data transmission information of each target virtual queue based on the data transmission volume, and transmits the data transmission information in parallel through the target virtual queue. The embodiment of the present invention can determine the required number of virtual queues to be allocated and select a suitable target virtual queue by detecting the received data transmission request and creating a corresponding request object. This process not only ensures the accurate allocation of data transmission volume, but also realizes the parallel transmission of multiple virtual queues by configuring special data transmission information for each target virtual queue. This efficient data transmission mechanism significantly improves the data transmission bandwidth between the host and the hardware accelerator, thereby greatly enhancing the acceleration effect of the application and ensuring the efficiency of data processing and overall performance optimization.

[0174] The embodiment of the present invention further provides an electronic device, such as Figure 5 As shown, it includes a processor 501, a communication interface 502, a memory 503 and a communication bus 504, wherein the processor 501, the communication interface 502, and the memory 503 communicate with each other through the communication bus 504.

[0175] Memory 503, used for storing computer programs;

[0176] The processor 501, when used to execute the program stored in the memory 503, can implement the following steps:

[0177] In case of detecting receiving a data transfer request, creating a data transfer request object;

[0178] Determining the number of virtual queues to be allocated based on the data transmission request object;

[0179] Determine a target virtual queue based on the number of virtual queues to be allocated;

[0180] Determine the data transmission volume corresponding to the target virtual queue;

[0181] The data transmission information of each of the target virtual queues is configured based on the data transmission amount, and the data transmission information is transmitted in parallel through the target virtual queues.

[0182] Among them, the memory and the processor are connected in a bus manner, and the bus may include any number of interconnected buses and bridges, and the bus connects various circuits of one or more processors and memories together. The bus can also connect various other circuits such as peripheral devices, voltage regulators, and power management circuits together, which are all well known in the art, so this article will not further describe them. The bus interface provides an interface between the bus and the transceiver. The transceiver can be one element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices on a transmission medium. The data processed by the processor can be transmitted on a wired medium or on a wireless medium through an antenna. Further, the antenna also receives data and transmits the data to the processor. The processor is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. The memory can be used to store data used by the processor when performing operations.

[0183] The communication bus mentioned in the above terminal can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0184] The communication interface is used for communication between the above terminal and other devices.

[0185] The memory may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0186] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0187] In another embodiment of the present invention, a computer-readable storage medium is provided. The computer-readable storage medium stores instructions. When the instructions are executed on a computer, the computer executes the data transmission method described in any one of the above embodiments.

[0188] In another embodiment of the present invention, a computer program product including instructions is provided. When the computer program product is run on a computer, the computer executes the data transmission method described in any one of the above embodiments.

[0189] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive Solid State Disk (SSD)), etc.

[0190] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0191] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0192] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A data transmission method, characterized in that: The method comprises: In case of detecting receiving a data transfer request, creating a data transfer request object; Determining the number of virtual queues to be allocated based on the data transmission request object; Determine the target virtual queue based on the number of virtual queues to be allocated, and the data transmission volume corresponding to each of the target virtual queues; configuring data transmission information of each of the target virtual queues based on the data transmission amount, and transmitting the data transmission information in parallel through the target virtual queues; Among them, each of the target virtual queues corresponds to a descriptor chain, and the descriptor chain includes a head descriptor and an intermediate descriptor. The head descriptor in the descriptor chain is used to point to the first data transmission information of the target virtual queue, and the first data transmission information includes a data transmission operation, a data transmission address, a data transmission length, and a data transmission status. The intermediate descriptor is used to point to the second data transmission information of the target virtual queue, and the second data transmission information includes a data transmission address and a data transmission length.

2. The method according to claim 1, characterized in that Determining the number of virtual queues to be allocated based on the data transmission request object comprises: Get the number of descriptors supported by each device virtual queue in the virtual queue list of the target device; Determine a minimum number of descriptors based on the number of descriptors supported by all of the device virtual queues; The number of virtual queues to be allocated is generated based on the data transmission length corresponding to the data transmission request object, the minimum number of descriptors, and the preset page size.

3. The method according to claim 1, characterized in that The determining of the target virtual queue based on the number of virtual queues to be allocated, and the data transmission volume corresponding to each of the target virtual queues include: Traversing the virtual queue linked list of the target device based on the number of virtual queues to be allocated and a preset traversal rule, determining a target virtual queue, and recording a queue number corresponding to the target virtual queue based on a preset bitmap data structure, and marking a state corresponding to the target virtual queue as a non-idle state; The preset traversal rule includes that the target virtual queue is in a non-abnormal state and an idle state, the non-abnormal state is determined based on the current actual transmission state of the target virtual queue, and the idle state is pre-configured based on the target device; The data transmission volume corresponding to each target virtual queue is determined based on the number corresponding to the target virtual queue.

4. The method according to claim 3, characterized in that Determining the data transmission volume corresponding to each target virtual queue based on the number corresponding to the target virtual queue includes: Obtaining an average data transmission volume corresponding to each of the target virtual queues based on the number and data transmission length corresponding to the target virtual queues; If the data transmission length exceeds the sum of the average data transmission amount and the number of virtual queues to be allocated, it is determined that the data transmission amount is exceeded; The excess data transmission amount is allocated to a first target virtual queue among the target virtual queues, wherein the first target virtual queue has the highest priority in the activation sequence corresponding to all the target virtual queues.

5. The method according to claim 3, characterized in that: The step of traversing the virtual queue linked list of the target device based on the number of virtual queues to be allocated and a preset traversal rule to determine the target virtual queue comprises: If the number of target virtual queues satisfying the preset traversal rules obtained by traversing the virtual queue linked list of the target device is zero, the data transmission status corresponding to the data transmission request object is set to a data transmission failure status, a data transmission request end notification is sent to the target device, and the data transmission request is stopped.

6. The method according to claim 1, characterized in that After the step of transmitting the data transmission information in parallel through the target virtual queue, the method comprises: When it is detected that any one of the target virtual queues is in an abnormal state, setting the data transmission state corresponding to the data transmission request object to a data transmission failure state, and setting the bit position corresponding to the current target virtual queue to a first identification value; Controlling the remaining target virtual queues to stop transmission operations based on the data transmission failure state and the first identification value; When it is detected that the bit position corresponding to any one of the target virtual queues remaining in the preset bitmap data structure variable is the first identification value and the preset bitmap data structure variable is the preset value, the data transmission request object is in a data transmission failure state.

7. The method according to claim 6, characterized in that After the step of setting the data transmission state corresponding to the data transmission request object to a data transmission failure state and setting the bit position corresponding to the current target virtual queue to a first identification value when any one of the target virtual queues among all the target virtual queues is detected to be in an abnormal state, the method includes: When all the target virtual queues receive the transmission failure status, a data transmission request end notification is sent to the target device.

8. The method according to claim 1, characterized in that After the step of transmitting the data transmission information in parallel through the target virtual queue, the method comprises: When it is detected that any one of the target virtual queues completes the transmission of the data transmission information, and the preset bitmap data structure variable is set to a preset value when the bit position corresponding to the current target virtual queue in the preset bitmap data structure variable is set to a first identification value, the data transmission request object is in a data transmission success state.

9. The method according to claim 1, characterized in that: The data transmission request object includes a preset bitmap data structure variable, and the preset bitmap data structure variable is used to record the virtual queue number of each device in the virtual queue linked list corresponding to the target device; The parallel transmission of the data transmission information through the target virtual queue includes: Determining whether the data transmission request object is in an abnormal state; If not, determining whether the data transmission state corresponding to the target virtual queue is in an abnormal state; If not, determining whether the data transmission information corresponding to the target virtual queue has been transmitted; If not, a descriptor of the target virtual queue is configured, and the data transmission information is transmitted in parallel through the target virtual queue.

10. The method according to claim 9, characterized in that After the step of determining whether the data transmission request object is in an abnormal state, the method includes: If so, the data transmission termination operation is performed; The data transmission termination operation includes setting the target virtual queue to an idle state, and / or deleting the number corresponding to the target virtual queue in the preset bitmap data structure variable.

11. The method according to claim 10, characterized in that The descriptor for configuring the target virtual queue includes: determining the number of target descriptors for each of the target virtual queues; The first data transmission information corresponding to the header descriptor and the second data transmission information corresponding to the intermediate descriptor are configured.

12. The method according to claim 1, characterized in that After the step of creating a data transmission request object in case of detecting receiving a data transmission request, the method comprises: Create a work queue object, where the work queue object is used to schedule data transmission information corresponding to each target virtual queue to different CPUs.

13. The method according to claim 12, characterized in that The configuring the data transmission information of each of the target virtual queues based on the data transmission amount, and transmitting the data transmission information in parallel through the target virtual queues includes: The data transmission information of each target virtual queue is configured based on the data transmission amount, the target virtual queue is added to the work queue object, and the data transmission information is transmitted through the work queue object.

14. The method according to claim 1, characterized in that Before the step of creating a data transmission request object in the case of detecting receiving a data transmission request, the method comprises: Get the initial number of virtual queues supported by the hardware accelerator and the number of descriptors corresponding to the virtual queues.

15. An electronic device, characterized in that: include: A transceiver, a memory, a processor, and a program stored on the memory and executable on the processor; The processor is used to read the program in the memory to implement the data transmission method as described in any one of claims 1-14.

16. A readable storage medium for storing a program, characterized in that: When the program is executed by a processor, the data transmission method according to any one of claims 1 to 14 is implemented.

17. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, a data transmission method as described in any one of claims 1 to 14 is implemented.

Images