Data processing method, device and equipment, computer storage medium and computer program product

Abstract

The embodiment of the invention discloses a data processing method, and the method comprises the steps: determining sending task information corresponding to to-be-sent data in response to a data processing request, and determining a target transmission rate corresponding to the to-be-sent data; determining a sending opportunity of a control instruction based on the target transmission rate; and sending the control instruction to a communication unit based on the sending opportunity, so as to obtain the sending task information in response to the control instruction through the communication unit and execute the sending of the to-be-sent data based on the sending task information, thereby enabling the transmission rate of the to-be-sent data to approach the target transmission rate when the to-be-sent data is sent. The embodiment of the invention further discloses a data processing device and equipment, a computer storage medium and a computer program product.

Description

Technical Field

[0001] The present application relates to data processing technology in the field of computer technology, and in particular to a data processing method, device, equipment, computer storage medium and computer program product. Background Art

[0002] Due to the advantages of high throughput, low latency, and low CPU utilization of Remote Direct Memory Access (RDMA), RDMA is increasingly deployed in cloud data centers to improve the performance of distributed computing. As more and more tenants migrate their applications to the cloud, some large cloud providers are considering providing RDMA communication services to tenants. When multiple tenants share RDMA resources, there will be resource competition problems, so cloud service providers must provide performance isolation measures to avoid interference between tenant applications. Rate limiting is an important performance isolation measure that can be used to limit the bandwidth resources that each tenant can use. At present, the RDMA rate limiting function is mainly implemented in hardware network cards or user space. RDMA network cards integrate hardware rate limiters and can be used in conjunction with single root I / O virtualization (SR-IOV) technology. By setting the bandwidth of each logical network port, the traffic bound to the logical network port can be limited. FreeFlow is a paravirtualization solution for container environments. It implements the RDMA paravirtualization backend (Hypervisor) in a container in the user space, intercepts and proxies all RDMA requests from tenant containers, and implements a token-based rate limiter on the data path. However, the RDMA flow rate limit solution in the above-mentioned related technologies has the problem of poor scalability and high implementation complexity. Summary of the invention

[0003] In order to solve the above technical problems, the embodiments of the present application hope to provide a data processing method, device, equipment, computer storage medium and computer program product, which solve the problem of poor scalability in implementing traffic speed limiting in related technologies and reduce the implementation complexity.

[0004] The technical solution of this application is implemented as follows:

[0005] A data processing method, applied to a kernel module, comprising:

[0006] In response to the data processing request, determining sending task information corresponding to the data to be sent, and determining a target transmission rate corresponding to the data to be sent;

[0007] Determining a timing for sending a control instruction based on the target transmission rate;

[0008] The control instruction is sent to the communication unit based on the sending timing, so that the communication unit obtains the sending task information in response to the control instruction and executes the sending of the data to be sent based on the sending task information, so that the transmission rate of the data to be sent approaches the target transmission rate.

[0009] In the above solution, in response to the data processing request, determining the sending task information corresponding to the data to be sent includes:

[0010] Receive the address of the sending task information sent by the user mode;

[0011] The sending task information is acquired based on the address of the sending task information.

[0012] In the above scheme, the method further includes:

[0013] In response to the interface creation request, receiving basic parameters related to the communication interface; wherein the communication interface is used by the communication unit to transmit the data to be sent to the opposite end device;

[0014] The communication interface is created based on the basic parameters.

[0015] In the above scheme, the method further includes:

[0016] According to the target mapping relationship, the identifier of the communication interface and the data transmission rate of the communication interface are stored.

[0017] In the above scheme, the method further includes:

[0018] Receive the identifier of the communication interface fed back by the communication unit.

[0019] In the above scheme, the method further includes:

[0020] Determine a target storage space, and send the address of the target storage space to the communication unit; wherein the target storage space is used to store the sending task information, and each communication interface has a unique corresponding target storage space.

[0021] In the above solution, determining the target transmission rate corresponding to the data to be sent includes:

[0022] Receiving an identifier of a target communication interface corresponding to each sending task information;

[0023] Based on the identifier of the target communication interface and the target mapping relationship, a target transmission rate of the target communication interface is determined, and the target communication interface is used by the communication unit to transmit the data to be sent to the target opposite-end device.

[0024] In the above solution, determining the timing of sending the control instruction based on the target transmission rate includes:

[0025] Determining an initial sending time for each of the control instructions based on the target transmission rate;

[0026] Based on the initial sending time, a sending timing of each control instruction is determined.

[0027] In the above solution, determining the initial sending time for each of the control instructions based on the target transmission rate includes:

[0028] Determining the initial sending time of each control instruction based on the size of the sending task information and the target transmission rate;

[0029] Accordingly, determining the sending timing of each control instruction based on the initial sending time includes:

[0030] The sending timing of each control instruction is determined based on the initial sending time and the length of the hash round.

[0031] In the above scheme, the method further comprises:

[0032] Based on the initial sending time and the length of the hash wheel, determine the storage location of each sending task information in the hash wheel;

[0033] The sending task information is stored in the hash wheel based on the storage location.

[0034] In the above solution, the control instruction carries the identification of the target communication interface;

[0035] The communication unit is used to determine the address of the unique corresponding target storage space based on the identifier of the target communication interface, and obtain the sending task information according to the address of the target storage space;

[0036] The target communication interface is used to transmit the data to be sent to the target opposite-end device, and the target storage space is used to store the sending task information.

[0037] In the above solution, the communication unit is used to obtain the corresponding sending task information based on the address of the unique corresponding target storage space.

[0038] A data processing device, comprising:

[0039] A first determining unit, configured to determine, in response to a data processing request, sending task information corresponding to the data to be sent, and determine a target transmission rate corresponding to the data to be sent;

[0040] The second determining unit is further used to determine the sending timing of the control instruction based on the target transmission rate;

[0041] A sending unit is used to send the control instruction to the communication unit based on the sending timing, so that the communication unit obtains the sending task information in response to the control instruction and executes the sending of the data to be sent based on the sending task information, so that the transmission rate of the data to be sent approaches the target transmission rate.

[0042] A data processing device, the device comprising: a processor, a memory and a communication bus;

[0043] The communication bus is used to realize the communication connection between the processor and the memory;

[0044] The processor is used to execute the data processing program in the memory to implement the steps of the above-mentioned data processing method.

[0045] A computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of the above-mentioned data processing method.

[0046] A computer program product comprises a computer program, wherein the computer program implements the above method when executed by a processor.

[0047] The data processing method, apparatus, device, computer storage medium and computer program product provided in the embodiments of the present application can determine the sending task information corresponding to the data to be sent in response to a data processing request, and determine the target transmission rate corresponding to the data to be sent, determine the sending timing of the control instruction based on the target transmission rate, and send the control instruction to the communication unit based on the sending timing, so that the communication unit obtains the sending task information in response to the control instruction and executes the sending of the data to be sent based on the sending task information, so that the transmission rate of the data to be sent when it is sent approaches the target transmission rate. In this way, the sending task information corresponding to the data to be sent can be determined by the kernel module, and the communication unit can send the data to be sent according to the sending timing determined by using the transmission rate. It is not necessary to rely entirely on the user state as in the related art, and the data to be sent can be sent at a transmission rate close to the target transmission rate, which solves the problem of poor scalability in implementing traffic speed limit in the related art, reduces the implementation complexity, and can provide speed limit accuracy close to hardware speed limit. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A flowchart of a data processing method provided in an embodiment of the present application;

[0049] Figure 2 A flowchart of another data processing method provided in an embodiment of the present application;

[0050] Figure 3 A flowchart of another data processing method provided in an embodiment of the present application;

[0051] Figure 4 A schematic diagram of a speed limiter corresponding to a data processing method provided in an embodiment of the present application;

[0052] Figure 5 A schematic diagram of the structure of a data processing device provided in an embodiment of the present application;

[0053] Figure 6 A schematic diagram of the structure of a data processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0054] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0055] It should be understood that the "embodiments of the present application" or "the aforementioned embodiments" mentioned throughout the specification mean that specific features, structures or characteristics related to the embodiments are included in at least one embodiment of the present application. Therefore, "in the embodiments of the present application" or "in the aforementioned embodiments" appearing throughout the specification may not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics may be combined in one or more embodiments in any suitable manner. In the various embodiments of the present application, the size of the sequence numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. The above-mentioned sequence numbers of the embodiments of the present application are for description only and do not represent the advantages and disadvantages of the embodiments.

[0056] Unless otherwise specified, any step in the embodiments of the present application may be performed by a processor of the device. It is also worth noting that the embodiments of the present application do not limit the order in which the device executes the following steps. In addition, the methods used to process data in different embodiments may be the same method or different methods. It should also be noted that any step in the embodiments of the present application can be independently executed by the device, that is, when the device executes any step in the following embodiments, it may not rely on the execution of other steps.

[0057] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0058] The present application embodiment provides a data processing method, which can be applied to a kernel module. Figure 1 As shown, the method may include the following steps:

[0059] Step 101: In response to a data processing request, determine the sending task information corresponding to the data to be sent, and determine the target transmission rate corresponding to the data to be sent.

[0060] The data processing request may be a request for data transmission initiated by a user through a user state application; it should be noted that the data processing request may be first sent by a user through a user state application to a user state, and then the user state may forward the data processing request to a kernel module. In a feasible implementation, the user state may refer to a user state driver, and the kernel module may refer to a kernel module in an operating system, or may refer to an RDMA kernel driver module in an operating system, or may refer to a kernel module newly added to the kernel state of an operating system.

[0061] In other embodiments of the present application, the sending task information may refer to a work queue element (Work Queue Element, WQE); the sending task information may be obtained by the kernel module according to the address of the acquired WQE. Specifically, the target transmission rate may refer to the transmission rate of the target communication interface corresponding to the WQE.

[0062] Step 102: Determine the timing of sending the control instruction based on the target transmission rate.

[0063] Specifically, the kernel module may first determine the initial sending time of the control instruction according to the target transmission rate, and determine the sending timing of the control instruction according to the initial sending time.

[0064] It should be noted that the control instruction may carry the identifier of the target communication interface; that is, the control instruction is used to send the identifier of the target communication interface corresponding to the target WQE in the WQEs that can be sent currently.

[0065] Step 103: Send a control instruction to the communication unit based on the sending timing, so that the communication unit obtains the sending task information in response to the control instruction and executes the sending of the data to be sent based on the sending task information, so that the transmission rate of the data to be sent approaches the target transmission rate.

[0066] Among them, the communication unit may refer to an RDMA network card; specifically, when the sending timing of the corresponding control instruction is reached, the control instruction may be sent to the RDMA network card; further, the RDMA network card may obtain the target WQE from the kernel module according to the information related to the target WQE carried in the control instruction, and execute the sending of the data to be sent according to the target WQE.

[0067] It should be noted that, because the kernel module determines the sending timing according to the target transmission rate, the transmission rate of the RDMA network card when sending the data to be sent can be close to the target transmission rate according to the sending timing, thereby providing a speed limit accuracy close to the hardware speed limit.

[0068] In other embodiments of the present application, the "determining, in response to the data processing request, the sending task information corresponding to the data to be sent" in the above step 101 can be implemented in the following manner:

[0069] A1. Receive the address of the sending task information sent by the user state.

[0070] The user-mode driver may send the address of the WQE to the kernel module through the target function. In a feasible implementation, the target function may refer to ioctl. It should be noted that, Figure 2 As shown, when ibv_post_send is executed, the user-mode driver sends the address of WQE to the kernel module through ioctl.

[0071] A2. Obtain the sending task information based on the address of the sending task information.

[0072] Specifically, the kernel module may obtain the WQE from a corresponding address according to the address of the received WQE.

[0073] In other embodiments of the present application, the method may further include:

[0074] In response to the interface creation request, receiving basic parameters related to the communication interface;

[0075] The communication interface is used by the communication unit to transmit the data to be sent to the opposite end device.

[0076] Create a communication interface based on basic parameters.

[0077] The communication interface may refer to a work queue pair (Queue Pair, QP). Specifically, Figure 3The user-mode application shown first calls ibv_create_cq to create a completion queue, and then calls ibv_create_qp to create a QP. At this time, the completion queue to be bound needs to be passed in. After verifying the legitimacy of the basic parameters about the QP, the user-mode driver will forward the interface creation request and basic parameters to the kernel module; then the kernel module receives the basic parameters about the QP sent by the user-mode driver in response to the interface creation request, and creates structures such as ib_cq and ib_qp based on the basic parameters to complete the creation of the QP. It should be noted that the basic parameters may include parameters such as the data transmission rate and the address of the target host.

[0078] In other embodiments of the present application, the method may further include:

[0079] According to the target mapping relationship, the identifier of the communication interface and the data transmission rate of the communication interface are stored.

[0080] The target mapping relationship may refer to a mapping relationship between a QP identifier and a data transmission rate of the QP; in a feasible implementation manner, the QP identifier may refer to a QP_number, which is used to uniquely identify the QP.

[0081] It should be noted that the QP_number and the data transmission rate of the QP for which the target mapping relationship is established may be recorded in the hash table.

[0082] In other embodiments of the present application, the method may further include:

[0083] The identifier of the communication interface fed back by the receiving communication unit.

[0084] The QP may be jointly created by the kernel module and the RDMA network card of the target host; and the RDMA network card may feed back the QP_number of the QP to the kernel module.

[0085] In other embodiments of the present application, the method may further include:

[0086] Determine the target storage space, and send the address of the target storage space to the communication unit; wherein the target storage space is used to store the sending task information; each communication interface has a unique corresponding target storage space.

[0087] The target storage space may refer to a buffer. Specifically, when the user-mode driver sends an interface creation request to the kernel module, it may also send the number of WQEs that need to be accommodated in the QP Buffer to the kernel module; further, Figure 3 As shown, after the kernel module creates the QP, it can also apply for the QP Buffer in the memory space; and the kernel module will also transmit the address of the applied QP Buffer to the RDMA network card.

[0088] It should be noted that WQE may include a send queue (Send Queue, SQ) WQE and a receive queue (Receive Queue, RQ) WQE; then, QP Buffer may be used to store SQ WQE and RQ WQE.

[0089] In other embodiments of the present application, the “determining the target transmission rate corresponding to the data to be sent” in the above step 101 can be implemented in the following manner:

[0090] B1. Receive the identifier of the target communication interface corresponding to each sending task information.

[0091] Among them, Figure 2 The user mode driver can send the QP_number of the target QP corresponding to each WQE to the kernel module through the target function. It should be noted that when ibv_post_send is executed, the target QP corresponding to each sending task information will be specified.

[0092] B2. Determine a target transmission rate of the target communication interface based on the identifier of the target communication interface and the target mapping relationship.

[0093] The target communication interface is used by the communication unit to transmit the data to be sent to the target peer device.

[0094] Specifically, Figure 4 The Timestamper in the rate limiter of the kernel module shown can look up the target transmission rate of the target QP from the hash table according to the received QP_number of the target QP and the target mapping relationship.

[0095] In other embodiments of the present application, the above step 102 may be implemented in the following manner:

[0096] C1. Determine the initial sending time for each control instruction based on the target transmission rate.

[0097] The initial sending time may refer to the earliest time when the control instruction is allowed to be sent; specifically, the initial sending time may be obtained after processing the target transmission rate.

[0098] C2. Determine the sending timing of each control instruction based on the initial sending time.

[0099] The sending timing may refer to the timing when the control instruction is finally allowed to be sent; specifically, the sending timing may be determined according to the initial sending time. Of course, the sending timing may also be determined according to the initial sending timing and the size of the storage module used to store the WQE.

[0100] In other embodiments of the present application, the above-mentioned determining the initial sending time for each control instruction based on the target transmission rate includes:

[0101] For each sending task information, an initial sending time of each control instruction is determined based on the size of the sending task information and the target transmission rate.

[0102] The kernel module can calculate based on the size of WQE and the target transmission rate to obtain the initial sending time of each control instruction. Specifically, the formula T i =max(T i-1 +L i-1 / R f ,NOW) to determine the initial sending time; where T i Refers to the initial sending time of the ith control instruction, T i-1 It refers to the i-1th WQE at bandwidth R f The time when the transmission on the virtual link ends, R f is the target transmission rate, L i-1 is the size of the i-1th WQE, and NOW is the current time.

[0103] In other embodiments of the present application, the above-mentioned determining the sending timing of each control instruction based on the initial sending time includes:

[0104] The sending timing of each control instruction is determined based on the initial sending time and the length of the hash round.

[0105] The hash wheel is a storage module for storing WQE. Specifically, the initial sending time and the length of the hash wheel can be calculated to obtain the sending timing of each control instruction. In a feasible implementation, the initial sending time can be divided by the length of the hash wheel horizon, and the quotient obtained can be used as the round (i.e., sending timing) at which the control instruction should be sent.

[0106] In other embodiments of the present application, the method may further include:

[0107] Based on the initial sending time and the length of the hash wheel, determine the storage location of each sending task information in the hash wheel;

[0108] Each sending task information is stored in a hash wheel based on the storage location.

[0109] Specifically, the initial sending time and the length of the hash wheel can be calculated to obtain the storage position of each WQE in the hash wheel. In a feasible implementation, the initial sending time can be divided by the hash wheel length horizon, and the remainder obtained can be used as the index corresponding to the WQE in the hash wheel (i.e., the storage position of the WQE in the hash wheel).

[0110] In other embodiments of the present application, the control instruction carries an identifier of the target communication interface;

[0111] The communication unit is used to determine the address of the unique corresponding target storage space based on the identifier of the target communication interface, and obtain the sending task information according to the address of the target storage space;

[0112] The target communication interface is used to transmit the data to be sent to the target peer device, and the target storage space is used to store the sending task information.

[0113] Among them, when the timing for sending the control instruction is reached, the kernel module can obtain the sending task information corresponding to the control instruction from the hash wheel, and store the sending task information in the target storage space. In addition, the kernel module can carry the identifier of the target communication interface in the control instruction, and send the control instruction carrying the identifier of the target communication interface to the communication unit; in this way, the communication unit can determine the address of the unique corresponding target storage space based on the identifier of the target communication interface and the address of the target storage space, and obtain the corresponding sending task information according to the address of the unique corresponding target storage space, and then obtain the data to be sent based on the address of the data to be sent in the obtained sending task information, and transmit the data to be sent to the target opposite device through the target communication interface based on the identifier of the target communication interface.

[0114] Specifically, the kernel module Figure 4 The scheduler in the speed limiter shown in the figure maintains the time variable t and uses the t value as an index to traverse the time wheel. When traversing to a slot of the hash wheel, if there are multiple WQEs on the slot, the quotient of each WQE will be reduced by one in turn, and it is determined that the target WQE with a quotient of 0 can be given to the RDMA network card; at this time, it is considered that the control instruction corresponding to the target WQE has reached the sending time, and the target WQE can be obtained from the corresponding slot of the hash wheel and stored in the QP Buffer.

[0115] In the embodiment of the present application, the QP_number of the target QP corresponding to the target WQE may be carried in the control instruction, and the control instruction is sent to the RDMA network card.

[0116] In other embodiments of the present application, the communication unit is used to obtain corresponding sending task information based on the address of the uniquely corresponding target storage space.

[0117] Specifically, the RDMA network card can obtain the target WQE according to the address of the QP Buffer, obtain the address of the data to be sent from the target WQE, obtain the data to be sent from the address of the data to be sent, and then transmit the obtained data to be sent.

[0118] It should be noted that the RDMA network card can send a task information acquisition instruction carrying the address of the unique corresponding target storage space to the kernel module, and the kernel module can send the target WQE corresponding to the address of the unique corresponding target storage space to the RDMA network card in response to the task information acquisition instruction. Of course, the RDMA network card can also actively obtain the target WQE from the corresponding position according to the address of the unique corresponding target storage space.

[0119] The data processing method provided in the embodiment of the present application can determine the sending task information corresponding to the data to be sent through the kernel module, and realize the sending of the data to be sent by the communication unit according to the sending timing determined by the transmission rate. It does not need to rely entirely on the user state as in the related art, and can allow the data to be sent to be sent at a transmission rate close to the target transmission rate. It solves the problem of poor scalability in implementing traffic speed limiting in the related art, reduces the implementation complexity, and can provide speed limit accuracy close to hardware speed limit.

[0120] Based on the above embodiments, the present invention provides a data processing device, which can be applied to Figure 1 In the data processing method provided in the corresponding embodiment, refer to Figure 5 As shown, the data processing device 2 may include: a determining unit 21, a processing unit 22 and a sending unit 23, wherein:

[0121] The determining unit 21 is used to determine the sending task information corresponding to the data to be sent and determine the target transmission rate corresponding to the data to be sent in response to the data processing request;

[0122] The processing unit 22 is used to determine the sending time of the control instruction based on the target transmission rate;

[0123] The sending unit 23 is used to send a control instruction to the communication unit based on the sending timing, so that the communication unit obtains the sending task information in response to the control instruction and executes the sending of the data to be sent based on the sending task information, so that the transmission rate of the data to be sent approaches the target transmission rate.

[0124] In other embodiments of the present application, the determining unit 21 is further configured to perform the following steps:

[0125] The address for receiving the sending task information sent by the user state;

[0126] Get the sending task information based on the address of the sending task information.

[0127] In other embodiments of the present application, the processing unit 22 is further configured to perform the following steps:

[0128] In response to the interface creation request, receiving basic parameters related to the communication interface; wherein the communication interface is used by the communication unit to transmit the data to be sent to the opposite end device;

[0129] Create a communication interface based on basic parameters.

[0130] In other embodiments of the present application, the processing unit 22 is further configured to store the identifier of the communication interface and the data transmission rate of the communication interface according to the target mapping relationship.

[0131] In other embodiments of the present application, refer to Figure 5 The data processing device 2 may further include: a receiving unit 24, wherein:

[0132] The receiving unit 24 is configured to receive the identifier of the communication interface fed back by the communication unit.

[0133] In other embodiments of the present application, the sending unit 23 is also used to determine the target storage space and send the address of the target storage space to the communication unit; wherein the target storage space is used to store the sending task information, and each communication interface has a unique corresponding target storage space.

[0134] In other embodiments of the present application, the determining unit 21 is further configured to perform the following steps:

[0135] Receiving an identifier of a target communication interface corresponding to each sending task information;

[0136] Based on the identifier of the target communication interface and the target mapping relationship, the target transmission rate of the target communication interface is determined; the target communication interface is used by the communication unit to transmit the data to be sent to the target opposite-end device.

[0137] In other embodiments of the present application, the processing unit 22 is further configured to perform the following steps:

[0138] determining an initial sending time for each control instruction based on a target transmission rate;

[0139] Based on the initial sending time, the sending timing of each control instruction is determined.

[0140] In other embodiments of the present application, the processing unit 22 is further configured to determine an initial sending time of each control instruction based on the size of the sending task information and the target transmission rate.

[0141] In other embodiments of the present application, the processing unit 22 is further configured to determine a sending timing of each control instruction based on an initial sending time and a length of a hash round.

[0142] In other embodiments of the present application, the processing unit 22 is further configured to perform the following steps:

[0143] Based on the initial sending time and the length of the hash wheel, determine the storage location of each sending task information in the hash wheel;

[0144] Each sending task information is stored in a hash wheel based on the storage location.

[0145] In other embodiments of the present application, the control instruction carries an identification of the target communication interface;

[0146] The communication unit is used to determine the address of the unique corresponding target storage space based on the identifier of the target communication interface, and obtain the sending task information according to the address of the target storage space;

[0147] The target communication interface is used to transmit the data to be sent to the target peer device, and the target storage space is used to store the sending task information.

[0148] In other embodiments of the present application, the communication unit is used to obtain corresponding sending task information based on the address of the uniquely corresponding target storage space.

[0149] It should be noted that the specific description of the steps performed by each unit can be found in Figure 1 The data processing method provided in the corresponding embodiment will not be described in detail here.

[0150] The data processing device provided in the embodiments of the present application can determine the sending task information corresponding to the data to be sent through the kernel module, and realize the sending of the data to be sent by the communication unit according to the sending timing determined by the transmission rate. It does not need to rely entirely on the user state as in the related art, and can allow the data to be sent to be sent at a transmission rate close to the target transmission rate. It solves the problem of poor scalability in implementing traffic speed limiting in the related art, reduces the implementation complexity, and can provide speed limit accuracy close to hardware speed limit.

[0151] Based on the above embodiments, the embodiments of the present application provide a data processing device, which may refer to a kernel module. The data processing device may be applied to Figure 1 In the data processing method provided in the corresponding embodiment, refer to Figure 6 As shown, the data processing device 3 may include: a processor 31, a memory 32 and a communication bus 33, wherein:

[0152] The communication bus 33 is used to realize the communication connection between the processor 31 and the memory 32;

[0153] The processor 31 is used to execute the data processing program in the memory 32 to implement the following steps:

[0154] In response to the data processing request, determining the sending task information corresponding to the data to be sent, and determining the target transmission rate corresponding to the data to be sent;

[0155] Determining a timing for sending a control instruction based on a target transmission rate;

[0156] A control instruction is sent to the communication unit based on the sending timing, so that the communication unit obtains the sending task information in response to the control instruction and executes the sending of the data to be sent based on the sending task information, so that the transmission rate of the data to be sent approaches the target transmission rate.

[0157] In other embodiments of the present application, the processor 31 is used to execute the data processing program in the memory 32 in response to the data processing request, determine the sending task information corresponding to the data to be sent, so as to implement the following steps:

[0158] The address for receiving the sending task information sent by the user state;

[0159] Get the sending task information based on the address of the sending task information.

[0160] In other embodiments of the present application, the processor 31 is used to execute the data processing program in the memory 32, and can also implement the following steps:

[0161] In response to the interface creation request, receiving basic parameters related to the communication interface; wherein the communication interface is used by the communication unit to transmit the data to be sent to the opposite end device;

[0162] Create a communication interface based on basic parameters.

[0163] In other embodiments of the present application, the processor 31 is used to execute the data processing program in the memory 32, and can also implement the following steps:

[0164] According to the target mapping relationship, the identifier of the communication interface and the data transmission rate of the communication interface are stored.

[0165] In other embodiments of the present application, the processor 31 is used to execute the data processing program in the memory 32, and can also implement the following steps:

[0166] The identifier of the communication interface fed back by the receiving communication unit.

[0167] In other embodiments of the present application, the processor 31 is used to execute the data processing program in the memory 32, and can also implement the following steps:

[0168] Determine the target storage space, and send the address of the target storage space to the communication unit; wherein the target storage space is used to store the sending task information, and each communication interface has a unique corresponding target storage space.

[0169] In other embodiments of the present application, the processor 31 is used to execute the data processing program in the memory 32 to determine the target transmission rate corresponding to the data to be sent, so as to implement the following steps:

[0170] Receiving an identifier of a target communication interface corresponding to each sending task information;

[0171] Based on the identifier of the target communication interface and the target mapping relationship, the target transmission rate of the target communication interface is determined; the target communication interface is used by the communication unit to transmit the data to be sent to the target opposite-end device.

[0172] In other embodiments of the present application, the processor 31 is used to execute the data processing program in the memory 32 to determine the sending timing of the control instruction based on the target transmission rate, so as to implement the following steps:

[0173] determining an initial sending time for each control instruction based on a target transmission rate;

[0174] Based on the initial sending time, the sending timing of each control instruction is determined.

[0175] In other embodiments of the present application, the processor 31 is used to execute the data processing program in the memory 32 to determine the initial sending time for each control instruction based on the target transmission rate, so as to implement the following steps:

[0176] Based on the size of the sending task information and the target transmission rate, the initial sending time of each control instruction is determined.

[0177] In other embodiments of the present application, the processor 31 is used to execute the data processing program in the memory 32 to determine the sending timing of each control instruction based on the initial sending time, so as to implement the following steps:

[0178] The sending timing of each control instruction is determined based on the initial sending time and the length of the hash round.

[0179] In other embodiments of the present application, the processor 31 is used to execute the data processing program in the memory 32, and can also implement the following steps:

[0180] Based on the initial sending time and the length of the hash wheel, determine the storage location of each sending task information in the hash wheel;

[0181] Each sending task information is stored in a hash wheel based on the storage location.

[0182] In other embodiments of the present application, the control instruction carries an identification of the target communication interface;

[0183] The communication unit is used to determine the address of the unique corresponding target storage space based on the identifier of the target communication interface, and obtain the sending task information according to the address of the target storage space;

[0184] The target communication interface is used to transmit the data to be sent to the target peer device, and the target storage space is used to store the sending task information.

[0185] In other embodiments of the present application, the communication unit is used to obtain corresponding sending task information based on the address of the uniquely corresponding target storage space.

[0186] It should be noted that the specific description of the steps executed by the processor can be referred to Figure 1 The data processing method provided in the corresponding embodiment will not be described in detail here.

[0187] The data processing device provided by the embodiments of the present application can determine the sending task information corresponding to the data to be sent through the kernel module, and realize the sending of the data to be sent by the communication unit according to the sending timing determined by the transmission rate. It does not need to rely entirely on the user state as in the related art, and can allow the data to be sent to be sent at a transmission rate close to the target transmission rate. It solves the problem of poor scalability in implementing traffic speed limiting in the related art, reduces the implementation complexity, and can provide speed limit accuracy close to hardware speed limit.

[0188] Based on the foregoing embodiments, the embodiments of the present application provide a computer-readable storage medium, which stores one or more programs, and the one or more programs can be executed by one or more processors to implement Figure 1 The corresponding embodiments provide steps of the data processing method.

[0189] Based on the foregoing embodiments, the embodiments of the present application provide a computer program product, including a computer program, which can be executed by the processor 31 to complete Figure 1 The corresponding embodiments provide steps of the data processing method.

[0190] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of hardware embodiments, software embodiments, or embodiments in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage and optical storage, etc.) that contain computer-usable program code.

[0191] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data transmission device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data transmission device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0192] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data transmission device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0193] These computer program instructions may also be loaded onto a computer or other programmable data transmission device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0194] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application.

Claims

1. A data processing method, characterized in that: Applied to a kernel module, the method comprises: In response to the data processing request, determining sending task information corresponding to the data to be sent, and determining a target transmission rate corresponding to the data to be sent; Determining a timing for sending a control instruction based on the target transmission rate; The control instruction is sent to the communication unit based on the sending timing, so that the communication unit obtains the sending task information in response to the control instruction and executes the sending of the data to be sent based on the sending task information, so that the transmission rate of the data to be sent approaches the target transmission rate.

2. The method according to claim 1, characterized in that The step of determining, in response to the data processing request, the sending task information corresponding to the data to be sent includes: Receive the address of the sending task information sent by the user mode; The sending task information is acquired based on the address of the sending task information.

3. The method according to claim 1, characterized in that The method further comprises: In response to the interface creation request, receiving basic parameters related to the communication interface; wherein the communication interface is used by the communication unit to transmit the data to be sent to the opposite end device; The communication interface is created based on the basic parameters.

4. The method according to claim 3, characterized in that The method further comprises: According to the target mapping relationship, the identifier of the communication interface and the data transmission rate of the communication interface are stored.

5. The method according to claim 3, characterized in that: The method further comprises: Receive the identifier of the communication interface fed back by the communication unit.

6. The method according to claim 3, characterized in that The method further comprises: Determine a target storage space, and send the address of the target storage space to the communication unit; wherein the target storage space is used to store sending task information, and each communication interface has a unique corresponding target storage space.

7. The method according to claim 1, characterized in that The determining a target transmission rate corresponding to the data to be sent includes: Receiving an identifier of a target communication interface corresponding to each sending task information; Based on the identifier of the target communication interface and the target mapping relationship, a target transmission rate of the target communication interface is determined, and the target communication interface is used by the communication unit to transmit the data to be sent to the target opposite-end device.

8. The method according to claim 7, characterized in that The determining of a sending timing of a control instruction based on the target transmission rate includes: Determining an initial sending time for each of the control instructions based on the target transmission rate; Based on the initial sending time, a sending timing of each control instruction is determined.

9. The method according to claim 8, characterized in that The determining, based on the target transmission rate, an initial sending time for each of the control instructions comprises: Determining the initial sending time of each control instruction based on the size of the sending task information and the target transmission rate; Accordingly, determining the sending timing of each control instruction based on the initial sending time includes: The sending timing of each control instruction is determined based on the initial sending time and the length of the hash round.

10. The method according to claim 8, characterized in that The method further comprises: Based on the initial sending time and the length of the hash wheel, determine the storage location of each sending task information in the hash wheel; The sending task information is stored in the hash wheel based on the storage location.

11. The method according to claim 1, characterized in that The control instruction carries an identification of a target communication interface; The communication unit is used to determine the address of the unique corresponding target storage space based on the identifier of the target communication interface, and obtain the sending task information according to the address of the target storage space; The target communication interface is used to transmit the data to be sent to the target opposite-end device, and the target storage space is used to store the sending task information.

12. The method according to claim 11, characterized in that The communication unit is used to obtain corresponding sending task information based on the address of the unique corresponding target storage space.

13. A data processing device, characterized in that: The device comprises: a determination unit, configured to determine, in response to a data processing request, transmission task information corresponding to the data to be transmitted, and determine a target transmission rate corresponding to the data to be transmitted; A processing unit, configured to determine a timing for sending a control instruction based on the target transmission rate; A sending unit is used to send the control instruction to the communication unit based on the sending timing, so that the communication unit obtains the sending task information in response to the control instruction and executes the sending of the data to be sent based on the sending task information, so that the transmission rate of the data to be sent approaches the target transmission rate.

14. A data processing device, characterized in that: The device comprises: a processor, a memory and a communication bus; The communication bus is used to realize the communication connection between the processor and the memory; The processor is used to execute the data processing program in the memory to implement the steps of the data processing method according to any one of claims 1 to 12.

15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps of the data processing method according to any one of claims 1 to 12.

16. A computer program product, comprising a computer program, characterized in that: The computer program implements the method according to any one of claims 1 to 12 when executed by a processor.

Images