Sketch data processing method and device based on cooperation of programmable switch and DPU, and medium

Abstract

The application discloses a Sketch data processing method and device based on cooperation of a programmable switch and a DPU, and a medium, and the method comprises the following steps: converting a Sketch data processing task into a basic operation set; processing the basic operation set through an automatic task segmentation algorithm to obtain an optimal unloading scheme of the basic operation set on a fast path and a common path; cooperating between the fast path and the common path according to the optimal unloading scheme of the basic operation set on the fast path and the common path to complete the Sketch data processing task; and in a running process, a Sketch data stream first reaches the fast path, the fast path and the common path respectively perform corresponding Sketch data processing operations, and meanwhile, the fast path controls the rate of sending Sketch key-value pair data to the common path to prevent saturation of the common path.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of computer networks, and particularly relates to a Sketch data processing method, device and medium based on cooperation of programmable switches and DPUs BACKGROUND

[0002] Sketch is an emerging approximate network measurement algorithm that measures traffic at linear rates using data structures inside switches. These data are periodically collected and processed to ensure that network management applications can detect and resolve network events, such as attack flow detection. However, existing solutions treat Sketch data processing as a trade-off, either sacrificing scalability, i.e., improving the performance of running any type of Sketch data processing task by dedicating a large number of servers, or maintaining high scalability by offloading tasks to switches, but losing generality due to the limited functions of switches. SUMMARY

[0003] The present application aims to overcome the deficiencies of the prior art and provides a Sketch data processing method, device and medium based on cooperation of programmable switches and DPUs, which comprehensively utilizes the common path located on the intelligent network card and the fast path located on the TOR (Top-of-Rack) switch, maximizes the offloading of tasks to the fast path, and cooperates the fast path and the common path to achieve scalable and general Sketch data processing.

[0004] In a first aspect, an embodiment of the present application provides a Sketch data processing method based on cooperation of programmable switches and DPUs, which comprises:

[0005] converting a Sketch data processing task into a basic operation set;

[0006] processing the basic operation set by an automatic task segmentation algorithm to obtain an optimal offloading scheme of the basic operation set on the fast path and the common path;

[0007] performing cooperation between the fast path and the common path according to the optimal offloading scheme of the basic operation set on the fast path and the common path to complete the Sketch data processing task; in the running process, a Sketch data stream first arrives at the fast path, and the fast path and the common path respectively perform corresponding Sketch data processing operations, while the fast path controls the rate of sending Sketch key-value pair data to the common path to prevent the common path from being saturated.

[0008] In a second aspect, an electronic device is provided, comprising a memory and a processor, the memory being coupled to the processor; wherein the memory is configured to store program data, and the processor is configured to execute the program data to implement the Sketch data processing method based on cooperation between a programmable switch and a DPU.

[0009] In a third aspect, a computer-readable storage medium is provided, which stores a computer program, and the program is executed by a processor to implement the Sketch data processing method based on cooperation between a programmable switch and a DPU.

[0010] Compared with the prior art, the present application has the following beneficial effects:

[0011] The present application provides a Sketch data processing method based on cooperation between a programmable switch and a DPU, which converts the Sketch data processing task into a basic operation set, obtains the optimal unloading scheme of the basic operation set on the fast path and the normal path, unloads the Sketch data processing task to the data plane switch by cooperating the work of the fast path and the normal path, maintains a large throughput and a small delay under a small core consumption. The present application has higher performance and lower overhead. The method is simple and convenient, flexible to implement, and has strong practicability.

[0012] The present application fully utilizes the cooperation between the fast path and the intelligent network card normal path on the switch, sends the part of Sketch data processed by the fast path to the normal path, controls the sending rate of the fast path, prevents the saturation of the normal path, generates the Sketch data processing result that can be directly queried, can process any kind of user input task, and realizes the scalability and universality. BRIEF DESCRIPTION OF DRAWINGS

[0013] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0014] Figure 1 is a Sketch data processing method based on cooperation between a programmable switch and a DPU provided by the present application;

[0015] Figure 2 is a Sketch data processing task provided by the present application;

[0016] Figure 3is an automatic segmentation flowchart of a task provided by an embodiment of the present application;

[0017] Figure 4 is an example of hash representation in flow number estimation provided by an embodiment of the present application;

[0018] Figure 5 is an example diagram of internal path coordination provided by an embodiment of the present application;

[0019] Figure 6 is a schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0020] The present application will be described in detail below with reference to the drawings and specific embodiments. The present embodiment is implemented on the premise of the technical solution of the present application, and gives a detailed implementation manner and specific operation process. Obviously, the described embodiments are only part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0021] As shown in Figure 1 The present application provides a Sketch data processing method based on programmable switch and DPU cooperation, which constructs a processing method for automatically offloading Sketch tasks to fast path and normal path while coordinating the performance of the two, and the workflow can be divided into the following three steps:

[0022] Step S1: task transformation. Step S1 accepts any user task as input, abstracts the input task into a set of basic operations in a unified way in the face of highly heterogeneous tasks, which can shield the low-level hardware implementation and allow users to intuitively express tasks.

[0023] Specifically, the flow of step S1 task transformation includes the following sub-steps:

[0024] S1-1: analyze the form of Sketch data to understand its characteristics. Sketch data can be directly regarded as a set of key-value pairs, where the key represents the index of the value in the Sketch data structure, and the value represents the value of the Sketch counter of the key index.

[0025] S1-2: analyze the Sketch data processing requirements to understand their characteristics. The user's processing requirements for Sketch data can be summarized as the following three points:

[0026] (1) Select and process a part of Sketch data, for example, check the data collected in the ingress switch to identify DDoS flow, detect events in the data of a certain period, or merge the data of multiple periods and multiple switches.

[0027] (2) Decode the encoded Sketch data.

[0028] (3) Determine the target of flow statistics in the Sketch data, including: the number of packets / bytes of a flow, the existence statistics indicating whether a given flow is recorded by the Sketch data, the maximum / minimum statistics extracting the maximum or minimum value of a given flow from the Sketch data, and the distinct statistics counting the number of different values of a given flow in the Sketch data.

[0029] S1-3: Divide the operations into 3 categories, corresponding to the 3 Sketch data processing requirements in S1-2.

[0030] Operation category 1: Data filtering, aiming to filter out the Sketch data of interest, including operations: (1) Filtering, filtering some parts of the Sketch data according to the user-specified conditions, for example, filtering out the counters with a value equal to zero; (2) Merging, aggregating the Sketch data collected from different switches or time periods.

[0031] Operation category 2: Data result, mainly decoding the Sketch data, including the operation of decoding, decoding the collected Sketch data to restore the original Sketch data.

[0032] Operation category 3: Data conversion, converting the Sketch data into flow-level statistics, including operations: (1) Existence, checking whether the Sketch data records a flow; (2) Maximum / minimum, obtaining the maximum / minimum value of a flow; (3) Flow number estimation, obtaining the number of different flows recorded by the Sketch data.

[0033] Step S2: Process the basic operation set through an automatic task partitioning algorithm to obtain the optimal offloading scheme of the basic operation set on the fast path and the normal path. Given a set of tasks, let the user first indicate which servers and smart NICs support which tasks, for example, the user can assign two servers for each task. When a task is assigned to multiple servers, the smart NIC on each server will become an instance of the normal path based on the task processing Sketch data. In addition, different tasks share the same fast path.

[0034] For each task, step S2 needs to perform four sub-steps:

[0035] S2-1: Decompose each input task into a set of operations and analyze the resource consumption of each operation.

[0036] Offloading operations to the fast path can improve the performance and scalability of the program. Since the resources of the fast path are limited, it is necessary to know the resource consumption of each operation type to prevent the saturation of the fast path resources during offloading. The analysis of the operation resource consumption is completed by the following four stages.

[0037] Stage 1: For each operation, it is necessary to determine whether it can be implemented in the fast path. If an operation meets one of the following two rules, it cannot be implemented on the fast path:

[0038] 1. It uses floating-point variables or loops;

[0039] 2. It uses an instruction that is not supported by the ASIC pipeline of the switch.

[0040] Through these rules, only the decode operation among the six operations described in S1-3 cannot be implemented in the fast path due to its complex decoding algorithm. That is, the filter operation, the merge operation, the existence operation, the maximum / minimum operation, and the flow estimation operation can be deployed in the fast path.

[0041] Stage 2: Determine the implementation of the operation in the fast path.

[0042] The input Sketch data is in the form of key-value pairs, each of which corresponds to a packet header and a parser in the fast path. When the key-value pair arrives at the fast path, the parser in the fast path extracts its value and fills these values into the packet header input of the subsequent operation.

[0043] The following is the specific implementation of the five operations among the six operations described in S1-3 that can be implemented in the fast path:

[0044] The filter operation uses a match-action table, and each rule in the table matches a group of fields in the input key-value pair.

[0045] The merge operation requires a register array in the fast path. For two groups of Sketch data that need to be merged, first place the first group of data in the register array, with the key of the data as the index of the array and the value stored in the corresponding register. Then for each key-value pair of the second group of data, index to its corresponding register through the key, and then add the value to the value stored in the register to obtain the merged data.

[0046] The existence operation uses a match-action table, and each rule in the table matches a specific key or value. If a key-value pair in the Sketch data matches a rule, the operation records this pair. If a rule is not matched after timeout, the operation returns a value indicating that nothing is found.

[0047] The max / min operation uses a single register in the fast path. When a key-value pair arrives at the fast path, its value is compared with the register value, which is initialized to zero (for max operation) or MAXINT (for min operation). If the value is higher / lower than the register value, the register records the value. When all values go through the register, the value in the register is the result of the operation.

[0048] The stream estimate operation uses an array of registers to build a hash table. For each distinct value v, the fast path hashes v to obtain an index that addresses a hash table entry. If the entry is empty, v is inserted in the entry. The final operation returns the number of non-empty entries as the number of distinct values.

[0049] For the stream estimate operation, a key point is how to mitigate hash collisions. Due to the skewness of traffic in production networks, most Sketch counters record as zero, and only a few hash table entries record as nonzero values. Therefore, this method, when a collision occurs, the fast path discards the new value if it is zero, otherwise, a new hash function is invoked to address another entry. If the new entry is empty, the new value is inserted in the entry. Otherwise, it repeats the above process. However, it is challenging to repeatedly address different entries because the fast path disables loops. In response, this method utilizes recirculation, i.e., re-circulating the new value on the fast path to access the hash table again. This approach has three benefits. (1) It does not use loops, which well adapts to the one-pass nature of the fast path. (2) It avoids consuming normal switch throughput because the value is circulated in an internal pipeline, which is isolated from the normal pipeline that processes Sketch data. (3) Since most entries record as zero, the colliding value can quickly find an empty entry after a few recirculations. Figure 4 Operations are shown facing hash collisions.

[0050] Phase 3: Given the above implementation, this method analyzes the switch resource consumption of each operation type by using a static compilation approach. This approach invokes the switch compiler to compile the implementation of each operation. The compiler reports the consumption of each type of switch resource. This method uses this approach because the reported consumption is fixed and does not change due to runtime dynamics (e.g., traffic workload variations).

[0051] S2-2: Extract the execution dependencies between operations from each input task. It is necessary to preserve these dependencies when partitioning the tasks because they determine the correctness of the Sketch data processing.

[0052] Define the execution dependency between any pair of Sketch data processing operations, for any two operations Oi and O j , O j → O i The dependency relationship represents that the execution of O j depends on the execution of O i , that is, O j must run after O i , and if one of the three conditions in the task occurs, O j → O i exists: 1. The read operation is followed by the write operation or the write operation is followed by the write operation, O j modifies the data read or written by O i ; 2. The write operation is followed by the read operation, O j reads the data written by O i ; 3. Control dependency, O j outputs a variable in the if-else statement, which determines whether to execute O j .

[0053] The method extracts all execution dependencies between operations invoked by each input task. It builds the control flow of the task, enumerates each operation in the control flow, and determines the input and output of the operation. Then enumerate each pair of operations. If the input and output of these operations meet a condition, it is determined that the two operations are dependent on each other, and the pair of operations is filled into a set Φ, which records the dependency relationship.

[0054] S2-3: Optimal task partitioning, according to the analysis result information and the dependency information, the task is divided into two parts, including the operation unloaded to the fast path and the other operation on the normal path. It has 3 goals, including (1) maximizing the operation unloaded to the fast path to maximize the scalability; (2) preserving the switch resource limit to avoid deployment failure; (3) preserving the execution dependency between operations to ensure the correctness of Sketch data processing.

[0055] These goals are achieved by treating the task partitioning problem as an optimization challenge. It takes the first of the above 3 goals as the optimization goal, aiming to find the best decision. The remaining two goals are encoded as constraints in the optimization process.

[0056] The following is a description of the optimization model:

[0057] Input: The input consists of the following 3 parts: (1) A set of operations that make up the task, denoted by . Each operation (1) One of the five operations (i.e., filtering operation, merging operation, existence operation, maximum / minimum value operation, and flow estimation operation). (2) The amount of switch resources required for each operation, i.e., the analysis result of switch resource consumption. (3) The dependencies between operations, which are recorded in Φ.

[0058] Output: The output is a set {x(O)}: if O is on the fast path, then x(O) = 1; otherwise, x(O) = 0.

[0059] Objective: For each task, the objective is to offload as many operations as possible to the fast path.

[0060] Limitations: To ensure successful deployment, the following limitations need to be added:

[0061] 1. Switch resource consumption limits: For all operations offloaded to the fast path (i.e., operations satisfying x(O) = 1), their cumulative resource consumption should be less than the total resource limit of the fast path. The cumulative resource consumption is obtained by summing the analysis results. The total resource limit of the fast path can be directly obtained from the switch compiler report.

[0062] 2. Dependency constraints are implemented. To maintain dependencies, this method employs a two-pronged approach to constrain each dependency in Φ. j ∈O i Case 1: x(O) i ) = 1. Operation O i Placed on the fast path. Depends on O. i Operation O j It can be placed on either the fast path or the normal path. (1) If x(O j ) = 1, that is, O j It is also placed on the fast path, restricting the placement of O. j The first switch pipeline stage must occur at the O placement i After the last level. (2) If x(O) j ) = 0, that is, O j If it is located on a normal path, no additional restrictions are introduced because the normal path is always called after the fast path, satisfying O(n log n). j →O i Case 2: x(O) i ) = 0. Operation O i It is placed on a normal path. Therefore, x(O j The condition ) = 0 is always true. Otherwise, O j Must be in O iThe previous execution, which violates the dependency. Therefore, the method limits the common path to have O j The previous execution O i .

[0063] S2-4: Operation deployment. After the optimal task partitioning in S2-3, the operations corresponding to the tasks are partitioned into two groups, i.e., the operations offloaded to the fast path, and the operations located on the common path.

[0064] For the set For the operations offloaded to the fast path, the method converts the operations into a data plane program acceptable to the fast path. Specifically, this workflow has five steps: (1) Each operation is converted into a code element corresponding to its fast path implementation. (2) The elements in the fast path control flow are chained in the original execution order of the operations specified in the task. (3) The format of the input data in the header is defined. At the beginning, the fast path extracts the values from the input data and records them in the header. Then, it passes the header to the elements in the control flow to perform the operations. (4) The method inserts coordination logic at the end of the fast path to implement the correct inter-device task execution. (5) The data plane program is input to the switch compiler, which converts the data plane program into the underlying configuration and loads it onto the switch ASIC pipeline.

[0065] For the set For the operations located on the common path, the method converts the operations of the task into a C++ program and executes it on the common path. The conversion steps are as follows: (1) According to the order between the operations in the task, link these operations in the C++ program in order. (2) Run the C++ program directly on the smart NIC CPU kernel of the common path. If the common path has multiple instances (when the user allocates multiple servers for the task), each instance runs the same C++ program.

[0066] Step S3: Fast path and common path collaboration. To handle the performance gap between the O(10 tbps) fast path and the O(100 gbps) common path, prevent the common path from saturating when processing a large amount of Sketch data, transfer the intermediate results of Sketch data processing in the fast path to the common path for subsequent processing, and use state packet processing and send rate control to minimize the collaboration overhead.

[0067] The distribution of operations can be divided into three cases: (1) all operations are performed in the fast path, in which case the fast path needs to generate state packets to load operation results and send the state packets to the normal path; (2) both operations are performed in the fast path and the normal path, in which case the fast path needs to send the intermediate results of Sketch data processing to the normal path through state packets; (3) all operations are performed in the normal path, and the fast path only needs to transmit the Sketch data in the form of key-value pairs to the normal path

[0068] This step uses S3-1 state packet processing and S3-2 transmission rate control in the fast path to minimize coordination overhead.

[0069] S3-1: State packet processing.

[0070] In the fast path, a synthetic state data packet is created without affecting the normal switch throughput. The purpose of these state packets is to retrieve the results of the offloaded operations, i.e. merge, maximum and minimum, whose results are stored in registers. Each state packet has a register address, which is indexed to the correct register, and loads the register data on its packet header field. If all state packets have been sent to the normal path, the fast path completes the state packet processing task.

[0071] In order to prevent the processing of state packets from occupying the normal switch bandwidth and interfering with the processing of Sketch data, the fast path only processes state packets in the internal pipeline, which is isolated from the normal bandwidth. At the same time, in order to avoid overloading the internal pipeline, the method limits the number of state data packets. According to previous research, due to the influence of traffic skew, most Sketch data records are zero. Therefore, if the register record is zero, there is no need to transfer its value. Therefore, the fast path only generates state packets for non-zero registers. In addition, the method minimizes the length of the state packet, only recording the register index and value.

[0072] S3-2: Transmission rate control.

[0073] Whether the fast path sends state packets or Sketch data to the normal path, the transmission rate needs to be limited to avoid saturation of the normal path. Let C i denote the processing capacity of the i-th normal path instance, and there are K instances processing the normal path Sketch data of a given task in parallel. The fast path is connected to the K normal path instances through K ports. In this case, its rate control includes three stages.

[0074] Stage 1: The fast path decides which instance to send the key-value pair (denoted by P) corresponding to the state packet or Sketch data to. By default, it uses a round-robin method to assign P to one of the K instances. It enables users to use other methods, e.g., assigning key-value pairs based on the hash result of the key.

[0075] Stage 2: For the ith normal path instance, the fast path records the time when it sends the first P to this instance as

[0076] Stage 3: When sending the subsequent P to the ith instance, the fast path records the current time as t i . Let l denote the size of P. If then releasing P can overload the ith instance. The fast path simply recycles P until the inequality does not hold. Once it releases P, it changes t i from 0 to t i .

[0077] Figure 5 The cooperation of the internal paths in this method is shown, and the fast path runs the merge operation to merge the Sketch data of the two periods into four registers. At T1, the fast path creates the first state packet O1 to carry the data of the first register. Assume there are K = 2 normal path instances. The fast path selects the first instance to send P1 by round-robin. At T2, P2 goes through a similar workflow and is sent to the second instance. At T3, the fast path observes that the register data is zero, so it does not need to send P3. At T4, the fast path finds that sending P4 can overwhelm the first instance (i.e., l / (T4-T1) > C1). Therefore, it stops sending P4 by recycling. At T5, l / (T5-T1) ≤ C1, at which time the fast path determines that P4 can be safely sent.

[0078] Embodiment 1

[0079] This method is implemented on a 32x400Gbps Intel Tofino2 programmable switch and a 400Gbps NVIDIA BlueField-3 DPU, respectively establishing the fast path and the normal path on the two devices. The core of it is executed on a separate server, receiving input tasks for Sketch data processing, and the core interacts with the fast path and the normal path through the data plane driver.

[0080] The method has two core modules, (1) it presents the Sketch data processing operation library to the user and accepts input tasks. Then it performs automatic task segmentation to split each task, and unloads the problem through Gurobi optimal solution. (2) Through the decision of the solution, the method uses two drivers to deploy operations, the first driver sends the unloading end operation to the switch operating system, and compiles and installs these operations on the fast path in the application-specific integrated circuit pipeline, and the second driver sends other operations to the SoC core of the smart network card, and executes the operations through DOCA.

[0081] The fast path of the switch application-specific integrated circuit pipeline receives the Sketch data abstractly expressed by the key-value pair. For each value pair, the value is parsed and recorded in the header, which matches the header type to determine which task operation processes the pair. The state packet is created using the packet generator and the fast path, and the exchange internal timestamp is used to support the sending rate control.

[0082] The common path runs on the smart network card SoC CPU core, which parses the received state packet or key-value pair, extracts its data to execute the remaining operations in the Sketch data processing. Once all operations are completed, it will encapsulate the final result in the RDMA (Remote Direct Memory Access) request, and these data are written to the host memory as the input of the application program.

[0083] Correspondingly, the application also provides an electronic device, including: one or more processors; a memory for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the Sketch data processing method based on programmable switch and DPU cooperation as described above. As Figure 6 shown, a hardware structure diagram of any data processing capable device where the device of the embodiment is located, in addition to Figure 6 the processor, memory and network interface, the any data processing capable device where the device of the embodiment is located usually includes other hardware according to the actual function of the any data processing capable device, which will not be described here.

[0084] Correspondingly, the application further provides a computer readable storage medium, which stores computer instructions, and the instructions are executed by a processor to implement the Sketch data processing method based on the cooperation between the programmable switch and the DPU. The computer readable storage medium can be an internal storage unit of any device with data processing capability, such as a hard disk or a memory. The computer readable storage medium can also be an external storage device, such as a plug-in hard disk, a smart media card (SMC), an SD card, a flash card, etc. Further, the computer readable storage medium can include both the internal storage unit of any device with data processing capability and the external storage device. The computer readable storage medium is used to store the computer program and other programs and data required by the device with data processing capability, and can also be used to temporarily store data that has been output or will be output.

[0085] The above embodiments are only used to illustrate the technical solutions of the present application, rather than limit them. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements for some technical features, and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A Sketch data processing method based on programmable switch cooperating with DPU, characterized in that, The method comprises: transforming a Sketch data processing task into a basic operation set; processing the basic operation set through an automatic task segmentation algorithm to obtain an optimal unloading scheme of the basic operation set on a fast path and a normal path; including: performing resource analysis and dependency relationship extraction on the basic operation set, dividing the basic operation set into operations unloaded on the fast path and operations on the normal path according to the resource analysis result and the dependency relationship extraction result; and deploying the operations on the fast path and the operations on the normal path obtained through the division; according to the optimal unloading scheme of the basic operation set on the fast path and the normal path, performing cooperation between the fast path and the normal path to complete the Sketch data processing task; in the running process, the Sketch data stream first reaches the fast path, and the fast path and the normal path respectively perform corresponding Sketch data processing operations, while the fast path controls the rate of sending Sketch key-value pair data to the normal path to prevent the normal path from being saturated.

2. The Sketch data processing method based on programmable switch and DPU cooperation according to claim 1, characterized in that, The process of transforming the Sketch data processing task into the basic operation set comprises: analyzing the characteristics of the Sketch data; the Sketch data is a set of key-value pairs, the key represents the index of the value in the Sketch data structure, and the value represents the value of the Sketch counter of the key index; dividing the Sketch data processing requirements into: processing selected Sketch data, decoding encoded Sketch data, and obtaining stream-level Sketch data statistical results; dividing the operation types and specific operations of the Sketch data processing according to the Sketch data processing requirements; including: a first operation type: data filtering; the data filtering is used to filter out the required Sketch data, including filtering operations and merging operations; a second operation type: data decoding; the data decoding is decoding the encoded Sketch data, including decoding operations; a third operation type: data conversion; the data conversion is used to convert the Sketch data into stream-level statistical data, including existence operations, maximum / minimum value operations, and stream number estimation operations.

3. The Sketch data processing method based on programmable switch and DPU cooperation according to claim 1 or 2, characterized in that, The process of performing resource analysis on the basic operation set comprises: determining whether the basic operation can be implemented on the fast path, thereby determining the basic operations that can be implemented on the fast path; the determination process is: if the operation satisfies any one of the following two rules, it is not implemented on the fast path: (1) using a floating-point variable or a loop; (2) using an instruction not supported by a switch application-specific integrated circuit pipeline; wherein the input of the operation is the Sketch data abstracted as key-value pairs, each key-value pair is composed of a data packet header and a parser, the switch resources consumed by each type of operation are further measured using a static compilation method, a switch compiler is called to compile the implementation of each operation, and the compiler reports the consumption of each type of switch resource.

4. The Sketch data processing method based on programmable switch and DPU cooperation according to claim 1, characterized in that, The process of performing dependency relationship extraction on the basic operation set comprises: First, define the dependency relationship between any pair of Sketch data processing operations, O i and O j , dependency O j → O i indicates that the execution of O j depends on O i , O j must be executed after O i , O j → O i exists if one of the following three conditions occurs in a task: 1) read operation behind write operation or write operation behind write operation, O i modification O j data read or written; 2) write operation is behind read operation, O i read O j data written down; 3) control dependence, O i Output a variable in the if-else statement, which determines whether to execute O j Extract all execution dependencies between operations of each input task call, build the control flow of the task, enumerate each operation in the control flow, and determine the input and output of the operation, then enumerate each pair of operations, if the input of one operation matches the output of another operation, it can be determined that the two operations are dependent on each other, and the pair of operations is filled into a set Φ, thereby recording the dependency relationship.

5. The Sketch data processing method based on programmable switch and DPU cooperation according to claim 1, characterized in that, The process of dividing the basic operation set into operations offloaded to the fast path and operations on the normal path according to the resource analysis result and the dependency extraction result includes: The process of dividing the basic operation set into operations offloaded to the fast path and operations on the normal path according to the resource analysis result and the dependency extraction result includes: The optimization target is set as: maximizing the offloading of operations in the basic operation set to the fast path; The first constraint condition is set as: meeting the switch resource limit; The second constraint condition is set as: following the execution dependency between operations, and these targets are achieved by converting the basic operation set division problem into an optimization problem; The input is: (1) each task consisting of a set of operations is represented by a ; each operation corresponds to one of the operations that can be implemented in the fast path. (2) a part of the switch resource consumed by each operation, that is, the analysis result of the switch resource consumption; (3) a set Φ recording the dependency between different operations; The output is a set {x(O)}: if O is located on the fast path, x(O) = 1; otherwise, x(O) = 0; The goal of each task is to offload as much of the operation as possible onto the fast path: The constraint needs to ensure the successful deployment of operations, including: 1) switch resource consumption, for all operations offloaded to the fast path, that is, operations satisfying x(O) = 1, the cumulative resource consumption should be less than the capacity of the fast path, the cumulative resource consumption is obtained by adding the resource consumption analysis result, and the capacity of the fast path is directly obtained from the report of the switch compiler; 2) Perform dependencies, to maintain dependencies, maintain each dependency O in Φ j → O i ; Case 1: x(O i ) = 1, that is, operation O i It is placed on the fast path, where it depends on O. i Operation O j It can be placed on either the fast path or the normal path; if x(O j ) = 1, that is, O j If it is also placed on the fast path, then the placement of O needs to be restricted. j The first switch pipeline stage must be located in the O-place i After the last level; if x(O j ) = 0, that is, O j If it is located on a normal path, no additional restrictions are introduced; Case 2: x(O i ) = 0, i.e. operation O i is placed on the normal path, in which case it must be guaranteed that x(O j ) = 0 holds.

6. The Sketch data processing method based on programmable switch and DPU cooperation according to claim 1, characterized in that, The process of deploying the operations on the fast path and the operations on the normal path obtained by division includes: For a set converting its operations into a fast path acceptable data plane program, including: Each operation is converted into a code element corresponding to the fast path implementation thereof; The elements in the fast path control flow are chained according to the original execution order of the operations in the task; The format of the input data in the header is defined, at the beginning, the fast path extracts the value from the input data and records it in the header, then it passes the header to the elements in the control flow to perform operations; Coordination logic is inserted into the end of the fast path to implement correct inter-device task execution; The data plane program is input into the switch compiler, and the switch compiler converts the data plane program into a bottom layer configuration and loads the bottom layer configuration onto the switch application-specific integrated circuit pipeline; For a set The operations of the task are converted into a C++ program and executed on the normal path, different operations are linked in the C++ program according to the order between the operations in the task; the C++ program is directly run on the smart NIC CPU kernel of the normal path, and if the normal path has multiple instances, the same C++ program is run for each instance.

7. The Sketch data processing method based on programmable switch and DPU cooperation according to claim 1, characterized in that, The process of cooperation between the fast path and the normal path according to the optimal offloading scheme of the basic operation set on the fast path and the normal path to complete the Sketch data processing task includes: State packet processing: creating a synthetic state packet on the fast path, the state packet is used to retrieve the results of offloaded operations, that is, merge, maximum and minimum, the results of these operations are stored in registers, each state packet has a register address, and is indexed to the correct register through the address, and loads the register data on the header field of the packet, if all the state packets have been sent to the normal path, the fast path completes the state packet processing task; Rate control: The rate of sending state packets or Sketch data needs to be controlled to avoid saturating the normal path, and to let C i denote the processing capacity of the ith normal path instance, and there are K instances of the normal path to process the given task in parallel, the fast path is connected to the K instances through K ports, the fast path first decides which instance to send the state packet or key-value pair to, use P to denote the state packet or key-value pair, use a loop to assign P to one of the K instances, while also allowing the user to use other methods for assignment; second, for the ith normal path instance, the fast path records the time when P is first sent to the instance as When subsequent P is sent to the ith instance, the fast path records the current time as t i , let l denote the size of P, if then releasing P may overload the ith instance, at this time the fast path reloops P until the inequality does not hold, once it releases P, it changes to t i .

8. An electronic device comprising a memory and a processor, characterized in that The memory is coupled with the processor; wherein the memory is configured to store program data, and the processor is configured to execute the program data to implement the Sketch data processing method based on cooperation of a programmable switch and a DPU according to any one of claims 1-7.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, The program is executed by the processor to implement the Sketch data processing method based on programmable switch cooperation with DPU as claimed in any one of claims 1-7.

Images