A pipeline-oriented network processing workflow scheduling method, device and medium

Abstract

The present invention discloses a pipeline-oriented network processing workflow scheduling method, device, and medium, wherein the method includes: generating a DAG and generating a global resource usage document; serializing the network processing workflow; training a network work scheduling model; and forming a network workflow scheduling strategy. The pipeline-oriented network processing workflow scheduling method proposed by the present invention can handle network processing workflows of variable length and network processing resources of variable scale, and utilizes a reinforcement learning model to simultaneously implement intelligent network processing resource planning and intelligent work scheduling for network processing workflows, thereby improving the overall utilization of network processing resources and reducing the waste of network processing resources. The present invention can be widely applied to fields such as reinforcement learning, parallel computing, and work scheduling.

Description

Technical Field

[0001] The present invention relates to the fields of reinforcement learning, parallel computing and work scheduling, and in particular to a pipeline-oriented network processing workflow scheduling method, device and medium. Background Art

[0002] With the development of software-defined networking (SDN) technology, more and more network processing facilities, devices, and components are opening up programming interfaces to users. These devices typically deploy user-defined network data processing flows into a processing pipeline, which then processes user-specified network packets.

[0003] Network processing processes that can be executed in parallel can generally be described using network processing workflows. To reduce the configuration and deployment costs of hierarchical layers, it is particularly important to improve resource utilization of network processing workflows through workflow scheduling and reduce the number of hierarchical layers. Among traditional workflow scheduling methods, column scheduling algorithms, such as the method used by H. Topcuoglu et al. in "Performance-Effective and Low-Complexity Task Scheduling for Heterogeneous Computing," are widely used in workflow scheduling. These algorithms often require serializing workflows and then using heuristic algorithms such as the earliest completion time algorithm to schedule the workflows.

[0004] In recent years, the development of artificial intelligence technologies such as machine learning and deep learning has provided new approaches for scheduling workflow DAGs (Database Availability Groups). As an important machine learning technique, reinforcement learning has been widely used in workflow DAG scheduling in recent years. In "Adaptive DAG tasksscheduling with deep reinforcement learning," Qing Wu et al. used a policy gradient-based deep reinforcement learning method to implement workflow DAG scheduling on multiple processors, achieving better scheduling results than existing scheduling algorithms. Generally speaking, once the state representation, action space, and reward generation mechanism for workflow DAG scheduling are defined, a reinforcement learning model can be used to generate more intelligent workflow scheduling strategies.

[0005] Currently, many existing scheduling technologies suffer from insufficient flexibility. Most existing workflow scheduling methods are unable to handle complex constraints or struggle to implement workflow scheduling in scenarios with variable resource quantities. Furthermore, many scheduling algorithms rely on a sequence of scheduled tasks. However, the generation process for these tasks considers fewer factors and is less intelligent. This generated sequence may not be conducive to generating optimal scheduling strategies. Summary of the Invention

[0006] In order to solve at least one of the technical problems existing in the prior art to a certain extent, the present invention aims to provide a pipeline-oriented network processing workflow scheduling method, device and medium.

[0007] The technical solution adopted in the present invention is:

[0008] A pipeline-oriented network processing workflow scheduling method includes the following steps:

[0009] S1. Generation of DAG and Global Resource Usage Document: Based on the network processing workflow description document, the network processing workflow operation process and the global resource usage of each network processing task are analyzed to form a directed acyclic graph (DAG) document of the network processing workflow and a global resource usage document;

[0010] S2. Serialization of network processing workflows: Further dependency analysis and synthesis are performed on the network processing workflow DAG document and global resource usage document to form a comprehensive dependency graph. A topological sorting method based on importance is then used to serialize the integrated dependency DAG nodes to form a sequence of tasks to be scheduled.

[0011] S3. Training of the network work scheduling model: Using the sequence of work to be scheduled generated in step S2 as the initial sequence, a network processing workflow scheduling Q network is trained using a policy gradient-based reinforcement learning method in combination with constraints;

[0012] S4. Formation of network workflow scheduling strategy: Taking the sequence of tasks to be scheduled generated in step S2 as the initial sequence, the trained network processing workflow scheduling Q network is used to obtain scheduling actions in sequence, and the final network processing workflow scheduling strategy is formed after integration.

[0013] Furthermore, the step S1 includes:

[0014] S11, analyzing the network processing workflow description document, obtaining multiple tasks included in the network processing workflow and each task T i The computing resource requirements R i ;

[0015] S12. Analyze the network processing workflow description document to obtain the work sequence dependency between each task; construct a network processing workflow DAG with each task in the network processing workflow as a node and the execution sequence of each task as a directed edge; combine the network processing workflow DAG with the computing resource requirements of the task represented by each node to form a DAG document of the network processing workflow;

[0016] S13. Analyze the network processing workflow description document to obtain the usage requirements of each task for specific global resources and form a global resource usage document.

[0017] Furthermore, each line of the DAG document consists of a triple (T i ,R i ,B i ) means that B i For network processing workflow DAG, the work is represented by T i The set of tasks represented by the nodes pointed to by the edges starting from the node;

[0018] Each line of the global resource usage document consists of a tuple (T i ,G i ) indicates that T i For work, G i For work T i A collection of information about the global resources used; G i Each element in is a tuple (X,u), representing the work T i Use global resource X through method u.

[0019] Furthermore, the step S2 includes:

[0020] S21. Analyze the DAG document and the global resource usage document, and generate dependency graphs of different types of dependency relationships based on the definitions of different types of dependency relationships according to the constraint conditions. The dependency graph is a directed graph, whose nodes are tasks in the network processing workflow, and the tasks represented by the end nodes of the edges are dependent on the tasks represented by the start nodes of the edges. Combine all dependency graphs of different types of dependency relationships to form a dependency graph set V.

[0021] S22. Merge all dependency graphs in the dependency graph set V to generate a comprehensive dependency graph Z. During the merging process, convert all dependency relationships into a priority order of the tasks, and merge the dependency graphs according to the priority order.

[0022] S23. Calculate the importance of each task using an importance calculation algorithm based on the computing resource requirements of each task in the DAG document and the dependency graph set V.

[0023] S24. Generate a topological sort A with importance sorting based on the comprehensive dependency graph Z and the importance of the workZ .

[0024] Furthermore, the step S24 includes:

[0025] S241. Obtain tasks represented by all nodes that are not end nodes of edges in the comprehensive dependency graph Z, and sort these tasks in descending order according to their importance.

[0026] S242, adding the sorted tasks to the topological sort list A, and deleting the nodes representing these tasks and their associated edges from the comprehensive dependency graph Z;

[0027] S243, repeat the above steps S241-S242 until Get topological sort A with importance sorting Z , and use it as the work sequence to be scheduled.

[0028] Furthermore, the step S3 includes:

[0029] S31. Initialize the learning environment of the reinforcement learning model and the Q network parameters in the agent using the work sequence to be scheduled as the initial work sequence to be scheduled and the empty scheduling strategy as the initial strategy. The scheduling strategy is a sequence, each level represents a planned pipeline level, and each level can place multiple network processing tasks in a disordered manner.

[0030] S32, the agent takes out the current work sequence to be scheduled W and the current scheduling strategy S from the environment, and obtains the first work to be scheduled T from the work sequence to be scheduled m As the current work to be scheduled;

[0031] S33, Agent obtains work T from DAG document m The computing resource requirements R m , and combines the comprehensive dependency graph Z and the current scheduling strategy S to calculate the work T m Features F under the current scheduling policy S m ; Combination R m With F m , which constitutes the current work to be scheduled T m The static state K=[R m ,F m ];

[0032] S34. The agent searches for the computing resource requirements of each job in the current work sequence W to be scheduled, and combines them to form the resource requirement sequence W′ of the current work sequence to be scheduled; and calculates the used computing resource characteristics S′ of each level of the current scheduling strategy S in combination with the constraints;

[0033] S35. The agent constructs the current state H = (W′, S′, K);

[0034] S36, Agent combines the current scheduling strategy S in the current state H, the static state K, and the current scheduled work T m , and the constraints in the scheduling process construct masks M1 and M2; where mask M1 is a binary vector whose length is the same as the number of scheduling action types that can be taken for the current scheduled work. The pth bit of M1 is 1, indicating that the current scheduling action of type p can be taken, and the pth bit is 0, indicating that the current scheduling action of type p cannot be taken. Mask M2 is a binary vector with the same length as the number of planned levels of the current scheduling strategy S, and the qth bit is 0, indicating that the current scheduled work T m It cannot be placed at the qth level of the current scheduling strategy S. The qth bit is 1, indicating that the current scheduled work T m It can be placed at the qth level of the current scheduling strategy S;

[0035] S37, the agent inputs the current state H into the Q network, and obtains two Q value vectors Q1 and Q2; each element in Q1 is the current task to be scheduled T m Take the Q value of each scheduling action type. Each element in Q2 is the current work to be scheduled T m The Q value placed at each planned level of the current scheduling strategy S;

[0036] S38, Agent maps the obtained Q value vectors Q1 and Q2 into positive value vectors Q ′ 1 and Q ′ 2, and multiply them by elements with masks M1 and M2 respectively to obtain decision vectors D1 and D2; according to the decision vector D1, the current scheduled work T is processed in a random manner or according to the preset decision rule. m The scheduling action type and generate the scheduling action P m Possible scheduling action types include adding a new layer and placing it in the newly planned layer, and placing it in the planned layer of the current scheduling policy S. If the scheduling action type is to place it in the planned layer of the current scheduling policy S, it is also necessary to calculate the layer to be placed according to the decision vector D2 in a random manner or a preset decision rule, and add it to the scheduling action P m Detailed description of

[0037] S39, Agent will schedule action P m Send to the environment; the environment will work T m Remove the current work sequence to be scheduled and follow the specific scheduling action P m Will work T m After adding the current scheduling strategy S, a penalty p is generated m , and record the quadruple Y=(W,S,P m ,p m) and then write it into the replay cache to complete the policy update;

[0038] S310, repeat steps S32 to S39 until the queue of jobs to be scheduled is empty;

[0039] S311, using the order of each task in the to-be-scheduled task sequence as the scheduling order and the quadruple in the replay buffer as data, using the policy gradient algorithm to train the Q network in the agent;

[0040] S312. Repeat steps S32 to S311 until the policy gradient algorithm and the Q network in the agent converge, and store the trained Q network as the network processing workflow scheduling Q network.

[0041] Furthermore, the step S4 includes:

[0042] S41. Initialize the reinforcement learning model learning environment using the scheduled work sequence as the initial scheduled work sequence and the empty scheduling strategy as the initial strategy. The scheduling strategy is a sequence, with each level representing a planned pipeline level, and each level can place multiple network processing tasks in a disordered manner. Use the parameter values in the network processing workflow scheduling Q network as the initial values of the corresponding parameters of the Q network.

[0043] S42, take out the current work sequence to be scheduled W, the current scheduling strategy S, and obtain the first work to be scheduled T from the work sequence to be scheduled m As the current work to be scheduled;

[0044] S43. Get job T from DAG document m The computing resource requirements R m , and combines the comprehensive dependency graph Z and the current scheduling strategy S to calculate the work T m Features F under the current scheduling policy S m ; Combination R m With F m , which constitutes the current work to be scheduled T m The static state K=[R m ,F m ];

[0045] S44. Find the computing resource requirements of each job in the current work sequence W to be scheduled, and combine them to form the resource requirement sequence W′ of the current work sequence to be scheduled; and calculate the used computing resource characteristics S′ of each level of the current scheduling strategy S in combination with the constraints.

[0046] S45. Construct the current state H = (W′, S′, K);

[0047] S46, combining the current scheduling strategy S in the current state H, the static state K, and the current scheduled work T mAnd the constraints are used to construct masks M1 and M2; mask M1 is a binary vector whose length is the same as the number of scheduling actions that can be taken by the current scheduled task. The p-th bit of M1 is 1, indicating that the current scheduling action of type p can be taken, and the p-th bit is 0, indicating that the current scheduling action of type p cannot be taken. Mask M2 is a binary vector with the same length as the number of planned levels of the current scheduling strategy S, and the q-th bit is 0, indicating that the current scheduled task T m It cannot be placed at the qth level of the current scheduling strategy S. The qth bit is 1, indicating that the current scheduled work T m It can be placed at the qth level of the current scheduling strategy S;

[0048] S47, input the current state H into the Q network, and obtain two Q value vectors Q1 and Q2; each element in Q1 is the current work to be scheduled T m Take the Q value of each scheduling action type. Each element in Q2 is the current work to be scheduled T m The Q value placed at each planned level of the current scheduling strategy S;

[0049] S48, mapping the obtained Q value vectors Q1 and Q2 to positive value vectors Q ′ 1 and Q ′ 2, and multiply them by the elements of mask M1 and M2 respectively to get decision vectors D1 and D2. According to the decision vector D1, the current scheduled work T is processed according to certain decision rules. m The scheduling action type and generate the scheduling action P m ; Possible scheduling action types include adding a new level and placing it in the newly planned level, and placing it in the planned level of the current scheduling strategy S. If the scheduling action type is to place it in the planned level of the current scheduling strategy S, it is also necessary to calculate the level to be placed according to the decision vector D2 according to certain decision rules and add it to the scheduling action P m ;

[0050] S49, work T m Remove the work sequence to be scheduled and assign P m Work T m Add the current scheduling strategy S to complete the strategy update;

[0051] S410, repeat steps S42 to S49 until the queue of jobs to be scheduled is empty;

[0052] S411: Output the current scheduling strategy S, which is the final scheduling strategy for the processed network processing workflow. The hierarchy contained in the scheduling strategy is the planned network processing resources, and the placement scheme of each task on the planned resources in the scheduling strategy is the scheduling scheme for the network processing workflow.

[0053] Furthermore, the network processing workflow scheduling Q network includes two encoders E based on recurrent neural networks w and E s , a fully connected neural network N and an output neural network D based on a recurrent neural network;

[0054] Among them, the encoder E w Encode the resource requirement sequence of the scheduled work sequence to obtain the code h w ; Encoder E s Encode the characteristics of the computing resources used at each level in the scheduling strategy to obtain the encoding h s ; Encode h w 、h s After being combined with the obtained static state K and input into the neural network N, the vector Q1 and output code h are obtained. o ; Encode h o The characteristic sequence of computing resources occupied by each element of the current scheduling strategy or the relevant representation of the sequence is input into the neural network D to obtain the vector Q2.

[0055] Another technical solution adopted in the present invention is:

[0056] A pipeline-oriented network processing workflow scheduling device, comprising:

[0057] at least one processor;

[0058] at least one memory for storing at least one program;

[0059] When the at least one program is executed by the at least one processor, the at least one processor implements the above method.

[0060] Another technical solution adopted in the present invention is:

[0061] A computer-readable storage medium stores a program executable by a processor, wherein the program executable by the processor is used to perform the method described above when executed by the processor.

[0062] The beneficial effects of the present invention are: the present invention can process network processing workflows of variable length and network processing resources of variable scale, and use the reinforcement learning model to simultaneously realize intelligent network processing resource planning and intelligent work scheduling for network processing workflows, thereby improving the overall utilization of network processing resources and reducing the waste of network processing resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following introduction is made to the drawings of the embodiments of the present invention or the related technical solutions in the prior art. It should be understood that the drawings introduced below are only for the convenience of clearly describing some embodiments of the technical solutions of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative work.

[0064] Figure 1 This is a flow chart of a pipeline-oriented network processing workflow scheduling method according to an embodiment of the present invention;

[0065] Figure 2 This is a diagram of a multi-stage pipeline process for processing messages using a protocol-independent switch fabric chip based on a P4 program in an embodiment of the present invention;

[0066] Figure 3 This is a structural diagram of a Q network that can be used to schedule P4 program basic blocks to a multi-stage pipeline of a protocol-independent switching architecture chip in an embodiment of the present invention. DETAILED DESCRIPTION

[0067] The embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention and are not to be construed as limiting the present invention. The step numbers in the following embodiments are provided for ease of explanation only and do not limit the order of the steps. The order of execution of the steps in the embodiments can be adaptively adjusted according to the understanding of those skilled in the art.

[0068] In the description of the present invention, it should be understood that descriptions involving orientations, such as up, down, front, back, left, right, etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings. They are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, they cannot be understood as limitations on the present invention.

[0069] In the description of the present invention, "several" means one or more, "many" means more than two, "greater than," "less than," and "exceed" are understood to exclude the number itself, while "above," "below," and "within" are understood to include the number itself. The use of "first" and "second" in the description is solely for the purpose of distinguishing technical features and should not be construed as indicating or implying relative importance, implicitly specifying the number of the indicated technical features, or implicitly specifying the order of the indicated technical features.

[0070] In the description of the present invention, unless otherwise clearly defined, terms such as setting, installing, and connecting should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meanings of the above terms in the present invention based on the specific content of the technical solution.

[0071] The Protocol Independent Switch Architecture (PISA) chip is a new type of switching chip. Users can use the P4 language to write network protocols and deploy them on the PISA chip to quickly implement new network protocols. The PISA chip contains a multi-stage pipeline for processing message forwarding. Each stage of the pipeline contains computing resources such as TCAM, HASH, and ALU. Figure 2 As shown, a P4 program can be converted into multiple program basic blocks that can be executed in parallel and have dependencies, and then deployed at various levels of the PISA chip pipeline to handle message forwarding. To minimize the number of pipeline levels and improve the utilization of each pipeline level and overall resources, it is necessary to rationally schedule the P4 program basic blocks to each level of the PISA chip pipeline, minimizing the length of the PISA chip pipeline occupied.

[0072] This example uses the scheduling of P4 program basic blocks on a PISA chip pipeline as a scenario to specifically illustrate a pipeline-oriented network processing workflow scheduling method proposed in this invention. This example proposes a pipeline-oriented network processing workflow scheduling method that intelligently serializes network processing workflows and uses reinforcement learning neural networks to flexibly embed network processing workflows, temporary scheduling schemes, and temporary planned hierarchies, as well as intelligently generate the final scheduling strategy. Ultimately, this method achieves a processing resource planning scheme and a network processing workflow scheduling scheme with high resource utilization for network processing workflows.

[0073] like Figure 1 As shown, the present embodiment provides a pipeline-oriented network processing workflow scheduling method, including the steps of generating a DAG and global resource usage document, serializing the network processing workflow, training a network work scheduling model, and forming a network workflow scheduling strategy. This embodiment uses the P4 program input by the user as the network processing workflow description document and the P4 program basic blocks as the tasks in the network processing workflow. The method proposed by the present invention is applied through the following steps to implement the generation of a scheduling strategy for the P4 program basic blocks on the PISA chip pipeline:

[0074] S1. Generation of DAG and Global Resource Usage Document: Based on the P4 program, the running process of the P4 program workflow and the global variable resource usage of each P4 program basic block are analyzed to form the P4 program's directed acyclic graph (DAG) document and global variable resource usage document. The specific method steps are as follows:

[0075] S1-1. Use the P4 language compiler to analyze the P4 program and obtain multiple P4 program basic blocks contained in the P4 program and each program basic block T i The computing resource requirements R i Computational resource requirements R i =[TCAM i ,HASH i ,ALU i ], where TCAM i 、HASH i and ALU i They are program basic blocks T i The demand for computing resources such as TCAM, HASH, and ALU;

[0076] S1-2. Use the P4 language compiler to analyze the P4 program and obtain the execution order dependency of each program basic block. Use the program basic block as a node and the execution order of each program basic block as a directed edge to build the P4 program DAG. Combine the P4 program DAG with the computing resource requirements of the program basic block represented by each node to form the DAG document of the P4 program. Each line of the DAG document consists of a triple (T i ,R i ,B i ) means that B i The P4 program DAG is represented by a program basic block T i The set of program basic blocks represented by the nodes pointed to by the edges starting from the node;

[0077] S1-3. Use the P4 language compiler to analyze the P4 program, obtain the read and write requirements of the global variable resources of each program block, and form a global variable resource usage document. Each line of the document consists of a tuple (T i ,G i ) indicates that T i is the basic block of the program, G i is the basic block T of the program i A collection of information about the global variable resources used. i Each element in is a tuple (X,u), representing a basic block T of the program i Use global variable X in mode u, where u can be a read or write operation.

[0078] S2. Serialization of network processing workflow: Further dependency analysis and synthesis are performed on the P4 program DAG document and global variable resource usage document to form a comprehensive dependency graph. A topological sorting-based method is then used to serialize the synthesized dependency DAG nodes to form a sequence of basic blocks of the P4 program to be scheduled. The specific steps are as follows:

[0079] S2-1. Analyze the P4 program DAG document generated in step S1-2 and the global variable resource usage document generated in step S1-3. Generate dependency graphs for four types of dependencies, including control dependency, read-after-write data dependency, write-after-read data dependency, and write-after-write data dependency, based on the constraints. The dependency graph is a directed graph whose nodes are P4 program basic blocks. The program basic block represented by the end node of an edge depends on the program basic block represented by the start node of the edge. Combine all dependency graphs of different dependency types to form a dependency graph set V.

[0080] S2-2 merge step S2-1 generated dependency graph set V all dependency graph, generate a comprehensive dependency graph Z, in the merging process, all dependencies are converted into the order of the basic blocks of the program, and merge the dependency graph according to the order;

[0081] S2-3. Based on the computing resource requirements of each program basic block on TCAM, HASH, and ALU in the P4 program DAG document generated in step S1-2, the out-degree of each node in the dependency graph of the four types of dependency relationships, including control dependency, read-after-write data dependency, write-after-read data dependency, and write-after-write data dependency, and the minimum depth required for the basic block to meet the write-after-read data dependency and write-after-write data dependency, use the superior-inferior solution distance algorithm to calculate the importance of each program basic block;

[0082] S2-4. Generate a topological sorting V with importance sorting based on the comprehensive dependency graph Z of step S2-2 and the importance of the program basic blocks of step S2-3 z Obtain the program basic blocks represented by all nodes in the integrated dependency graph Z that are not the end nodes of the edges, and sort these program basic blocks in descending order according to the importance of each program basic block. Add the sorted program basic blocks to the topological sort list A, and delete the nodes representing these program basic blocks and their associated edges from the integrated dependency graph Z. Repeat the above process until Get topological sort A with importance sorting z , which is used as the basic block sequence of the P4 program to be scheduled.

[0083] Before proceeding further, you need to Figure 3 Define the Q network structure used in the subsequent steps. Figure 3 As shown, the Q network used in this embodiment to implement P4 program resource planning and basic block scheduling includes two variational autoencoders E based on a bidirectional long short-term memory (Bidirectional Long Short-Term Memory, BiLSTM) network encoder and decoder. w and E s, a fully connected neural network N and an LSTM-based output neural network D. Variational autoencoder E w and E s The input sequence is encoded using the Transformer encoding layer and then fed into the BiLSTM encoder.

[0084] S3. Training of the network work scheduling model: Using the sequence of basic blocks of the P4 program to be scheduled generated in step S2 as the initial sequence, a P4 program resource planning and basic block scheduling Q network is trained using a policy gradient-based reinforcement learning method in combination with constraints. The specific steps are as follows:

[0085] S3-1. Initialize the reinforcement learning model learning environment and the Q network parameters in the agent using the P4 program basic block sequence to be scheduled generated in step S2-4 as the initial program basic block sequence to be scheduled and the empty scheduling strategy as the initial strategy. The scheduling strategy is a sequence, with each level representing a planned PISA chip pipeline level. Each level can place multiple network processing tasks in an unordered manner.

[0086] S3-2. The agent takes the current basic block sequence W of the program to be scheduled and the current scheduling policy S from the environment, and obtains the first basic block T of the program to be scheduled from the basic block sequence of the program to be scheduled m As the basic block of the program currently to be scheduled;

[0087] S3-3. The agent obtains the program basic block T from the P4 program DAG document generated in step S1-2 m The computing resource requirements R m , and calculate the program basic block T from the comprehensive dependency graph Z generated in step S2-2 m The minimum level M that can be placed in the current scheduling strategy S m . Calculate the length L of the basic block sequence of the current program to be scheduled w and the length L of the current scheduling strategy S s , construct the basic block feature F of the current program to be scheduled m =[M m ,L w ,L s ] Combination R m With F m , construct the static state K of the basic block of the current program to be scheduled = [R m ,F m ]=[TCAM i ,HASH i ,ALU i ,M m ,L w ,L s ];

[0088] S3-4. The agent searches for the computing resource requirements of each basic block in the current basic block sequence W to be scheduled, and combines them to form the resource requirement sequence W′ of the current basic block sequence to be scheduled. The e-th element in the W′ sequence is a triple [TCAM e ,HASH e ,ALU e ], which represents the computing resource requirement R of the e-th program basic block in the current sequence of program basic blocks to be scheduled e Combined with the single-level computing resource limit and cross-level computing resource limit in the constraints, calculate the computing resource characteristics S′ of each level of the current scheduling strategy S. The characteristic S′ is a sequence, and the e-th element in the sequence is a triple [TCAM e ,HASH e ,ALU e ], which represents the amount of TCAM, HASH, and ALU computing resources remaining available at the e-th level of the current scheduling strategy S while satisfying the single-level computing resource limit and cross-level computing resource limit in the constraints;

[0089] S3-5. The agent constructs the current state H = (W′, S′, K);

[0090] S3-6.Agent combines the current scheduling strategy S in the current state H and L in the static state K m , the basic block of the program currently to be scheduled T m The constraints in the scheduling process are used to construct masks M1 and M2. The constraints that can be considered include single-level computing resource limitations, cross-level computing resource limitations, control dependency restrictions on the hierarchical order of program basic blocks, and different types of data dependencies on the hierarchical order of program basic blocks. Mask M1 is a binary vector of length 2, the first bit of which indicates the creation of a new level and the addition of program basic block T m Place it in the new level that meets the constraints. The second position indicates that the program basic block T m Placed in the existing level of the current scheduling strategy S. The pth bit of M1 is 1, indicating that the current scheduling action of type p can be taken, and the pth bit is 0, indicating that the current scheduling action of type p cannot be taken. Mask M2 is a binary vector with the same length as the number of existing levels of the current scheduling strategy S. The qth bit is 0, indicating that the current basic block T to be scheduled m It cannot be placed at the qth level of the current scheduling strategy S. The qth bit is 1, indicating that the current scheduled program basic block T m It can be placed at the qth level of the current scheduling strategy S;

[0091] S3-7.Agent inputs the current state H into the Q network and obtains two Q value vectors Q1 and Q2. Each element in Q1 is the basic block T of the current program to be scheduled.m Take the Q value of each scheduling action type. Each element in Q2 is the basic block T of the current program to be scheduled. m The Q value placed at each level of the current scheduling strategy S. Figure 3 As shown, the process of inputting the Q network to obtain two Q value vectors is:

[0092] S3-7-1. Use the variational autoencoder E in the Q network to convert the resource requirement sequence W′ of the basic block sequence of the current program to be scheduled w Encoding, take the hidden state of the final time step of the BiLSTM encoder as the encoding h w ;

[0093] S3-7-2. Use the variational autoencoder E in the Q network to calculate the feature sequence S′ of the used computing resources at each level obtained by the current scheduling strategy S s The Transformer encoding layer in the code is encoded to obtain the temporary representation f of S′ s ′, and feed it into the variational autoencoder E s The BiLSTM encoder takes the hidden state of the final time step of the BiLSTM encoder as the encoding h s ;

[0094] S3-7-3. The code h obtained in step S3-7-1 w 、The code h obtained in step S3-7-2 s The static state K obtained in step S3-3 is input into the fully connected neural network N in the Q network to obtain the output code h o and the Q value vector Q1 described in step S3-7;

[0095] S3-7-4. Encode the output obtained in step S3-7-3 h o and the temporary representation f obtained in step S3-7-2 s ′Input the LSTM output neural network D in the Q network to obtain the Q value vector Q2 described in step S3-7.

[0096] S3-8.Agent uses the Sigmoid function to map the Q value vectors Q1 and Q2 into positive value vectors Q′1 and Q′2, and multiplies them element-wise with the masks M1 and M2 respectively to obtain decision vectors D1 and D2. According to the decision vector D1, the current scheduled program basic block T is processed in a random manner or according to the dual Q network action generation rule. m The scheduling action type and generate the scheduling action P m , its possible action type is to convert the program basic block T m Place it in a new layer that does not violate the constraints, and place the program basic block T mPlace it in the existing level of the current scheduling strategy S. If the scheduling action type is placed in the existing level of the current scheduling strategy S, it is also necessary to calculate the level to be placed according to the decision vector D2 in a random manner or the dual Q network action generation rule, and add it to the scheduling action P m In the specific description of

[0097] S3-9.Agent will schedule action P m The environment sends the basic block T to be scheduled. m Remove the basic block sequence of the current program to be scheduled and follow the specific scheduling action P m The basic block T m After adding the current scheduling strategy S, a penalty p is generated m , and record the quadruple Y=(W,S,P m ,p m ) and then write it into the replay cache. The specific steps are:

[0098] S3-9-1. If action P n To convert the program basic block T m Put it into the existing level of the current scheduling strategy S, then the program basic block T m Put in the scheduling action P m The specific description of the level. If the action P m To convert the program basic block T m Put in the new level, then use the greedy algorithm to create a new level in the current scheduling strategy S until the program basic block T m Can be placed into the new hierarchy without violating any constraints;

[0099] S3-9-2. Calculate the penalty p m , whose value is the opposite of the number of newly created levels in step S3-9-1. If step S3-9-1 does not create a new level in the current scheduling strategy S, the penalty is 0;

[0100] S3-9-3. Record the quaternion Y = (W, S, P m ,p m ) and then write it to the replay cache.

[0101] S3-10. Repeat steps S3-2 to S3-9 until the basic block queue to be scheduled is empty;

[0102] S3-11. Use the order of each basic block in the sequence of basic blocks to be scheduled generated in step S2-4 as the scheduling order, and use the four-tuple in the replay buffer as data to train the Q network in the Agent using the policy gradient algorithm. Each time, first train the variational autoencoder E w and E S, and then train the fully connected neural network N and the output neural network D;

[0103] S3-12. Repeat steps S3-2 to S3-11 until the Q network in the Agent converges, and store the trained Q network as the P4 program resource planning and basic block scheduling Q network.

[0104] S4. Formation of workflow scheduling strategy: Using the basic block sequence of the program to be scheduled generated in step S2 as the initial sequence, the trained P4 program resource planning and basic block scheduling Q network is used to obtain scheduling actions in sequence, and the final network processing workflow scheduling strategy is formed after integration. The specific method steps are as follows:

[0105] S4-1. Initialize the reinforcement learning model learning environment using the P4 program basic block sequence to be scheduled generated in step S2-4 as the initial program basic block sequence to be scheduled and the empty scheduling strategy as the initial strategy. The scheduling strategy is a sequence, each level represents a planned PISA chip pipeline level, and each level can place multiple network processing tasks in an unordered manner. The parameter values in the P4 program resource planning and basic block scheduling Q network obtained in step S3-12 are used as the initial values of the corresponding parameters of the Q network;

[0106] S4-2. Take out the current basic block sequence W of the program to be scheduled, the current scheduling policy S, and obtain the first basic block T of the program to be scheduled from the basic block sequence of the program to be scheduled m As the basic block of the program currently to be scheduled;

[0107] S4-3. Obtain the program basic block T from the P4 program DAG document generated in step S1-2 m The computing resource requirements R m , and calculate the program basic block T from the comprehensive dependency graph Z generated in step S2-2 m The minimum level M that can be placed in the current scheduling strategy S m . Calculate the length L of the basic block sequence of the current program to be scheduled w and the length L of the current scheduling strategy S s , construct the basic block feature F of the current program to be scheduled m =[M m ,L w ,L s ] Combination R m With F m , construct the static state K of the basic block of the current program to be scheduled = [R m ,F m ]=[TCAM i ,HASH i ,ALU i ,M m ,L w ,Ls ];

[0108] S4-4. Find the computing resource requirements of each job in the basic block sequence W of the current program to be scheduled, and combine them to form the resource requirement sequence W' of the basic block sequence of the current program to be scheduled. The e-th element in the W' sequence is a triple [TCAM e ,HASH e ,ALU e ], which represents the computing resource requirement R of the e-th program basic block in the current sequence of program basic blocks to be scheduled e Combined with the single-level computing resource limit and cross-level computing resource limit in the constraints, calculate the characteristic sequence S′ of the computing resources used at each level of the current scheduling strategy S. The e-th element in the S′ sequence is a triple [TCAM e ,HASM e ,ALU e ], which represents the amount of TCAM, HASH, and ALU computing resources remaining available at the e-th level of the current scheduling strategy S while satisfying the single-level computing resource limit and cross-level computing resource limit in the constraints;

[0109] S4-5. Construct the current state H = (W', S', K);

[0110] S4-6. Combine the current scheduling strategy S in the current state H and the L in the static state K m , the basic block of the program currently to be scheduled T m The constraints considered in step S3-6 are used to construct masks M1 and M2. The constraints that can be considered include single-level computing resource restrictions, cross-level computing resource restrictions, restrictions on the hierarchical order of program basic blocks due to control dependencies, and restrictions on the hierarchical order of program basic blocks due to different types of data dependencies. Mask M1 is a binary vector of length 2, the first bit of which indicates that a new level is created and the program basic block T is added. m Place it in the new level that meets the constraints. The second position indicates that the program basic block T m Placed in the existing level of the current scheduling strategy S. The pth bit of M1 is 1, indicating that the current scheduling action of type p can be taken, and the pth bit is 0, indicating that the current scheduling action of type p cannot be taken. Mask M2 is a binary vector with the same length as the number of existing levels of the current scheduling strategy S. The qth bit is 0, indicating that the current basic block T to be scheduled m It cannot be placed at the qth level of the current scheduling strategy S. The qth bit is 1, indicating that the current scheduled program basic block T m It can be placed at the qth level of the current scheduling strategy S;

[0111] S4-7. Input the current state H into the Q network to obtain two Q value vectors Q1 and Q2. Each element in Q1 is the basic block T of the current program to be scheduled. m Take the Q value of each scheduling action type. Each element in Q2 is the basic block T of the current program to be scheduled. m The Q value placed at each level of the current scheduling strategy S. Figure 3 As shown, the process of inputting the Q network to obtain two Q value vectors is:

[0112] S4-7-1. Use the variational autoencoder E in the Q network to convert the resource requirement sequence W′ of the basic block sequence of the current program to be scheduled w Encoding, take the hidden state of the final time step of the BiLSTM encoder in the encoder as the encoding h w ;

[0113] S4-7-2. Encode the feature sequence S′ of the used computing resources at each level obtained by the current scheduling strategy S using another Transformer encoding layer in the Q network to obtain a temporary representation f of S′ s ′, and feed it into the variational autoencoder E in the Q network s The BiLSTM encoder takes the hidden state of the final time step of the BiLSTM encoder as the encoding h s ;

[0114] S4-7-3. The code h obtained in step S4-7-1 w 、The code h obtained in step S4-7-2 s The static state K obtained in step S4-3 is input into the fully connected neural network N in the Q network to obtain the output code h o and the Q value vector Q1 described in step S4-7;

[0115] S4-7-4. Encode the output obtained in step S4-7-3 h o and the temporary representation f obtained in step S4-7-2 s ′Input the LSTM output neural network D in the Q network to obtain the Q value vector Q2 described in step S4-7.

[0116] S4-8. Use the Sigmoid function to map the Q value vectors Q1 and Q2 into positive value vectors Q′1 and Q′2, and multiply them element-wise with the masks M1 and M2 respectively to obtain decision vectors D1 and D2. According to the decision vector D1, the dual Q network action generation rule is used to obtain the basic block T of the program to be scheduled. m The scheduling action type and generate the scheduling action P m , its possible action type is to convert the program basic block T m Place it in a new layer that does not violate the constraints, and place the program basic block T mPlace it in the existing level of the current scheduling strategy S. If the scheduling action type is placed in the existing level of the current scheduling strategy S, it is also necessary to calculate the level to be placed according to the decision vector D2 according to the dual Q network action generation rule and add it to the scheduling action P m In the specific description of

[0117] S4-9. The basic block T of the program to be scheduled m Remove the basic block sequence of the program to be scheduled and assign P to it according to the specific scheduling action. m Program basic block T m Add the current scheduling strategy S to complete the environment update. m The process of joining the current scheduling strategy S is: if action P m To convert the program basic block T m Put it into the existing level of the current scheduling strategy S, then the program basic block T m Put in the scheduling action P m The specific description of the level. If the action P m To convert the program basic block T m Put in the new level, then use the greedy algorithm to create a new level in the current scheduling strategy s until the program basic block T m Can be placed into the new hierarchy without violating any constraints;

[0118] S4-10. Repeat steps S4-2 to S4-9 until the basic block queue to be scheduled is empty;

[0119] S4-11. Output the current scheduling strategy s, which is the final scheduling strategy for the program basic blocks generated by the processed P4 program on the multi-stage pipeline of the PISA chip. The pipeline hierarchy contained in the scheduling strategy is the PISA pipeline planning hierarchy for the input P4 program, and the placement of each program basic block at each pipeline level in the scheduling strategy is the scheduling plan for the P4 program basic blocks.

[0120] In summary, compared with the prior art, the embodiments of the present invention have at least the following advantages and beneficial effects:

[0121] (1) The present invention achieves intelligent serialization of network processing workflows. While most workflow serialization algorithms only apply a topological sorting algorithm to the workflow DAG to generate a sequence of tasks to be scheduled, the method proposed in the present invention applies a topological sorting algorithm that incorporates task importance to a comprehensive dependency graph that integrates multiple dependencies to generate a sequence of tasks to be scheduled. The generated sequence of tasks to be scheduled more comprehensively considers the computational resource requirements and dependencies of each task in the workflow, thereby increasing the possibility of obtaining a scheduling strategy with higher resource utilization.

[0122] (2) The present invention realizes the simultaneous intelligent planning of processing resources and workflow scheduling. Most workflow scheduling algorithms only consider the scheduling of work on a fixed amount of processing resources, but do not consider the planning of resources. The Q network used in the method proposed in the present invention realizes the flexible embedding of variable-scale planned processing resources and variable-length to-be-scheduled work sequences through two encoders based on recurrent neural networks, so that resource planning and work scheduling can be carried out simultaneously and produce more intelligent resource planning and work scheduling strategies, thereby reducing resource waste.

[0123] (3) The present invention enables the flexible introduction of constraints in the workflow scheduling process. The method proposed in the present invention introduces a mask that represents the validity of scheduling actions during the generation of scheduling actions, thereby enabling a unified representation of complex constraints. Users can generate a mask based on the constraints, the current work to be scheduled, and the current scheduling policy, thereby avoiding the generation of scheduling policies that violate constraints during resource planning and work scheduling.

[0124] The present invention also provides a pipeline-oriented network processing workflow scheduling device, comprising:

[0125] at least one processor;

[0126] at least one memory for storing at least one program;

[0127] When the at least one program is executed by the at least one processor, the at least one processor implements Figure 1 The method shown.

[0128] A pipeline-oriented network processing workflow scheduling device of this embodiment can execute a pipeline-oriented network processing workflow scheduling method provided by the method embodiment of the present invention, can execute any combination of implementation steps of the method embodiment, and has the corresponding functions and beneficial effects of the method.

[0129] The present application also discloses a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs Figure 1 The method shown.

[0130] This embodiment also provides a storage medium that stores instructions or programs that can execute a pipeline-oriented network processing workflow scheduling method provided by an embodiment of the method of the present invention. When the instructions or program are run, any combination of implementation steps of the method embodiment can be executed, and the corresponding functions and beneficial effects of the method can be obtained.

[0131] In some optional embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the present invention is provided in an exemplary manner for the purpose of providing a more comprehensive understanding of the technology. The disclosed method is not limited to the operation and logic flow presented herein. Optional embodiments are contemplated in which the order of the various operations is changed and the sub-operations described as a part of a larger operation are performed independently.

[0132] Furthermore, although the present invention is described in the context of functional modules, it should be understood that, unless otherwise indicated, one or more of the functions and / or features described may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in separate physical devices or software modules. It will also be understood that a detailed discussion of the actual implementation of each module is not necessary for understanding the present invention. More specifically, given the properties, functions, and internal relationships of the various functional modules in the devices disclosed herein, the actual implementation of the module will be understood within the ordinary skill of an engineer. Therefore, a person skilled in the art using ordinary skill will be able to implement the present invention set forth in the claims without undue experimentation. It will also be understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.

[0133] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0134] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0135] More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or more wires (electronic devices), a portable computer disk cartridge (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering, or processing in another suitable manner as necessary, and then stored in a computer memory.

[0136] It should be understood that various parts of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above-described embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0137] In the above description of this specification, reference to the terms "one embodiment / example," "another embodiment / example," or "certain embodiments / examples" means that the specific features, structures, materials, or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0138] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to the embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the claims and their equivalents.

[0139] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the above embodiments. Those skilled in the art can make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of this application.

Claims

1. A pipeline-oriented network processing workflow scheduling method, characterized in that: The following steps are involved: S1. Generation of DAG and Global Resource Usage Document: Based on the network processing workflow description document, the network processing workflow operation process and the global resource usage of each network processing task are analyzed to form a directed acyclic graph (DAG) document of the network processing workflow and a global resource usage document; S2. Serialization of network processing workflows: Further dependency analysis and integration are performed on the network processing workflow DAG document and global resource usage documents to form a comprehensive dependency graph. Based on the importance of the tasks and topological sorting, the integrated dependency DAG nodes are serialized to form a sequence of tasks to be scheduled. S3. Training of the network work scheduling model: Using the sequence of work to be scheduled as the initial sequence, and combining constraints with a policy gradient-based reinforcement learning method, a network processing workflow scheduling Q network is trained; S4. Formation of network workflow scheduling strategy: Taking the sequence of tasks to be scheduled as the initial sequence, the trained network processing workflow scheduling Q network is used to obtain scheduling actions in sequence, and the final network processing workflow scheduling strategy is formed after integration. The step S4 comprises: S41, using the work sequence to be scheduled as the initial work sequence to be scheduled and the air scheduling strategy as the initial strategy, completing the initialization of the reinforcement learning model learning environment; using the parameter values in the network processing workflow scheduling Q network as the initial values of the corresponding parameters of the Q network; S42, take out the current work sequence to be scheduled W, the current scheduling strategy S, and obtain the first work to be scheduled T from the work sequence to be scheduled m As the current work to be scheduled; S43. Get job T from DAG document m The computing resource requirements R m , and combines the comprehensive dependency graph Z and the current scheduling strategy S to calculate the work T m Features F under the current scheduling policy S m ; Combination R m With F m , which constitutes the current work to be scheduled T m The static state K=[R m ,F m ]; S44. Find the computing resource requirements of each job in the current work sequence W to be scheduled, and combine them to form the resource requirement sequence W′ of the current work sequence to be scheduled; and calculate the used computing resource characteristics S′ of each level of the current scheduling strategy S in combination with the constraints. S45. Construct the current state H = (W′, S′, K); S46, combining the current scheduling strategy S in the current state H, the static state K, and the current scheduled work T m and constraints to construct masks M1 and M2; S47, input the current state H into the Q network to obtain two Q value vectors Q1 and Q2; S48, mapping the obtained Q value vectors Q1 and Q2 to positive value vectors Q ′ 1 and Q ′ 2, and multiply them by elements with masks M1 and M2 respectively to obtain decision vectors D1 and D2; according to the decision vector D1, the current scheduled work T is processed in a random manner or according to the preset decision rule. m The scheduling action type and generate the scheduling action P m If the scheduling action type is to place it in the planned level of the current scheduling strategy S, it is also necessary to calculate the level to be placed according to the decision vector D2 in a random manner or a preset decision rule, and add it to the scheduling action P m ; S49, work T m Remove the work sequence to be scheduled and follow the specific scheduling action P m Will work T m Add the current scheduling strategy S to complete the strategy update; S410, repeat steps S42 to S49 until the queue of jobs to be scheduled is empty; S411. Output the current scheduling strategy S, which is the final scheduling strategy of the processed network processing workflow.

2. A pipeline-oriented network processing workflow scheduling method according to claim 1, characterized in that: The step S1 comprises: S11, analyzing the network processing workflow description document, obtaining multiple tasks included in the network processing workflow and each task T i The computing resource requirements R i ; S12. Analyze the network processing workflow description document to obtain the work sequence dependency between each task; construct a network processing workflow DAG with each task in the network processing workflow as a node and the execution sequence of each task as a directed edge; combine the network processing workflow DAG with the computing resource requirements of the task represented by each node to form a DAG document of the network processing workflow; S13. Analyze the network processing workflow description document to obtain the usage requirements of each task for specific global resources and form a global resource usage document.

3. A pipeline-oriented network processing workflow scheduling method according to claim 2, characterized in that: Each line of the DAG document consists of a triple (T i ,R i ,B i ) means that B i For network processing workflow DAG, the work is represented by T i The set of tasks represented by the nodes pointed to by the edges starting from the node; Each line of the global resource usage document consists of a tuple (T i ,G i ) indicates that T i For work, G i For work T i A collection of information about the global resources used; G i Each element in is a tuple (X,u), representing the work T i Use global resource X through method u.

4. The pipeline-oriented network processing workflow scheduling method according to claim 1, characterized in that: The step S2 includes: S21. Analyze the DAG document and the global resource usage document, and generate dependency graphs of different types of dependency relationships based on the definitions of different types of dependency relationships according to the constraint conditions. The dependency graph is a directed graph, whose nodes are tasks in the network processing workflow, and the tasks represented by the end nodes of the edges are dependent on the tasks represented by the start nodes of the edges. Combine all dependency graphs of different types of dependency relationships to form a dependency graph set V. S22. Merge all dependency graphs in the dependency graph set V to generate a comprehensive dependency graph Z. During the merging process, convert all dependency relationships into a priority order of the tasks, and merge the dependency graphs according to the priority order. S23. Calculate the importance of each job based on the computing resource requirements of each job in the DAG document and the dependency graph set V; S24. Generate a topological sort A with importance sorting based on the comprehensive dependency graph Z and the importance of the work Z .

5. A pipeline-oriented network processing workflow scheduling method according to claim 4, characterized in that: The step S24 includes: S241. Obtain tasks represented by all nodes that are not end nodes of edges in the comprehensive dependency graph Z, and sort these tasks in descending order according to their importance. S242, adding the sorted tasks to the topological sort list A, and deleting the nodes representing these tasks and their associated edges from the comprehensive dependency graph Z; S243, repeat the above steps S241-S242 until Get topological sort A with importance sorting Z , and use it as the work sequence to be scheduled.

6. The pipeline-oriented network processing workflow scheduling method according to claim 1, characterized in that: The step S3 comprises: S31, using the work sequence to be scheduled as the initial work sequence to be scheduled and the empty scheduling strategy as the initial strategy, completing the initialization of the reinforcement learning model learning environment and the initialization of the Q network parameters in the agent; S32, the agent takes out the current work sequence to be scheduled W and the current scheduling strategy S from the environment, and obtains the first work to be scheduled T from the work sequence to be scheduled m As the current work to be scheduled; S33, Agent obtains work T from DAG document m The computing resource requirements R m , and combines the comprehensive dependency graph Z and the current scheduling strategy S to calculate the work T m Features F under the current scheduling policy S m ; Combination R m With F m , which constitutes the current work to be scheduled T m The static state K=[R m ,F m ]; S34. The agent searches for the computing resource requirements of each job in the current work sequence W to be scheduled, and combines them to form the resource requirement sequence W′ of the current work sequence to be scheduled; and calculates the used computing resource characteristics S′ of each level of the current scheduling strategy S in combination with the constraints; S35. Agent constructs the current state H = (W′, S′, K); S36, Agent combines the current scheduling strategy S in the current state H, the static state K, and the current scheduled work T m , and the constraints in the scheduling process to construct masks M1 and M2; S37, the agent inputs the current state H into the Q network and obtains two Q value vectors Q1 and Q2; S38, Agent maps the obtained Q value vectors Q1 and Q2 into positive value vectors Q ′ 1 and Q ′ 2, and multiply them by elements with masks M1 and M2 respectively to obtain decision vectors D1 and D2; according to the decision vector D1, the current scheduled work T is processed in a random manner or according to the preset decision rule. m The scheduling action type and generate the scheduling action P m If the scheduling action type is to place it in the planned level of the current scheduling strategy S, it is also necessary to calculate the level to be placed according to the decision vector D2 in a random manner or a preset decision rule, and add it to the scheduling action P m ; S39, Agent will schedule action P m Send to the environment; the environment will work T m Remove the current work sequence to be scheduled and follow the specific scheduling action P m Will work T m After adding the current scheduling strategy S, a penalty p is generated m , and record the quadruple Y=(W,S,P m ,p m ) and then write it into the replay cache to complete the policy update; S310, repeat steps S32 to S39 until the queue of jobs to be scheduled is empty; S311, using the order of each task in the to-be-scheduled task sequence as the scheduling order and the quadruple in the replay buffer as data, using the policy gradient algorithm to train the Q network in the agent; S312, repeat steps S32 to S311 until the policy gradient algorithm converges to the Q network in the agent, and store the trained Q network as the network processing workflow scheduling Q network.

7. A pipeline-oriented network processing workflow scheduling method according to claim 1 or 6, characterized in that: The network processing workflow scheduling Q network contains two encoders E based on recurrent neural networks w and E s , a fully connected neural network N and an output neural network D based on a recurrent neural network; Among them, the encoder E w Encode the resource requirement sequence of the scheduled work sequence to obtain the code h w ; Encoder E s Encode the characteristics of the computing resources used at each level in the scheduling strategy to obtain the encoding h s ; Encode h w 、h s After being combined with the obtained static state K and input into the neural network N, the vector Q1 and output code h are obtained. o ; Encode h o The characteristic sequence of computing resources occupied by each element of the current scheduling strategy or the relevant representation of the sequence is input into the neural network D to obtain the vector Q2.

8. A pipeline-oriented network processing workflow scheduling device, characterized in that: include: at least one processor; at least one memory for storing at least one program; When the at least one program is executed by the at least one processor, the at least one processor implements the method according to any one of claims 1 to 7.

9. A computer-readable storage medium storing a program executable by a processor, characterized in that: The processor-executable program is used to perform the method according to any one of claims 1 to 7 when executed by the processor.

Images