Test task scheduling method and system of control system based on graph model and deep reinforcement learning

Abstract

The invention belongs to the technical field of automatic testing, and particularly relates to a test task scheduling method and system of a control system based on a graph model and deep reinforcement learning. The method comprises the following steps: analyzing a test task demand, and then constructing a Single-rate DAG task scheduling model; according to a Single-rate DAG task scheduling model, generating a directed graph as the input of a GNN; establishing a heat map about the edge based on the directed graph and the GNN; generating a task scheduling sequence according to the heat map, and planning a scheduling scheme according to the task scheduling sequence; calculating a loss function, and updating the graph neural network according to the loss function; repeatedly establishing the heat map to calculate the loss function until a high-quality solution conforming to an expected target can be planned or a difference value between a mean value and a minimum value of the planned test scheme makespan is smaller than a certain threshold value, and ending iteration; and step 7, outputting an optimized solution. The method is used for solving the problem that a planning result of a high-correlation test task scheduling process is often high in randomness and is difficult to meet actual requirements.

Description

Technical Field

[0001] The present invention belongs to the field of automatic testing technology, and specifically relates to a test task scheduling method and a system for a control system based on a graphical model and deep reinforcement learning. Background Art

[0002] Control systems often have complex functions and structures, numerous subsystems, and multiple inputs and outputs. The corresponding test tasks are complex and numerous. In order to reduce the test execution time and minimize the idle time of test resources, parallel testing methods should be used for testing, that is, allowing parallel tasks. Parallel testing can execute multiple test items at the same time, but two different test items require the same resources, and there are often complex logical relationships between tasks, that is, constraints between tasks. Therefore, for a large number of test tasks, it is necessary to introduce a scheduling algorithm for test task planning.

[0003] Existing algorithms can plan for test task sets and take into account various conflicts between tasks. However, existing methods lack good modeling of the scheduling process of highly correlated test tasks, which leads to an increase in the number of iterations. At the same time, the planning results are often highly random and difficult to apply to actual needs. Therefore, a test task scheduling method for control systems based on graphical models and deep reinforcement learning is proposed. Summary of the invention

[0004] The present invention provides a test task scheduling method for a control system based on a graphical model and deep reinforcement learning, so as to solve the problem in the prior art that there is a lack of good modeling of a highly correlated test task scheduling process, which leads to an increase in the number of iterations, and at the same time, the planning results are often highly random and difficult to apply to actual needs.

[0005] The present invention provides a test task scheduling system for a control system based on a graphical model and deep reinforcement learning, and implements a test task scheduling method for a control system based on a graphical model and deep reinforcement learning.

[0006] The invention also provides a computer device.

[0007] The present invention also provides a computer-readable storage medium.

[0008] The present invention is achieved through the following technical solutions:

[0009] A test task scheduling method for a control system based on a graphical model and deep reinforcement learning, the test task scheduling method comprising the following steps:

[0010] Step 1: Analyze the test task requirements and then build a single-rate DAG task scheduling model;

[0011] Step 2: Generate a directed graph as the input of GNN based on the Single-rate DAG task scheduling model;

[0012] Step 3: Create a heat map about edges based on the directed graph and GNN;

[0013] Step 4: Generate task scheduling ranking based on the heat map, and plan the scheduling plan based on the task scheduling ranking;

[0014] Step 5: Calculate the loss function and update the graph neural network based on the loss function;

[0015] Step 6: Repeat steps 3-5 until a high-quality solution that meets the expected goal can be planned or the difference between the mean and minimum values ​​of the make span of the test solution planned in step 4 is less than a certain threshold, and the iteration ends;

[0016] Step 7: Output the optimized solution.

[0017] Furthermore, the step 1 is specifically to obtain a test task set T = {t1, t2, t3, ..., t n} and the test resource set R = {r1,r2,r3,...,r m}; Each test task t i The attributes of the predecessor task set Successor Task Set Duration Subsequent communication time Testing resources etc. Test resources i With attributes Indicates the available quantity of the test resource;

[0018] As the entry point of the DAG task scheduling model,

[0019] As the export of DAG task scheduling model, single-rate DAG is built on this basis; thus, the start and end marks of the task are clearly defined;

[0020] Furthermore, the step 1 is specifically:

[0021] The DAG task scheduling model consists of a set of nodes and edges of DAG, denoted as G = (V, E); nodes represent tasks in the system, and edges represent constraints between tasks;

[0022] V is the set of all nodes, expressed as V = {v1,...,v |V|}, where |V| is the total number of nodes. In the DAG task scheduling model, |V|=n+2; each node vi is used to represent a corresponding task t i , node v i The weight of task t i own information;

[0023] E is the set of all edges, where each edge e i,j ∈E represents v i and v j The dependency relationship between them, when there is e in DAG i,j When v i Execution is completed and v i v cannot be executed before the output of j ; Each edge e i,j There are weights to express the relationship between tasks.

[0024] Furthermore, the step 2 is specifically to simplify the DAG task scheduling model;

[0025] Traverse all tasks. If there is no constraint relationship between two tasks, including direct constraint relationship and indirect constraint relationship, add an edge between the two tasks to indicate that there is no dependency relationship between the two tasks.

[0026] Add a new weight to the edge to distinguish the type of edge; assign different weights to the original edge and the newly added edge;

[0027] In the newly generated directed graph, the meaning of the nodes has not changed, but the meaning of the edges has changed, indicating that there is a possibility of continuous execution of the two tasks. When After execution, the next task to be executed is It is possible to exist;

[0028] Define and initialize a graph neural network GNN with trainable parameters θ; construct a graph neural network consisting of a GNN backbone and an MLP decoder to extract node features and edge features of the graph and map them to edge weight outputs of the graph;

[0029] Specifically, step 3 is to input the given directed graph into the GNN model, and the GNN model will give a set of edge weights as output according to the current parameters, and use this as a basis to establish a heat map W = [w ij ] (n+2)(n+2) , where w i,j Represents edge The weight of high-quality solutions.

[0030] Furthermore, step 4 is specifically to use the test task as the first task in the sorting when generating task scheduling, as the starting point of task scheduling, and generate a schedulable task set T according to the current task sorting state. allow ;

[0031] The specific rule is: when all predecessor tasks of a task have been sorted, the current task can be added to T allow ; Based on the last scheduled task Generate T with heatmap W allow The corresponding state transition probability in Based on the state transition probability, select the next scheduling task; move the selected scheduling task from T allow , and update T according to the method described above. allow , and select the next scheduling task until all test tasks in all test task sets are sorted;

[0032] On the basis of completing the sorting, test resources and start execution time are allocated to the tasks, and the task scheduling plan is planned.

[0033] Furthermore, for the allocation of test resources, the earliest available task allocation time of each required test resource is calculated according to the greedy algorithm, and the earliest available resource of each test resource free time is selected to allocate the test task;

[0034] According to the allocated test resources, calculate the start time of the test. The specific calculation formula is as follows:

[0035]

[0036] Among them, the predecessor task set with a higher priority than the current task is used Represents; Task t j The start time is expressed as Task j The end time is expressed as The subsequent communication time of the task is recorded as Indicates whether the task needs to communicate with the predecessor task and the current task. Indicates that the test resources of the two tasks are the same and no communication is required. It indicates that communication time needs to be considered; Indicates the earliest idle time of all selected test resources; That is, it is the maximum value of the start time of the previous test task, the maximum value of the predecessor task execution and data transmission completion, and the earliest idle time of all selected test resources. The test start time planned by this formula avoids the conflict problem caused by decision-making during task scheduling, or obtains the optimal scheduling result to the maximum extent;

[0037] Based on the above method, multiple groups of test scheduling plans are generated as the basis for subsequent updates of the graph neural network.

[0038] Furthermore, the step 5 is specifically as follows: statistically calculate the test time makespan of the current test scheduling scheme; use the REINFORCE algorithm to calculate the loss function, update the trainable parameter θ of the graph neural network according to the calculation result, and perform gradient clipping to avoid the gradient explosion problem.

[0039] A test task scheduling system for a control system based on a graphical model and deep reinforcement learning, the test task scheduling system uses the test task scheduling method for a control system based on a graphical model and deep reinforcement learning as described above, the test task scheduling system includes:

[0040] The model building module analyzes the test task requirements and then builds the Single-rate DAG task scheduling model;

[0041] The directed graph generation module generates a directed graph as the input of GNN based on the Single-rate DAG task scheduling model;

[0042] Heat map building module, which builds heat maps about edges based on directed graphs and GNNs;

[0043] The scheduling scheme planning module generates task scheduling ranking based on the heat map and plans the scheduling scheme based on the task scheduling ranking;

[0044] The calculation module calculates the loss function and updates the graph neural network based on the loss function;

[0045] The iteration module uses the heat map building module, the scheduling scheme planning module, and the calculation module to iterate until a high-quality solution that meets the expected goal can be planned or the difference between the mean and minimum values ​​of the make span of the test scheme planned in step 4 is less than a certain threshold, and the iteration ends;

[0046] Output module, outputs the optimized solution.

[0047] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the above method is implemented.

[0048] A computer-readable storage medium stores a computer program, and the computer program implements the above method when executed by a processor.

[0049] The beneficial effects of the present invention are:

[0050] The present invention establishes a single-DAG task scheduling model, clarifies the dependencies between complex tasks, and facilitates the generation and iteration of planning schemes in subsequent scheduling algorithms.

[0051] The present invention adopts graph neural network to process data with complex relationships. Test task scheduling involves complex relationships such as dependencies between tasks and resource allocation. Graph neural network can effectively mine and utilize this information.

[0052] The present invention enables reinforcement learning to learn and optimize through continuous trial and error. By combining with graph neural networks, it can better utilize the features extracted by graph neural networks to guide the reinforcement learning algorithm to make intelligent scheduling decisions.

[0053] The present invention models and studies the tasks and resources in the system from a global perspective, and can better consider the global information of the entire test task scheduling system, thereby making more reasonable decisions.

[0054] Compared with the traditional scheduling method, the present invention can obtain stable optimization results within a lower number of iterations in the case of iterative samples and is not prone to falling into local convergence. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 It is a flow chart of the method of the present invention.

[0056] Figure 2 It is a convergence curve schematic diagram of the present invention.

[0057] Figure 3 It is a schematic diagram of the change process of the difference between the mean value and the minimum value of the makespan of the present invention.

[0058] Figure 4 It is a schematic diagram of the loss function change process of the present invention.

[0059] Figure 5 It is the task scheduling Gantt chart of the present invention. DETAILED DESCRIPTION

[0060] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0061] It should be understood that when used in this specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof.

[0062] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, the singular forms "a", "an" and "the" are intended to include plural forms unless the context clearly indicates otherwise.

[0063] The following is attached to this application specification Figure 1-5 , the technical solutions in the embodiments of the present application are clearly and completely described. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present application.

[0064] In the following description, many specific details are set forth to facilitate a full understanding of the present application, but the present application may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present application. Therefore, the present application is not limited to the specific embodiments disclosed below.

[0065] Embodiment 1

[0066] This embodiment provides a test task scheduling method for a control system based on a graphical model and deep reinforcement learning. Figure 1 As shown, the test task scheduling method includes the following steps:

[0067] Step 1: Analyze the test task requirements and then build a single-rate DAG task scheduling model;

[0068] Step 2: Generate a directed graph as the input of GNN based on the Single-rate DAG task scheduling model;

[0069] Step 3: Create a heat map of edges based on the directed graph of the current instance and GNN;

[0070] Step 4: Generate task scheduling ranking based on the heat map, and plan the scheduling plan based on the task scheduling ranking;

[0071] Step 5: Calculate the loss function and update the graph neural network based on the loss function;

[0072] Step 6: Repeat steps 3-5 until a high-quality solution that meets the expected goal can be planned or the difference between the mean and minimum values ​​of the make span of the test solution planned in step 4 is less than a certain threshold, and the iteration ends;

[0073] Step 7: Output the optimized high-quality solution.

[0074] Furthermore, the step 1 analyzes the test task requirements and obtains the test task set T = {t1, t2, t3, ..., t n} and the test resource set R = {r1,r2,r3,...,r m}; Each test task t i The attributes of the predecessor task set Successor Task Set Duration Subsequent communication time

[0075] Testing resources etc. Test resources i With attributes Indicates the available quantity of the test resource;

[0076] Add two tasks to the task set as the start and end marks of the test task; T = T∪{t0,t n+1}. Two tasks t0 and t n+1 Neither of them occupies any resources, and the execution time and communication time of the task are both 0;

[0077] As the entry point of the DAG task scheduling model, As the export of DAG task scheduling model, single-rate DAG can be constructed based on it, that is, DAG with a single source node or all source nodes entering at the same time; thus, the start and end flags of the task are clearly defined;

[0078] Furthermore, the step 1 of constructing the Single-rate DAG task scheduling model is specifically as follows:

[0079] The DAG task scheduling model consists of a set of nodes and edges of DAG, denoted as G = (V, E); nodes represent tasks in the system, and edges represent constraints between tasks;

[0080] V is the set of all nodes, expressed as V = {v1,...,v |V|}, where |V| is the total number of nodes. In the DAG task scheduling model, |V|=n+2; each node v i is used to represent a corresponding task t i , node vi The weight of task t i Its own information, including the resources occupied by the task, the execution time of the task, etc.

[0081] E is the set of all edges, where each edge e i,j ∈E represents v i and v j The dependency relationship between them, when there is e in DAG i,j When v i Execution is completed and v i v cannot be executed before the output of j ; Each edge e i,j There are weights to express the relationship between tasks, such as communication time.

[0082] Furthermore, the step 2 is specifically to simplify the DAG task scheduling model and remove redundant edges; for example, if there is an edge e 1,2 、e 1,3 、e 2,3 Then the redundant edge e 1,3 Needs to be removed.

[0083] Traverse all tasks. If there is no constraint relationship between two tasks, including direct constraint relationship and indirect constraint relationship, add an edge between the two tasks to indicate that there is no dependency relationship between the two tasks.

[0084] Add a new weight to the edge to distinguish the type of edge; assign different weights to the original edge and the newly added edge;

[0085] In the newly generated directed graph, the meaning of the nodes has not changed, but the meaning of the edges has changed, indicating that there is a possibility of continuous execution of the two tasks. When After execution, the next task to be executed is It is possible to exist;

[0086] Define and initialize a graph neural network GNN with trainable parameters θ; construct a graph neural network consisting of a GNN backbone and an MLP decoder to extract node features and edge features of the graph and map them to edge weight outputs of the graph;

[0087] Specifically, step 3 is to input the directed graph of a given instance into the GNN model. The GNN model will give a set of edge weights as output based on the current parameters, and use this as a basis to establish a heat map W = [w ij ] (n+2)(n+2) , where w i,j Represents edge The weight of high-quality solutions.

[0088] Furthermore, step 4 is specifically to use the test task as the first task in the sorting when generating task scheduling, as the starting point of task scheduling, and generate a schedulable task set T according to the current task sorting state. allow ;

[0089] The specific rule is: when all predecessor tasks of a task have been sorted, the current task can be added to T allow ; Based on the last scheduled task Generate T with heatmap W allow The corresponding state transition probability in Based on the state transition probability, select the next scheduling task; move the selected scheduling task from T allow , and update T according to the method described above. allow , and select the next scheduling task until all test tasks in all test task sets are sorted;

[0090] On the basis of completing the sorting, test resources and start execution time are allocated to the tasks, and the task scheduling plan is planned.

[0091] Furthermore, for the allocation of test resources, the earliest available task allocation time of each required test resource is calculated according to the greedy algorithm, and the earliest available resource of each test resource free time is selected to allocate the test task;

[0092] According to the allocated test resources, calculate the start time of the test. The specific calculation formula is as follows:

[0093]

[0094] Among them, the predecessor task set with a higher priority than the current task is used Represents; Task t j The start time is expressed as Task j The end time is expressed as The subsequent communication time of the task is recorded as Indicates whether the task needs to communicate with the predecessor task and the current task, b trafficjpredi =0 means that the test resources of the two tasks are the same and no communication is required. It indicates that communication time needs to be considered; Indicates the earliest idle time of all selected test resources; That is, it is the maximum value of the start time of the previous test task, the maximum value of the predecessor task execution and data transmission completion, and the earliest idle time of all selected test resources. The test start time planned by this formula avoids the conflict problem caused by decision-making during task scheduling, or obtains the optimal scheduling result to the maximum extent;

[0095] Based on the above method, multiple groups of test scheduling plans are generated as the basis for subsequent updates of the graph neural network.

[0096] Furthermore, the step 5 is specifically as follows: statistically calculate the test time makespan of the current test scheduling scheme; use the REINFORCE algorithm to calculate the loss function, update the trainable parameter θ of the graph neural network according to the calculation result, and perform gradient clipping to avoid the gradient explosion problem.

[0097] It can be seen from the above that the embodiment of the present invention implements the test task scheduling method of the control system based on the graphical model and deep reinforcement learning, and obtains a new planning and scheduling method, called the Single-rate DAG task scheduling model, which is used to plan the scheduling scheme according to the task scheduling sequence.

[0098] A test task instance with 50 test tasks and 6 test resources was selected for training. The convergence curve during training is shown in Figure 2 shown.

[0099] The change process of the difference between the mean and minimum values ​​of the test scheme makespan is as follows Figure 3 shown.

[0100] The change process of the loss function is as follows Figure 4 shown.

[0101] The final planning scheme is as follows Figure 5 shown.

[0102] Embodiment 2

[0103] This embodiment uses the test task scheduling method for a control system based on a graphical model and deep reinforcement learning as described in Example 1, and provides a test task scheduling system for a control system based on a graphical model and deep reinforcement learning, wherein the test task scheduling system includes a model building module, a directed graph generation module, a heat map establishment module, a scheduling scheme planning module, a calculation module, an iteration module, and an output module;

[0104] The model building module analyzes the test task requirements and then builds the Single-rate DAG task scheduling model;

[0105] The directed graph generation module generates a directed graph as the input of GNN based on the Single-rate DAG task scheduling model;

[0106] The heat map building module builds a heat map about the edge based on the directed graph and GNN of the current instance;

[0107] The scheduling scheme planning module generates task scheduling ranking based on the heat map and plans the scheduling scheme based on the task scheduling ranking;

[0108] The calculation module calculates the loss function and updates the graph neural network based on the loss function;

[0109] The iteration module uses the heat map building module, the scheduling scheme planning module, and the calculation module to iterate until a high-quality solution that meets the expected goal can be planned or the difference between the mean and minimum values ​​of the make span of the test scheme planned in step 4 is less than a certain threshold, and the iteration ends;

[0110] Output module outputs optimized high-quality solutions.

[0111] Furthermore, the model building module first analyzes the test task requirements, specifically, obtains the test task set T = {t1, t2, t3, ..., t n} and the test resource set R = {r1,r2,r3,...,r m}; Each test task t i The attributes of the predecessor task set Successor Task Set Duration Subsequent communication time Testing resources etc. Test resources i With attributes Indicates the available quantity of the test resource;

[0112] Add two tasks to the task set again as the start and end marks of the test task;

[0113] T=T∪{t0,t n+1}. Two tasks t0 and t n+1 Neither of them occupies any resources, and the execution time and communication time of the task are both 0; As the entry point of the DAG task scheduling model, As the export of DAG task scheduling model, single-rate DAG can be constructed based on it, that is, DAG with a single source node or all source nodes entering at the same time; thus, the start and end flags of the task are clearly defined;

[0114] Furthermore, the model building module finally builds a Single-rate DAG task scheduling model specifically as follows:

[0115] The DAG task scheduling model consists of a set of nodes and edges of DAG, denoted as G = (V, E); nodes represent tasks in the system, and edges represent constraints between tasks;

[0116] V is the set of all nodes, expressed as V = {v1,...,v |V|}, where |V| is the total number of nodes. In the DAG task scheduling model, |V|=n+2; each node v i is used to represent a corresponding task t i , node v i The weight of task t i Its own information, including the resources occupied by the task, the execution time of the task, etc.

[0117] E is the set of all edges, where each edge e i,j ∈E represents v i and v j The dependency relationship between them, when there is e in DAG i,j When v i Execution is completed and v i v cannot be executed before the output of j ; Each edge e i,j There are weights to express the relationship between tasks, such as communication time.

[0118] Furthermore, the working process of the directed graph generation module is specifically to simplify the DAG task scheduling model and remove redundant edges; for example, if there is an edge e 1,2 、e 1,3 、e 2,3 Then the redundant edge e 1,3 Needs to be removed.

[0119] Traverse all tasks. If there is no constraint relationship between two tasks, including direct constraint relationship and indirect constraint relationship, add an edge between the two tasks to indicate that there is no dependency relationship between the two tasks.

[0120] Add a new weight to the edge to distinguish the type of edge; assign different weights to the original edge and the newly added edge;

[0121] In the newly generated directed graph, the meaning of the nodes has not changed, but the meaning of the edges has changed, indicating that there is a possibility of continuous execution of the two tasks. When After execution, the next task to be executed is It is possible to exist;

[0122] Define and initialize a graph neural network GNN with trainable parameters θ; construct a graph neural network consisting of a GNN backbone and an MLP decoder to extract node features and edge features of the graph and map them to edge weight outputs of the graph;

[0123] The working process of the heat map building module is as follows: the directed graph of a given instance is input into the GNN model, and the GNN model will give a set of edge weights as output based on the current parameters, and use this as a basis to build a heat map W=[w ij ] (n+2)(n+2) , where w i,j Represents edge The weight of high-quality solutions.

[0124] Furthermore, the working process of the scheduling scheme planning module is specifically as follows: when generating task scheduling sorting, the test task is taken as the first task in the sorting, as the starting point of task scheduling, and a schedulable task set T is generated according to the current task sorting state. allow ;

[0125] The specific rule is: when all predecessor tasks of a task have been sorted, the current task can be added to T allow ; Based on the last scheduled task t ilast Generate a heatmap with T allow The corresponding state transition probability in Based on the state transition probability, select the next scheduling task; move the selected scheduling task from T allow , and update T according to the method described above. allow , and select the next scheduling task until all test tasks in all test task sets are sorted;

[0126] On the basis of completing the sorting, test resources and start execution time are allocated to the tasks, and the task scheduling plan is planned.

[0127] Furthermore, for the allocation of test resources, the earliest available task allocation time of each required test resource is calculated according to the greedy algorithm, and the earliest available resource of each test resource free time is selected to allocate the test task;

[0128] According to the allocated test resources, calculate the start time of the test. The specific calculation formula is as follows:

[0129]

[0130] Among them, the predecessor task set with a higher priority than the current task is used Represents; Task t j The start time is expressed as Task jThe end time is expressed as The subsequent communication time of the task is recorded as Indicates whether the task needs to communicate with the predecessor task and the current task. Indicates that the test resources of the two tasks are the same and no communication is required. It indicates that communication time needs to be considered; Indicates the earliest idle time of all selected test resources; That is, it is the maximum value of the start time of the previous test task, the maximum value of the predecessor task execution and data transmission completion, and the earliest idle time of all selected test resources. The test start time planned by this formula avoids the conflict problem caused by decision-making during task scheduling, or obtains the optimal scheduling result to the maximum extent;

[0131] Based on the above method, multiple groups of test scheduling plans are generated as the basis for subsequent updates of the graph neural network.

[0132] Furthermore, the working process of the calculation module is specifically as follows: statistics on the test time makespan of the current test scheduling scheme; use the REINFORCE algorithm to calculate the loss function, update the trainable parameters θ of the graph neural network according to the calculation results, and perform gradient clipping to avoid the gradient explosion problem.

[0133] It can be seen from the above that the embodiment of the present invention implements a test task scheduling system for a control system based on a graphical model and deep reinforcement learning, and obtains a new planning and scheduling method, called a Single-rate DAG task scheduling model, which is used to plan a scheduling scheme according to the task scheduling sequence.

[0134] Embodiment 3

[0135] An embodiment of the present invention provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the memory is used to store software programs and modules, and the processor executes various functional applications and data processing by running the software programs and modules stored in the memory. The memory and the processor are connected via a bus. Specifically, the processor implements any step in the first embodiment above by running the computer program stored in the memory.

[0136] It should be understood that in the embodiments of the present invention, the processor referred to may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0137] The memory may include a read-only memory, a flash memory, and a random access memory, and provides instructions and data to the processor. A part or all of the memory may also include a nonvolatile random access memory.

[0138] It can be seen from the above that the electronic device provided by the embodiment of the present invention can implement the test task scheduling method of the control system based on the graphical model and deep reinforcement learning as described in Example 1 by running a computer program, and obtain a new planning and scheduling method, called the Single-rate DAG task scheduling model, which is used to plan the scheduling scheme according to the task scheduling sorting.

[0139] It should be understood that if the above-mentioned integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The above-mentioned computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the above-mentioned computer program includes computer program code, and the above-mentioned computer program code can be in source code form, object code form, executable file or some intermediate form. The above-mentioned computer-readable medium may include: any entity or device capable of carrying the above-mentioned computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the above-mentioned computer-readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction.

[0140] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0141] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the above-mentioned device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0142] It should be noted that the methods and detailed examples provided in the above embodiments can be combined with the devices and equipment provided in the embodiments, and references can be made to each other, and no further details will be given.

[0143] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0144] In the embodiments provided by the present invention, it should be understood that the disclosed apparatus / terminal equipment and method can be implemented in other ways. For example, the apparatus / equipment embodiments described above are only illustrative, for example, the division of the above modules or units is only a logical function division, and in actual implementation, other division methods can be used, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0145] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions described in the above embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should be included in the protection scope of the present invention.

Claims

1. A test task scheduling method for a control system based on a graphical model and deep reinforcement learning, characterized in that: The test task scheduling method comprises the following steps: Step 1: Analyze the test task requirements and then build a single-rate DAG task scheduling model; Step 2: Generate a directed graph as the input of GNN based on the Single-rate DAG task scheduling model; Step 3: Create a heat map about edges based on the directed graph and GNN; Step 4: Generate task scheduling order based on the heat map, and plan the scheduling plan based on the task scheduling order; Step 5: Calculate the loss function and update the graph neural network based on the loss function; Step 6: Repeat steps 3-5 until a high-quality solution that meets the expected goal can be planned or the difference between the mean and minimum values ​​of the make span of the test solution planned in step 4 is less than a certain threshold, and the iteration ends; Step 7: Output the optimized solution.

2. The test task scheduling method according to claim 1, characterized in that: The step 1 specifically includes obtaining a test task set T = {t1, t2, t3, ..., t n } and the test resource set R = {r1,r2,r3,...,r m }; Each test task t i The attributes of the predecessor task set Successor Task Set Duration Subsequent communication time Testing resources etc. Test resources i With attributes Indicates the available quantity of the test resource; As the entry point of the DAG task scheduling model, As the export of DAG task scheduling model, single-rate DAG is built on this basis; thus, the start and end marks of the task are clearly defined.

3. The test task scheduling method according to claim 2, characterized in that: The step 1 specifically comprises: The DAG task scheduling model consists of a set of nodes and edges of DAG, denoted as G = (V, E); nodes represent tasks in the system, and edges represent constraints between tasks; V is the set of all nodes, expressed as V = {v1,...,v |V| }, where |V| is the total number of nodes. In the DAG task scheduling model, |V|=n+2; each node v i is used to represent a corresponding task t i , node v i The weight of task t i own information; E is the set of all edges, where each edge e i,j ∈E represents v i and v j The dependency relationship between them, when there is e in DAG i,j When v i Execution is completed and v i v cannot be executed before the output of j ; Each edge e i,j There are weights to express the relationship between tasks.

4. The test task scheduling method according to claim 3, characterized in that: Specifically, step 2 is to simplify the DAG task scheduling model; Traverse all tasks. If there is no constraint relationship between two tasks, including direct constraint relationship and indirect constraint relationship, add an edge between the two tasks to indicate that there is no dependency relationship between the two tasks. Add a new weight to the edge to distinguish the type of edge; assign different weights to the original edge and the newly added edge; In the newly generated directed graph, the meaning of the nodes has not changed, but the meaning of the edges has changed, indicating that there is a possibility of continuous execution of the two tasks. When After execution, the next task to be executed is It is possible to exist; Define and initialize a graph neural network GNN with trainable parameters θ; construct a graph neural network consisting of a GNN backbone and an MLP decoder to extract node features and edge features of the graph and map them to edge weight outputs of the graph; Specifically, step 3 is to input the given directed graph into the GNN model, and the GNN model will give a set of edge weights as output according to the current parameters, and use this as a basis to establish a heat map W = [w ij ] (n+2)(n+2) , where w i,j Represents edge The weight of high-quality solutions.

5. The test task scheduling method according to claim 4, characterized in that: Step 4 is to use the test task as the first task in the order and as the starting point of task scheduling when generating task scheduling order, and generate a schedulable task set T according to the current task ordering state. allow ; The specific rule is: when all predecessor tasks of a task have been sorted, the current task can be added to T allow ; Based on the last scheduled task Generate T with heatmap W allow The corresponding state transition probability in Based on the state transition probability, select the next scheduling task; move the selected scheduling task from T allow , and update T according to the method described above. allow , and select the next scheduling task until all test tasks in all test task sets are sorted; On the basis of completing the sorting, test resources and start execution time are allocated to the tasks, and the task scheduling plan is planned.

6. The test task scheduling method according to claim 5, characterized in that: For the allocation of test resources, the greedy algorithm is used to calculate the earliest time when each test resource can be used to allocate tasks, and the earliest available resource in the idle time of each test resource is selected to allocate test tasks; According to the allocated test resources, calculate the start time of the test. The specific calculation formula is as follows: Among them, the predecessor task set with a higher priority than the current task is used Represents; Task t j The start time is expressed as Task j The end time is expressed as The subsequent communication time of the task is recorded as Indicates whether the task needs to communicate with the predecessor task and the current task. Indicates that the test resources of the two tasks are the same and no communication is required. It indicates that communication time needs to be considered; Indicates the earliest idle time of all selected test resources; That is, it is the maximum value of the start time of the previous test task, the maximum value of the predecessor task execution and data transmission completion, and the earliest idle time of all selected test resources. The test start time planned by this formula avoids the conflict problem caused by decision-making during task scheduling, or obtains the optimal scheduling result to the maximum extent; Based on the above method, multiple groups of test scheduling plans are generated as the basis for subsequent updates of the graph neural network.

7. The test task scheduling method according to claim 6, characterized in that: Specifically, step 5 includes: calculating the test time makespan of the current test scheduling scheme; using the REINFORCE algorithm to calculate the loss function, updating the trainable parameter θ of the graph neural network based on the calculation result, and performing gradient clipping to avoid the gradient explosion problem.

8. A test task scheduling system for control systems based on graphical models and deep reinforcement learning, characterized in that: The test task scheduling system uses the test task scheduling method of the control system based on the graphical model and deep reinforcement learning as claimed in any one of claims 1 to 7, and the test task scheduling system includes: The model building module analyzes the test task requirements and then builds the Single-rate DAG task scheduling model; The directed graph generation module generates a directed graph as the input of GNN based on the Single-rate DAG task scheduling model; Heat map building module, which builds heat maps about edges based on directed graphs and GNNs; The scheduling scheme planning module generates task scheduling ranking based on the heat map and plans the scheduling scheme based on the task scheduling ranking; The calculation module calculates the loss function and updates the graph neural network based on the loss function; The iteration module uses the heat map building module, the scheduling scheme planning module, and the calculation module to iterate until a high-quality solution that meets the expected goal can be planned or the difference between the mean and minimum values ​​of the make span of the test scheme planned in step 4 is less than a certain threshold, and the iteration ends; Output module, outputs the optimized solution.

9. A computer device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images