Test task scheduling method and system of control system based on graph model and deep reinforcement learning

Abstract

The present application belongs to the field of automatic test technology, and particularly relates to a test task scheduling method and system of a control system based on a graph model and deep reinforcement learning. The test task demand is analyzed, and a Single-rate DAG task scheduling model is constructed; a directed graph is generated as the input of GNN according to the Single-rate DAG task scheduling model; a heat map about edges is established based on the directed graph and GNN; a task scheduling sequence is generated according to the heat map, and a scheduling scheme is planned according to the task scheduling sequence; a loss function is calculated, and the graph neural network is updated according to the loss function; the heat map is repeatedly constructed to the loss function is calculated until a high-quality solution meeting the expected target can be planned or the difference between the average and the minimum of the makespan of the planned test scheme is less than a certain threshold, and the iteration is ended; and step 7: the optimized solution is output. The present application is used to solve the problem that the planning result of the test task scheduling process with high correlation often has relatively large randomness and is difficult to be applied to actual demand.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of automatic testing, and particularly relates to a test task scheduling method and system for a control system based on a graph model and deep reinforcement learning. BACKGROUND

[0002] A control system is often complex in function and structure, has many subsystems, and has multiple inputs and outputs, and its corresponding test tasks are complex and numerous. In order to reduce test execution time and maximize idle time of test resources, parallel test methods should be used for testing, that is, parallelism between tasks is allowed. Parallel testing can execute multiple test items at the same time, but two different test items need the same resources, and there are complex logical relationships between tasks, that is, constraints between tasks. Therefore, for a large number of test tasks, a scheduling algorithm needs to be introduced for test task planning.

[0003] Existing algorithms can plan for a test task set and can also consider various conflicts between tasks. However, the existing method lacks good modeling of the high-correlation test task scheduling process, which increases the number of iterations, and the planning result often has large randomness, which is difficult to apply to actual needs. Therefore, a test task scheduling method for a control system based on a graph model and deep reinforcement learning is proposed. SUMMARY

[0004] The application provides a test task scheduling method for a control system based on a graph model and deep reinforcement learning, which solves the problem that the existing technology lacks good modeling of the high-correlation test task scheduling process, which increases the number of iterations, and the planning result often has large randomness, which is difficult to apply to actual needs.

[0005] The application provides a test task scheduling system for a control system based on a graph model and deep reinforcement learning, which implements the test task scheduling method for the control system based on the graph model and the deep reinforcement learning.

[0006] The application also provides a computer device.

[0007] The application also provides a computer-readable storage medium.

[0008] The application is implemented through the following technical solutions:

[0009] A test task scheduling method for a control system based on a graph model and deep reinforcement learning, the test task scheduling method comprising the following steps:

[0010] Step 1: analyzing test task requirements and constructing a Single-rate DAG task scheduling model;

[0011] Step 2: Based on the Single-rate DAG task scheduling model, generate a directed graph as the input to the GNN;

[0012] Step 3: Construct a heatmap of edges based on the directed graph and GNN;

[0013] Step 4: Generate a task scheduling order based on the heatmap, and plan a scheduling scheme based on the task scheduling order;

[0014] Step 5: Calculate the loss function and update the graph neural network based on the loss function;

[0015] Step 6: Repeat steps 3-5 until a high-quality solution that meets the expected goal can be planned, or the difference between the mean and minimum value of the test scheme makespan planned in step 4 is less than a certain threshold, then the iteration ends;

[0016] Step 7: Output the optimized solution.

[0017] Furthermore, step 1 specifically involves obtaining the test task set T = {t1, t2, t3, ..., t}. n} and the test resource set R = {r1, r2, r3, ..., r m}; where each test task t i The attributes include the precursor mission set Subsequent task set Duration Subsequent communication time Test resources Wait, test resource r i Has attributes This indicates the number of available test resources;

[0018] As the entry point for the DAG task scheduling model,

[0019] As the exit point of the DAG task scheduling model, a single-rate DAG is constructed based on it; thus, the start and end markers of the task are clearly defined.

[0020] Furthermore, step 1 specifically involves:

[0021] The DAG task scheduling model consists of a set of nodes and an edge set of the DAG, denoted as G = (V, E); nodes represent tasks in the system, and edge sets represent the constraints between tasks.

[0022] V is the set of all nodes, represented as V = {v1, ..., v}. |V|}, where |V| is the total number of nodes. In the DAG task scheduling model, |V| = n + 2; each node vi is used to represent a corresponding task t i , the weight of node v i is the information of task t i itself;

[0023] E is the set of all edges, where each edge e i,j ∈E represents the dependency relationship between v i and v j , when e i,j exists in the DAG, v i cannot be executed before the execution of v i is completed and the output of v j arrives; each edge e i,j exists with a weight to express the relationship between tasks.

[0024] Further, the step 2 is specifically to simplify the DAG task scheduling model;

[0025] Traverse all tasks, if there is no any constraint relationship between two tasks, including direct constraint relationship and indirect constraint relationship, then a new edge is added between the two tasks, indicating that there is no any dependency relationship between the two tasks;

[0026] A new weight is added to the edge to distinguish the type of the edge; different weights are given to the original edge and the newly added edge;

[0027] In the newly generated directed graph, the meaning of the node does not change, but the meaning of the edge changes, indicating the possibility of continuous execution of the two tasks, if e exists, it means that after the execution of v , the next task to be executed is v , which may exist;

[0028] Define a graph neural network GNN with trainable parameters θ and initialize it; the graph neural network is composed of a GNN backbone and an MLP decoder, which extracts the node features and edge features of the graph and maps them as the edge weight output of the graph;

[0029] The step 3 is specifically to input the given directed graph into the GNN model, and the GNN model will give a set of edge weights as output according to the current parameters, and based on this, a heat map W=[w ij ] (n+2)(n+2) is established on the edge of the directed graph, where w i,j represents the weight of the edge that belongs to the high-quality solution.

[0030] Further, step 4 is specifically, when generating the task scheduling sequence, the test task is taken as the first task in the sequence as the starting point of the task scheduling, and a schedulable task set T is generated according to the current task sequence state allow ;

[0031] The specific rule is that when the predecessor tasks of a certain task have all been sequenced, the current task can be added to T allow ; according to the last scheduled task and the heat map W to generate T allow The corresponding state transition probability Among them Based on the state transition probability, the next scheduling task is selected; the selected scheduling task is deleted from T allow , and T allow is updated according to the method described above, and the selection of the next scheduling task is performed until all test tasks in the test task set are sequenced.

[0032] On the basis of completing the sequencing, test resources and start execution time are allocated to the tasks, and a task scheduling scheme is planned.

[0033] Further, for the allocation of test resources, according to the greedy algorithm, the earliest allocable task time of the available resources of each required test resource is calculated, and the earliest available resource of each test resource is selected to allocate the test task.

[0034] According to the allocated test resources, the start time of the test is calculated, and the specific calculation formula is as follows,

[0035]

[0036] Among them, the predecessor task set with a higher priority than the current task is represented by T ; the start time of task t j is represented as The end time of task t j is represented as The subsequent communication time of the task is recorded as Indicates whether the task and the predecessor task and the current task need to communicate, indicates that the test resources of the two tasks are the same and do not need to communicate, indicates that the communication time needs to be considered; indicates the earliest idle time of all selected test resources; That is, the maximum value of the start time of the last test task, the execution and transmission of the predecessor task, and the earliest idle time of all selected test resources; the test start time planned by the formula avoids the conflict problem caused by decision-making in task scheduling, or obtains the optimal scheduling result to the maximum extent.

[0037] According to the above method, a plurality of test scheduling schemes are generated as the basis for subsequent updating of the graph neural network.

[0038] Further, the step 5 is specifically, the test time makespan of the current test scheduling scheme is counted; the loss function is calculated using the REINFORCE algorithm, the trainable parameters θ of the graph neural network are updated according to the calculation result, and the gradient clipping is performed to avoid the problem of gradient explosion.

[0039] A test task scheduling system of a control system based on a graph model and deep reinforcement learning, the test task scheduling system uses the test task scheduling method of the control system based on the graph model and deep reinforcement learning as described above, and the test task scheduling system comprises,

[0040] A model construction module analyzes the test task demand and then constructs a Single-rate DAG task scheduling model;

[0041] A directed graph generation module generates a directed graph as the input of the GNN according to the Single-rate DAG task scheduling model;

[0042] A heat map establishment module establishes a heat map about the edge based on the directed graph and the GNN;

[0043] A scheduling scheme planning module generates a task scheduling order according to the heat map and plans a scheduling scheme according to the task scheduling order;

[0044] A calculation module calculates a loss function and updates the graph neural network according to the loss function;

[0045] An iteration module iterates using the heat map establishment module, the scheduling scheme planning module and the calculation module until a high-quality solution meeting the expected target can be planned or the difference between the average and the minimum of the test scheme makespan planned in step 4 is less than a certain threshold, and the iteration ends;

[0046] An output module outputs the optimized solution.

[0047] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, and the processor implements the method as described above when executing the computer program.

[0048] A computer readable storage medium, the computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the method as described above.

[0049] The beneficial effects of the present application are:

[0050] The application establishes a single-DAG task scheduling model, clearly defines the complex tasks and the dependency relationship between the tasks, and facilitates the generation and iteration of planning schemes in subsequent scheduling algorithms.

[0051] The application adopts a graph neural network to process data with complex relationships, and the test task scheduling involves complex relationships such as dependency relationships between tasks and resource allocation, and the graph neural network can effectively mine and utilize these information.

[0052] The application enables reinforcement learning to learn and optimize in continuous trial and error, and through the combination of the graph neural network, the features extracted by the graph neural network can be better utilized to guide the intelligent scheduling decision of the reinforcement learning algorithm.

[0053] The application models and learns the tasks and resources in the system from a global perspective, can better consider the global information of the entire test task scheduling system, and thus make more reasonable decisions.

[0054] Compared with the traditional scheduling method, the application can obtain stable optimization results in a lower number of iterations under the condition of the first iteration sample, and is not prone to local convergence. BRIEF DESCRIPTION OF DRAWINGS

[0055] Figure 1 is a method flowchart of the application.

[0056] Figure 2 is a convergence curve schematic diagram of the application.

[0057] Figure 3 is a variation process schematic diagram of the difference between the mean value and the minimum value of the makespan of the application.

[0058] Figure 4 is a loss function variation process schematic diagram of the application.

[0059] Figure 5 is a task scheduling Gantt chart of the application. DETAILED DESCRIPTION

[0060] In the following description, specific details are set forth such as particular system architectures, techniques, etc. in order to provide a thorough understanding of the embodiments of the present application. However, persons skilled in the art will understand that the present application can be implemented in other embodiments without these specific details. In other cases, well-known system, device, circuit and method details are omitted so as not to obscure the description of the present application with unnecessary details.

[0061] It should be understood that the term "include" as used in this specification and in the following claims indicates the presence of the described features, integers, steps, operations, elements, and / or components but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof.

[0062] It should also be understood that the terms used in the present application specification are only for the purpose of describing particular embodiments and are not intended to limit the present application. As used in the present application specification and the appended claims, "a", "an", and "the" are intended to include the plural forms unless the context clearly indicates otherwise.

[0063] The following will be described in detail with reference to the accompanying drawings of the present application specification Figures 1-5 The technical solutions in the embodiments of the present application are described clearly and completely. Obviously, the described embodiments are only some of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0064] In the following description, many specific details are set forth in order to provide a thorough understanding of the present application. However, the present application can be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the present application, so the present application is not limited to the specific embodiments disclosed below.

[0065] Embodiment one

[0066] The embodiment provides a test task scheduling method of a control system based on a graph model and deep reinforcement learning, as shown in the figure, the test task scheduling method comprises the following steps: Figure 1 As shown in the figure, the test task scheduling method comprises the following steps:

[0067] Step 1: analyze the test task demand, and then build a Single-rate DAG task scheduling model;

[0068] Step 2: according to the Single-rate DAG task scheduling model, generate a directed graph as the input of GNN;

[0069] Step 3: based on the directed graph of the current instance and GNN, establish a heat map (heat-map) about the edge;

[0070] Step 4: generate a task scheduling order according to the heat map, and plan a scheduling scheme according to the task scheduling order;

[0071] Step 5: calculate the loss function, and update the graph neural network according to the loss function;

[0072] Step 6: Repeat steps 3-5 until a high-quality solution that meets the expected goal can be planned, or the difference between the mean and minimum value of the test scheme makespan planned in step 4 is less than a certain threshold, then the iteration ends;

[0073] Step 7: Output the optimized, high-quality solution.

[0074] Furthermore, step 1, which analyzes the test task requirements, specifically involves obtaining a test task set T = {t1, t2, t3, ..., t...}. n} and the test resource set R = {r1, r2, r3, ..., r m}; where each test task t i The attributes include the precursor mission set Subsequent task set Duration Subsequent communication time

[0075] Test resources Wait, test resource r i Has attributes This indicates the number of available test resources;

[0076] Add two tasks to the task set as start and end markers for the test tasks; T = T∪{t0,t...} n+1 Two tasks t0 and t... n+1 Neither consumes any resources, and the task execution time and communication time are both 0;

[0077] As the entry point for the DAG task scheduling model, As the exit point of the DAG task scheduling model, a single-rate DAG can be constructed based on this, that is, a DAG with a single source node or all source nodes entering at the same time; thus, the start and end markers of the task are clearly defined.

[0078] Furthermore, step 1, constructing the Single-rate DAG task scheduling model, specifically involves:

[0079] The DAG task scheduling model consists of a set of nodes and an edge set of the DAG, denoted as G = (V, E); nodes represent tasks in the system, and edge sets represent the constraints between tasks.

[0080] V is the set of all nodes, represented as V = {v1, ..., v}. |V|}, where |V| is the total number of nodes. In the DAG task scheduling model, |V| = n + 2; each node v i Used to represent a corresponding task t i Node vi The weight is task t i The task itself contains information such as the resources it consumes and the execution time of the task.

[0081] E is the set of all edges, where each edge e i,j ∈E represents v i and v j The dependency relationship between them, when there is e in the DAG i,j At that time, in v i Execution completed and v i v cannot be executed until the output of is reached. j ; Each edge e i,j Weights exist to express the relationships between tasks, such as communication time.

[0082] Furthermore, step 2 specifically involves simplifying the DAG task scheduling model by removing redundant edges; for example, if there is an edge e... 1,2 e 1,3 e 2,3 Then redundant edge e 1,3 It needs to be removed.

[0083] Iterate through all tasks. If there are no constraints between two tasks, including direct and indirect constraints, then add an edge between the two tasks to indicate that there are no dependencies between them.

[0084] Add a new weight to the edge to distinguish the edge type; assign different weights to the original edges and the newly added edges;

[0085] In the newly generated directed graph, the meaning of the nodes remains unchanged, but the meaning of the edges changes, indicating that there is a possibility that two tasks can be executed consecutively. When, it means at After execution, the next task to begin execution is It is possible;

[0086] Define and initialize a graph neural network (GNN) with trainable parameters θ; the GNN backbone and MLP decoder are used to form the graph neural network, which extracts the node features and edge features of the graph and maps them to the edge weights of the graph.

[0087] Step 3 specifically involves inputting the directed graph of the given instance into the GNN model. The GNN model will output a set of edge weights based on the current parameters, and use these weights to construct a heatmap W = [w...] on the edges of the directed graph. ij ] (n+2)(n+2) , where w i,j Representing an edge The weights that belong to high-quality solutions.

[0088] Further, step 4 is specifically, when generating the task scheduling sequence, the test task is taken as the first task in the sequence as the starting point of the task scheduling, and the schedulable task set T is generated according to the current task sequence state allow ;

[0089] The specific rule is that when the predecessor task of a certain task has been sequenced, the current task can be added to T allow ; according to the last scheduled task and the hot map W to generate T allow The corresponding state transition probability Among them Based on the state transition probability, the next scheduling task is selected; the selected scheduling task is deleted from T allow , and T allow is updated according to the method described above, and the selection of the next scheduling task is performed until all test tasks in the test task set are sequenced.

[0090] On the basis of completing the sequencing, the test resources and the start execution time are allocated to the tasks, and the task scheduling scheme is planned.

[0091] Further, for the allocation of test resources, according to the greedy algorithm, the earliest allocable task time of the available resources of each required test resource is calculated, and the earliest available resource of each test resource is selected to allocate the test task.

[0092] According to the allocated test resources, the start time of the test is calculated, and the specific calculation formula is as follows,

[0093]

[0094] Among them, the predecessor task set with a priority higher than the current task is represented by ; the start time of task t j is represented by The end time of task t j is represented by The subsequent communication time of the task is recorded as b trafficjpredi = 0 represents that the test resources of the two tasks are the same and no communication is needed, which means that the communication time needs to be considered; which represents the earliest idle time of all selected test resources; The maximum value of the start time of the previous test task, the maximum value of the predecessor task execution and data transmission completion, and the earliest idle time of all selected test resources is the maximum value of the three; the test start time planned by the formula avoids the conflict problem caused by decision-making in task scheduling, or obtains the optimal scheduling result to the maximum extent;

[0095] According to the above method, a plurality of test scheduling schemes are generated as the basis for subsequent updating of the graph neural network.

[0096] Further, the step 5 is specifically to count the test time makespan of the current test scheduling scheme; the loss function is calculated using the REINFORCE algorithm, the trainable parameters θ of the graph neural network are updated according to the calculation result, and gradient clipping is performed to avoid the problem of gradient explosion.

[0097] As can be seen from the above, the embodiment of the application realizes the test task scheduling method of the control system based on the graph model and deep reinforcement learning, obtains a new planning and scheduling method, called Single-rate DAG task scheduling model, which is used to plan and schedule the scheduling scheme according to the task scheduling order.

[0098] A test task instance with 50 test tasks and 6 test resources is selected for training. The convergence curve in the training process is as shown in Figure 2 .

[0099] The change process of the difference between the mean value and the minimum value of the test scheme makespan is as shown in Figure 3 .

[0100] The change process of the loss function is as shown in Figure 4 .

[0101] The final planning scheme is as shown in Figure 5 .

[0102] Embodiment two

[0103] The embodiment uses the test task scheduling method of the control system based on the graph model and deep reinforcement learning as described in embodiment one, and provides a test task scheduling system of the control system based on the graph model and deep reinforcement learning, which comprises a model construction module, a directed graph generation module, a heat map establishment module, a scheduling scheme planning module, a calculation module, an iteration module and an output module.

[0104] The model construction module analyzes the test task demand and then constructs the Single-rate DAG task scheduling model;

[0105] The directed graph generation module generates a directed graph as the input of the GNN according to the Single-rate DAG task scheduling model;

[0106] a heat-map establishment module, which establishes a heat-map about edges based on the directed graph of the current instance and the GNN;

[0107] a scheduling scheme planning module, which generates a task scheduling order according to the heat-map and plans a scheduling scheme according to the task scheduling order;

[0108] a calculation module, which calculates a loss function and updates the graph neural network according to the loss function;

[0109] an iteration module, which iterates using the heat-map establishment module, the scheduling scheme planning module and the calculation module until a high-quality solution meeting the expected target can be planned or the difference between the mean and the minimum of the makespan of the test schemes planned in step 4 is less than a certain threshold, and the iteration ends;

[0110] an output module, which outputs the optimized high-quality solution.

[0111] Further, the model construction module first analyzes the test task requirements, specifically, obtains a test task set T = {t1, t2, t3,..., t n} and a test resource set R = {r1, r2, r3,..., r m}; wherein each test task t i has a predecessor task set a successor task set a duration a successor communication time a test resource , etc., and the test resource r i has an attribute representing the available number of the test resource;

[0112] Again, two tasks are added to the task set as the start and end markers of the test tasks;

[0113] T = T U {t0, t n+1}. The two tasks t0 and t n+1 do not occupy any resources, and the execution time and communication time of the tasks are both 0; as the entrance of the DAG task scheduling model, as the exit of the DAG task scheduling model, and based on this, a single-rate DAG can be constructed, i.e., a DAG with a single source node or all source nodes entering at the same time; in this way, the start and end markers of the tasks are determined;

[0114] Further, the model construction module finally constructs the single-rate DAG task scheduling model, specifically,

[0115] The DAG task scheduling model consists of a set of nodes and an edge set of the DAG, denoted as G = (V, E); nodes represent tasks in the system, and edge sets represent the constraints between tasks.

[0116] V is the set of all nodes, represented as V = {v1, ..., v}. |V|}, where |V| is the total number of nodes. In the DAG task scheduling model, |V| = n + 2; each node v i Used to represent a corresponding task t i Node v i The weight is task t i The task itself contains information such as the resources it consumes and the execution time of the task.

[0117] E is the set of all edges, where each edge e i,j ∈E represents v i and v j The dependency relationship between them, when there is e in the DAG i,j At that time, in v i Execution completed and v i v cannot be executed until the output of is reached. j ; Each edge e i,j Weights exist to express the relationships between tasks, such as communication time.

[0118] Furthermore, the directed graph generation module specifically works by simplifying the DAG task scheduling model and removing redundant edges; for example, if edge e exists... 1,2 e 1,3 e 2,3 Then redundant edge e 1,3 It needs to be removed.

[0119] Iterate through all tasks. If there are no constraints between two tasks, including direct and indirect constraints, then add an edge between the two tasks to indicate that there are no dependencies between them.

[0120] Add a new weight to the edge to distinguish the edge type; assign different weights to the original edges and the newly added edges;

[0121] In the newly generated directed graph, the meaning of the nodes remains unchanged, but the meaning of the edges changes, indicating that there is a possibility that two tasks can be executed consecutively. When, it means at After execution, the next task to begin execution is It is possible;

[0122] A graph neural network GNN with trainable parameters θ is defined and initialized; the graph neural network is composed of a GNN backbone and an MLP decoder, which extracts node features and edge features of the graph and maps them as edge weight output of the graph;

[0123] The working process of the heat map establishment module is specifically that the directed graph of a given instance is input into the GNN model, and the GNN model will give a set of edge weights as output according to the current parameters, and the heat map W = [w ij ] (n+2)(n+2) , where w i,j represents the weight of the edge belonging to the high-quality solution.

[0124] Further, the working process of the scheduling scheme planning module is specifically that when generating the task scheduling sequence, the test task is taken as the first task in the sequence as the starting point of task scheduling, and a schedulable task set T allow is generated according to the current task sequence state.

[0125] The specific rule is that when the predecessor tasks of a certain task have all been sequenced, the current task can be added to T allow ; according to the last scheduled task t ilast and the state transition probability corresponding to the heat map generation T allow , where The next scheduling task is selected based on the state transition probability; the selected scheduling task is deleted from T allow , and T allow is updated according to the method described above, and the selection of the next scheduling task is performed until all test tasks in the test task set are sequenced.

[0126] On the basis of completing the sequencing, the test resources and the start execution time are allocated to the tasks to plan the task scheduling scheme.

[0127] Further, for the allocation of test resources, the earliest allocable task time of the available resources of each kind of test resource required is calculated according to the greedy algorithm, and the earliest available resource of each kind of test resource is selected to allocate the test task.

[0128] According to the allocated test resources, the start time of the test is calculated, and the specific calculation formula is as follows,

[0129]

[0130] where the predecessor task set with a higher priority than the current task is represented by ; the start time of task t j is represented by j ​​The end time is represented as The subsequent communication time of the task is recorded as It is represented whether the task needs to communicate with the predecessor task and the current task, It is represented whether the test resources of the two tasks are the same and whether communication is needed, It is represented that the communication time needs to be considered; It is represented that the earliest idle time of all selected test resources is selected; That is, the maximum value of the start time of the last test task, the maximum value of the execution and data transmission completion of the predecessor task, and the earliest idle time of all selected test resources; the test start time planned by the formula avoids the conflict problem caused by decision-making in task scheduling, or obtains the optimal scheduling result to the maximum extent;

[0131] According to the above method, a plurality of test scheduling schemes are generated, which are used as the basis for subsequent updating of the graph neural network.

[0132] Further, the working process of the calculation module is specifically that the test time makespan of the current test scheduling scheme is counted; the loss function is calculated using the REINFORCE algorithm, the trainable parameters θ of the graph neural network are updated according to the calculation result, and gradient clipping is performed to avoid the problem of gradient explosion.

[0133] As can be seen from the above, the embodiment of the application realizes a test task scheduling system of a control system based on a graph model and deep reinforcement learning, obtains a new planning and scheduling method, which is called a Single-rate DAG task scheduling model, and is used for planning and scheduling a scheduling scheme according to task scheduling ordering.

[0134] Embodiment three

[0135] The embodiment of the application provides an electronic device, which comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the memory is used for storing software programs and modules, and the processor executes various functional applications and data processing by running the software programs and modules stored in the memory. The memory and the processor are connected through a bus. Specifically, the processor realizes any step in the above embodiment one by running the above computer program stored in the memory.

[0136] It should be appreciated that in the embodiments of the present application, the processor can be a central processing unit (CPU), and can also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic components, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0137] The memory can include read-only memory, flash memory, and random access memory, and provide instructions and data to the processor. Part or all of the memory can also include non-volatile random access memory.

[0138] As can be seen from the above, the electronic device provided by the embodiments of the present application can realize the test task scheduling method of the control system based on the graph model and deep reinforcement learning as described in Embodiment One by running the computer program, obtain a new planning and scheduling method, called Single-rate DAG task scheduling model, for planning and scheduling scheme according to task scheduling order.

[0139] It should be understood that the integrated modules / units described above, if realized in the form of software function units and sold or used as independent products, can be stored in a computer readable storage medium. Based on this understanding, all or part of the processes in the above-mentioned embodiment methods can also be completed by a computer program instructing related hardware, and the above-mentioned computer program can be stored in a computer readable storage medium. When the processor executes the computer program, the steps of the above-mentioned various method embodiments can be realized. The above-mentioned computer program includes computer program code, which can be in the form of source code, object code, executable file or some intermediate form, etc. The above-mentioned computer readable medium can include any entity or device capable of carrying the above-mentioned computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the above-mentioned computer readable storage medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction.

[0140] The above description of disclosed embodiments enables one of ordinary skill in the art to make or use the application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein can be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0141] It can be clearly understood by a person skilled in the art that, for the convenience and conciseness of description, only the division of the above functional units and modules is taken as an example, and in actual application, the above functions can be completed by different functional units and modules according to needs, that is, the internal structure of the above device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit. In addition, the specific names of each functional unit and module are only for the convenience of mutual distinction, and are not used to limit the protection scope of the application. The specific working process of the units and modules in the above system can refer to the corresponding process in the foregoing method embodiments, and will not be described here.

[0142] It should be noted that the method and its details provided by the above embodiments can be combined into the device and equipment provided by the embodiments, and mutual reference will not be described here.

[0143] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized in electronic hardware or in a combination of computer software and electronic hardware. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0144] In the embodiments provided by the present application, it should be understood that the disclosed device / terminal equipment and method can be implemented in other ways. For example, the above-described device / device embodiments are only schematic, for example, the division of the above modules or units is only a logical function division, and an actual implementation can be different, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0145] The above examples are only used to illustrate the technical solutions of the present application, and are not intended to limit the present application; although the present application has been described in detail with reference to the foregoing examples, those skilled in the art should understand that the technical solutions recorded in the foregoing examples can be modified, or some technical features can be replaced by equivalent features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A test task scheduling method for a control system based on graph models and deep reinforcement learning, characterized in that, The test task scheduling method includes the following steps: Step 1: Analyze the requirements of the test tasks and then construct a Single-rate DAG task scheduling model; Step 2: Based on the Single-rate DAG task scheduling model, generate a directed graph as the input to the GNN; Step 3: Construct a heatmap of edges based on the directed graph and GNN; Step 4: Generate a task scheduling order based on the heatmap, and plan a scheduling scheme based on the task scheduling order; Step 5: Calculate the loss function and update the graph neural network based on the loss function; Step 6: Repeat steps 3-5 until a high-quality solution that meets the expected goal is planned, or the difference between the mean and minimum value of the test scheme makespan planned in step 4 is less than a certain threshold, then the iteration ends; Step 7: Output the optimized solution; Step 2 specifically involves simplifying the DAG task scheduling model. Iterate through all tasks. If there are no constraints between two tasks, including direct and indirect constraints, then add an edge between the two tasks to indicate that there are no dependencies between them. Add a new weight to the new edge to distinguish the edge type; assign different weights to the original edges and the newly added edges; In the newly generated directed graph, the meaning of the nodes remains unchanged, but the meaning of the edges changes, indicating that there is a continuous execution between two tasks. When, it means at After execution, the next task to begin execution is It exists; Define and initialize a graph neural network (GNN) with trainable parameters θ; the GNN backbone and MLP decoder are used to form the graph neural network, which extracts the node features and edge features of the graph and maps them to the edge weights of the graph. Step 3 specifically involves inputting the given directed graph into the GNN model. The GNN model will output a set of edge weights based on the current parameters, and use these weights to construct a heatmap W = [w...] on the edges of the directed graph. ij ] (n+2)(n+2) , where w i,j Representing an edge The weights that belong to high-quality solutions.

2. The test task scheduling method according to claim 1, characterized in that, Step 1 specifically involves obtaining the test task set T = {t1, t2, t3, ..., t}. n } and the test resource set R = {r1, r2, r3, ..., r m }; where each test task t i The attributes include the precursor mission set Subsequent task set Duration Subsequent communication time and test resources Test resource r i Has attributes This indicates the number of available test resources; As the entry point for the DAG task scheduling model, As the exit point of the DAG task scheduling model, a single-rate DAG is constructed based on it; thus, the start and end markers of the task are clearly defined.

3. The test task scheduling method according to claim 2, characterized in that, Specifically, step 1 is as follows: The DAG task scheduling model consists of a set of nodes and an edge set of the DAG, denoted as G = (V, E); nodes represent tasks in the system, and edge sets represent the constraints between tasks. V is the set of all nodes, represented as V = {v1, ..., v}. |V| }, where |V| is the total number of nodes. In the DAG task scheduling model, |V| = n + 2; each node v i Used to represent a corresponding task t i Node v i The weight is task t i Its own information; E is the set of all edges, where each edge e i,j ∈E represents v i and v j The dependency relationship between them, when there is e in the DAG i,j At that time, in v i Execution completed and v i v cannot be executed until the output of is reached. j ; Each edge e i,j Weights exist to express the relationships between tasks.

4. The test task scheduling method according to claim 1, characterized in that, Step 4 specifically involves, during the task scheduling and sorting process, taking the test task as the first task in the sort and using it as the starting point for task scheduling, and generating a set of schedulable tasks T based on the current task sorting status. allow ; The specific rule is as follows: when all the predecessor tasks of a certain task have been sorted, the current task is added to T. allow Based on the previously scheduled task Generating T from heatmap W allow Corresponding state transition probability in Based on the state transition probability, select the next scheduled task; move the selected scheduled task from T. allow Delete it and update T according to the method described above. allow Then, select the next scheduled task, until all test tasks in all test task sets have been sorted; Based on the sorting, allocate test resources and start execution time to the tasks, and plan the task scheduling scheme.

5. The test task scheduling method according to claim 4, characterized in that, For the allocation of test resources, based on the greedy algorithm, the earliest available task time for each type of test resource is calculated, and the earliest available resource with the idle time for each type of test resource is selected to allocate test tasks. Based on the allocated test resources, calculate the test start time using the following formula. Among them, the set of predecessor tasks with higher priority than the current task is used Indicates; task t j The start time is expressed as Task t j The end time is expressed as The subsequent communication time of the task is denoted as Indicates whether communication is required between the current task and its predecessor task. This indicates that the two tasks have the same test resources and do not need to communicate. This indicates that communication time needs to be considered; Indicates the earliest idle time for all selected test resources; It is the maximum value of the start time of the previous test task, the maximum value of the completion of the predecessor task in executing and transmitting data, and the earliest idle time of all selected test resources; the test start time planned by this formula avoids the conflict problem caused by decision-making during task scheduling, or obtains the optimal scheduling result within the maximum limit. Based on the above method, multiple test scheduling schemes are generated as the basis for subsequent updates to the graph neural network.

6. The test task scheduling method according to claim 5, characterized in that, Step 5 specifically involves: calculating the test time makespan of the current test scheduling scheme; using the REINFORCE algorithm to calculate the loss function; updating the trainable parameters θ of the graph neural network based on the calculation results; and performing gradient pruning to avoid the gradient explosion problem.

7. A test task scheduling system for a control system based on graph models and deep reinforcement learning, characterized in that, The test task scheduling system uses the test task scheduling method for control systems based on graph models and deep reinforcement learning as described in any one of claims 1-6, and the test task scheduling system includes, The model building module analyzes the test task requirements and then builds a Single-rate DAG task scheduling model. The directed graph generation module generates a directed graph based on the Single-rate DAG task scheduling model, which serves as the input to the GNN. The heatmap creation module generates heatmaps about edges based on directed graphs and GNNs. The scheduling scheme planning module generates a task scheduling order based on the heatmap and plans a scheduling scheme based on the task scheduling order. The computation module calculates the loss function and updates the graph neural network based on the loss function. The iterative module uses the heatmap building module, scheduling scheme planning module, and calculation module to iterate until a high-quality solution that meets the expected goal can be planned, or the difference between the mean and minimum value of the test scheme makespan planned in step 4 is less than a certain threshold, at which point the iteration ends. The output module outputs the optimized solution.

8. A computer device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements the method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method described in any one of claims 1-6.

Images