Linear integer programming method and system for intelligent power grid data analysis job scheduling

Abstract

The invention discloses a linear integer programming method and system for intelligent power grid data analysis job scheduling, and relates to the field of power grid data analysis job scheduling, and the method comprises the steps: carrying out the modeling of an intelligent power grid data analysis job as a periodic job, and defining a system model; establishing a joint calculation and communication task scheduling problem based on the system model; a joint calculation and communication task scheduling problem is converted from a nonlinear optimization problem to a linear optimization problem; a branch cutting algorithm is improved, the linear optimization problem is solved, and planning of intelligent power grid data analysis job scheduling is completed; according to the method, data transmission delay can be reduced, the response speed of an intelligent power grid system is improved, the process of searching an optimal solution is effectively accelerated by using an improved branch cutting algorithm, the search space is reduced, and the efficiency of the algorithm is improved; and actual calculation and communication resource constraints, such as non-preemptive scheduling of machines and sharing of network channels, are considered, so that the model and the solution are closer to the actual application scene of the intelligent power grid.

Description

Technical Field

[0001] The present invention relates to the field of power grid data analysis job scheduling, and in particular to a linear integer programming method and system for smart grid data analysis job scheduling. Background Art

[0002] With the rapid development of information technology, smart grid, as an important part of the modernization of power system, is gradually becoming a key infrastructure for global energy management. By integrating advanced information and communication technologies, smart grid realizes real-time monitoring and control of power production, distribution and consumption, greatly improving the efficiency and reliability of power grid. Among them, the widespread application of smart meters and sensors enables the power grid to collect massive amounts of data, which is crucial for the operation status analysis, fault diagnosis, energy consumption optimization, etc. of the power grid.

[0003] However, the massive growth of data in smart grids has brought great challenges to data processing and analysis. Traditional data processing systems are usually designed for general applications, focusing on data partitioning and computing, rarely optimizing data transmission performance, and failing to fully consider the particularity of smart grid data. In smart grid systems, real-time analysis of data is crucial to ensure stable operation of the grid and respond to emergencies. Therefore, how to efficiently process and analyze this data has become an urgent problem to be solved. In order to minimize the completion time of data analysis jobs, traditional works are designed for general applications and focus on computing task placement or network flow scheduling. However, the separation of scheduling computing and communication tasks leads to inefficient job processing performance when the data volume is huge. Specifically, the data transmission scheduling problem considering data volume has been well studied. Since their goal is to minimize network congestion rather than reduce the overall job completion time, the joint optimization of computing and data transmission is not considered. To overcome this, some researchers have recently begun to break the barriers and try to coordinate computing and communication tasks, such as the Symbiosis framework and the Firebird framework, but these works aim to reduce the completion time of computing or communication tasks rather than optimize the entire job. Some researchers have considered this and proposed a joint computing and communication task scheduling problem, but it cannot be directly applied to smart grids, so there is still a lack of smart grid application solutions for this problem. At the same time, the inherent causal relationship between computing task placement and data transmission conditions in data analysis operations will greatly increase the complexity of the integrated scheduling problem. Summary of the invention

[0004] In view of the above-mentioned problems, the present invention is proposed.

[0005] Therefore, the technical problem solved by the present invention is: real-time analysis and processing performance optimization of massive data in smart grids, especially solving the problems of data transmission delay and long job completion time by jointly scheduling computing tasks and data transmission.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0007] In a first aspect, an embodiment of the present invention provides a linear integer programming method for scheduling smart grid data analysis jobs, including:

[0008] Model the smart grid data analysis job as a periodic job and define the system model;

[0009] Based on the system model, establish the joint computing and communication task scheduling problem;

[0010] Transform the joint computing and communication task scheduling problem from a nonlinear optimization problem to a linear optimization problem;

[0011] Improve the branch cutting algorithm, solve the linear optimization problem, and complete the planning of smart grid data analysis job scheduling.

[0012] As a preferred solution of the linear integer programming method for smart grid data analysis job scheduling,

[0013] The smart grid data analysis operation is modeled as a periodic operation, and the system model is defined including:

[0014] Each job is represented by a double-weighted directed acyclic graph G = {V, E, P, Q, R}, where V is the computing task set and v j ∈V represents the jth computing task; task v j The execution duration j time slots, P = {p j |1≤j≤|V|} is the execution time set of computing tasks; E is the double-weight edge set, edge e uv ∈E represents the dependency relationship between computing tasks u and v. The two weights of an edge represent the internal transmission time. and external transmission time

[0015] In the job, a set of available machines M = {α i |1≤i≤M} to process data, the machines share a set of network resources N={β k |1≤k≤N}; divide the time axis T={τ|1≤τ≤max} into multiple identical time slots, assume that the processing time of each computing task and the transmission time of each data are predetermined, and the computing tasks and network flows are non-preemptive.

[0016] As a preferred solution of the linear integer programming method for smart grid data analysis job scheduling,

[0017] The joint computing and communication task scheduling problem based on the system model includes:

[0018] Considering computing resource constraints, causality constraints, priority constraints and communication resource constraints, the optimization goal is set to minimize the job completion time, that is, the maximum completion time among all communication and computing tasks.

[0019] As a preferred solution of the linear integer programming method for smart grid data analysis job scheduling,

[0020] The transformation of the joint computing and communication task scheduling problem from a nonlinear optimization problem to a linear optimization problem includes:

[0021] The concept of general flow and auxiliary virtual channels are introduced into the joint computing and communication task scheduling problem. The computation disjointness and communication disjointness constraints are linearized, as well as the causality and priority constraints are linearized. The joint computing and communication task scheduling problem is transformed from a nonlinear optimization problem to a linear optimization problem.

[0022] As a preferred solution of the linear integer programming method for smart grid data analysis job scheduling,

[0023] The converting of the joint computing and communication task scheduling problem from a nonlinear optimization problem to a linear optimization problem also includes:

[0024] The linearized constraint problem is expressed as:

[0025] P2:minC max

[0026] st(1),(3),(11)-(20),

[0027]

[0028]

[0029]

[0030] As a preferred solution of the linear integer programming method for smart grid data analysis job scheduling,

[0031] The improved branch and cut algorithm for solving the linear optimization problem includes:

[0032] Considering the topological relationship between computing tasks and network flows in a dual-weight directed acyclic graph, an improved branch and cut algorithm is proposed to solve linear optimization problems.

[0033] As a preferred solution of the linear integer programming method for smart grid data analysis job scheduling,

[0034] The improved branch cutting algorithm comprises:

[0035] Given a job G = {V, E, P, Q, R}, resource sets {M} and {N}, use a heuristic algorithm to solve it, set upper and lower bounds, an initial priority matrix, chained priority constraints in G, initial interval constraints, and calculate the affinity and priority of tasks / communication flows; repeatedly use the branch and cut method to calculate the current solution, update the interval constraints, until the optimal solution is found.

[0036] In a second aspect, an embodiment of the present invention provides a linear integer programming system for scheduling smart grid data analysis jobs, including:

[0037] The system modeling module is used to model the smart grid data analysis job as a periodic job and define the system model;

[0038] Scheduling problem establishment module, used to establish joint computing and communication task scheduling problems based on the system model;

[0039] A conversion module, used to convert the joint computing and communication task scheduling problem from a nonlinear optimization problem to a linear optimization problem;

[0040] The solution module is used to improve the branch cutting algorithm, solve the linear optimization problem, and complete the planning of smart grid data analysis job scheduling.

[0041] In a third aspect, an embodiment of the present invention provides a computing device, including:

[0042] Memory and processor;

[0043] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions. When the one or more programs are executed by the one or more processors, the one or more processors implement the linear integer programming method for smart grid data analysis job scheduling as described in any embodiment of the present invention.

[0044] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the linear integer programming method for smart grid data analysis job scheduling.

[0045] Beneficial effects of the present invention: The present invention particularly optimizes the data volume of data analysis tasks in smart grids, and takes into account the coupling relationship between data transmission and computing tasks, which is crucial for processing large-scale data; by jointly scheduling computing tasks and data transmission, it can reduce data transmission delays and improve the response speed of smart grid systems, which is particularly important for smart grid applications that require real-time control and monitoring; utilizing the topological information of the data analysis job graph and using an improved branch cutting algorithm to effectively accelerate the process of finding the optimal solution, reduce the search space, and improve the efficiency of the algorithm; taking into account actual computing and communication resource constraints, such as non-preemptive scheduling of machines and sharing of network channels, the model and solution are closer to the actual application scenarios of smart grids. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0047] Figure 1 It is an overall flow chart of the linear integer programming method for smart grid data analysis job scheduling described in the first embodiment of the present invention. DETAILED DESCRIPTION

[0048] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0049] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0050] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0051] Example 1

[0052] Reference Figure 1 , which is the first embodiment of the present invention, provides a linear integer programming method for scheduling smart grid data analysis jobs, including:

[0053] S1: Model the smart grid data analysis job as a periodic job and define the system model;

[0054] S2: Based on the system model, establish the joint computing and communication task scheduling problem;

[0055] S3: Convert the joint computing and communication task scheduling problem from a nonlinear optimization problem to a linear optimization problem;

[0056] S4: Improve the branch cutting algorithm, solve the linear optimization problem, and complete the planning of smart grid data analysis job scheduling.

[0057] In the embodiment of the present application, in the above step S1, the smart grid data analysis operation is modeled as a periodic operation, and the definition of the system model includes:

[0058] Each job is represented by a dual-weight directed acyclic graph G = {V, E, P, Q, R}, where V is the set of computing tasks and v j ∈V represents the jth computing task; task v j The execution duration j time slots, P = {p j ||1≤j≤|V|} is the execution time set of computing tasks; E is the set of double-weighted edges, edge e uv ∈E represents the dependency relationship between computing tasks u and v. The two weights of an edge represent the internal transmission time. and external transmission time

[0059] In the job, a set of available machines M = {α i ||1≤i≤M} to process data, and the machines share a set of network resources N={β k |1≤k≤N}; divide the time axis T={τ|1≤τ≤max} into multiple identical time slots, assume that the processing time of each computing task and the transmission time of each data are predetermined, and the computing tasks and network flows are non-preemptive.

[0060] In the embodiment of the present application, in the above step S2, establishing the joint computing and communication task scheduling problem based on the system model includes:

[0061] Considering computing resource constraints, causality constraints, priority constraints and communication resource constraints, the optimization goal is set to minimize the job completion time, that is, the maximum completion time among all communication and computing tasks.

[0062] Specifically, the optimization goal is to minimize the job completion time, which is expressed as:

[0063]

[0064] Where X jiτ represents a binary computation task placement decision variable; if task v j At time τ, start on machine α i Execute on, X jiτ =1; otherwise X jiτ = 0. fkτ is the binary network flow scheduling decision variable, Y fkτ = 1 means that flow f starts at time τ on the communication channel β k So the constraints are satisfied:

[0065]

[0066]

[0067] Specifically, the computing resource constraints are disjoint constraints, that is, the execution time of computing tasks on the same machine will not overlap, including constraints (3) and (4):

[0068]

[0069]

[0070] The causal constraint is that the flow f uv The occurrence of depends on the placement of tasks u and v with sequential constraints, expressed as:

[0071]

[0072] The priority constraint is that there is a priority relationship between two adjacent upstream and downstream computing tasks. A task starts to execute only after all its predecessor tasks and upstream necessary flows have finished. According to the location of the tasks with priority constraints on the same machine, two situations need to be considered, namely, the situation where tasks u and v are placed on the same machine or on different machines, and the constraints are (6), (7), and (8) respectively:

[0073]

[0074]

[0075]

[0076] The communication resource constraints are disjoint constraints, that is, the flow transmissions on the same network channel will not overlap, including constraints (9) and (10):

[0077]

[0078]

[0079] In the embodiment of the present application, in the above step S3, converting the joint computing and communication task scheduling problem from a nonlinear optimization problem to a linear optimization problem includes:

[0080] The concept of general flow and auxiliary virtual channels are introduced into the joint computing and communication task scheduling problem. The computation disjointness and communication disjointness constraints are linearized, as well as the causality and priority constraints are linearized. The joint computing and communication task scheduling problem is transformed from a nonlinear optimization problem to a linear optimization problem.

[0081] Specifically, the nonlinear constraint problem can be expressed as

[0082] P1: minC max

[0083] st(1)-(10)

[0084] It should be noted that a general flow is an internal or external flow transmitted between two tasks with sequential constraints. In order to build a unified flow scheduling framework compatible with the two types of flows, an auxiliary virtual channel is introduced, which is contention-free for all internal flow transmissions.

[0085] Therefore, the external flow is transmitted through the physical network link N, while the internal flow is transmitted through the auxiliary virtual channel Since each generic flow must be transmitted, constraint (11) is obtained:

[0086]

[0087] In the computationally disjoint constraint linearization, two types of auxiliary variables are introduced to describe the placement of computational tasks. jj′i ∈{0, 1} represents two computing tasks v j and v j′ Whether it is placed on the same machine i, if on the same machine ψ jj′i = 1. Binary priority indicator variable σ jj′ ∈{0, 1} represents the priority relationship between two computing tasks. If task v j No later than task v j′ At the beginning, then σ jj′ =1. jj′i and σ jj′ And the reconstruction of constraint (4) is:

[0088]

[0089]

[0090]

[0091] In the linearization of the communication disjointness constraint, an auxiliary binary variable χ is introduced ff′k and φ ff′ . χ ff′k indicates whether both flows are placed on the kth communication channel, while the priority indicator variable φ ff′ Indicates the priority relationship between two flows. ff′k and φ ff′ The constraints and the reconstruction of constraint (10) are as follows:

[0092]

[0093]

[0094]

[0095] The linearization of the causality constraint is achieved by introducing an auxiliary variable ψ jj′i ,The causal relationship between the placement of computing tasks and the occurrence of network flows in constraint (6) can be rewritten as:

[0096]

[0097] After linearization of the priority constraints, we can include constraints (19) and (20):

[0098]

[0099]

[0100] The linearized constraint problem is expressed as:

[0101] P2:minC max

[0102] st(1),(3),(11)-(20),

[0103]

[0104]

[0105]

[0106] In the embodiment of the present application, the branch and cut algorithm is improved in the above step S4 to solve the linear optimization problem including:

[0107] Considering the topological relationship between computing tasks and network flows in a dual-weight directed acyclic graph, an improved branch and cut algorithm is proposed to solve the optimization problem.

[0108] Specifically, the topological relationship between computing tasks and network flows in a dual-weight directed acyclic graph includes:

[0109] Branching is performed based on chain priority constraints, that is, based on the priority relationship between computing and communication tasks, some search branches are directly pruned to reduce the search space; task interval constraints are updated, that is, in the dual-weight directed acyclic graph of each job, the earliest and latest start time of each computing task or network flow can be inferred based on the current upper and lower bounds of the job and the processing and transmission time of other computing tasks and flows along its longest branch; the symmetry of the solution space is utilized, one of the symmetry features comes from the homogeneity of machines and channels, and setting the machine affinity value of each task in advance can eliminate a large number of symmetric solutions; the other symmetry feature comes from the symmetry of nodes or edges, adding different priorities to equivalent computing tasks or network flows to reduce redundant searches.

[0110] The steps of the improved branch and cut algorithm are as follows: given a job G = {V, E, P, Q, R}, and resource sets {M} and {N}, first use the heuristic algorithm to solve, set the upper and lower bounds, the initial priority matrix, the chain priority constraints in G, the initial interval constraints, and calculate the affinity and priority of the task / communication flow. Then repeatedly use the branch and cut method to calculate the current solution and update the interval constraints until the optimal solution is found.

[0111] The above is a schematic scheme of the linear integer programming method for smart grid data analysis job scheduling of this embodiment. It should be noted that the technical scheme of the linear integer programming system for smart grid data analysis job scheduling and the technical scheme of the linear integer programming method for smart grid data analysis job scheduling described above belong to the same concept. For the details of the technical scheme of the linear integer programming system for smart grid data analysis job scheduling not described in detail in this embodiment, please refer to the description of the technical scheme of the linear integer programming method for smart grid data analysis job scheduling described above.

[0112] The linear integer programming system for smart grid data analysis job scheduling in this embodiment includes:

[0113] The system modeling module is used to model the smart grid data analysis job as a periodic job and define the system model;

[0114] Scheduling problem establishment module, used to establish joint computing and communication task scheduling problems based on the system model;

[0115] A conversion module, used to convert the joint computing and communication task scheduling problem from a nonlinear optimization problem to a linear optimization problem;

[0116] The solution module is used to improve the branch cutting algorithm, solve the linear optimization problem, and complete the planning of smart grid data analysis job scheduling.

[0117] This embodiment further provides a computing device, which is applicable to the case of a linear integer programming method for scheduling smart grid data analysis jobs, including:

[0118] Memory and processor; the memory is used to store computer executable instructions, and the processor is used to execute computer executable instructions to implement the linear integer programming method for smart grid data analysis job scheduling proposed in the above embodiment.

[0119] This embodiment also provides a storage medium on which a computer program is stored. When the program is executed by a processor, the linear integer programming method for scheduling smart grid data analysis jobs as proposed in the above embodiment is implemented.

[0120] The storage medium proposed in this embodiment and the linear integer programming method for smart grid data analysis job scheduling proposed in the above embodiment belong to the same inventive concept. The technical details not described in detail in this embodiment can be referred to the above embodiment, and this embodiment has the same beneficial effects as the above embodiment.

[0121] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A linear integer programming method for scheduling smart grid data analysis jobs, characterized in that: include: Model the smart grid data analysis job as a periodic job and define the system model; Based on the system model, establish the joint computing and communication task scheduling problem; Transform the joint computing and communication task scheduling problem from a nonlinear optimization problem to a linear optimization problem; Improve the branch cutting algorithm, solve the linear optimization problem, and complete the planning of smart grid data analysis job scheduling.

2. The linear integer programming method for smart grid data analysis job scheduling according to claim 1, characterized in that: The smart grid data analysis operation is modeled as a periodic operation, and the system model is defined including: Each job is represented by a dual-weight directed acyclic graph G = {V, E, P, Q, R}, where V is the computing task set and v j ∈V represents the jth computing task; task v j The execution duration j time slots, P = {p j |1≤j≤|V|} is the execution time set of computing tasks; E is the double-weight edge set, edge e uv ∈E represents the dependency relationship between computing tasks u and v. The two weights of an edge represent the internal transmission time. and external transmission time In the job, a set of available machines M = {α i |1≤i≤M} to process data, the machines share a set of network resources N={β k |1≤k≤N}; divide the time axis T={τ|1≤τ≤max} into multiple identical time slots, assume that the processing time of each computing task and the transmission time of each data are predetermined, and the computing tasks and network flows are non-preemptive.

3. The linear integer programming method for smart grid data analysis job scheduling according to claim 2, characterized in that: The system model-based joint computing and communication task scheduling problem includes: Considering computing resource constraints, causality constraints, priority constraints and communication resource constraints, the optimization goal is set to minimize the job completion time, that is, the maximum completion time among all communication and computing tasks.

4. The linear integer programming method for smart grid data analysis job scheduling according to claim 3, characterized in that: The transformation of the joint computing and communication task scheduling problem from a nonlinear optimization problem to a linear optimization problem includes: The concept of general flow and auxiliary virtual channels are introduced into the joint computing and communication task scheduling problem. The computation disjointness and communication disjointness constraints are linearized, as well as the causality and priority constraints are linearized. The joint computing and communication task scheduling problem is transformed from a nonlinear optimization problem to a linear optimization problem.

5. The linear integer programming method for smart grid data analysis job scheduling according to claim 4, characterized in that: The converting of the joint computing and communication task scheduling problem from a nonlinear optimization problem to a linear optimization problem also includes: The linearized constraint problem is expressed as: P2:minC max st(1),(3),(11)-(20), 6. The linear integer programming method for smart grid data analysis job scheduling according to claim 5, characterized in that: The improved branch and cut algorithm for solving the linear optimization problem includes: Considering the topological relationship between computing tasks and network flows in a dual-weight directed acyclic graph, an improved branch and cut algorithm is proposed to solve linear optimization problems.

7. The linear integer programming method for smart grid data analysis job scheduling according to claim 6, characterized in that: The improved branch cutting algorithm comprises: Given a job G = {V, E, P, Q, R}, resource sets {M} and {N}, use a heuristic algorithm to solve it, set upper and lower bounds, an initial priority matrix, chained priority constraints in G, initial interval constraints, and calculate the affinity and priority of tasks / communication flows; repeatedly use the branch and cut method to calculate the current solution, update the interval constraints, until the optimal solution is found.

8. A system using the linear integer programming method for scheduling smart grid data analysis tasks as claimed in any one of claims 1 to 7, characterized in that: include: The system modeling module is used to model the smart grid data analysis job as a periodic job and define the system model; Scheduling problem establishment module, used to establish joint computing and communication task scheduling problems based on the system model; A conversion module, used to convert the joint computing and communication task scheduling problem from a nonlinear optimization problem to a linear optimization problem; The solution module is used to improve the branch cutting algorithm, solve the linear optimization problem, and complete the planning of smart grid data analysis job scheduling.

9. A computing device comprising: Memory and processor; The memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions. When the computer executable instructions are executed by the processor, the steps of the linear integer programming method for smart grid data analysis job scheduling as described in any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the steps of the linear integer programming method for smart grid data analysis job scheduling as described in any one of claims 1 to 7.

Images