A business path scheduling method, device and storage medium based on virtual power plant

Abstract

The present invention discloses a service path scheduling method, device and storage medium based on a virtual power plant, including: determining all paths that meet constraint conditions for each service based on the network topology of the virtual power plant; inputting the paths into a pre-trained first DQN network model to obtain a main routing path; obtaining a backup routing path with the lowest primary-backup routing similarity based on a pre-trained second DQN network model and the main routing path. By implementing the present invention, all alternative routing solutions that meet multiple constraints are first found for the source and destination nodes of each service based on the network topology of the virtual power plant; then, based on the DQN network algorithm, the main routing path and the backup routing path that meets the minimum routing similarity and has the best possible indicators are determined. In this way, the method can ensure the stable execution of the deterministic delay service of the virtual power plant and improve the reliability of the virtual power plant service.

Description

Technical Field

[0001] The present invention relates to the field of virtual power plant communication technology, and in particular to a service path scheduling method, device and storage medium based on a virtual power plant. Background Art

[0002] As the integration point of energy supply and energy consumption, virtual power plants have various business types, mainly including power frequency regulation, power peak regulation, large power grid stability control and power market transactions. Since these businesses involve power control and transactions, they have extremely high requirements for business latency. Traditional network deployment methods and path scheduling methods are often based on pre-set paths, which make it difficult to actively adapt to the QoS (Quality of Service) requirements of the business, and thus lack latency sensitivity and reliability.

[0003] In order to meet the latency-sensitive requirements of virtual power plant services, a network architecture based on SDN (Software Defined Network) can be adopted. The network architecture based on SDN can program the network from the overall level of the network, realize intelligent wide-area path scheduling, actively perceive the QoS requirements such as bandwidth and latency of the service and the status of communication network resources, select reliable service paths while balancing the computing time, policy execution time, and path latency, and realize rapid and reasonable allocation and processing of resources between and within domains.

[0004] However, the service path selected by the existing technology is usually only one, and the backup routing scheme in the case of node damage in the routing is not considered, which has certain limitations. Summary of the invention

[0005] In view of this, an embodiment of the present invention provides a service path scheduling method, device and storage medium based on a virtual power plant to solve the technical problem that the prior art does not consider the backup routing solution in the case of node damage in the routing.

[0006] The technical solution proposed by the present invention is as follows:

[0007] A first aspect of an embodiment of the present invention provides a service path scheduling method based on a virtual power plant, comprising: determining all paths that meet constraint conditions for each service based on the network topology of the virtual power plant; inputting the paths into a pre-trained first DQN network model to obtain a main routing path; and obtaining a backup routing path with the lowest primary-backup routing similarity based on a pre-trained second DQN network model and the main routing path.

[0008] Optionally, all paths that meet constraint conditions for each service are determined based on the network topology of the virtual power plant, including: applying a traversal algorithm to the network topology of the virtual power plant to determine all paths that meet constraint conditions for each service; the constraint conditions include bandwidth constraints and delay constraints.

[0009] Optionally, the path is input into a pre-trained first DQN network model to obtain a main routing path, including: determining a main routing objective function based on a set of services, a total delay of each service, and a link balance; determining an immediate reward function of the main routing based on the main routing objective function; determining a loss function of the first DQN network based on the immediate reward function of the main routing; training the first DQN network based on the first DQN network loss function to obtain a trained first DQN network model; and inputting the path into the trained first DQN network model to obtain the main routing path.

[0010] Optionally, the main routing objective function is determined based on the set of services, the total delay of each service, and the link balance, including: summing the propagation delay, transmission delay and processing delay of the service to obtain the total delay of each service; calculating the link balance according to the sum of the importance of the services carried on each link and the total number of links; and determining the main routing objective function according to the set of services, the total delay of each service and the link balance.

[0011] Optionally, obtaining a backup routing path with the lowest primary-backup routing similarity based on a pre-trained second DQN network model and a main routing path includes: inputting the path and the main routing path into a pre-trained second DQN network model, and determining the backup routing path with the lowest primary-backup routing similarity based on the primary-backup routing similarity calculated when the second DQN network model is running.

[0012] Optionally, the path satisfying the constraint condition is represented by the following matrix:

[0013]

[0014] Where k represents the kth service, v i and v j They respectively represent the starting point and end point of the path in the network topology of the virtual power plant.

[0015] Optionally, the state space of the DQN network is composed of the row number index of each business, and the row number index is the matrix C k The index of each row in; the action space of the DQN network is represented by the indicator matrix C k The actions of moving the current row up, moving the current row down, and keeping the current row unchanged are composed.

[0016] A second aspect of an embodiment of the present invention provides a service path scheduling device based on a virtual power plant, comprising: a first path determination module, used to determine all paths that meet constraint conditions for each service based on the network topology of the virtual power plant; a main path determination module, used to input the path into a pre-trained first DQN network model to obtain a main routing path; and a backup path determination module, used to obtain a backup routing path with the lowest primary-backup routing similarity based on a pre-trained second DQN network model and the main routing path.

[0017] A third aspect of an embodiment of the present invention provides a computer-readable storage medium, which stores computer instructions, and the computer instructions are used to enable the computer to execute the business path scheduling method based on a virtual power plant as described in the first aspect of the embodiment of the present invention and any one of the first aspects.

[0018] A fourth aspect of an embodiment of the present invention provides an electronic device, comprising: a memory and a processor, the memory and the processor being communicatively connected to each other, the memory storing computer instructions, and the processor executing the computer instructions to execute the business path scheduling method based on a virtual power plant as described in the first aspect of the embodiment of the present invention and any one of the first aspects.

[0019] The technical solution provided by the present invention has the following effects:

[0020] The service path scheduling method, device and storage medium based on the virtual power plant provided by the embodiment of the present invention first find all alternative routing solutions that meet multiple constraints for the source and destination nodes of each service based on the network topology of the virtual power plant; then determine the main routing path and the backup routing path that satisfies the minimum routing similarity and the best possible indicators based on the DQN network algorithm. Therefore, the service path scheduling method can ensure the stable execution of the deterministic delay service of the virtual power plant by determining the main and backup routing paths, thereby improving the reliability of the virtual power plant service. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0022] Figure 1 is a flow chart of a service path scheduling method based on a virtual power plant according to an embodiment of the present invention;

[0023] Figure 2is a flow chart of a service path scheduling method based on a virtual power plant according to another embodiment of the present invention;

[0024] Figure 3 is a training flow chart of a DQN network according to an embodiment of the present invention;

[0025] Figure 4 is a structural block diagram of a service path scheduling device based on a virtual power plant according to an embodiment of the present invention;

[0026] Figure 5 is a schematic diagram of the structure of a computer-readable storage medium provided according to an embodiment of the present invention;

[0027] Figure 6 is a schematic diagram of the structure of an electronic device provided according to an embodiment of the present invention. DETAILED DESCRIPTION

[0028] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0029] The terms "first", "second", "third", "fourth", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0030] According to an embodiment of the present invention, a business path scheduling method based on a virtual power plant is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0031] In this embodiment, a service path scheduling method based on a virtual power plant is provided, which can be used in electronic devices such as computers, mobile phones, tablet computers, etc. Figure 1 is a flow chart of a service path scheduling method based on a virtual power plant according to an embodiment of the present invention. Figure 1 As shown, the method comprises the following steps:

[0032] Step S101: Determine all paths that meet the constraints for each service based on the network topology of the virtual power plant. Specifically, when constructing the network topology of the virtual power plant, first obtain the physical structure of the network in the virtual power plant, and then abstract it to obtain the network topology architecture. The network topology can be represented as G(V,E). Among them, V represents the set of nodes in the network; E represents the set of edges in the network. The node can be a network node such as a router or switch in the network.

[0033] The path between two points in the network is represented by e ij The starting point and end point of the path are v i and v j , where e ij ∈E,v i v j ∈V. The services that need to be sent in the network can be expressed as L, that is, L is the set of services that need to be sent in the virtual power plant, and l represents the total number of services in the set.

[0034] In order to determine the optimal path for service transmission, all paths for service transmission can be determined first, that is, the path between the source node and the destination node of the service. At the same time, in order to reduce the amount of subsequent calculations, the path that meets the constraints is determined by traversal. For example, a depth-first traversal method or a breadth-first traversal method can be used. The present invention does not limit the specific method for determining the path. For each service, the corresponding path that meets the constraints can be determined. For example, for the kth service, all its paths that meet the constraints can be represented by a matrix:

[0035]

[0036] Where k represents the kth service, v i and v j They respectively represent the starting point and end point of the path in the network topology of the virtual power plant.

[0037] In this matrix, each row represents a path that satisfies the constraints. The total number of rows in the matrix is ​​R, which means that it contains a total of R paths that satisfy the constraints. The number of columns in the matrix n represents the number of nodes in the network. Each row of the matrix v i to v j The space between them represents the node sequence of each path, followed by 0 as padding.

[0038] The constraints that the path satisfies include bandwidth constraints and delay constraints. The bandwidth constraint is expressed as:

[0039]

[0040] Where b k represents the bandwidth occupied by service k, F ij Represents the remaining bandwidth capacity of edge (i, j).

[0041] The delay constraint is expressed as:

[0042]

[0043] Where, T k is the delay of the alternative path for service k, T k ={t1,t2,....,t R}, t1, t2, ...., t R Corresponding to the above matrix C k The delay of each path in k is the delay of all paths that meet the constraints of the k-th service, is the preset minimum delay value of service k, is the preset maximum delay value of service k.

[0044] Since the path constraints include delay constraints, that is, the corresponding delay needs to meet certain conditions, this service can also be called a deterministic delay service.

[0045] Step S102: Input the path into the pre-trained first DQN network model to obtain the main routing path. According to the above steps, it can be known that by traversing, all paths that meet the constraint conditions between any two nodes can be determined. In order to determine the main routing path from these paths, a DQN (Deep Q-learning) algorithm can be used to pre-train a DQN network model, and then the determined path is input into the DQN network model, so that the optimal main path between the two nodes selected by the first DQN network model can be obtained.

[0046] Among them, the DQN algorithm is a classic reinforcement learning algorithm, and reinforcement learning refers to the influence of the environment through the agent, that is, the initiator of the action. After the environment accepts the action, the state changes, and a reinforcement signal (reward or punishment) is generated and fed back to the agent. The agent selects the next action based on the reinforcement signal and the current state of the environment. The principle of selection is to increase the probability of receiving positive reinforcement (reward). The selected action not only affects the immediate reinforcement value, but also affects the state of the environment at the next moment and the final reinforcement value. In reinforcement learning, there are two basic elements: state and action. Performing a certain action in a certain state is a strategy. What the learner needs to do is to obtain a good strategy through continuous exploration and learning.

[0047] Therefore, in order to enable the first DQN network model to select the optimal main path from the determined paths, the state space and action space of the DQN algorithm can be defined based on the determined paths. The state space of the DQN network is composed of the row number index of each business, and the row number index is the matrix C k The index of each row in; the action space of the DQN network is represented by the indicator matrix C k The actions of moving the current row up, moving the current row down, and keeping the current row unchanged are composed.

[0048] Specifically, the state space of the DQN algorithm is defined as S = {p1, p2, ...., p l}, where l is the total number of services in the network, p k (k=1,2,....,l) represents the row number index of the kth service, that is, the index of a row in the path matrix, p k The value space of (k=1,2,....,l) is all the row numbers in the matrix, which does not exceed the number of alternative paths R corresponding to the service. k Represents the set of indices of all row vectors in the matrix, p k ∈P k (k=1,2,....,l).

[0049] Define the action space of each business as a k ={-1,0,1}, -1 means the current row moves up, the corresponding index p k (k=1,2,....,l) is subtracted by 1, 1 means moving the current row down, and the corresponding index p k (k=1,2,....,l) is added by 1, 0 means the current row remains unchanged, and the corresponding index p k (k=1,2,....,l) also remains unchanged. In this way, the size of the entire action space is 3 l , that is, for each state S, there are 3 l optional actions to choose from. The entire action space is: A = {a1, a2, ...., a l}.

[0050] Step S103: Based on the pre-trained second DQN network model and the main routing path, a backup routing path with the lowest primary and backup routing similarity is obtained. In order to ensure the reliable operation of the business, it is necessary to allocate a reasonable backup route for each business. The backup route should have as low a degree of overlap as possible with the links and nodes between the main routes on the basis of taking into account the constraints of bandwidth and latency. Therefore, the path and the main routing path are input into the second DQN network model, and based on the primary and backup routing similarity calculated when the second DQN network model is running, the backup routing path with the lowest primary and backup routing similarity is determined. That is, when the second DQN network model is running, the node and link overlap of the primary and backup routes is used as an optimization target, so that the nodes and links of the generated backup routing path have the lowest degree of overlap with the main routing path.

[0051] The primary and backup routes are calculated using the following formula:

[0052]

[0053] In the formula, e′ k and e k They represent the number of overlapping links and the total number of links in the main routing path of service k and the optional backup routing path generated when the second DQN network model is running, respectively, v′ k and v k They respectively represent the number of overlapping nodes and the total number of nodes in the main routing path of business k and the optional backup routing path generated when the second DQN network model is running.

[0054] The service path scheduling method based on a virtual power plant provided in an embodiment of the present invention first finds all alternative routing solutions that meet multiple constraints for the source and destination nodes of each service based on the network topology of the virtual power plant; then, based on the DQN network algorithm, determines the main routing path and the backup routing path that satisfies the minimum routing similarity and the best possible indicators. Therefore, the service path scheduling method can ensure the stable execution of the deterministic delay service of the virtual power plant by determining the main and backup routing paths, thereby improving the reliability of the virtual power plant service.

[0055] In one embodiment, if Figure 2 As shown, the path is input into the pre-trained first DQN network model to obtain the main routing path, including the following steps:

[0056] Step S201: Determine the main routing objective function based on the set of services, the total delay of each service, and the link balance. Specifically, the total delay of the service includes the propagation delay, transmission delay, and processing delay of the service. Therefore, the total delay of the service is calculated using the following formula:

[0057] t=t r +n×(t s +td +t f )

[0058] In the formula, t r represents the propagation delay of the service IP packet in the link, n represents the number of nodes on the service link, t s Indicates the sending delay of the service IP packet on the node, t d Indicates the processing delay of the service IP packet on the node, t f Indicates the queuing delay of IP packets on the node. Since the service is transmitted in segments, each segment can be regarded as an IP packet.

[0059] Among them, t r The calculation formula is:

[0060]

[0061] Where M is the link length, n1 is the fiber refractive index, and c is the speed of light.

[0062] Link balance refers to the degree of balance of services carried on links in the network. Link balance is calculated by the sum of the importance of services carried on each link and the total number of links. That is, link balance is expressed as:

[0063]

[0064] Where, L i is the sum of the importance of the services carried on the ith link, and m is the total number of links in the entire network. Under normal circumstances, the smaller the link balance D is, the more average the importance of the services carried on each link is, and the smaller the impact of link interruption is. The importance of each service can be pre-assigned according to the actual situation, and then the sum of the service importance is determined based on the assignment.

[0065] When determining the main routing objective function, the above delay and balance can be comprehensively considered to obtain the main routing path with the best delay and balance. Therefore, the main routing objective function is expressed by the following formula:

[0066]

[0067] Where L represents the set of deterministic delay services in the virtual power plant, T k is the total delay of service k on the link, is the preset maximum delay of service k, is the preset minimum delay of service k, D is the balance of all links in the whole network, α and β are two constant coefficients, and the values ​​of the constant coefficients can be adjusted to normalize the two target factors according to the needs of the actual scenario.

[0068] Step S202: Determine the immediate reward function of the main route based on the main route objective function. Specifically, the immediate reward function of the main route is expressed as V target Represents the main routing objective function.

[0069] Step S203: Determine the loss function of the first DQN network based on the immediate reward function of the main route. Specifically, based on the above state space, action space and immediate reward function, the loss function can be expressed as:

[0070] L(θ)=E[(Target QQ(s,a;θ)) 2 ]

[0071] TargetQ=r+γmax a′ Q(s′,a′;θ′)

[0072] Among them, s' is the next state after s executes a.

[0073] Step S204: Train the first DQN network based on the first DQN network loss function to obtain a trained first DQN network model. Figure 3 As shown, the training process is implemented in the following steps:

[0074] 1) Initialize the capacity N of the memory pool. The memory pool can be understood as a sample pool. Because the S, A, R, and S' data are needed when training the model, sufficient sample data must be collected before training. N here can be selected based on experience.

[0075] 2) Initialize the neural network corresponding to the value function Q with random weights θ.

[0076] 3) Use weight θ′=θ to initialize the neural network corresponding to the target value function targetQ. Among them, the value function Q and the target value function targetQ are both output values ​​of the neural network. There are many nodes in a neural network, and each node has a different weight parameter. Through 2) and 3), a weight parameter is randomly assigned to each node. The weight parameters of all nodes in the entire neural network are collectively referred to as θ. In the subsequent training process, the weight parameters of these nodes will be continuously updated, that is, θ will be continuously updated to output more accurate Q and targetQ. Here, the neural network corresponding to targetQ is actually the neural network corresponding to Q. It’s just that the θ value is different. Generally, Q uses the real-time updated θ value, and targetQ uses the old θ value.

[0077] 4) Initialize the state S of the entire network.

[0078] 5) Based on the current state S, select action A, obtain the corresponding reward value R, the state S' after the action is completed, and save the relevant parameters S, A, R, S' into the memory pool.

[0079] 6) Determine whether the amount of data stored in the memory pool exceeds the observed value. If not, go to 5); if the data is sufficient, go to 7).

[0080] 7) Start training:

[0081] ① Randomly select a part of the data from the memory pool as training samples;

[0082] ② Use the randomly sampled state S' as a training sample and obtain the Q value of the corresponding state through the output value of the neural network.

[0083] ③ Calculate the targetQ value corresponding to the Q value according to the following formula:

[0084] TargetQ=r+γmax a′ Q(s′,a′;θ′)

[0085] 8) Use the Q value and targetQ value to update according to the loss function.

[0086] 9) Update every C steps.

[0087] 10) Determine whether the maximum number of steps T has been reached. If not, return to step 5); otherwise, go to 11).

[0088] 11) Determine whether the maximum number of training rounds M has been reached. If not, return to step 4); otherwise, end.

[0089] Step S205: input the path into the trained first DQN network model to obtain a main routing path.

[0090] In one embodiment, for the second DQN network model, in order to enable it to use the primary and backup route similarity as the optimization target during operation, the loss function of the first DQN network model can be changed on the basis of the first DQN network model. According to the first DQN network model, the loss function is related to the immediate reward function or the objective function. Therefore, the primary and backup route similarity can be added on the basis of the primary route objective function to obtain the backup route objective function. That is, the backup route objective function is expressed by the following formula:

[0091]

[0092] After obtaining the backup route objective function, the immediate reward function of the backup route can be determined in the same way. That is, the immediate reward function of the backup route is expressed as By substituting the immediate reward function of the backup route into the above loss function calculation formula, the loss function of the second DQN network can be obtained. Therefore, except for the different loss functions, the first DQN network and the second DQN network have the same structure and training method. Only the primary and backup routes are added to the loss function of the second DQN network. Therefore, by inputting the path and the main route path into the pre-trained second DQN network model, the backup route path with the lowest similarity to the main route path can be obtained.

[0093] The service path scheduling method based on the virtual power plant provided by the embodiment of the present invention takes into account the delay optimization problem of deploying a large number of services in the network, sets the maximum delay and the minimum delay for each service, and then controls the service delay to the middle value of its constraint interval to enable nodes and links to carry more services, sets different importance for different services, and then uses the variance of the service importance index carried on the link to represent the link balance. At the same time, a traversal method is used to solve all paths that meet multiple constraints between the source point and the destination node for each service to be configured, and then the DQN algorithm is used to quickly find the main routing solution with approximately optimal indicators for each service to be configured and the backup routing solution with the minimum routing similarity and the best possible indicators for each service to be configured, which can ensure the stable execution of the deterministic delay service of the virtual power plant and improve the reliability of the virtual power plant service.

[0094] The embodiment of the present invention also provides a service path scheduling device based on a virtual power plant, such as Figure 4 As shown, the device comprises:

[0095] The first path determination module is used to determine all paths that meet the constraint conditions for each business based on the network topology of the virtual power plant; the specific content can be found in the corresponding part of the above method embodiment, which will not be repeated here.

[0096] The main path determination module is used to input the path into the pre-trained first DQN network model to obtain the main routing path; the specific content refers to the corresponding part of the above method embodiment, which will not be repeated here.

[0097] The backup path determination module is used to obtain a backup routing path with the lowest primary and backup routing similarity based on the pre-trained second DQN network model and the primary routing path. For details, please refer to the corresponding part of the above method embodiment, which will not be repeated here.

[0098] The service path scheduling device based on the virtual power plant provided by the embodiment of the present invention first finds all alternative routing solutions that meet multiple constraints for the source and destination nodes of each service based on the network topology of the virtual power plant; then, based on the DQN network algorithm, determines the main routing path and the backup routing path that satisfies the minimum routing similarity and the best possible indicators. Therefore, the service path scheduling device can ensure the stable execution of the deterministic delay service of the virtual power plant by determining the main and backup routing paths, thereby improving the reliability of the virtual power plant service.

[0099] For a detailed description of the functions of the service path scheduling device based on a virtual power plant provided in an embodiment of the present invention, please refer to the description of the service path scheduling method based on a virtual power plant in the above embodiment.

[0100] The embodiment of the present invention also provides a storage medium, such as Figure 5 As shown, a computer program 601 is stored thereon, and when the instruction is executed by the processor, the steps of the business path scheduling method based on the virtual power plant in the above embodiment are implemented. The storage medium also stores audio and video stream data, feature frame data, interaction request signaling, encrypted data, and preset data size, etc. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory (Flash Memory), a hard disk (Hard Disk Drive, abbreviated: HDD) or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above-mentioned types of memory.

[0101] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the storage medium can be a disk, an optical disk, a read-only memory (ROM), a random access memory (RAM), a flash memory, a hard disk drive (HDD) or a solid-state drive (SSD), etc.; the storage medium can also include a combination of the above-mentioned types of memory.

[0102] The embodiment of the present invention further provides an electronic device, such as Figure 6As shown, the electronic device may include a processor 51 and a memory 52, wherein the processor 51 and the memory 52 may be connected via a bus or other means. Figure 6 The example of connecting through bus is taken in the following.

[0103] The processor 51 may be a central processing unit (CPU). The processor 51 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, or a combination of the above chips.

[0104] The memory 52 is a non-transient computer-readable storage medium that can be used to store non-transient software programs, non-transient computer executable programs and modules, such as the corresponding program instructions / modules in the embodiment of the present invention. The processor 51 executes various functional applications and data processing of the processor by running the non-transient software programs, instructions and modules stored in the memory 52, that is, to implement the business path scheduling method based on the virtual power plant in the above method embodiment.

[0105] The memory 52 may include a program storage area and a data storage area, wherein the program storage area may store an application required for operating the device and at least one function; the data storage area may store data created by the processor 51, etc. In addition, the memory 52 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other non-volatile solid-state storage device. In some embodiments, the memory 52 may optionally include a memory remotely arranged relative to the processor 51, and these remote memories may be connected to the processor 51 via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0106] The one or more modules are stored in the memory 52 and when executed by the processor 51, perform the following steps: Figure 1 -3 shows a business path scheduling method based on a virtual power plant in the embodiment shown in FIG.

[0107] For details of the above electronic equipment, please refer to Figures 1 to 3 The corresponding related descriptions and effects in the illustrated embodiments can be understood and will not be repeated here.

[0108] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A service path scheduling method based on a virtual power plant, characterized in that: include: Determine all paths that meet the constraints for each business based on the network topology of the virtual power plant; Inputting the path into a pre-trained first DQN network model to obtain a main routing path; Obtaining a backup routing path with the lowest primary-backup routing similarity based on the pre-trained second DQN network model and the primary routing path; Inputting the path into the pre-trained first DQN network model to obtain the main routing path includes: Determine the main routing objective function based on the service set, the total delay of each service, and the link balance; Determine an immediate reward function of the main route based on the main route objective function; Determine the loss function of the first DQN network based on the immediate reward function of the main route; Training the first DQN network based on the first DQN network loss function to obtain a trained first DQN network model; Inputting the path into the trained first DQN network model to obtain a main routing path; The main routing objective function is determined based on the service set, the total delay of each service, and the link balance, including: The total delay of each service is obtained by summing up the propagation delay, sending delay and processing delay of the service; The link balance is calculated based on the sum of the importance of the services carried on each link and the total number of links; Determine the main routing objective function based on the service set, the total delay of each service, and the link balance; The main routing objective function is expressed by the following formula: Where L represents the set of deterministic delay services in the virtual power plant, T k is the total delay of service k on the link, is the preset maximum delay of service k, is the preset minimum delay of service k, D is the balance of all links in the whole network, α and β are two constant coefficients. According to the needs of the actual scenario, the values ​​of the constant coefficients are adjusted to normalize the two target factors.

2. The service path scheduling method based on virtual power plant according to claim 1 is characterized in that: Based on the network topology of the virtual power plant, all paths that meet the constraints for each business are determined, including: A traversal algorithm is used for the network topology of the virtual power plant to determine all paths that meet constraint conditions for each service; the constraint conditions include bandwidth constraints and delay constraints.

3. The service path scheduling method based on virtual power plant according to claim 1 is characterized in that: Based on the pre-trained second DQN network model and the main routing path, a backup routing path with the lowest primary and backup routing similarity is obtained, including: The path and the main routing path are input into a pre-trained second DQN network model, and based on the main and backup routing similarities calculated when the second DQN network model is running, a backup routing path with the lowest main and backup routing similarity is determined.

4. The service path scheduling method based on virtual power plant according to claim 1 is characterized in that: The path that satisfies the constraint conditions is represented by the following matrix: Where k represents the kth service, v i and v j They respectively represent the starting point and end point of the path in the network topology of the virtual power plant.

5. The service path scheduling method based on virtual power plant according to claim 4 is characterized in that: The state space of the DQN network is composed of the row number index of each business, and the row number index is the matrix C k The index of each row in; The action space of the DQN network is represented by the indicator matrix C k The actions of moving the current row up, moving the current row down, and keeping the current row unchanged are composed.

6. A service path scheduling device based on a virtual power plant, characterized in that: include: A first path determination module is used to determine all paths that meet constraint conditions for each service based on the network topology of the virtual power plant; A main path determination module, used for inputting the path into a pre-trained first DQN network model to obtain a main routing path; A backup path determination module, used for obtaining a backup routing path with the lowest primary and backup routing similarity based on a pre-trained second DQN network model and a primary routing path; Inputting the path into the pre-trained first DQN network model to obtain the main routing path includes: Determine the main routing objective function based on the service set, the total delay of each service, and the link balance; Determine an immediate reward function of the main route based on the main route objective function; Determine the loss function of the first DQN network based on the immediate reward function of the main route; Training the first DQN network based on the first DQN network loss function to obtain a trained first DQN network model; Inputting the path into the trained first DQN network model to obtain a main routing path; The main routing objective function is determined based on the service set, the total delay of each service, and the link balance, including: The total delay of each service is obtained by summing up the propagation delay, sending delay and processing delay of the service; The link balance is calculated based on the sum of the importance of the services carried on each link and the total number of links; Determine the main routing objective function based on the service set, the total delay of each service, and the link balance; The main routing objective function is expressed by the following formula: Where L represents the set of deterministic delay services in the virtual power plant, T k is the total delay of service k on the link, is the preset maximum delay of service k, is the preset minimum delay of service k, D is the balance of all links in the whole network, α and β are two constant coefficients. According to the needs of the actual scenario, the values ​​of the constant coefficients are adjusted to normalize the two target factors.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable the computer to execute the service path scheduling method based on a virtual power plant as described in any one of claims 1 to 5.

8. An electronic device, characterized in that: include: A memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the business path scheduling method based on a virtual power plant as described in any one of claims 1 to 5 by executing the computer instructions.

Images