Communication resource scheduling method and device, equipment and storage medium

Abstract

The invention provides a communication resource scheduling method and device, equipment and a storage medium, and the method is applied to a twin network layer, and comprises the steps: collecting physical data of a physical network layer in real time; acquiring a communication resource allocation strategy based on the physical data; issuing the communication resource allocation strategy to the physical network layer; wherein the physical network layer comprises a virtual power plant layer, a 6G core network and a 6G edge network. Therefore, the QoS requirement of the regulation and control service of the source load storage equipment in the virtual power plant scene is met, the system can be simplified, and the communication resource scheduling efficiency can be improved.

Description

Technical Field

[0001] The present invention relates to the field of wireless communication technology, and in particular to a communication resource scheduling method, device, equipment and storage medium. Background Art

[0002] Virtual power plant is a technology and management system that integrates distributed energy. It has great advantages in the access and management of distributed energy. With the development of distributed energy, higher requirements are placed on the communication resource scheduling capabilities of virtual power plants.

[0003] The existing method of scheduling communication resources through virtual power plants is relatively traditional. For example, in a multi-input and multi-output power communication system, the resource scheduling method is determined by maximizing the deterministic rate model and resource scheduling constraints, and then resource scheduling is performed, or the communication resource information of the current cell and adjacent cells is obtained in real time, and dynamic scheduling is performed.

[0004] For scenarios where virtual power plants have dense source, load and storage equipment and very tight communication resources, the above methods are difficult to meet the quality of service (QoS) requirements for this scenario during the scheduling process, or require higher computing power and complex algorithms, which may increase the complexity and implementation difficulty of the system. Summary of the invention

[0005] The present invention provides a communication resource scheduling method, device, equipment and storage medium, which are used to solve the defects of the prior art that the communication resource scheduling method in the scenario where the source, load and storage equipment of the virtual power plant are dense and the communication resources are very tight is difficult to meet the QoS requirements and the system is complex. It can meet the QoS requirements of the control business of the source, load and storage equipment in the virtual power plant scenario, simplify the system, and improve the communication resource scheduling efficiency.

[0006] The present invention provides a communication resource scheduling method, which is applied to a twin network layer, comprising: Collect physical data of the physical network layer in real time; Based on the physical data, obtaining a communication resource allocation strategy; Sending the communication resource allocation strategy to the physical network layer; Among them, the physical network layer includes a virtual power plant layer, a 6G core network and a 6G edge network.

[0007] According to a communication resource scheduling method provided by the present invention, obtaining a communication resource allocation strategy based on the physical data specifically includes: With the goal of maximizing system throughput and maximizing system fairness, based on the physical data and constraints of each distributed subnet, determine the communication resource allocation strategy of each distributed source, load and storage device in each distributed subnet; The constraints include: The power occupied by all communication resource blocks is not higher than the total power of the base stations in the distributed subnet; The number of distributed source-load-storage devices corresponding to any communication resource block at any time is 1; The scheduling time of any distributed source, load and storage device meets the QoS requirements; The throughput between any communication resource block and any distributed source-load-storage device is greater than or equal to 0.

[0008] According to a communication resource scheduling method provided by the present invention, the communication resource allocation strategy of each distributed source, load and storage device in each distributed subnet is determined based on the physical data and constraints of each distributed subnet with the goal of maximizing system throughput and maximizing system fairness, specifically including: Determine the reward function based on the system throughput, system fairness index and the waiting time of the distributed source-load-storage device; Based on the reward function, the physical data of each distributed subnet and the constraints, the communication resource allocation strategy of each distributed source-load-storage device in each distributed subnet is determined according to the multi-agent deep deterministic policy gradient MADDPG algorithm.

[0009] According to a communication resource scheduling method provided by the present invention, the reward function is determined according to the following formula:

[0010] In the formula, R is the reward function value; R1 is the first reward function, which is determined according to the system throughput; R2 is the second reward function, which is determined according to the waiting time of the distributed source-load-storage device; R3 is the third reward function, which is determined according to the system fairness index.

[0011] According to a communication resource scheduling method provided by the present invention, the system fairness index is determined according to the power allocated to each of the distributed source-load-storage devices.

[0012] According to a communication resource scheduling method provided by the present invention, the first reward function is obtained by normalizing the system throughput; the second reward function is obtained by normalizing the waiting time of the distributed source-load-storage device.

[0013] The present invention also provides a communication resource scheduling device, which is applied to a twin network layer, comprising: The acquisition module is used to collect physical data of the physical network layer in real time; An acquisition module, used for acquiring a communication resource allocation strategy based on the physical data; A sending module, used for sending the communication resource allocation strategy to the physical network layer; Among them, the physical network layer includes a virtual power plant layer, a 6G core network and a 6G edge network.

[0014] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the communication resource scheduling method described above is implemented.

[0015] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the communication resource scheduling method as described in any one of the above is implemented.

[0016] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the communication resource scheduling method as described above is implemented.

[0017] The communication resource scheduling method, device, equipment and storage medium provided by the present invention combine the virtual power plant with the 6G core network and the 6G edge network into a physical network layer, construct a twin network layer corresponding to the physical network layer, and the twin network layer collects the physical data of the physical network layer in real time to generate a corresponding communication resource allocation strategy, and sends the communication resource allocation strategy to the physical network layer, thereby meeting the QoS requirements of the control services of the source, load and storage equipment in the virtual power plant scenario, and can simplify the system and improve the efficiency of communication resource scheduling. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0019] Figure 1 It is a flow chart of the communication resource scheduling method provided by the present invention; Figure 2 It is a schematic diagram of the network architecture of the virtual power plant modeling based on digital twin provided by the present invention; Figure 3 It is a structural schematic diagram of the communication resource scheduling device provided by the present invention; Figure 4It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0021] Combine the following Figure 1-Figure 4 The present invention describes the communication resource scheduling method, device, equipment and storage medium provided by the present invention.

[0022] Figure 1 A flow chart of the communication resource scheduling method provided by the present invention is as follows: Figure 1 As shown, the method is applied to the twin network layer and includes the following steps: Step 100: Collect physical data of the physical network layer in real time.

[0023] Step 101: Acquire a communication resource allocation strategy based on physical data.

[0024] Step 102: Send the communication resource allocation strategy to the physical network layer.

[0025] Among them, the physical network layer includes the virtual power plant layer, 6G core network and 6G edge network.

[0026] Specifically, the current virtual power plant communication resource scheduling mainly focuses on maximizing throughput and reducing task scheduling delays. In this scenario, source, load and storage equipment are dense and communication resources are very tight.

[0027] As for intensive source-load-storage equipment, the high bandwidth, low latency, and high energy efficiency of the sixth-generation mobile communication technology (6-Generation, 6G) network can meet the virtual power plant's control requirements for source-load-storage equipment.

[0028] Digital twins are currently widely used in the industrial field, with positive effects such as real-time monitoring, prediction, and performance optimization. Using digital twins for communication resource scheduling in virtual power plants can perceive resource allocation in real time and make timely resource scheduling.

[0029] Based on the above analysis, the present invention provides a communication resource scheduling method based on digital twin and 6G distributed subnet architecture.

[0030] The 6G distributed subnet architecture based on digital twins in the embodiment of the present invention includes a twin network layer and a physical network layer. The physical network layer can be divided into three layers, namely the virtual power plant layer, the 6G core network and the 6G edge network. The virtual power plant layer is responsible for the dispatching and control of the power system. Various power equipment and renewable energy can be connected to the virtual power plant through the micro base station of the 6G edge network, and each source load storage device can send the real-time status to the virtual power plant through the uplink. The twin network layer corresponds to the physical network layer, and a high-fidelity virtual power plant model can be established, and the virtual power plant model can be managed and updated in real time.

[0031] In the solution provided by the embodiment of the present invention, first, the twin network layer can collect physical data of the physical network layer in real time, such as the number of distributed subnets corresponding to the virtual power plant layer, the number of distributed source load storage devices in each distributed subnet, the number of communication resource blocks in each distributed subnet, the bandwidth of each communication resource block, etc.

[0032] Then, the twin network layer can obtain the communication resource allocation strategy based on the physical data collected in the physical network layer. In the process of obtaining the communication resource allocation strategy, different algorithms can be designed according to the actual QoS requirements. For example, the communication resource allocation strategy can be determined with the goal of maximizing system throughput.

[0033] After the twin network layer obtains the communication resource allocation strategy, it can send the communication resource allocation strategy to the virtual power plant in the physical network layer, thereby realizing the regulation of communication resources.

[0034] The communication resource scheduling method provided by the present invention combines the virtual power plant with the 6G core network and the 6G edge network as a physical network layer, constructs a twin network layer corresponding to the physical network layer, and the twin network layer collects the physical data of the physical network layer in real time to generate a corresponding communication resource allocation strategy, and sends the communication resource allocation strategy to the physical network layer, thereby meeting the QoS requirements of the control services of the source, load and storage equipment in the virtual power plant scenario, and can simplify the system and improve the efficiency of communication resource scheduling.

[0035] According to a communication resource scheduling method provided by the present invention, a communication resource allocation strategy is obtained based on physical data, specifically comprising: With the goal of maximizing system throughput and system fairness, the communication resource allocation strategy of each distributed source, load and storage device in each distributed subnet is determined based on the physical data and constraints of each distributed subnet; The constraints include: The power occupied by all communication resource blocks is not higher than the total power of the base stations in the distributed subnet; The number of distributed source-load-storage devices corresponding to any communication resource block at any time is 1; The scheduling time of any distributed source, load and storage device meets the QoS requirements; The throughput between any communication resource block and any distributed source-load-storage device is greater than or equal to 0.

[0036] Specifically, since the scenario considered by the solution of the present invention is a resource-scarce scenario, and the control services of the virtual power plant are also sensitive to latency requirements, the fairness of resource allocation in the embodiment of the present invention is also an indicator that needs to be considered while maximizing throughput.

[0037] If the maximum throughput is required, resources only need to be allocated to the source, load and storage with better channel conditions to achieve the maximum throughput. However, the control services of the virtual power plant are also sensitive to latency requirements. If resources are not allocated all the time, control tasks cannot be issued, resulting in failure to be regulated in time. Therefore, while considering the throughput, it is also necessary to consider the fairness of scheduling each source, load and storage device.

[0038] Therefore, in an embodiment of the present invention, in the process of obtaining the communication resource allocation strategy, the goal can be to maximize the system throughput and maximize the system fairness, and based on the physical data and constraints of each distributed subnet, the communication resource allocation strategy of each distributed source, load and storage device in each distributed subnet can be determined.

[0039] In an embodiment of the present invention, a mathematical model for determining a communication resource allocation strategy includes a plurality of distributed subnets, each of which includes a wireless base station and a plurality of distributed source-load-storage devices at different distances from the wireless base station, each of which includes a plurality of resource blocks.

[0040] It can be assumed that the model mainly includes N distributed subnets, in each distributed subnet there is a wireless base station and K distributed source-load-storage devices at different distances from the wireless base station; in each subnet, it is divided into M resource blocks, the bandwidth of each resource block is B, each resource block can only be allocated to one source-load-storage device, but one source-load-storage device can occupy multiple resource blocks at the same time.

[0041] In subnet n, the signal-to-interference-to-noise ratio between source-load-storage device k and resource block m is It can be expressed as:

[0042] In the formula, represents the power between resource block m and source-load-storage device k in subnet n, represents the channel gain between resource block m in subnet n and source-load-storage device k, is the power spectral density of Gaussian white noise per unit bandwidth, Represents the bandwidth between resource block m in subnet n and source load storage device k.

[0043]

[0044] In the formula, is the power of the transmitting antenna, in dBi, is the power of the receiving antenna, in dBi, is the path loss of source load storage device k in subnet n.

[0045]

[0046] In the formula, is the carrier frequency, is the distance from the source load storage device k to the base station.

[0047] In the control services of virtual power plants, the request packet length of each control service is relatively short, the impact of the transmission error rate on reliability cannot be ignored, and it needs to be transmitted with extremely low latency and extremely high reliability. Therefore, the traditional Shannon formula cannot accurately model the control services of virtual power plants.

[0048] Because in the virtual power plant scenario, the equipment does not move. Assuming that the channel of the model is a quasi-static flat fading channel and both the transmitter and the receiver know the information state of the channel, the maximum throughput between resource block m and source load storage device k in subnet n is It can be expressed as:

[0049] In the formula, represents the bandwidth between source load storage device k and resource block m in subnet n, represents the signal-to-interference-noise ratio of the source load storage device k and resource block m in subnet n, represents the channel dispersion, is the length of the transmitted data packet, represents the inverse function of the Q function, Indicates the decoding error rate of the downlink.

[0050]

[0051] Then the throughput of source load storage device k in subnet n is:

[0052] In the formula, If it is 0, it means that the source load storage device k does not use the resource block m of subnet n. If it is 1, it means that the source load storage device k is connected to the resource block m of the subnet n.

[0053] Then the throughput of the subnet can be expressed as:

[0054] In the embodiment of the present invention, the Jain fairness index can be used as an index for evaluating the fairness of the system. The range of the index is between 0 and 1, where 1 represents complete fairness and close to 0 represents extreme unfairness. The calculation formula of the Jain fairness index is as follows:

[0055] In the formula, is the amount of resources obtained by source load storage device k in subnet n.

[0056] The value of indicates a certain degree of fairness. The closer the value is to 1, the better the fairness. =1, indicating complete fairness, that is, all source-load-storage devices obtain equal amounts of resources; when Close to 0, it indicates extreme unfairness, that is, resources are occupied by very few source-load-storage devices, and most source-load-storage devices obtain very few resources.

[0057] In the embodiment of the present invention, the constraint conditions may include: (1) The power occupied by all communication resource blocks is not higher than the total power of the base stations in the distributed subnet. The formula is as follows:

[0058] (2) The number of distributed source-load-storage devices corresponding to any communication resource block at any time is 1, and the formula is as follows:

[0059] (3) The scheduling time of any distributed source-load-storage device meets the QoS requirements. The QoS requirements can be met by setting the bit error rate threshold. The formula is as follows:

[0060] (4) The throughput between any communication resource block and any distributed source-load-storage device is greater than or equal to 0, as shown in the following formula:

[0061] The embodiment of the present invention takes maximizing system throughput and maximizing system fairness as optimization goals, and adopts different algorithms to determine the communication resource allocation strategy of each distributed source, load and storage device in each distributed subnet based on the physical data and constraints of each distributed subnet, so as to maintain high throughput and system fairness while meeting QoS requirements.

[0062] According to a communication resource scheduling method provided by the present invention, with the goal of maximizing system throughput and maximizing system fairness, based on the physical data and constraints of each distributed subnet, a communication resource allocation strategy for each distributed source, load and storage device in each distributed subnet is determined, specifically including: Determine the reward function based on the system throughput, system fairness index and the waiting time of the distributed source-load-storage device; Based on the reward function, the physical data of each distributed subnet and the constraints, the communication resource allocation strategy of each distributed source-load-storage device in each distributed subnet is determined according to the multi-agent deep deterministic policy gradient (MADDPG) algorithm.

[0063] Specifically, deep reinforcement learning can be used to determine the communication resource allocation strategy of each distributed source, load and storage device in each distributed subnet. In the embodiment of the present invention, a multi-agent deep deterministic policy gradient (MADDPG) algorithm is used to allocate communication resources.

[0064] The reinforcement learning problem is usually formulated as a Markov decision process defined by a five-tuple: , each parameter represents the state space, action space, state transition probability, reward function and reward discount factor respectively.

[0065] Because each resource block can be allocated to different sources, loads and storage devices, and the power size of the resource block can be adjusted, the state space is defined as follows:

[0066] in Indicates the resource block allocation at time t, Indicates the resource block allocation power at time t.

[0067] The action space is defined as follows:

[0068] An integer between 0 and m, indicating that the resource block is unloaded to the source and storage device in time slot t , Indicates that the power change of the resource block adjusted in time slot t is .

[0069] Each agent of MADDPG can have a different reward value at each moment. In the embodiment of the present invention, the average waiting time and the system fairness index can be used together as indicators to measure the fairness of the system, that is, the reward function can be determined according to the system throughput, the system fairness index and the waiting time of the distributed source-load-storage device. For example, the system throughput, the system fairness index and the average waiting time of the distributed source-load-storage device are assigned different weight coefficients to obtain the reward function.

[0070] In some implementations, more emphasis may be placed on system throughput, so the reward function is determined according to the following formula:

[0071] Wherein, R is the reward function value; R1 is the first reward function, which is determined according to the system throughput; R2 is the second reward function, which is determined according to the waiting time of the distributed source-load-storage device; R3 is the third reward function, which is determined according to the system fairness index.

[0072] Among them, the average waiting time of distributed source-load-storage equipment is as follows:

[0073] In the formula, represents the waiting time of device k in subnet n during the i-th scheduling, represents the time when device k in subnet n is scheduled for the i-th time, represents the average waiting time of device k in subnet n, Indicates the average waiting time of subnet n.

[0074] According to a communication resource scheduling method provided by the present invention, the first reward function can be obtained by normalizing the system throughput; the second reward function can be obtained by normalizing the waiting time of the distributed source-load-storage device.

[0075] That is, the first reward function is defined as follows:

[0076] in represents the total throughput of the system, and They represent the minimum and maximum values ​​of the total system throughput respectively. The first reward function is obtained by normalizing the system throughput.

[0077] The second reward function is defined as follows:

[0078] in Indicates the average waiting time of the source load storage device in the system. and They represent the minimum and maximum values ​​of the average waiting time respectively. The second reward function is obtained by normalizing the waiting time of the distributed source-load-storage equipment.

[0079] According to a communication resource scheduling method provided by the present invention, the system fairness index can be determined according to the power allocated to each distributed source load storage device, that is, the Jain fairness index As shown below:

[0080] In the formula, is the power allocated to the source-load-storage device k in subnet n.

[0081] The third reward function is determined according to the system fairness index, that is, the third reward function is defined as follows:

[0082] The MADDPG algorithm is a deep reinforcement learning algorithm designed for multi-agent environments. Each agent has an independent policy network (actor) and a shared value network (critic). The policy network is responsible for selecting actions, and the value network evaluates the quality of actions. Agents learn the optimal strategy through mutual collaboration and competition.

[0083] In the process of training through the MADDPG algorithm, the policy network and value network of each agent are first initialized, and the experience replay pool is set; at each time step, the agent selects an action based on the current policy network, obtains a new state and reward after executing the action, and stores the current experience (state, action, reward, new state) in the experience replay pool. Then, samples are randomly drawn from the experience replay pool to update the value network and policy network, and the policy network of each agent is updated through the policy gradient algorithm. The above steps are repeated until the policy converges.

[0084] In the embodiment of the present invention, the MADDPG algorithm is utilized and multi-agent reinforcement learning is used for centralized training, thereby achieving the goals of rapid convergence, maximizing throughput and fairness, improving the throughput of source-load-storage equipment in the virtual power plant scenario, and preventing the starvation problem by ensuring fairness, thereby improving the reliability of communication resource scheduling in the virtual power plant when resources are scarce.

[0085] The communication resource scheduling method provided by the present invention is further explained below through embodiments in specific application scenarios.

[0086] The content of this embodiment includes two parts: 6G virtual power plant modeling and resource scheduling algorithm based on digital twin.

[0087] 1. 6G virtual power plant modeling based on digital twins

[0088] Figure 2 A schematic diagram of the network architecture of a virtual power plant modeling based on digital twins provided by the present invention is shown in FIG. Figure 2 As shown in the figure, the model is divided into a twin network layer and a physical network layer. The twin network layer is divided into a digital twin sublayer and a resource allocation sublayer. The digital twin sublayer has a data storage module, a twin management module, and a twin mapping module. The data storage module provides functions such as data acquisition, storage, and management, and is responsible for real-time acquisition and storage of data from the virtual power plant. The twin management module provides the basic functions for the operation of the digital twin layer and is responsible for managing and updating the virtual model. The twin mapping model establishes a high-fidelity virtual power plant model based on the basic information of the data warehouse, providing state monitoring, scheduling strategy generation, and policy delivery functions for the virtual power plant. Based on the mapping of the physical state by the digital twin sublayer, the resource allocation sublayer forms a complete three-step resource allocation function closed loop of state monitoring, scheduling strategy generation, and policy deployment.

[0089] The physical network layer is divided into three layers. The first layer is the virtual power plant layer, which is responsible for the dispatching and control of the power system. The second layer is the 6G core network, and the third layer is the 6G edge network. Various power equipment and renewable energy are connected to the virtual power plant through the micro base station of the 6G edge network. Each source, load and storage device sends the real-time status to the virtual power plant through the uplink. The digital twin layer collects the information of the virtual power plant in the physical network layer, calculates the communication resource allocation strategy, and sends the strategy to the virtual power plant to realize the regulation of communication resources.

[0090] 2. Communication resource scheduling method

[0091] (1) Mathematical model

[0092] The model mainly includes N distributed subnets, in each of which there is a wireless base station and K distributed source-load-storage devices at different distances from the base station. The distributed source-load-storage devices are connected to the 6G distributed subnet through the wireless base station.

[0093] Assume that in each sub-network, there are M resource blocks, the bandwidth of each resource block is B, each resource block can only be allocated to one source, load and storage device, but one source, load and storage device can occupy multiple resource blocks at the same time.

[0094] In subnet n, the signal-to-interference-noise ratio between source-load-storage device k and resource block m can be expressed as

[0095] In the formula, represents the power between resource block m and source-load-storage device k in subnet n, represents the channel gain between resource block m in subnet n and source-load-storage device k, is the power spectral density of Gaussian white noise per unit bandwidth, Represents the bandwidth between resource block m in subnet n and source load storage device k.

[0096]

[0097] In the formula, is the power of the transmitting antenna, in dBi, is the power of the receiving antenna, in dBi, is the path loss of source load storage device k in subnet n.

[0098]

[0099] In the formula, is the carrier frequency, is the distance from the source load storage device k to the base station.

[0100] In the control services of virtual power plants, the request packet length of each control service is relatively short, the impact of the transmission error rate on reliability cannot be ignored, and it needs to be transmitted with extremely low latency and extremely high reliability. Therefore, the traditional Shannon formula cannot accurately model the control services of virtual power plants.

[0101] Because in the virtual power plant scenario, the equipment does not move. Assuming that the channel of the model is a quasi-static flat fading channel and both the transmitter and the receiver know the information state of the channel, the maximum throughput between resource block m and terminal k in subnet n is It can be expressed as:

[0102] In the formula, represents the bandwidth between source load storage device k and resource block m in subnet n, represents the signal-to-interference-noise ratio of the source load storage device k and resource block m in subnet n, represents the channel dispersion, is the length of the transmitted data packet, represents the inverse function of the Q function, Indicates the decoding error rate of the downlink.

[0103]

[0104] Then the throughput of source load storage device k in subnet n is:

[0105] In the formula, If it is 0, it means that the source load storage device k does not use the resource block m of subnet n. If it is 1, it means that the source load storage device k is connected to the resource block m of the subnet n.

[0106] Then the throughput of the subnet can be expressed as:

[0107] Since the scenario considered in this embodiment is a resource-scarce scenario, and the control services of the virtual power plant are also sensitive to latency requirements, the fairness of resource allocation is also an indicator that needs to be considered while maximizing throughput. If the maximum throughput needs to be achieved, resources only need to be allocated to the source-load-storage device with better channel conditions to achieve the maximum throughput. The control services of the virtual power plant are also sensitive to latency requirements. If resources are not allocated all the time, control tasks cannot be issued, resulting in failure to be regulated in time. Therefore, while considering the throughput, it is also necessary to consider the fairness of scheduling each source-load-storage device.

[0108] This embodiment uses the average waiting time and the Jain fairness index to jointly measure the fairness index of the system.

[0109] The average waiting times are as follows:

[0110] In the formula, represents the waiting time of device k in subnet n during the i-th scheduling, represents the time when device k in subnet n is scheduled for the i-th time, represents the average waiting time of device k in subnet n, Indicates the average waiting time of subnet n.

[0111] The Jain Fairness Index is shown below:

[0112] in is the power allocated to the source-load-storage device k in subnet n.

[0113] This embodiment aims to maximize system throughput and system fairness, and the optimization objectives are as follows:

[0114] The constraints are:

[0115] The constraints are as follows: the power occupied by a resource block must not be higher than the total power of the base station; a resource block can only be allocated to one device at a time; the scheduling time of the device needs to meet the QoS requirements; and the throughput needs to be greater than or equal to 0.

[0116] (2) Resource allocation mechanism based on deep reinforcement learning

[0117] The reinforcement learning problem is usually formulated as a Markov decision process defined by a five-tuple: , respectively represent the state space, action space, state transition probability, reward function and reward discount factor. Because each resource block can be allocated to different source-load-storage devices, and the power size of the resource block can be adjusted, the state space is defined as follows:

[0118] in Indicates the resource block allocation at time t, Indicates the resource block allocation power at time t.

[0119] The action space is defined as follows:

[0120] An integer between 0 and m, indicating that the resource block is unloaded to the source and storage device in time slot t , Indicates that the power change of the resource block adjusted in time slot t is .

[0121] The reward function is defined as follows: Each agent in MADDPG can have different reward values ​​at each moment. In this embodiment, the reward value of each agent takes into account the three indicators of global throughput, fairness, and waiting time.

[0122] The first reward function is defined as follows:

[0123] in represents the total throughput of the system, and They represent the minimum and maximum values ​​of the total system throughput respectively.

[0124] The second reward function is defined as follows:

[0125] in Indicates the average waiting time of the source load storage device in the system. and Respectively represent the minimum and maximum values ​​of the average waiting time.

[0126] The third reward function is defined as follows:

[0127] in ,in It represents the sum of the power allocated to the kth source-load-storage device. The Jain fairness index is used to measure the fairness of the system according to the allocated resources.

[0128] This example focuses more on the system throughput, so the reward of the agent is defined as:

[0129] The algorithm flow of MADDPG is as follows: Resource allocation strategy based on deep reinforcement learning: 1 for episode = 1 to E do

[0130] Initialize environment settings, such as source load storage device location, resource allocation, etc.

[0131] for time=1 to N do

[0132] For each agent i, Get the action that the agent needs to perform

[0133] Execute gableSoftMax to discretize the action

[0134] Execute an action , and get new rewards and status

[0135] The experience tuple Store into the experience replay buffer

[0136] for agent=1 to M do

[0137] Randomly extract S groups of experience tuples from the experience replay buffer

[0138] By minimizing the loss function Update the critic network

[0139] pass Sampling policy gradients to update actor networks

[0140] Update target network parameters

[0141] end for

[0142] end for

[0143] 2 end for

[0144] In this embodiment, the allocation mechanism of communication resources of the virtual power plant based on digital twins maintains a good balance in terms of system throughput, fairness and average scheduling delay, and can maintain high throughput and system fairness under the premise of ensuring reliability, average scheduling delay and other QoS requirements. This framework and algorithm can well meet the QoS requirements of the control business of source-load-storage equipment in the virtual power plant scenario, and achieve better performance. This embodiment proposes that the virtual power plant communication resource scheduling algorithm based on digital twins has certain advantages in control business.

[0145] The communication resource scheduling device provided by the present invention is described below. The communication resource scheduling device described below and the communication resource scheduling method described above can be referenced to each other.

[0146] Figure 3 A schematic diagram of the structure of the communication resource scheduling device provided by the present invention, such as Figure 3 As shown, the device is applied to the twin network layer, including: The collection module 300 is used to collect physical data of the physical network layer in real time; An acquisition module 310, configured to acquire a communication resource allocation strategy based on physical data; The sending module 320 is used to send the communication resource allocation strategy to the physical network layer; Among them, the physical network layer includes the virtual power plant layer, 6G core network and 6G edge network.

[0147] According to a communication resource scheduling device provided by the present invention, based on physical data, a communication resource allocation strategy is obtained, which specifically includes: With the goal of maximizing system throughput and system fairness, the communication resource allocation strategy of each distributed source, load and storage device in each distributed subnet is determined based on the physical data and constraints of each distributed subnet; The constraints include: The power occupied by all communication resource blocks is not higher than the total power of the base stations in the distributed subnet; The number of distributed source-load-storage devices corresponding to any communication resource block at any time is 1; The scheduling time of any distributed source, load and storage device meets the QoS requirements; The throughput between any communication resource block and any distributed source-load-storage device is greater than or equal to 0.

[0148] According to a communication resource scheduling device provided by the present invention, with the goal of maximizing system throughput and maximizing system fairness, based on the physical data and constraints of each distributed subnet, a communication resource allocation strategy for each distributed source, load and storage device in each distributed subnet is determined, specifically including: Determine the reward function based on the system throughput, system fairness index and the waiting time of the distributed source-load-storage device; Based on the reward function, the physical data of each distributed subnet and the constraints, the communication resource allocation strategy of each distributed source-load-storage device in each distributed subnet is determined according to the multi-agent deep deterministic policy gradient (MADDPG) algorithm.

[0149] According to a communication resource scheduling device provided by the present invention, the reward function is determined according to the following formula:

[0150] Wherein, R is the reward function value; R1 is the first reward function, which is determined according to the system throughput; R2 is the second reward function, which is determined according to the waiting time of the distributed source-load-storage device; R3 is the third reward function, which is determined according to the system fairness index.

[0151] According to a communication resource scheduling device provided by the present invention, the system fairness index is determined according to the power allocated to each distributed source load storage device.

[0152] According to a communication resource scheduling device provided by the present invention, the first reward function is obtained by normalizing the system throughput; the second reward function is obtained by normalizing the waiting time of the distributed source load storage device.

[0153] Figure 4 A schematic diagram of the structure of an electronic device provided by the present invention, such as Figure 4 As shown, the electronic device may include: a processor 410, a communications interface 420, a memory 430 and a communication bus 440, wherein the processor 410, the communications interface 420 and the memory 430 communicate with each other through the communication bus 440. The processor 410 may call the logic instructions in the memory 430 to execute the communication resource scheduling method provided by the above methods, and the method includes: Collect physical data of the physical network layer in real time; Based on physical data, obtain communication resource allocation strategy; Send communication resource allocation strategy to the physical network layer; Among them, the physical network layer includes the virtual power plant layer, 6G core network and 6G edge network.

[0154] In addition, the logic instructions in the above-mentioned memory 830 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0155] On the other hand, the present invention further provides a computer program product, the computer program product comprising a computer program, the computer program can be stored on a non-transitory computer-readable storage medium, and when the computer program is executed by a processor, the computer can execute the communication resource scheduling method provided by the above methods, the method comprising: Collect physical data of the physical network layer in real time; Based on physical data, obtain communication resource allocation strategy; Send communication resource allocation strategy to the physical network layer; Among them, the physical network layer includes the virtual power plant layer, 6G core network and 6G edge network.

[0156] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the communication resource scheduling method provided by the above methods is implemented, and the method includes: Collect physical data of the physical network layer in real time; Based on physical data, obtain communication resource allocation strategy; Send communication resource allocation strategy to the physical network layer; Among them, the physical network layer includes the virtual power plant layer, 6G core network and 6G edge network.

[0157] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0158] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0159] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A communication resource scheduling method, characterized in that: Applied to the twin network layer, including: Collect physical data of the physical network layer in real time; Based on the physical data, obtaining a communication resource allocation strategy; Sending the communication resource allocation strategy to the physical network layer; Among them, the physical network layer includes a virtual power plant layer, a 6G core network and a 6G edge network.

2. The communication resource scheduling method according to claim 1, characterized in that: The acquiring of a communication resource allocation strategy based on the physical data specifically includes: With the goal of maximizing system throughput and maximizing system fairness, based on the physical data and constraints of each distributed subnet, determine the communication resource allocation strategy of each distributed source, load and storage device in each distributed subnet; The constraints include: The power occupied by all communication resource blocks is not higher than the total power of the base stations in the distributed subnet; The number of distributed source-load-storage devices corresponding to any communication resource block at any time is 1; The scheduling time of any distributed source, load and storage device meets the QoS requirements; The throughput between any communication resource block and any distributed source-load-storage device is greater than or equal to 0.

3. The communication resource scheduling method according to claim 2, characterized in that: The communication resource allocation strategy of each distributed source-load-storage device in each distributed subnet is determined based on the physical data and constraints of each distributed subnet with the goal of maximizing system throughput and maximizing system fairness, specifically including: Determine the reward function based on the system throughput, system fairness index and the waiting time of the distributed source-load-storage device; Based on the reward function, the physical data of each distributed subnet and the constraints, the communication resource allocation strategy of each distributed source-load-storage device in each distributed subnet is determined according to the multi-agent deep deterministic policy gradient MADDPG algorithm.

4. The communication resource scheduling method according to claim 3, characterized in that: The reward function is determined according to the following formula: In the formula, R is the reward function value; R1 is the first reward function, which is determined according to the system throughput; R2 is the second reward function, which is determined according to the waiting time of the distributed source-load-storage device; R3 is the third reward function, which is determined according to the system fairness index.

5. The communication resource scheduling method according to claim 3 or 4, characterized in that: The system fairness index is determined according to the power allocated to each of the distributed source-load-storage devices.

6. The communication resource scheduling method according to claim 4, characterized in that: The first reward function is obtained by normalizing the system throughput; the second reward function is obtained by normalizing the waiting time of the distributed source-load-storage device.

7. A communication resource scheduling device, characterized in that: Applied to the twin network layer, including: The acquisition module is used to collect physical data of the physical network layer in real time; An acquisition module, used for acquiring a communication resource allocation strategy based on the physical data; A sending module, used for sending the communication resource allocation strategy to the physical network layer; Among them, the physical network layer includes a virtual power plant layer, a 6G core network and a 6G edge network.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the communication resource scheduling method according to any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the communication resource scheduling method according to any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the communication resource scheduling method according to any one of claims 1 to 6 is implemented.

Images