Data transmission path generation method and device of power distribution network, terminal equipment and storage medium

Abstract

The invention discloses a data transmission path generation method and device of a power distribution network, terminal equipment and a storage medium, which can predict future network state data based on a to-be-selected data transmission path and current network state data of each node, so that the data transmission path has predictability when being selected; according to the method, the influence of the to-be-selected data transmission paths on the future data transmission of the power distribution network can be considered based on the predicted future network state data, and a state reward function is introduced, so that the conditions of future network congestion and data throughput after each to-be-selected data transmission path is executed can be quantified, and the data transmission efficiency is improved. According to the method, the advantages and disadvantages of different paths can be compared, so that the final transmission path is selected by maximizing the data throughput, network congestion possibly occurring in the future is avoided, it is guaranteed that the power distribution network is not congested after transmitting interaction data, and the transmission efficiency and performance of the power distribution network can be guaranteed.

Description

Technical Field

[0001] The present invention relates to the technical field of data transmission in a distribution network, and in particular to a method, an apparatus, a terminal device and a storage medium for generating a data transmission path in a distribution network. Background Art

[0002] The distribution terminals and multiple business platforms in the distribution Internet of Things work together to realize the intelligent, automated and efficient operation of the distribution system. In the communication network of the distribution network, due to the complexity of the network topology, there are often multiple optional transmission paths when data is transmitted from the distribution terminal to each business platform. The nodes in the transmission path play the role of data forwarding and routing in the network, that is, the nodes can receive the operation data from the distribution terminal and forward it to the downstream node according to the routing information until the operation data reaches the destination business platform.

[0003] When obtaining the data transmission path when the distribution terminal transmits to the business platform, the existing technology usually adopts a static routing selection method or a heuristic path optimization algorithm to obtain the data transmission path, but both are only based on the current network state of the node. The existing technology only focuses on the local optimal solution under the current network state, and does not take into account the impact of path selection on future network congestion. It may cause congestion and insufficient throughput in the future network, thereby failing to guarantee the transmission efficiency and performance of the distribution network. Summary of the invention

[0004] The embodiments of the present invention provide a method, apparatus, terminal device and storage medium for generating a data transmission path of a distribution network, which can predict future network status data, thereby having foresight when selecting a data transmission path, and can maximize data throughput through the expected reward value of future network status data to select a final transmission path, thereby avoiding possible network congestion in the future. This can effectively solve the problem in the prior art that the impact of path selection on future network congestion is not taken into account, which may cause congestion and insufficient throughput in the future network.

[0005] An embodiment of the present invention provides a method for generating a data transmission path of a distribution network, comprising:

[0006] Acquire current network status data corresponding to each node between the power distribution terminal and an Internet of Things platform in a current time period, and each data transmission path to be selected formed between each node; wherein the current network status data includes: bandwidth and load;

[0007] For each data transmission path to be selected, the current network status data of the data transmission path to be selected and each node are input into the state prediction model, so that the state prediction model generates the future network status data of each node in the next time period according to the network status characteristics corresponding to the current network status data and the path characteristics corresponding to the data transmission path to be selected; wherein the future network status data is used to characterize the operating state of the node in the distribution network after the interactive data is transmitted through the data transmission path to be selected;

[0008] For each data transmission path to be selected, each future network state data corresponding to the selected data transmission path is simulated based on the Monte Carlo simulation method to generate an expected reward value corresponding to each future network state data;

[0009] For each data transmission path to be selected, the expected reward value corresponding to the data transmission path to be selected and each future network state data is substituted into the state reward function to generate the state reward value corresponding to the data transmission path to be selected; wherein the state reward function is used to evaluate the data throughput of the distribution network in the next time period;

[0010] The data transmission path to be selected corresponding to the maximum state reward value is used as the target data transmission path; wherein the target data transmission path is used to transmit the interactive data between the power distribution terminal and the Internet of Things platform.

[0011] Preferably, the training process of the state prediction model includes:

[0012] The data transmission path samples and the historical network status data samples of each node are used as training samples;

[0013] Taking several training samples and the actual future network state data of each training sample as input and the predicted future network state data of each training sample as output, the state prediction model to be trained is iteratively trained until the model converges to generate a preset state prediction model.

[0014] Preferably, the current network status data further includes: propagation delay and packet loss rate; the network status characteristics include: bandwidth characteristics, load characteristics, propagation delay characteristics and packet loss rate characteristics; the path characteristics include: path length characteristics and link characteristics;

[0015] The state prediction model generates future network state data of each node in the next time period according to the network state characteristics of the current network state data and the path characteristics corresponding to the data transmission path to be selected, including:

[0016] The state prediction model captures the long-term dependency and short-term dependency of the current network state data in the current time period, and generates bandwidth characteristics, load characteristics, propagation delay characteristics and packet loss rate characteristics corresponding to the current time period according to the long-term dependency and short-term dependency;

[0017] By sliding multiple convolution kernels on the data transmission path to be selected, the context information of the data transmission path to be selected is captured, and then convolution and pooling operations are performed on the context information to extract the path length features and link features corresponding to the data transmission path to be selected;

[0018] According to the bandwidth characteristics, load characteristics, propagation delay characteristics, packet loss rate characteristics, path length characteristics and link characteristics, future network status data of each node in the next time period is generated.

[0019] Preferably, generating future network status data of each node in the next time period according to the bandwidth characteristics, load characteristics, propagation delay characteristics, packet loss rate characteristics, path length characteristics and link characteristics includes:

[0020] Based on the attention mechanism, different weights are assigned to the bandwidth feature, load feature, propagation delay feature, packet loss rate feature, path length feature and link feature respectively;

[0021] According to the weights corresponding to the bandwidth feature, load feature, propagation delay feature, packet loss rate feature, path length feature and link feature, respectively, weighted summation is performed on the bandwidth feature, load feature, propagation delay feature, packet loss rate feature, path length feature and link feature to generate a fused feature vector;

[0022] Based on the fused feature vectors, the future network status data of each node in the next time period is generated.

[0023] Preferably, the step of substituting the expected reward value corresponding to the data transmission path to be selected and each future network state data into the state reward function to generate the state reward value corresponding to the data transmission path to be selected includes:

[0024] Substituting the data transmission path to be selected and the current network status data of each node into a preset action reward function to generate an action reward value corresponding to the data transmission path to be selected; wherein the action reward value is used to evaluate the data throughput of the distribution network in the current time period;

[0025] According to the state reward function, the action reward value and the expected reward value corresponding to each future network state data, the state reward value corresponding to the data transmission path to be selected is generated.

[0026] Preferably, generating the state reward value corresponding to the data transmission path to be selected according to the state reward function, the action reward value and the expected reward value corresponding to each future network state data includes:

[0027] According to the following state reward function, the state reward value corresponding to the data transmission path to be selected is calculated:

[0028] Q(S t ,a t )=R t +γQ∑ n (S t+1 ,a t );

[0029] Among them, Q(S t ,a t ) indicates that the current network status data S t Execute the data transmission path a to be selected t The corresponding state reward value, R t Indicates the data transmission path to be selected a t The corresponding action reward value, γ is the discount factor, Q(S t+1 ,a t ) indicates that the data transmission path a to be selected t The future network status data S of the node after execution t+1 The corresponding expected reward value, n is the number of nodes.

[0030] Preferably, it also includes: a strategy network and a value network;

[0031] The strategy network is used to generate various data transmission paths to be selected between various nodes according to the current network status data corresponding to various nodes in the current time period;

[0032] Substituting the expected reward value corresponding to the data transmission path to be selected and each future network state data into the state reward function to generate the state reward value corresponding to the data transmission path to be selected includes:

[0033] Through the value network, a state reward value corresponding to the data transmission path to be selected is generated according to the data transmission path to be selected and each future network state data.

[0034] Based on the above method embodiments, the present invention provides corresponding device embodiments.

[0035] An embodiment of the present invention provides a data transmission path generation device for a distribution network, comprising: a data acquisition module, a state prediction module, a simulation module, a reward value generation module, and a data transmission path determination module;

[0036] The data acquisition module is used to acquire the current network status data corresponding to the current time period of each node between the power distribution terminal and an Internet of Things platform, and each data transmission path to be selected formed between each node; wherein the current network status data includes: bandwidth and load;

[0037] The state prediction module is used to input the current network state data of the data transmission path to be selected and each node into the state prediction model for each data transmission path to be selected, so that the state prediction model generates the future network state data of each node in the next time period according to the network state characteristics corresponding to the current network state data and the path characteristics corresponding to the data transmission path to be selected; wherein the future network state data is used to characterize the operating state of the node in the distribution network after the interactive data is transmitted through the data transmission path to be selected;

[0038] The simulation module is used to simulate each future network state data corresponding to each data transmission path to be selected based on the Monte Carlo simulation method, and generate an expected reward value corresponding to each future network state data;

[0039] The reward value generating module is used for substituting the expected reward value corresponding to the data transmission path to be selected and each future network state data into the state reward function for each data transmission path to be selected, so as to generate the state reward value corresponding to the data transmission path to be selected; wherein the state reward function is used to evaluate the data throughput of the distribution network in the next time period;

[0040] The data transmission path determination module is used to use the to-be-selected data transmission path corresponding to the maximum state reward value as the target data transmission path; wherein the target data transmission path is used to transmit interactive data between the power distribution terminal and the Internet of Things platform.

[0041] Based on the above method embodiments, the present invention provides corresponding terminal device embodiments.

[0042] Another embodiment of the present invention provides a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, and when the processor executes the computer program, a method for generating a data transmission path for a distribution network described in the above-mentioned embodiment of the invention is implemented.

[0043] Based on the above method embodiments, the present invention provides corresponding storage medium item embodiments.

[0044] Another embodiment of the present invention provides a storage medium, wherein the computer-readable storage medium includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute a method for generating a data transmission path for a distribution network as described in the above-mentioned embodiment of the invention.

[0045] The following beneficial effects are achieved by implementing the present invention:

[0046] The embodiment of the present invention provides a data transmission path generation method, device, terminal device and storage medium of a distribution network. When extracting a target data transmission path from a plurality of data transmission paths to be selected, the present invention can predict the future network state data based on the current network state data of the data transmission path to be selected and each node through a state prediction model; then the influence of each data transmission path to be selected on the next time period can be considered based on the future network state data corresponding to each data transmission path to be selected; further, each future network state data is simulated by a Monte Carlo simulation method, and after the corresponding expected reward value is generated, the state reward value of each data transmission path to be selected can be obtained according to the expected reward value of the data transmission path to be selected and the corresponding future network state data. Since the state reward function can be used to evaluate the data throughput of the distribution network in the next time period, the influence of each data transmission path to be selected on the congestion of the future network can be quantified through the calculated state reward value, and finally the data transmission path to be selected corresponding to the maximum state reward value is used as the target data transmission path. Compared with the prior art, the present invention can predict the future network state data through the state prediction model, so that it has foresight when selecting the data transmission path, so that the path selection is not only based on the current state, but also takes into account the possible network conditions in the future. And a state reward function is introduced, so that the present invention can quantitatively evaluate the impact of each data transmission path to be selected on future network congestion and data throughput. By calculating the state reward value, the present invention can compare the advantages and disadvantages of different paths, so as to select the target data transmission path corresponding to the maximum state reward value, and then maximize the data throughput to select the final transmission path, avoiding possible network congestion in the future, and ensuring that the distribution network will not be congested after transmitting interactive data, thereby ensuring the transmission efficiency and performance of the distribution network. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 It is a flowchart of a method for generating a data transmission path of a distribution network provided by an embodiment of the present invention.

[0048] Figure 2 It is a schematic diagram of the path selection optimization process of reinforcement learning provided by an embodiment of the present invention.

[0049] Figure 3It is a structural schematic diagram of a data transmission path generating device for a distribution network provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0050] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0051] like Figure 1 FIG. 1 is a flow chart of a method for generating a data transmission path of a distribution network provided by an embodiment of the present invention. The method for generating a data transmission path of a distribution network comprises:

[0052] Step S1: obtaining current network status data corresponding to a current time period of each node between a power distribution terminal and an Internet of Things platform, and each data transmission path to be selected formed between each node; wherein the current network status data includes: bandwidth and load;

[0053] Step S2: for each data transmission path to be selected, the current network status data of the data transmission path to be selected and each node are input into the state prediction model, so that the state prediction model generates the future network status data of each node in the next time period according to the network status characteristics corresponding to the current network status data and the path characteristics corresponding to the data transmission path to be selected; wherein the future network status data is used to characterize the operating state of the node in the distribution network after the interactive data is transmitted through the data transmission path to be selected;

[0054] Step S3: for each data transmission path to be selected, simulating the future network state data corresponding to the selected data transmission path based on the Monte Carlo simulation method to generate an expected reward value corresponding to the future network state data;

[0055] Step S4: For each data transmission path to be selected, the expected reward value corresponding to the data transmission path to be selected and the future network state data is substituted into the state reward function to generate the state reward value corresponding to the data transmission path to be selected; wherein the state reward function is used to evaluate the data throughput of the distribution network in the next time period;

[0056] Step S5: The data transmission path to be selected corresponding to the maximum state reward value is used as the target data transmission path; wherein the target data transmission path is used to transmit the interactive data between the power distribution terminal and the Internet of Things platform.

[0057] For step S1, in a preferred embodiment, the distribution terminal is used to collect operating data of different distribution equipment in the distribution network; and in the distribution network, multiple Internet of Things platforms are used to store and process the operating data transmitted from the distribution terminal; schematically, the operating data of different distribution equipment is the interaction data between the distribution terminal and the Internet of Things platform.

[0058] In the power distribution network, the multi-node distributed architecture makes the routing selection of data transmission particularly complicated. Each distribution terminal not only needs to transmit the monitored real-time data to multiple IoT business platforms (such as power monitoring center, load management platform, etc.), but also needs to be flexibly scheduled according to network conditions and business needs to ensure the communication priority of key businesses. At the same time, the distribution terminal is not just a single data acquisition device, it also needs to interact with multiple business platforms. Therefore, how to achieve on-demand scheduling of the optimal path under the conditions of multiple node routing and multiple business platforms to ensure low latency and high reliability of data transmission is a problem that the present invention can solve.

[0059] Then, the current network status data corresponding to the current time period of each node between the power distribution terminal and an Internet of Things platform can be obtained. Schematically, the current network status data includes: bandwidth, load, propagation delay and packet loss rate; then, the various data transmission paths to be selected formed between the various nodes are obtained, so that the optimal transmission path can be extracted from the various data transmission paths to be selected as the transmission path for subsequent interactive data.

[0060] For step S2, in a preferred embodiment, the present invention combines deep reinforcement learning with network status prediction technology, which can not only select paths based on the current network status of the nodes, but also predict the future network status, thereby considering the impact of the data transmission path to be selected on the future data transmission of the distribution network based on the predicted future network status data.

[0061] Indicatively, Figure 2 As shown, the process of implementing the optimal path selection in the present invention is as follows:

[0062] Collect network status information of distribution terminals and use deep learning models to predict future network status;

[0063] The reinforcement learning algorithm selects the optimal path from multiple paths for data transmission, so that on-demand scheduling can be performed based on the needs of different business platforms.

[0064] The present invention can also optimize the path selection strategy according to the actual transmission effect feedback after data transmission, dynamically adjust the path and update the network parameters of the reinforcement learning to ensure the long-term optimization of the path selection and the high efficiency of data transmission.

[0065] Specifically, the present invention can dynamically optimize the path selection strategy through a reinforcement learning algorithm. First, the state space of the distribution network is constructed, and the following node states are defined as the state space of reinforcement learning. Then the state S t Includes the following information: bandwidth (B), load (N), propagation delay (L), and packet loss rate (P);

[0066] Therefore, the state S t can be represented as a vector:

[0067] s t =[L1,P1,B1,N1,Pr1,…,L n ,P n ,B n ,N n ,Pr n ];

[0068] Among them, n is the number of nodes in the network.

[0069] At each time step, the embodiment of the present invention needs to select an action, that is, which path to select for data transmission. Assuming that there are n available paths between the power distribution terminal and an IoT platform, the action space A t For: A t ={a1,a2,…,a n}, where a i Indicates the selection of the i-th path.

[0070] In a preferred embodiment, the present invention further comprises: a strategy network and a value network;

[0071] Then, the strategy network is used to generate each to-be-selected data transmission path formed between each node according to the current network status data corresponding to each node in the current time period;

[0072] Policy network, also called Actor network; Actor network is represented by π(S t |θ π ), which can be based on the current state S t Generate action a t , that is, choosing the path to take next, θ π is the network parameter.

[0073] The value network is also called the Critic network. The Critic network can be expressed as Q(S t ,a t |θ Q ), which is used to evaluate a given state and action pair (S t , a t ) value Q, that is, in state S t Take action at The expected future return, θ Q is a network parameter. The definition of reward can be based on the effect of path selection, including network performance indicators such as low latency, high bandwidth utilization, and low energy consumption. t , which is generally defined as the benefit after taking a specific path in a certain state, then the state reward function corresponding to the Critic network can be:

[0074] Q(S t ,a t )=R t +γQ∑ n (S t+1 ,a t );

[0075] Among them, Q(S t ,a t ) indicates that the current network status data S t Execute the data transmission path a to be selected t The corresponding state reward value, R t Indicates the data transmission path to be selected a t The corresponding action reward value, γ is the discount factor, Q(S t+1 ,a t ) indicates that the data transmission path a to be selected t The future network status data S of the node after execution t+1 The corresponding expected reward value, n is the number of nodes. It can be understood that the γ discount factor is used to balance the importance of current and future rewards.

[0076] Therefore, through the Critic network, the state reward value corresponding to the data transmission path to be selected can be generated according to the data transmission path to be selected and each future network state data.

[0077] The Actor network and the Critic network can dynamically optimize the path selection of the distribution terminal by collaborating with each other. In actual application, the optimal path can be selected for data transmission based on the current network status data corresponding to each node in the current time period.

[0078] Schematically, both the Actor network and the Critic network need to update parameters to optimize path selection, and the network update process is as follows:

[0079] Critic network, which can update θ by minimizing the TD error Q , the TD error is the gap between the Q value estimated by the current Critic network and the target Q value:

[0080]

[0081] Among them, y i =Q(S t ,a t ) is the target Q value, is the Q value estimate of the current Critic network, and N is the sample size in the training sample.

[0082] Actor network, which updates the Actor network parameters θ by maximizing the Q value output by the Critic network μ , the formula is:

[0083]

[0084] Among them, α is the learning rate, is the gradient of the Actor network parameters.

[0085] During the training process of the two networks, after each data transmission, the results are used as experience samples (s t ,a t ,r t ,s t+1 ) is stored in the experience replay buffer. Through repeated training, the reinforcement learning system can gradually optimize the path selection strategy and dynamically adapt to changes in network status, thereby achieving long-term selection of the optimal data transmission path. That is, after each path selection is completed, the parameters of the reinforcement learning agent can be optimized according to the detected network status and the calculation of the value network of the reinforcement learning agent, so as to achieve the purpose of continuous optimization.

[0086] Then the reinforcement learning process of the present invention is:

[0087] State collection and representation: First, the network state information of the distribution terminal is collected, including key indicators such as bandwidth (B), load (N), propagation delay (L), and packet loss rate (P). This state information is represented as a vector as the input state space of reinforcement learning.

[0088] Action selection: The action is generated according to the current state through the policy network (Actor network), that is, which path is selected for data transmission. The schematic action space consists of the available paths between the distribution terminal and the IoT platform.

[0089] Execute actions and observe results: Execute the selected action, that is, the data transmission path, and observe the results after execution. The results include actual transmission effects, such as latency, bandwidth utilization, energy consumption and other network performance indicators.

[0090] Calculate rewards: The value network (Critic network) evaluates the value of a given state and action, that is, the expected future reward. In an embodiment of the present invention, the expected future reward may be throughput.

[0091] Network parameter update: Critic network update: Update network parameters by minimizing the TD error (the gap between the current estimated Q value and the target Q value). Actor network update: Update network parameters by maximizing the Q value output by the Critic network to optimize the path selection strategy.

[0092] Experience replay and training:

[0093] After each data transfer, the results are stored as experience samples in the experience replay buffer.

[0094] Therefore, through repeated training and repetitive execution of the above process, the present invention can gradually optimize the path selection strategy through the Actor network and Critic network of reinforcement learning, and can dynamically adapt to changes in network status to obtain the optimal path and meet the throughput transmission requirements of the distribution network.

[0095] Furthermore, in the state prediction process of reinforcement learning, the present invention can use a state prediction model to predict the future network state data of each node in the next time period based on the current network state data corresponding to the current time period of each node, and then:

[0096] The network status characteristics include: bandwidth characteristics, load characteristics, propagation delay characteristics and packet loss rate characteristics; the path characteristics include: path length characteristics and link characteristics; it can be understood that the path length characteristics may involve the number of nodes or links on the path, and the link characteristics may include the bandwidth, quality, reliability and other attributes of the link.

[0097] When the state prediction model generates the future network state data of each node in the next time period according to the network state characteristics of the current network state data and the path characteristics corresponding to the data transmission path to be selected, the specific process is as follows:

[0098] The state prediction model captures the long-term dependency and short-term dependency of the current network state data in the current time period, and generates bandwidth characteristics, load characteristics, propagation delay characteristics and packet loss rate characteristics corresponding to the current time period according to the long-term dependency and short-term dependency;

[0099] By sliding multiple convolution kernels on the data transmission path to be selected, the context information of the data transmission path to be selected is captured, and then convolution and pooling operations are performed on the context information to extract the path length features and link features corresponding to the data transmission path to be selected;

[0100] According to the bandwidth characteristics, load characteristics, propagation delay characteristics, packet loss rate characteristics, path length characteristics and link characteristics, future network status data of each node in the next time period is generated.

[0101] Specifically, when obtaining the future network status data of each node in the next time period, different weights can be assigned to the bandwidth feature, load feature, propagation delay feature, packet loss rate feature, path length feature and link feature based on the attention mechanism;

[0102] Then, according to the weights corresponding to the bandwidth feature, load feature, propagation delay feature, packet loss rate feature, path length feature and link feature, respectively, the bandwidth feature, load feature, propagation delay feature, packet loss rate feature, path length feature and link feature are weighted and summed to generate a fused feature vector;

[0103] Finally, based on the fused feature vectors, the future network status data of each node in the next time period is generated.

[0104] The training process of the state prediction model can be:

[0105] The data transmission path samples and the historical network status data samples of each node are used as training samples;

[0106] Taking several training samples and the actual future network state data of each training sample as input and the predicted future network state data of each training sample as output, the state prediction model to be trained is iteratively trained until the model converges to generate a preset state prediction model.

[0107] Illustratively, the state prediction model can be an LSTM model. By predicting the future network state through the LSTM model, the changing trends of network load, delay, etc. can be judged in advance. Compared with the prior art that directly makes decisions based on the current network state, the present invention predicts the future state and can consider possible network fluctuations in the future in advance when making path selection, thereby improving the foresight and accuracy of decision-making.

[0108] Specifically, the state prediction model (LSTM model) of the present invention can more comprehensively understand the changing trend of the network state by capturing the long-term dependencies and short-term dependencies in the current network state data, so that the future network state can be predicted more accurately, thereby improving the accuracy and efficiency of path selection.

[0109] Not only network status features (such as bandwidth, load, propagation delay, packet loss rate) are considered, but also path features (such as path length, link quality) are combined. In addition, an attention mechanism is introduced to assign different weights to different features, so that the model pays more attention to key factors that have a significant impact on changes in network status. Finally, the fused feature vector is generated by weighted summation, which can more accurately reflect changes in network status and further improve the accuracy of prediction.

[0110] Accurate prediction of future network status provides important input information for reinforcement learning algorithms. By considering changes in future network status, reinforcement learning algorithms can optimize path selection strategies more dynamically, that is, they can select data transmission paths that perform better in future time periods.

[0111] Therefore, the present invention can predict future network status data through the state prediction model, so as to have foresight when selecting a data transmission path, so that the path selection is not only based on the current state, but also takes into account possible future network conditions.

[0112] For step S3, schematically, Monte Carlo simulation is a statistical method based on random sampling. During the simulation process, each data transmission path to be selected can be simulated multiple times, and each simulation uses the future network state data of each node as input to calculate the reward value that the path may obtain in the future. It can be understood that these reward values ​​can be network performance indicators such as transmission efficiency, bandwidth utilization, delay, and energy consumption of the path.

[0113] After multiple simulations, the average reward value obtained by each selected data transmission path in all simulations is calculated, that is, the expected reward value, which reflects the average performance of the path under future network conditions.

[0114] Based on the expected reward value, the quality of each data transmission path to be selected can be evaluated. The path with a higher expected reward value may achieve better transmission effect in the future network state.

[0115] For step S4, in order to better evaluate the data throughput of each to-be-selected data transmission path to the distribution network in the next time period, the embodiment of the present invention introduces a state reward function.

[0116] In a preferred embodiment, the step of substituting the expected reward value corresponding to the data transmission path to be selected and each future network state data into the state reward function to generate the state reward value corresponding to the data transmission path to be selected includes:

[0117] Substituting the data transmission path to be selected and the current network status data of each node into a preset action reward function to generate an action reward value corresponding to the data transmission path to be selected; wherein the action reward value is used to evaluate the data throughput of the distribution network in the current time period;

[0118] According to the state reward function, the action reward value and the expected reward value corresponding to each future network state data, the state reward value corresponding to the data transmission path to be selected is generated.

[0119] Schematically, the action reward value can be evaluated based on the data throughput of the current time period, which provides immediate feedback on the current operation, thereby identifying which paths are effective in the current state, thereby avoiding selecting those paths that perform poorly at the current state.

[0120] The state reward value combines the action reward value and the expected reward value of future network state data. It takes into account the potential performance of the path in the future time period. Therefore, the state reward value can make more comprehensive path selection decisions while pursuing immediate benefits while considering long-term planning.

[0121] By introducing the action reward value, the present invention can more accurately evaluate the performance of the data transmission path to be selected in the current time period, so as to screen out the path that performs well in the current state, providing a reliable basis for further calculating the state reward value.

[0122] At the same time, combined with the expected reward value of future network status data, the potential benefits of the path in the future time period can be more comprehensively evaluated, which improves the accuracy of path selection decisions, thereby selecting those paths that perform well at present and have potential in the future.

[0123] It is understandable that when evaluating the state reward value, it can be evaluated through the Critic network in the reinforcement learning framework, that is, the Critic network is a value function network, whose main task is to estimate the value function of the strategy. The value function measures the long-term return that can be obtained by taking a certain action in a given state.

[0124] For step S5, by calculating the expected reward value and the state reward value of each data transmission path to be selected, the optimal path can be selected for data transmission.

[0125] Therefore, the present invention can adjust the selection of the data transmission path in real time according to the change of the network state. When the network state changes, the reward value can be recalculated and a new optimal path can be selected, thereby ensuring the stability and efficiency of data transmission.

[0126] Moreover, the present invention can select the target data transmission path corresponding to the maximum state reward value, so it can maximize the data throughput to select the final transmission path, avoid possible network congestion in the future, and ensure that the distribution network will not be congested after transmitting interactive data, thereby ensuring the transmission efficiency and performance of the distribution network.

[0127] like Figure 3 As shown, based on the above-mentioned various embodiments of the method for generating a data transmission path of a power distribution network, the present invention provides a corresponding device embodiment;

[0128] An embodiment of the present invention provides a data transmission path generation device for a distribution network, comprising: a data acquisition module, a state prediction module, a simulation module, a reward value generation module, and a data transmission path determination module;

[0129] The data acquisition module is used to acquire the current network status data corresponding to the current time period of each node between the power distribution terminal and an Internet of Things platform, and each data transmission path to be selected formed between each node; wherein the current network status data includes: bandwidth and load;

[0130] The state prediction module is used to input the current network state data of the data transmission path to be selected and each node into the state prediction model for each data transmission path to be selected, so that the state prediction model generates the future network state data of each node in the next time period according to the network state characteristics corresponding to the current network state data and the path characteristics corresponding to the data transmission path to be selected; wherein the future network state data is used to characterize the operating state of the node in the distribution network after the interactive data is transmitted through the data transmission path to be selected;

[0131] The simulation module is used to simulate each future network state data corresponding to each data transmission path to be selected based on the Monte Carlo simulation method, and generate an expected reward value corresponding to each future network state data;

[0132] The reward value generating module is used for substituting the expected reward value corresponding to the data transmission path to be selected and each future network state data into the state reward function for each data transmission path to be selected, so as to generate the state reward value corresponding to the data transmission path to be selected; wherein the state reward function is used to evaluate the data throughput of the distribution network in the next time period;

[0133] The data transmission path determination module is used to use the to-be-selected data transmission path corresponding to the maximum state reward value as the target data transmission path; wherein the target data transmission path is used to transmit interactive data between the power distribution terminal and the Internet of Things platform.

[0134] It should be noted that the device embodiments described above are merely schematic, wherein the modules described as separate components may or may not be physically separated, and the components displayed as modules may or may not be physical modules, and may be located in one place or distributed on multiple network modules. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the accompanying drawings of the device embodiments provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines. Those of ordinary skill in the art can understand and implement it without paying creative labor.

[0135] Those skilled in the art can clearly understand that, for the sake of convenience and simplicity, the specific working process of the device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0136] Based on the above-mentioned various embodiments of the method for generating a data transmission path of a power distribution network, the present invention provides corresponding embodiments of terminal equipment items.

[0137] An embodiment of the present invention provides a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, a method for generating a data transmission path for a distribution network as described in any method embodiment of the present invention is implemented.

[0138] The terminal device may be a computing terminal device such as a desktop computer, a notebook, a palm computer, a cloud server, etc. The terminal device may include, but is not limited to, a processor and a memory.

[0139] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the terminal device, and uses various interfaces and lines to connect various parts of the entire terminal device.

[0140] The memory can be used to store the computer program, and the processor realizes various functions of the terminal device by running or executing the computer program stored in the memory and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function, etc.; the data storage area can store data created according to the use of the mobile phone, etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device or other volatile solid-state storage device.

[0141] Based on the above-mentioned various embodiments of the method for generating a data transmission path of a power distribution network, the present invention provides corresponding storage medium item embodiments.

[0142] An embodiment of the present invention provides a storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the computer-readable storage medium is located is controlled to execute a method for generating a data transmission path for a distribution network as described in any method embodiment of the present invention.

[0143] The storage medium is a computer-readable storage medium, and the computer program is stored in the computer-readable storage medium. When the computer program is executed by the processor, the steps of each method embodiment described above can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0144] The above is a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A method for generating a data transmission path of a distribution network, characterized in that: include: Acquire current network status data corresponding to each node between the power distribution terminal and an Internet of Things platform in a current time period, and each data transmission path to be selected formed between each node; wherein the current network status data includes: bandwidth and load; For each data transmission path to be selected, the current network status data of the data transmission path to be selected and each node are input into the state prediction model, so that the state prediction model generates the future network status data of each node in the next time period according to the network status characteristics corresponding to the current network status data and the path characteristics corresponding to the data transmission path to be selected; wherein the future network status data is used to characterize the operating state of the node in the distribution network after the interactive data is transmitted through the data transmission path to be selected; For each data transmission path to be selected, each future network state data corresponding to the selected data transmission path is simulated based on the Monte Carlo simulation method to generate an expected reward value corresponding to each future network state data; For each data transmission path to be selected, the expected reward value corresponding to the data transmission path to be selected and each future network state data is substituted into the state reward function to generate the state reward value corresponding to the data transmission path to be selected; wherein the state reward function is used to evaluate the data throughput of the distribution network in the next time period; The data transmission path to be selected corresponding to the maximum state reward value is used as the target data transmission path; wherein the target data transmission path is used to transmit the interactive data between the power distribution terminal and the Internet of Things platform.

2. A method for generating a data transmission path for a distribution network according to claim 1, characterized in that: The training process of the state prediction model includes: The data transmission path samples and the historical network status data samples of each node are used as training samples; Taking several training samples and the actual future network state data of each training sample as input and the predicted future network state data of each training sample as output, the state prediction model to be trained is iteratively trained until the model converges to generate a preset state prediction model.

3. A method for generating a data transmission path for a distribution network according to claim 2, characterized in that: The current network status data also includes: propagation delay and packet loss rate; the network status characteristics include: bandwidth characteristics, load characteristics, propagation delay characteristics and packet loss rate characteristics; the path characteristics include: path length characteristics and link characteristics; The state prediction model generates future network state data of each node in the next time period according to the network state characteristics of the current network state data and the path characteristics corresponding to the data transmission path to be selected, including: The state prediction model captures the long-term dependency and short-term dependency of the current network state data in the current time period, and generates bandwidth characteristics, load characteristics, propagation delay characteristics and packet loss rate characteristics corresponding to the current time period according to the long-term dependency and short-term dependency; By sliding multiple convolution kernels on the data transmission path to be selected, the context information of the data transmission path to be selected is captured, and then convolution and pooling operations are performed on the context information to extract the path length features and link features corresponding to the data transmission path to be selected; According to the bandwidth characteristics, load characteristics, propagation delay characteristics, packet loss rate characteristics, path length characteristics and link characteristics, future network status data of each node in the next time period is generated.

4. A method for generating a data transmission path for a distribution network according to claim 3, characterized in that: The generating of future network status data of each node in the next time period according to the bandwidth characteristics, load characteristics, propagation delay characteristics, packet loss rate characteristics, path length characteristics and link characteristics includes: Based on the attention mechanism, different weights are assigned to the bandwidth feature, load feature, propagation delay feature, packet loss rate feature, path length feature and link feature respectively; According to the weights corresponding to the bandwidth feature, load feature, propagation delay feature, packet loss rate feature, path length feature and link feature, respectively, weighted summation is performed on the bandwidth feature, load feature, propagation delay feature, packet loss rate feature, path length feature and link feature to generate a fused feature vector; Based on the fused feature vectors, the future network status data of each node in the next time period is generated.

5. The method for generating a data transmission path of a distribution network according to claim 4, characterized in that: Substituting the expected reward value corresponding to the data transmission path to be selected and each future network state data into the state reward function to generate the state reward value corresponding to the data transmission path to be selected includes: Substituting the data transmission path to be selected and the current network status data of each node into a preset action reward function to generate an action reward value corresponding to the data transmission path to be selected; wherein the action reward value is used to evaluate the data throughput of the distribution network in the current time period; According to the state reward function, the action reward value and the expected reward value corresponding to each future network state data, the state reward value corresponding to the data transmission path to be selected is generated.

6. A method for generating a data transmission path for a distribution network according to claim 5, characterized in that: The generating of the state reward value corresponding to the to-be-selected data transmission path according to the state reward function, the action reward value and the expected reward value corresponding to each future network state data comprises: According to the following state reward function, the state reward value corresponding to the data transmission path to be selected is calculated: Q(S t ,a t )=R t +γQ∑ n (S t+1 ,a t ); Among them, Q(S t ,a t ) indicates that the current network status data S t Execute the data transmission path a to be selected t The corresponding state reward value, R t Indicates the data transmission path to be selected a t The corresponding action reward value, γ is the discount factor, Q(S t+1 ,a t ) indicates that the data transmission path a to be selected t The future network status data S of the node after execution t+1 The corresponding expected reward value, n is the number of nodes.

7. The method for generating a data transmission path of a distribution network according to claim 1, characterized in that: Also includes: Strategy network and value network; The strategy network is used to generate various data transmission paths to be selected between various nodes according to the current network status data corresponding to various nodes in the current time period; Substituting the expected reward value corresponding to the data transmission path to be selected and each future network state data into the state reward function to generate the state reward value corresponding to the data transmission path to be selected includes: Through the value network, a state reward value corresponding to the data transmission path to be selected is generated according to the data transmission path to be selected and each future network state data.

8. A data transmission path generation device for a distribution network, characterized in that: include: Data acquisition module, state prediction module, simulation module, reward value generation module and data transmission path determination module; The data acquisition module is used to acquire the current network status data corresponding to the current time period of each node between the power distribution terminal and an Internet of Things platform, and each data transmission path to be selected formed between each node; wherein the current network status data includes: bandwidth and load; The state prediction module is used to input the current network state data of the data transmission path to be selected and each node into the state prediction model for each data transmission path to be selected, so that the state prediction model generates the future network state data of each node in the next time period according to the network state characteristics corresponding to the current network state data and the path characteristics corresponding to the data transmission path to be selected; wherein the future network state data is used to characterize the operating state of the node in the distribution network after the interactive data is transmitted through the data transmission path to be selected; The simulation module is used to simulate each future network state data corresponding to each data transmission path to be selected based on the Monte Carlo simulation method, and generate an expected reward value corresponding to each future network state data; The reward value generating module is used for substituting the expected reward value corresponding to the data transmission path to be selected and each future network state data into the state reward function for each data transmission path to be selected, so as to generate the state reward value corresponding to the data transmission path to be selected; wherein the state reward function is used to evaluate the data throughput of the distribution network in the next time period; The data transmission path determination module is used to use the to-be-selected data transmission path corresponding to the maximum state reward value as the target data transmission path; wherein the target data transmission path is used to transmit interactive data between the power distribution terminal and the Internet of Things platform.

9. A terminal device, characterized in that: It comprises a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, it implements a method for generating a data transmission path of a distribution network as described in any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium includes a stored computer program, wherein when the computer program is running, the device where the storage medium is located is controlled to execute the method for generating a data transmission path of a distribution network as described in any one of claims 1 to 7.

Images