A power private network and 5G public network selection method in a smart grid

Abstract

The application discloses a power special network and 5G public network network selection method in a smart grid, overcomes the problem that the communication quality of the smart grid cannot be guaranteed when the power special network and the 5G public network are fused in the prior art, and comprises the following steps: S1, deploying a power special network and 5G public network network selection cooperation module on a terminal and a master station; S2, measuring the reference signal receiving power (RSRP) of the physical cell to which the power special network and the 5G public network belong; S3, the terminal sends a service request to the master station through the power special network and the 5G public network respectively; S4, the master station acquires the service request and processes the same, and acquires the current state of the local network; S5, the master station selects a network and returns the selection result to the terminal through a response packet; S6, the terminal transmits service data to the master station by using the network selected by the master station; and S7, the network transmission quality is monitored, and it is judged whether the network selection process needs to be restarted according to the monitoring result. The 5G public network and the power special network are better fused, and the communication quality of the smart grid is guaranteed.

Description

Technical Field

[0001] The present invention relates to the technical field of 4G and 5G network integration, and in particular to a method for selecting a private power network and a 5G public network in a smart grid. Background Art

[0002] 5G, with its high speed, large capacity, and ultra-low latency, not only meets the personalized needs of individual users but also addresses the differentiated business needs of vertical industries such as smart manufacturing, smart healthcare, smart cities, and smart grids. Among 5G verticals, the power industry, as an essential foundation for national economy and people's livelihoods, is highly compatible with 5G. However, currently, insufficient 5G network signal coverage, weak signal penetration, and high data rates limit its application in power sector scenarios. Existing power wireless private networks offer security and low signal coverage costs. In terms of signal coverage, these networks effectively fill the gaps in 5G network coverage. A more realistic solution is to effectively integrate 5G and power wireless private networks. This can reduce network construction and operation costs while meeting the needs for agile response and precise control of massive resources in the integrated "power source, grid, load, and energy storage" system.

[0003] The power grid's wireless private network belongs to the power sector, while the 5G public network belongs to telecom operators. Although the 5G public network offers high speed, large capacity, and ultra-low latency, its users are not only smart grids but also general consumers and other vertical industries. Therefore, it cannot guarantee consistent high-quality communication services for smart grids. Therefore, even within the coverage area of ​​the 5G public network, smart grids must choose between the power grid wireless private network and the 5G public network as the transmission channel for power services based on the current network status. Summary of the Invention

[0004] The purpose of the present invention is to overcome the problem in the prior art that when the power private network and the 5G public network are integrated, it cannot guarantee that high-quality communication services can always be provided for the smart grid. A method for selecting the power private network and the 5G public network in the smart grid is provided. The method can select whether to use the power wireless private network or the 5G public network as the transmission bearer channel for the power business according to the current network status, so as to better integrate the 5G public network and the power private network and ensure the communication quality of the smart grid.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: a method for selecting a private power network and a 5G public network in a smart grid, comprising the following steps:

[0006] S1: Deploy the power wireless private network and 5G public network selection and collaboration module on the power wireless dual-mode terminal, and deploy the power wireless private network and 5G public network selection and collaboration module on the power business master station;

[0007] S2: measuring the reference signal receiving power (RSRP) of the physical cell to which the power wireless private network and the 5G public network belong;

[0008] S3: the power wireless dual-mode terminal sends a service request to the master station through the power wireless private network and the 5G public network respectively;

[0009] S4: the master station acquires the service request and processes it, and acquires the current state of the local network;

[0010] S5: the master station selects a network using a deep reinforcement learning algorithm and returns the selection result to the power wireless dual-mode terminal through a response packet;

[0011] S6: the power wireless dual-mode terminal transmits service data to the master station using the network selected by the master station;

[0012] S7: monitoring the network transmission quality and determining whether the network selection process needs to be restarted according to the monitoring result.

[0013] The application determines whether to use the power private network or the 5G public network as the power service communication bearer channel through the cooperation of the power wireless dual-mode terminal and the service master station, effectively improving the quality of communication and the resource utilization rate of the power wireless private network and the 5G public network.

[0014] As a preferred, in the step S1, deploying the power wireless private network and 5G public network selection cooperation module in the power wireless dual-mode terminal includes: designing the power wireless dual-mode terminal, deploying the power wireless private network and 5G public network selection cooperation module on the power wireless dual-mode terminal, the power wireless private network and 5G public network selection cooperation module accepting the communication transmission request of the service layer, selecting the communication bearer channel for the master station, and completing the network selection through signaling interaction with the master station.

[0015] The power wireless dual-mode terminal contains the baseband chip, radio frequency chip and antenna hardware of the power wireless private network, and also contains the 5G communication baseband chip, radio frequency chip and antenna hardware. The power wireless dual-mode terminal has the ability to transmit service data through LTE230 and 5G at the same time. The power wireless private network and 5G public network selection cooperation module sends and receives communication data to the communication bearer channel through the interface, and actively and periodically acquires the communication transmission state of the power wireless private network and the 5G network.

[0016] As a preferred, in the step S1, deploying the power wireless private network and 5G public network selection cooperation module in the power service master station includes:

[0017] A1: the master station simultaneously accesses the core network of the power wireless private network and the 5G public network, and the power service data is transmitted to the master station through the power wireless private network or the 5G public network;

[0018] A2: The master station accepts the communication transmission request of the power wireless dual-mode terminal, and selects a communication bearer channel according to the network status of the power wireless dual-mode terminal and the master station.

[0019] The power service master station accesses the core network of the power wireless private network and the core network of the 5G public network. The power service data is transmitted to the power service master station through the power wireless private network or the 5G public network. The power service master station can select one of the networks to transmit data to the power wireless dual-mode terminal. The power wireless private network and the 5G public network selection cooperation module is deployed on the power service master station. The power wireless private network and the 5G public network selection cooperation module accepts the communication transmission request of the power wireless dual-mode terminal and selects a communication bearer channel for the master station. The power wireless private network and the 5G public network selection cooperation module sends and receives communication data to the communication bearer channel through the interface, and actively or periodically acquires the communication transmission status of the power wireless private network core network and the 5G core network. The power wireless private network and the 5G public network selection cooperation module deployed on the master station and the power wireless private network and the 5G public network selection cooperation module deployed on the power wireless dual-mode terminal complete network selection through signaling interaction.

[0020] Preferably, in step S3, the sending service request further comprises:

[0021] S3.1: The power wireless dual-mode terminal service layer sends a communication transmission request to the power wireless private network and the 5G public network selection cooperation module;

[0022] S3.2: The power wireless private network and the 5G public network selection cooperation module obtains the reference signal received power (RSRP) of the power wireless private network and the 5G public network through the network layer interface;

[0023] S3.3: The power wireless private network and the 5G public network selection cooperation module encapsulates the information obtained in S3.1 and S3.2 according to the specified frame format, adds check information, and forms a probe data packet. The power wireless private network and the 5G public network selection cooperation module sends the probe data packet to the opposite master station through the power wireless private network and the 5G public network interface.

[0024] The communication transmission request includes the priority of the service, the bandwidth requirement, the time delay requirement, the duration, and the information of the opposite master station.

[0025] Preferably, step S4 is further represented as:

[0026] S4.1: The power service master station respectively accepts, identifies, verifies and analyzes the probe packet sent by the power wireless dual-mode terminal from the power wireless private network and the 5G public network two channels, and obtains the service request of the power wireless dual-mode terminal;

[0027] S4.2: The master station periodically samples the current load of the power wireless private network and the 5G public network;

[0028] S4.3: The service request and the current load situation are combined to form a comprehensive feature vector describing the transmission demand and the current state of the two networks.

[0029] The power service master station receives probe packets from dual-mode power wireless terminals over both the power wireless private network and the 5G public network. The power wireless private network and 5G public network selection and collaboration module within the power service master station verifies and identifies the probe packets. After identifying the probe packets, it obtains the corresponding network latency for the power wireless private network and the 5G public network. The master station's power wireless private network and 5G public network selection and collaboration module parses the probe packets, including the network RSRP, transmission service priority, bandwidth requirements, latency requirements, and duration. The master station regularly samples the current load of the power wireless private network and the 5G public network, including the number of connected peers, total bandwidth, used bandwidth, average latency, and average latency jitter. The dual-mode terminal's transmission requirements and the master station's network status are combined to form a 16-dimensional feature vector, describing the transmission requirements and the current network conditions at both ends.

[0030] Preferably, in step S5, the master station adopts a deep reinforcement learning algorithm to select a network further comprising:

[0031] S5.1: Define the state space s of deep reinforcement learning to describe the transmission requirements of the power wireless dual-mode terminal and define the comprehensive feature vector x of the current network, which has 16 dimensions in total.

[0032] S5.2: Define the action space of deep reinforcement learning as a = {0, 1}, where action a = 0 indicates using the 5G public network as the communication bearer channel, and action a = 1 indicates using the power wireless private network as the communication bearer channel;

[0033] S5.3: Define the reward function for deep reinforcement learning as R = ∑ i γ i r(s i ,a i ), where γ is the return discount factor, r(s i ,a i ) indicates the state s at the i-th request i Next take action a i The reward obtained when using action a i When the determined network successfully completes the communication transmission task, the return is 1, otherwise the return is 0;

[0034] S5.4: Use a deep reinforcement learning algorithm to train the model. The output is the probability of using the power wireless private network and the 5G public network as the communication bearer channel, as well as the evaluation value of the current state S.

[0035] S5.5: Construct a response packet and return the response packet to the wireless dual-mode terminal through two communication channels: the power wireless private network and the 5G public network.

[0036] The response packet includes the selected communication bearer channel and encryption algorithm configuration.

[0037] Preferably, in step S6, the transmission service data is further represented as:

[0038] S6.1: After sending a detection packet, the power wireless dual-mode terminal receives and parses the response packets on the power wireless private network and the 5G public network.

[0039] S6.2: The power wireless dual-mode terminal obtains the communication network and encryption algorithm configuration, and verifies the consistency of the data in the response packets received by the two networks;

[0040] S6.3: The power wireless dual-mode terminal processes the data according to the specified encryption algorithm configuration, and then sends the data using the communication bearer channel specified in the response packet.

[0041] Preferably, in step S7, monitoring the transmission quality is further expressed as:

[0042] S7.1: During service data transmission, the dual-mode terminal regularly samples the status characteristic data of the current transmission network and sends it to the power wireless private network and 5G public network selection collaboration module of the master station;

[0043] S7.2: The master station receives the network status characteristic data of the service data transmitted by the dual-mode terminal and determines whether the status characteristic index exceeds a specified threshold;

[0044] S7.3: When any one of the status characteristic indicators exceeds a specified threshold, the master station sends an instruction to the dual-mode terminal to restart the network selection process.

[0045] The state characteristic indicators include RSRP, network delay, packet loss rate and packet loss jitter. When any one of these indicators exceeds a specified threshold, the master station sends an instruction to the dual-mode terminal to restart the network selection process.

[0046] Therefore, the present invention has the following beneficial effects: 1. The modification of the basic hardware settings of the power wireless private network and 5G public network involved in the present invention is relatively small, and it belongs to the integration of the power private network and the 5G public network at the application layer, which has better realism and feasibility; 2. The present invention takes into account the network status on the mobile terminal side and the network status on the main station side. Compared with the method of selecting the network based on the network status measured on the mobile side, the feature information based on the present invention is more comprehensive and complete; 3. The present invention uses deep reinforcement learning to model the network selection problem. The result of the decision is that the utilization rate of the power private network and the 5G public network can obtain long-term maximized returns. Compared with the general classification learning method, it has obvious advantages, and also greatly reduces the amount of sample annotation and improves efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 1 is an operational flow chart of the method of the present invention.

[0048] Figure 2 This is a schematic diagram of the structure of the power wireless dual-mode terminal in the present invention.

[0049] Figure 3 This is the network architecture diagram of the power private network and 5G public network in the present invention.

[0050] Figure 4 2 is a structural diagram of the Actor-Critic neural network in an embodiment of the present invention. DETAILED DESCRIPTION

[0051] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments:

[0052] like Figure 1 In the illustrated embodiment, a method for selecting a power private network and a 5G public network in a smart grid can be seen, and its operation process is as follows: Step 1, deploying a power wireless private network and a 5G public network selection collaboration module on the power wireless dual-mode terminal, and deploying a power wireless private network and a 5G public network selection collaboration module on the power business master station; Step 2, measuring the reference signal received power RSRP of the physical cells to which the power wireless private network and the 5G public network belong; Step 3, the power wireless dual-mode terminal sends a service request to the master station through the power wireless private network and the 5G public network respectively; Step 4, the master station obtains and processes the service request, and obtains the current status of the local network; Step 5, the master station uses a deep reinforcement learning algorithm to select a network, and returns the selection result to the power wireless dual-mode terminal through a response packet; Step 6, the power wireless dual-mode terminal uses the network selected by the master station to transmit service data to the master station; Step 7, monitoring the network transmission quality, and judging whether the network selection process needs to be restarted based on the monitoring results.

[0053] The present invention determines whether to use the power private network or the 5G public network as the power business communication bearer channel through the collaboration of the power wireless dual-mode terminal and the business master station, effectively improving the communication quality and the resource utilization of the power wireless private network and the 5G public network.

[0054] The technical solution of this application is further illustrated below through specific examples:

[0055] Step 1: Deploy the power wireless private network and 5G public network selection collaboration module on the power wireless dual-mode terminal, and deploy the power wireless private network and 5G public network selection collaboration module on the power business master station.

[0056] 1. Deploy the power wireless private network and 5G public network on the power wireless dual-mode terminal and select the collaboration module.

[0057] Design a power wireless dual-mode terminal, which includes the baseband chip, radio frequency chip, and antenna hardware of the power wireless private network, and also includes the 5G communication baseband chip, radio frequency chip, and antenna hardware. The structural diagram of the power wireless dual-mode terminal in this embodiment is as follows Figure 2 As shown, the device includes a CPU 1, network port 2 connected to the CPU, serial port 3, power supply 4, 5G 5, LTE230 6, DDR 7, and eMMC 8. The power wireless dual-mode terminal is capable of simultaneously transmitting power service data via LTE230 and 5G. A power wireless private network and 5G public network selection and coordination module is deployed on the power wireless dual-mode terminal. This module accepts communication transmission requests from the service layer and selects a communication bearer channel for the service layer. This module sends and receives communication data to and from the communication bearer channel through an interface and actively and periodically obtains the communication transmission status of the power wireless private network and 5G network. The power wireless private network and 5G public network selection and coordination module deployed on the dual-mode terminal and on the power service master station complete network selection through signaling interaction.

[0058] 2. Deploy the power wireless private network and 5G public network selection collaboration module at the power business main station.

[0059] Deploy dual network access for the power business master station. The position of the master station in the power private network and 5G public network architecture is as follows: Figure 3 The power business master station is connected to both the core network of the power wireless private network and the core network of the 5G public network. Power business data is transmitted to the master station via the power wireless private network or the 5G public network. The master station selects one of the networks based on the overall network conditions to transmit the data to the power business terminal.

[0060] A power wireless private network and 5G public network selection and coordination module is deployed at the power service master station. This module accepts communication transmission requests from power wireless dual-mode terminals and selects communication bearer channels for these terminals. This module sends and receives communication data to and from the communication bearer channels through an interface, while proactively and regularly obtaining the communication transmission status of the power wireless private network core network and the 5G core network. The master station's power wireless private network and 5G public network selection and coordination module, deployed on the power wireless dual-mode terminals, completes network selection through signaling interaction.

[0061] Step 2: Measure the reference signal received power (RSRP) of the physical cells of the power wireless private network and the 5G public network.

[0062] The power wireless private network and 5G public network network selection collaboration module sends the measurement configuration to the power wireless private network and 5G public network interface through the interface; the dual-mode terminal underlying network interface will measure the current serving cell and determine whether to perform measurements on adjacent cells based on the configuration; the measurement report trigger type is periodic and event-based. When the measurement report conditions are met, the power wireless private network and 5G public network interface will send the measurement results to the power wireless private network and 5G public network network selection collaboration module; the power wireless private network and 5G public network network selection collaboration module caches the RSRP of the two networks.

[0063] Step 3: The power wireless dual-mode terminal sends a service request to the master station through the power wireless private network and the 5G public network respectively.

[0064] The power wireless dual-mode terminal constructs a detection data packet for the power wireless private network and the 5G public network. The data packet contains the RSRP of each network, service priority, bandwidth requirement, latency requirement, and duration, and then sends it to the master station via the power wireless private network and the 5G public network respectively. The specific steps are as follows:

[0065] The power wireless dual-mode terminal service layer sends a communication transmission request to the power wireless private network and 5G public network network selection and collaboration module, including the service priority, bandwidth requirements, latency requirements, duration, and information about the peer master station. The power wireless private network and 5G public network network selection and collaboration module retrieves the reference signal received power (RSRP) of the physical cell to which the power wireless private network and 5G public network are connected, obtained through the network layer interface. The power wireless private network and 5G public network network selection and collaboration module encapsulates the information obtained in the previous steps according to the specified frame format and adds verification information to form a detection data packet. The power wireless private network and 5G public network network selection and collaboration module sends the detection data packet to the peer master station through the power wireless private network and 5G public network interface.

[0066] Step 4: The power wireless private network and 5G public network selection collaboration module of the master station obtains and processes the service request, and obtains the current status of the local network.

[0067] The power wireless private network and 5G public network selection and collaboration module on the master station side collects the network latency of the power wireless private network and 5G public network detection packets, analyzes the RSRP and transmission requirements of the dual-mode terminal side network. This information, combined with the current network status of the power wireless private network and 5G public network on the master station side, forms a comprehensive description of the two networks.

[0068] Specifically:

[0069] The power business system master station receives detection packets sent by dual-mode mobile terminals through two channels: the power wireless private network and the 5G public network. The master station's power wireless private network and 5G public network selection and collaboration module verifies and identifies the detection packets. After identifying the detection packets, it obtains the corresponding power wireless private network and 5G public network delays. The power wireless private network and 5G public network selection and collaboration module parses the content of the detection packets, including the network RSRP, as well as the transmission service priority, bandwidth requirements, latency requirements, and duration. Since it is the RSRP of two networks, there are a total of six dimensions of features.

[0070] The master station regularly samples the current load of the power wireless private network and the 5G public network, including the number of connected peers, total bandwidth, used bandwidth, average latency, and average latency jitter, resulting in a total of five dimensions. The network status and transmission requirements of the dual-mode terminal and the master station obtained in the previous steps are combined to form a comprehensive feature vector x describing the requirements and the current network, with a total of 6 + 5 × 2 = 16 dimensions.

[0071] Step 5: The master station uses a deep reinforcement learning algorithm to select whether to use the power wireless private network or the 5G public network as the communication bearer channel, and returns the selection result to the power wireless dual-mode terminal through a response packet.

[0072] The power wireless private network and 5G public network selection collaboration module uses a deep reinforcement learning algorithm to select whether to use the power wireless private network or the 5G public network as the communication bearer channel for the wireless dual-mode terminal, and returns it to the wireless dual-mode terminal through a response packet, which contains the encryption algorithm configuration.

[0073] This example uses the deep reinforcement learning PPO algorithm based on the Actor-Critic framework. The specific steps are as follows:

[0074] 1: Define the state space s of deep reinforcement learning as the comprehensive feature vector x that describes the dual-mode terminal requirements and the current network, with a total of 16 dimensions.

[0075] 2: Define the action space of deep reinforcement learning as a = {0, 1}, where action a is 0, which means using 5G public network as the communication bearer channel, and action a is 1, which means using power wireless private network as the communication bearer channel.

[0076] 3: Define the reward function of deep reinforcement learning, r(s i ,a i ) represents the reward obtained by taking action a i in state s i at the i-th request, when the network successfully completes the communication transmission task determined by action a i , the reward r(s i ,a i ) is 1, otherwise r(s i ,a i ) is 0, that is:

[0077]

[0078] The goal of system decision is to maximize the long-term reward, and let γ be the reward discount factor, then the learning goal is:

[0079] R = ∑ i γ i r(s i ,a i ).

[0080] 4: Use the deep reinforcement learning PPO algorithm based on the Actor-Critic framework for model training, the Actor neural network and Critic neural network structure are shown in Figure 4 , where B represents the bias term and W represents the weight.

[0081] 5: The Actor subnetwork contains 5 layers of perceptron, which uses neural network linear function and activation function to realize, each linear function Linear is followed by a Relu activation function. The first linear function maps the 16-dimensional state vector to a 256-dimensional high-dimensional vector, and the input and output of the subsequent 3 linear functions are 256-dimensional, and the input of the last linear function is 256-dimensional and the output is 2-dimensional. The 2-dimensional probability is normalized by the softmax layer, which is the probability of using power wireless private network and 5G public network as the communication bearer channel.

[0082] 6: The Critic subnetwork and the Actor subnetwork share the first 3 layers of perceptron, the first linear function maps the 16-dimensional state vector to a 256-dimensional high-dimensional vector, and the input and output of the subsequent 3 linear functions are 256-dimensional, and the input of the last linear function is 256-dimensional and the output is 1-dimensional, which is the evaluation value of the current state s.

[0083] 7: Based on the Actor network and Critic network described above, the PPO algorithm is used to sample the environment and perform gradient updates. The parameter update formula is:

[0084]

[0085] Among them, J ppo (θ) is the PPO policy gradient, π θ and π old They are the current strategy and the historical strategy, is the estimated advantage function, λ is a hyperparameter, and KL is the divergence function.

[0086] 8: Construct a response packet and return it to the wireless dual-mode terminal through two communication channels: the power wireless private network and the 5G public network. The response packet contains the selected communication bearer channel and encryption algorithm configuration.

[0087] Step 6: The power wireless dual-mode terminal uses the communication bearer channel selected by the master station to transmit business data to the master station.

[0088] 1: After sending a detection packet, the power wireless dual-mode terminal receives a response packet in two communication networks, the power wireless private network and the 5G public network, and then parses the response packet.

[0089] 2: The power wireless dual-mode terminal obtains the network used for communication and the encryption algorithm configuration, and verifies the consistency of the data in the response packets received by the two networks.

[0090] 3: The power wireless dual-mode terminal processes the data according to the specified encryption algorithm configuration, and then sends the data using the communication bearer channel specified in the response packet.

[0091] Step 7: Monitor the network data transmission quality and determine whether to restart the network selection process based on the monitoring results.

[0092] During the transmission process, the power wireless dual-mode terminal regularly sends the network's received power RSRP, network delay, and packet loss rate information to the power wireless private network and 5G public network selection and collaboration module of the power business master station. When it exceeds a certain threshold, the network selection process is restarted.

[0093] Specifically:

[0094] In the process of service data transmission, the power wireless dual-mode terminal periodically samples the RSRP of the current transmission network, and the network state characteristics of the transmission, including network delay, packet loss rate and packet loss jitter; the power wireless dual-mode terminal sends the above data to the power wireless private network and the 5G public network selection cooperation module of the power service master station; the master station accepts the network state characteristics of the service data transmission of the dual-mode terminal, and judges whether the RSRP, network delay, packet loss rate and packet loss jitter indicators exceed the specified threshold.

[0095] When any one of the RSRP, network delay, packet loss rate and packet loss jitter indicators exceeds the specified threshold, the master station sends an instruction to the dual-mode terminal to restart the network selection process.

[0096] The power wireless private network and 5G public network involved in the present application have small modification in the setting of basic hardware, belong to the fusion of the power private network and the 5G public network at the application layer, and have better reality and feasibility. Meanwhile, the present application considers not only the network state of the mobile terminal side, but also the network condition of the master station side, and the feature information based on the present application is more comprehensive and complete compared with the method of selecting the network only according to the network state measured by the mobile side. The deep reinforcement learning is used to model the network selection problem, and the result of decision is to make the utilization rate of the power private network and the 5G public network obtain long-term maximum return, which has obvious advantages compared with the general classification learning method, and greatly reduces the labeling amount of samples and improves the efficiency.

[0097] The above-mentioned embodiment is only a preferred scheme of the present application, and does not limit the present application in any form. Other variants and modifications can be made without exceeding the technical scheme recorded in the claims.

Claims

1. A method for selecting a private power network and a 5G public network in a smart grid, characterized in that: The following steps are involved: S1: Deploy the power wireless private network and 5G public network selection and collaboration module on the power wireless dual-mode terminal, and deploy the power wireless private network and 5G public network selection and collaboration module on the power business master station; S2: Measure the reference signal received power (RSRP) of the physical cells of the power wireless private network and the 5G public network. S3: The power wireless dual-mode terminal sends a service request to the master station through the power wireless private network and the 5G public network respectively; S4: The master station obtains and processes the service request, and obtains the current status of the local network; S5: The master station uses the deep reinforcement learning (PPO) algorithm based on the actor-critic framework to select a network and returns the selection result to the power wireless dual-mode terminal via a response packet. The actor subnetwork contains five layers of perceptrons, each of which includes linear functions, each followed by a Relu activation function. The first linear function maps the 16-dimensional state vector to a 256-dimensional high-dimensional vector. The last linear function has a 256-dimensional input and a 2-dimensional output, representing the probability of using the power wireless private network and the 5G public network as the communication bearer channel. The critic subnetwork and the actor subnetwork share the first three layers of perceptrons and use the PPO algorithm to sample the environment and perform gradient updates. S6: The power wireless dual-mode terminal uses the network selected by the master station to transmit business data to the master station; S7: Monitor the network transmission quality and determine whether to restart the network selection process based on the monitoring result.

2. The method for selecting a private power network and a 5G public network in a smart grid according to claim 1, characterized in that: In the step S1, deploying the power wireless private network and 5G public network selection collaboration module on the power wireless dual-mode terminal includes: designing the power wireless dual-mode terminal, deploying the power wireless private network and 5G public network selection collaboration module on the power wireless dual-mode terminal, the power wireless private network and 5G public network selection collaboration module accepting the communication transmission request of the business layer, and completing the network selection with the power business master station through signaling interaction.

3. The method for selecting a private power network and a 5G public network in a smart grid according to claim 1 or 2, characterized in that: In step S1, deploying the power wireless private network and 5G public network selection collaboration module at the power business master station includes: A1: The power business master station is connected to both the power wireless private network and the 5G public network core network. Power business data is transmitted to the master station via the power wireless private network or the 5G public network. A2: The master station receives the communication transmission request from the power wireless dual-mode terminal and selects a communication bearer channel based on the power wireless dual-mode terminal and its own network conditions.

4. The method for selecting a private power network and a 5G public network in a smart grid according to claim 1, characterized in that: In step S3, sending the service request further includes: S3.1: The power wireless dual-mode terminal sends a communication transmission request; S3.2: The power wireless private network and 5G public network selection and collaboration module retrieves the reference signal received power (RSRP) of the power wireless private network and the 5G public network obtained through the network layer interface; S3.3: The power wireless private network and 5G public network selection collaboration module encapsulates the information obtained in steps S3.1 and S3.2 according to the specified frame format, and adds verification information to form a detection data packet, which is then sent to the peer master station through the power wireless private network and 5G public network interface.

5. The method for selecting a private power network and a 5G public network in a smart grid according to claim 1 or 4, characterized in that: The step S4 is further expressed as: S4.1: The power business master station receives, identifies, verifies, and parses the detection packets sent by the power wireless dual-mode terminal through the power wireless private network and the 5G public network, respectively, and obtains the service request of the power wireless dual-mode terminal; S4.2: The master station periodically samples the current load of the power wireless private network and the 5G public network; S4.3: The service request and the current load situation are combined to form a comprehensive feature vector describing the transmission demand and the current state of the two networks.

6. The method for selecting a private power network and a 5G public network in a smart grid according to claim 1, characterized in that: In step S5, the master station further adopts the deep reinforcement learning algorithm to select a network including: S5.1: Define the state space s for deep reinforcement learning to describe the transmission requirements of the power wireless dual-mode terminal and the comprehensive feature vector x of the current network, with a total of 16 dimensions; S5.2: Define the action space of deep reinforcement learning as a = {0, 1}; S5.3: Define the reward function for deep reinforcement learning as R = ∑ i γ i r(s i ,a i ), where γ is the return discount factor, r(s i ,a i ) indicates the state s at the i-th request i Next take action a i the rewards received; S5.4: Use a deep reinforcement learning algorithm to train the model. The output is the probability of using the power wireless private network and the 5G public network as the communication bearer channel, as well as the evaluation value of the current state S. S5.5: Construct a response packet and return the response packet to the power wireless dual-mode terminal through two communication channels: the power wireless private network and the 5G public network.

7. The method for selecting a private power network and a 5G public network in a smart grid according to claim 1 or 6, characterized in that: In step S6, the transmission of service data is further represented as: S6.1: After sending a detection packet, the power wireless dual-mode terminal receives and parses the response packets on the power wireless private network and the 5G public network. S6.2: The power wireless dual-mode terminal obtains the communication network and encryption algorithm configuration, and verifies the consistency of the data in the response packets received by the two networks; S6.3: The power wireless dual-mode terminal processes the data according to the specified encryption algorithm configuration, and then sends the data using the communication bearer channel specified in the response packet.

8. The method for selecting a private power network and a 5G public network in a smart grid according to claim 6, characterized in that: In step S7, monitoring the transmission quality is further expressed as: S7.1: During the service data transmission process, the power wireless dual-mode terminal regularly samples the status characteristic data of the current transmission network and sends it to the power service master station; S7.2: The power business master station receives the network status characteristic data of the business data transmitted by the power wireless dual-mode terminal and determines whether the status characteristic index exceeds a specified threshold; S7.3: When any one of the state characteristic indicators exceeds a specified threshold, the power service master station sends an instruction to the power wireless dual-mode terminal to restart the network selection process.

Images