A gateway selection method and system

Abstract

The application discloses a gateway selection method and system, and is applied to the technical field of electric power communication. The method comprises the following steps: selecting optional gateways from all target gateways based on the cache queue depth and fine-grained occupation degree of each target gateway; selecting candidate gateways from all optional gateways based on the current network state vector of the electric power communication network and the service priority of to-be-interacted data; and reasoning the interaction selection of the to-be-interacted data based on the cache queue depth and fine-grained occupation degree of the candidate gateways and the service priority of the to-be-interacted data, so as to obtain a gateway selection result. The application improves the accuracy of gateway selection in electric power communication.

Description

Technical Field

[0001] This invention relates to the field of power communication technology, and in particular to a gateway selection method and system. Background Technology

[0002] The communication performance between the main grid (transmission network) and the distribution network (distribution network) in a power grid is one of the foundations for ensuring the safe operation of the power system. Existing solutions for gateway selection in data interaction between the transmission and distribution networks are illustrated in the prior art published under CN110048950A. CN110048950A proposes a full-service load awareness method for power optical networks. This method collects data packet arrival rates through optical distribution nodes and calculates data packet waiting delays using queuing theory. It predicts the link traffic status for the next cycle using a weighted moving average mechanism; estimates the buffer queue length based on Little's rule; calculates normalized link load using range processing and calculates the total bandwidth utilization of the entire network; and periodically updates the network status, determining whether the bandwidth utilization meets the QoS requirements of new services. If it does, a suitable route is allocated; otherwise, access is rejected. This method aims to achieve dynamic allocation of network resources through traffic prediction and load statistics.

[0003] However, CN110048950A has the following problems: (1) Load perception is based on statistical prediction model to indirectly estimate link status, lacking real-time and deterministic perception of physical layer hard pipe time slot resources, resulting in delayed and inaccurate decision-making basis; (2) Routing allocation is only performed by simple comparison of bandwidth utilization and QoS threshold, which cannot adapt to dynamically changing network environment and differences in the priority of multiple services; (3) Only a single routing allocation result is output, which cannot realize parallel collaborative traffic distribution of multiple gateways, and it is difficult to cope with the load balancing requirements under sudden traffic. Summary of the Invention

[0004] This invention provides a gateway selection method and system to solve the technical problem of how to achieve multi-gateway collaborative selection under multiple service priority constraints in power communication networks based on fine-grained resource awareness of the fgOTN physical layer.

[0005] To address the aforementioned technical problems, embodiments of the present invention provide a gateway selection method applied to a power communication network including a transmission network, a target gateway, and a distribution network; the transmission network and the distribution network interact with each other through the target gateway; the gateway selection method includes: Based on the cache queue depth and fine-grained occupancy of each target gateway, selectable gateways are obtained from all the target gateways. Based on the current network state vector of the power communication network and the service priority of the data to be interacted, the candidate gateways among all the available gateways are selected. Based on the cache queue depth and fine-grained occupancy of the candidate gateways, as well as the service priority of the data to be interacted with, the interaction selection of the data to be interacted with is inferred to obtain the gateway selection result.

[0006] As one preferred embodiment, the step of filtering out the selectable gateways from all the target gateways based on the cache queue depth and fine-grained occupancy of each target gateway includes: Based on the fgOTN network, a power communication network including the transmission network, target gateway, and distribution network is constructed; A communication topology diagram containing communication nodes and communication links is constructed; the communication nodes include the transmission network, the distribution network, and the target gateway; the communication links are composed of multiple communication nodes. The operating parameters of the communication nodes and the communication links are analyzed to obtain the cache queue depth and fine-grained occupancy of each target gateway.

[0007] As one preferred embodiment, the process of selecting selectable gateways from all target gateways based on the cache queue depth and fine-grained occupancy of each target gateway includes: The cache queue depth of each target gateway is compared with a first preset threshold to obtain a first comparison result; The fine-grained occupancy of the communication link corresponding to each target gateway is compared with a second preset threshold to obtain a second comparison result; Based on the first comparison result and the second comparison result, the blocked gateway among all the target gateways is determined; By filtering out the blocked gateways, the selectable gateways from all the target gateways are obtained.

[0008] As one preferred embodiment, the process of filtering out candidate gateways from all available gateways based on the current network state vector of the power communication network and the service priority of the data to be interacted includes: Based on the cache queue depth and fine-grained occupancy of each target gateway, and the remaining bandwidth of the power communication network, the current network state vector of the power communication network is determined. The current network state vector is input into the decision neural network to obtain the value stream output scalar of the power communication network output by the decision neural network, and the advantage stream output vector of each of the optional gateways; Based on the value stream output scalar and the advantage stream output vector, as well as the business priority of the data to be interacted with, the candidate gateways are selected from all the available gateways.

[0009] As one preferred embodiment, the reasoning for the interaction selection of the data to be interacted based on the cache queue depth and fine-grained occupancy of the candidate gateways, as well as the service priority of the data to be interacted, to obtain the gateway selection result includes: Based on the cache queue depth and fine-grained occupancy of the candidate gateways, as well as the service priority of the data to be interacted with, the result of any of the data to be interacted with through any of the candidate gateways is inferred to obtain the data transmission latency, gateway load data and interaction topology overhead of all interaction results. The composite reward value for each interaction result is determined based on the data transmission latency, gateway load data, and interaction topology overhead of each interaction result. Based on the composite reward value of each interaction result, the gateway selection result for each piece of data to be interacted with is determined.

[0010] Another embodiment of the present invention provides a gateway selection system, comprising: The target gateway filtering module is used to filter out the selectable gateways among all the target gateways based on the cache queue depth and fine-grained occupancy of each target gateway. The optional gateway filtering module is used to filter out the candidate gateways from all the optional gateways based on the current network state vector of the power communication network and the service priority of the data to be interacted with. The gateway selection result determination module is used to reason about the interaction selection of the data to be interacted based on the cache queue depth and fine-grained occupancy of the candidate gateways and the service priority of the data to be interacted, and to obtain the gateway selection result.

[0011] As one preferred embodiment, the gateway selection system further includes: The power communication network construction module is used to construct a power communication network, including the transmission network, target gateway, and distribution network, based on the fgOTN network. A communication topology construction module is used to construct a communication topology map that includes communication nodes and communication links; the communication nodes include the transmission network, the distribution network, and the target gateway; the communication links are composed of multiple communication nodes; The operation parameter analysis module is used to analyze the operation parameters of the communication node and the communication link to obtain the cache queue depth and fine-grained occupancy of each target gateway.

[0012] As one preferred embodiment, the target gateway filtering module includes: The first comparison result determination unit is used to compare the cache queue depth of each target gateway with a first preset threshold to obtain a first comparison result; The second comparison result determination unit is used to compare the fine-grained occupancy of the communication link corresponding to each target gateway with a second preset threshold to obtain a second comparison result; A blocked gateway determination unit is configured to determine the blocked gateway among all the target gateways based on the first comparison result and the second comparison result; A blocking gateway screening unit is used to screen out the blocking gateways to obtain the selectable gateways from all the target gateways.

[0013] As one preferred embodiment, the optional gateway filtering module includes: The current network state vector determination unit is used to determine the current network state vector of the power communication network based on the cache queue depth and fine-grained occupancy of each target gateway and the remaining bandwidth of the power communication network. The decision neural network execution unit is used to input the current network state vector into the decision neural network to obtain the value stream output scalar of the power communication network output by the decision neural network, and the advantage stream output vector of each of the optional gateways; The optional gateway filtering unit is used to filter out candidate gateways from all the optional gateways based on the value stream output scalar, the advantage stream output vector, and the business priority of the data to be interacted with.

[0014] As one preferred embodiment, the gateway selection result determination module includes: The interactive reasoning unit is used to reason about the result of any interactive data being interacted with through any of the candidate gateways based on the cache queue depth and fine-grained occupancy of the candidate gateways and the service priority of the interactive data, so as to obtain the data transmission latency, gateway load data and interactive topology overhead of all interactive results. The composite reward value determination unit is used to determine the composite reward value of each interaction result based on the data transmission delay, gateway load data, and interaction topology overhead of each interaction result. The gateway selection result determination unit is used to determine the gateway selection result for each piece of data to be interacted based on the composite reward value of each interaction result.

[0015] This invention filters out congested gateways by considering the cache queue depth and fine-grained occupancy of each target gateway. It further utilizes the current network state vector and the priority of the data services to be interacted with for secondary gateway selection, ensuring that the candidate gateways simultaneously meet physical layer conditions and service level requirements. Based on this, it performs comprehensive reasoning on the interaction selection according to the cache queue depth, fine-grained occupancy, and service priority of the candidate gateways to obtain the gateway selection result. Then, based on this gateway selection result, data interaction is executed in parallel through each selected candidate gateway. This forms a complete decision-making and execution chain from physical resource awareness and service awareness filtering to parallel output from multiple gateways. This avoids policy oscillations caused by blind exploration, and effectively prevents overload of a single gateway through parallel traffic distribution across multiple gateways, thereby improving the overall accuracy of gateway selection. Attached Figure Description

[0016] Figure 1 This is one of the flowcharts illustrating the gateway selection method provided by the present invention; Figure 2 This is the second flowchart illustrating the gateway selection method provided by the present invention; Figure 3 This is a schematic diagram of the gateway selection system provided by the present invention.

[0017] Figure label: Among them, 301 is the target gateway filtering module; 302 is the optional gateway filtering module; 303 is the gateway selection result determination module; and 304 is the gateway selection module. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The purpose of providing these embodiments is to make the disclosure of the present invention more thorough and comprehensive. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0019] In the description of this application, the terms "first," "second," "third," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined with "first," "second," "third," etc., may explicitly or implicitly include one or more of that feature. In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0020] In the description of this application, it should be noted that, unless otherwise expressly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to fixed connections, detachable connections, or integral connections; they can refer to mechanical connections or electrical connections; they can refer to direct connections or indirect connections through an intermediate medium; and they can refer to the internal communication between two components. The terms "vertical," "horizontal," "left," "right," "upper," "lower," and similar expressions used herein are for illustrative purposes only and do not indicate or imply that the device or component referred to must have a specific orientation, or be constructed and operated in a specific orientation, and therefore should not be construed as limiting the invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.

[0021] In the description of this application, it should be noted that, unless otherwise defined, all technical and scientific terms used in this invention have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in this specification is for the purpose of describing specific embodiments only and is not intended to limit the invention. Those skilled in the art can understand the specific meaning of the above terms in this application based on the specific circumstances.

[0022] See Figure 1 , Figure 1 This is a flowchart illustrating an embodiment of the gateway selection method provided by the present invention. The gateway selection method is applied to a power communication network including a transmission network, a target gateway, and a distribution network; the transmission network and the distribution network interact with each other through the target gateway; such as Figure 1 As shown, this embodiment includes steps 100 to 400, and the specific steps are as follows: Step 100: Based on the cache queue depth and fine-grained occupancy of each target gateway, select the available gateways from all the target gateways; In the power communication network used in this invention, the transmission network and distribution network exchange data through target gateways. Target gateways can be deployed between the transmission and distribution networks, and they possess functions such as protocol conversion, data caching, and data forwarding, making them key devices connecting the two networks. Based on the cache queue depth and fine-grained occupancy of each target gateway, all target gateways are screened to obtain selectable gateways. For example, selectable gateways are those among the target gateways that meet the communication link time slot carrying capacity requirements and are not identified as congested.

[0023] In one specific implementation, the process of filtering target gateways based on the buffer queue depth and fine-grained occupancy of each target gateway is as follows: First, the buffer queue depth of each target gateway is collected in real time. This buffer queue depth represents the total number of bits of the data packets currently waiting to be forwarded by the target gateway, reflecting the local load of the target gateway. Simultaneously, the fine-grained occupancy of the uplink connecting each target gateway is obtained. This fine-grained occupancy represents the proportion of physical layer time slot resources occupied by the fgOTN communication link connecting the target gateway at the current moment (also known as time slot occupancy rate). Its value ranges from 0 to 1, and it is used to characterize the idle level of the fgOTN communication link. Unlike link bandwidth utilization, the fine-grained occupancy of fgOTN uses time slots as the smallest statistical unit (down to the 2.5Gbps time slot level), which can more accurately reflect the actual resource occupancy status of the physical layer hard pipe. Subsequently, the cache queue depth of each gateway is compared with a preset queue depth threshold, and the fine-grained occupancy of the link corresponding to that gateway is compared with a preset occupancy threshold. When the cache queue depth of a target gateway exceeds the queue depth threshold, or the fine-grained occupancy of its corresponding link exceeds the occupancy threshold, the target gateway is marked as an unavailable gateway. After traversing all target gateways, the target gateways that have not been marked as unavailable gateways are selected as optional gateways.

[0024] For example, the threshold for relay protection service-related data can be set to 0.7, while the threshold for regular data collection service-related data can be set to 0.9, to ensure that gateways corresponding to high-priority services operate in the light-load zone of the physical link. Through the gateway filtering method in this embodiment, a pre-verification of physical layer resource congestion status is achieved, fundamentally avoiding the transmission of data to already congested links, thereby transferring the trial-and-error cost of reinforcement learning from the physical environment to the filtering stage within the algorithm.

[0025] It should be noted that "fine-grained occupancy" and "time slot occupancy rate" are different expressions of the same physical parameter. Fine-grained occupancy focuses on describing the resource awareness characteristics of the fgOTN network with time slots as the smallest statistical granularity, while time slot occupancy rate focuses on describing the numerical meaning of this physical parameter in the time slot resource dimension. Both refer to the same physical quantity, namely the ratio of the number of currently occupied time slots to the total number of time slots in the fgOTN communication link.

[0026] By using gateway filtering based on cache queue depth and fine-grained occupancy, real-time alignment between the decision action space and the physical layer resource status was achieved. This avoided blindly sending data to congested gateways in subsequent decisions, thereby improving the certainty of decisions, reducing the number of invalid explorations, and laying a data foundation for the rapid convergence of subsequent reinforcement learning models.

[0027] Step 200: Based on the current network state vector of the power communication network and the service priority of the data to be interacted, filter out the candidate gateways from all the available gateways; The current network state vector is a set of parameters used to describe the current operating state of the power communication network. In this embodiment, the state vector may include parameters with multiple dimensions, such as the buffer queue depth of each target gateway, the fine-grained occupancy of the fgOTN (Fine-Grain Optical Transport Network) resources on the uplink corresponding to each target gateway, and the average remaining bandwidth of the entire communication network. Service priority is used to characterize the importance and real-time requirements of the data to be exchanged. Service types in the power communication network include relay protection services, stability control services, and data acquisition services, etc., and different services have different latency sensitivity and transmission reliability requirements. For example, relay protection services are extremely latency-sensitive services and have the highest priority; stability control services are next; and conventional data acquisition services have relatively low priority.

[0028] In one specific implementation, the process of selecting candidate gateways based on the current network state vector and the service priority of the data to be interacted with is as follows: First, the collected current network state vector is input into a pre-trained decision neural network. This decision neural network is built based on the Dueling DQN (Dueling Deep Q-Network, deep reinforcement learning for network structure improvement) architecture. The value stream output scalar of the decision neural network is used to evaluate the inherent value of the current network state itself, which is not affected by specific gateway selection actions. The advantage stream output vector contains multiple components, each representing the degree of advantage of selecting the corresponding gateway relative to the average level in the current network state. Subsequently, the value stream output scalar and the advantage stream output vector are aggregated to obtain the Q value of each optional gateway. Next, a corresponding mask threshold is determined according to the service priority of the data to be interacted with. A binary mask matrix is ​​generated based on this threshold, and gateways with Q values ​​lower than the threshold or physical link occupancy exceeding the threshold are eliminated. After filtering by the mask matrix, the remaining optional gateways constitute the set of candidate gateways. The above method enables the embedding of hard constraints for business priority awareness in the reinforcement learning decision-making process, ensuring that high-priority services are assigned to the gateway with the optimal physical link status.

[0029] By introducing a service priority-aware dynamic filtering mechanism, services of different levels can obtain differentiated gateway resource guarantees. In particular, high-priority time-sensitive services can be forcibly allocated to gateways with light physical link loads, thereby ensuring deterministic low-latency transmission of power-critical services.

[0030] Step 300: Based on the cache queue depth and fine-grained occupancy of the candidate gateways, and the service priority of the data to be interacted with, reason about the interaction selection of the data to be interacted with, and obtain the gateway selection result; In this embodiment, reasoning refers to the process of using a trained lightweight neural network model to quickly output a decision result based on the current input state data through forward computation.

[0031] In one specific implementation, after offline training is completed, the inference phase uses pre-defined neural network parameters. During inference, the current network state vector is collected in real time, including the cache queue depth, fine-grained occupancy, and remaining network bandwidth of each target gateway. The collected state vector is input into the neural network, and after forward propagation through the input layer and two shared hidden layers, the value stream output scalar and the advantage stream output vector are obtained. These two are aggregated to generate the Q-value of each candidate gateway. Subsequently, based on the cache queue depth and fine-grained occupancy of the candidate gateways, as well as the service priority of the data to be interacted with, a mask threshold is further determined. This threshold is used to generate a mask matrix to filter the Q-values. In the filtered legal action space, an ε-greedy strategy is used for decision-making: either a preset number of gateways are randomly selected with probability ε to ensure exploration, or a preset number of gateways with the largest Q-values ​​are selected with probability 1-ε as the output. The final output gateway selection result is a set of integers representing physical gateway numbers. Since the inference process only involves matrix linear operations of a three-layer fully connected network, the time for a single inference can be controlled in the microsecond range, which is much shorter than the time slot scheduling cycle of fgOTN, thus ensuring the real-time response of power services.

[0032] In another implementation, inference can also be performed based on a simplified rule engine. That is, when the neural network model is unavailable for some reason, a weighted score is directly performed based on the gateway's cache queue depth and fine-grained occupancy, and the best-scoring gateways are selected as the output to ensure availability in extreme cases.

[0033] This embodiment adopts an architecture that separates offline training and online inference, placing the complex training process in the background and deploying only a lightweight model on-site to perform forward computation. This significantly reduces the computing power threshold and inference latency for real-time decision-making, enabling the intelligent gateway optimization strategy to adapt to the scheduling cycle of fgOTN, ensuring the intelligence of decision-making while meeting the real-time requirements of power services.

[0034] In this embodiment, the gateway selection result is the final decision output for gateway selection, which is represented as a set of selected gateway identifiers. The data to be interacted with is exchanged through each candidate gateway. Based on the gateway selection result, the data to be interacted with is transmitted in parallel through multiple selected gateways according to a preset allocation strategy.

[0035] In one specific implementation, after outputting a one-dimensional vector containing multiple gateway indices, the corresponding fgOTN link information is searched in a pre-maintained network topology table based on the gateway number in the vector. Subsequently, fgOTN hard-pipe connections are established with each selected gateway and distribution network node, and the data stream to be exchanged is split according to a data allocation strategy. This data allocation strategy can be based on a weighted allocation of the current idle bandwidth ratio of each selected gateway; for example, the larger the idle bandwidth of a gateway, the more data is allocated, to avoid overloading any gateway. Next, each distribution network node transmits the split data in parallel to the corresponding selected gateway through the established fgOTN links. Upon receiving the data, the selected gateway performs protocol conversion and buffering, and finally forwards it to the power grid. During transmission, the transmission status of each link is continuously monitored. If a selected gateway fails or its link quality deteriorates sharply during transmission, a fast reselection mechanism is triggered, using the latest status data to re-execute the gateway selection process and select an alternative gateway to ensure the continuity of data transmission.

[0036] In another implementation, the data allocation strategy can also be a fixed allocation method based on the service type. For example, all relay protection service data can be transmitted through the gateway with the optimal latency, while the acquisition service data can be transmitted through other gateways to achieve differentiated services at the service level.

[0037] This embodiment maps the gateway selection results obtained through inference to the actual fgOTN link connection and data distribution operations, realizing the seamless integration of intelligent decision-making and physical execution. Furthermore, by using a multi-gateway parallel transmission method, it effectively utilizes the fine-grained slice resources of fgOTN, improving the transmission efficiency of master-slave collaborative data interaction and the overall load balancing capability.

[0038] At the gateway selection decision-making front end, this invention filters out congested gateways by considering the cache queue depth and fine-grained occupancy of each target gateway. It further utilizes the current network state vector and the priority of the data services to be interacted with for secondary gateway selection, ensuring that the candidate gateways simultaneously meet physical layer conditions and service level requirements. Based on this, a comprehensive reasoning process is performed on the interaction selection based on the cache queue depth, fine-grained occupancy, and service priority of the candidate gateways to obtain the gateway selection result. Then, according to this gateway selection result, data interaction is executed in parallel through each selected candidate gateway. This forms a complete decision-making and execution chain from physical resource awareness and service awareness filtering to parallel output from multiple gateways. This avoids policy oscillations caused by blind exploration, and effectively prevents overload of a single gateway through parallel traffic distribution across multiple gateways, thereby improving the overall accuracy of gateway selection.

[0039] In another embodiment of the gateway selection method provided by the present invention, the steps prior to step 100 specifically include: Step 10: Based on the fgOTN network, construct a power communication network including the transmission network, target gateway, and distribution network; Step 20: Construct a communication topology diagram including communication nodes and communication links; the communication nodes include the transmission network, the distribution network, and the target gateway; the communication links are composed of multiple communication nodes; Step 30: Analyze the operating parameters of the communication node and the communication link to obtain the cache queue depth and fine-grained occupancy of each target gateway.

[0040] In this embodiment, the fgOTN network is a fine-grained optical transport network. Based on traditional OTN, it further refines the time slot granularity of the optical channels, enabling it to provide hard-pipe carrying capacity down to 2.5Gbps or even smaller granularities. It is a key physical layer technology for achieving rigid isolation and deterministic low-latency transmission of power services. The communication topology diagram is used to describe the connection relationships between communication nodes and communication links in the power communication network.

[0041] In one specific implementation, the construction of the power communication network is as follows: based on fgOTN network technology, fgOTN equipment (e.g., fgOTN cross-connection equipment and fgOTN access equipment) is used to establish a physical connection between the transmission network and the distribution network, and multiple target gateways are deployed at the boundary positions of the transmission network and the distribution network, thereby forming a primary and secondary coordinated communication network with the target gateways as the interaction hubs.

[0042] In one specific implementation, the process of constructing the communication topology is as follows: First, the aggregation nodes on the transmission network side, the access nodes on the distribution network side, and all target gateways are defined as communication nodes. Then, any two communication nodes that can communicate directly through the fgOTN link are defined as a communication link. All communication nodes and communication links together constitute the communication topology G=(V,E), where V represents the set of nodes and E represents the set of links.

[0043] After the topology map is constructed, the fgOTN software-defined network control plane monitors the operational status of each communication node and the transmission quality of each communication link in real time. Operational parameters include, but are not limited to: the length of the data queue to be transmitted at the target gateway (i.e., the buffer queue depth), the fgOTN timeslot occupancy ratio of the communication link (i.e., fine-grained occupancy), the remaining bandwidth of the link, and the link propagation delay. The fgOTN software-defined network control plane has a global network view and can interact with the underlying fgOTN devices through southbound interfaces (e.g., OpenFlow or PCEP protocols), collecting the above operational parameters in real time and reporting them to the decision-making agent. The data collection period can be set according to business needs; for example, for protection-related services, the collection period can be set to milliseconds to ensure timely detection of transient fluctuations in physical layer resources.

[0044] This embodiment constructs a power communication network based on the fgOTN network and establishes a refined communication topology map, realizing digital modeling and real-time status awareness of physical layer hard pipeline resources. This provides a comprehensive and accurate data foundation for subsequent gateway selection, ensuring deep coupling between the decision-making process and the physical environment.

[0045] In another embodiment of the gateway selection method provided by the present invention, step 100 specifically includes: Step 110: Compare the cache queue depth of each target gateway with a first preset threshold to obtain a first comparison result; Step 120: Compare the fine-grained occupancy of the communication link corresponding to each target gateway with the second preset threshold to obtain the second comparison result; Step 130: Based on the first comparison result and the second comparison result, determine the blocking gateway among all the target gateways; Step 140: Filter out the blocked gateways to obtain the selectable gateways from all the target gateways.

[0046] This embodiment further refines the step "selecting the available gateways among all target gateways based on the cache queue depth and fine-grained occupancy of each target gateway" in claim 1.

[0047] In this embodiment, the time slot occupancy rate refers to the ratio of the number of occupied time slots in the fgOTN physical layer hard pipe to the total number of time slots, which can be directly characterized by fine-grained occupancy. A congested gateway is a gateway whose corresponding uplink fgOTN link's time slot occupancy rate exceeds a preset threshold, resulting in the inability to meet the transmission quality requirements of new services.

[0048] In one specific implementation, the process of selecting candidate gateways based on the cache queue depth and fine-grained occupancy of each target gateway is as follows: the cache queue depth of each target gateway is compared with a first preset threshold, and the fine-grained occupancy of the link corresponding to that gateway is compared with a second preset threshold; if the cache queue depth of the target gateway is greater than or equal to the first preset threshold, or the fine-grained occupancy of the target gateway is greater than or equal to the second preset threshold, then the target gateway is determined to be a congested gateway, and the congested gateway is removed from the candidate set.

[0049] This embodiment introduces a dynamic threshold based on business priority awareness to achieve accurate identification and screening of congested gateways, ensuring that core services always operate in the linear stable zone of the physical link. This effectively avoids data transmission timeouts and policy oscillations caused by physical layer congestion, and improves stability under sudden traffic surges.

[0050] See Figure 2 , Figure 2 This is a flowchart illustrating another embodiment of the gateway selection method provided by the present invention, as shown below. Figure 2 As shown, this embodiment includes steps 210 to 240, and the specific steps are as follows: Step 210: Based on the cache queue depth and fine-grained occupancy of each target gateway, and the remaining bandwidth of the power communication network, determine the current network state vector of the power communication network; Step 220: Input the current network state vector into the decision neural network to obtain the value stream output scalar of the power communication network output by the decision neural network, and the advantage stream output vector of each of the optional gateways; Step 230: Based on the value stream output scalar and the advantage stream output vector, as well as the business priority of the data to be interacted with, select the candidate gateways from all the available gateways.

[0051] The decision neural network in this embodiment is a deep reinforcement learning network built on the Dueling DQN architecture. The core improvement of Dueling DQN lies in its two-path network structure: one for estimating state value and the other for estimating action advantage. The value stream outputs a scalar V(s), used to evaluate the inherent value of the current network state s itself. This scalar is independent of specific actions and reflects the "goodness" or "badness" of the state. The advantage stream outputs a vector A(s,a), whose dimension is the same as the number of selectable gateways. Each component represents the advantage of taking the corresponding action (i.e., selecting the corresponding gateway) in state s relative to the average level of all actions in that state.

[0052] In one specific implementation, the process of selecting candidate gateways is as follows: First, the cache queue depth D_t of each target gateway, the fine-grained occupancy O_t of each uplink, and the average remaining network bandwidth B_t are concatenated in a preset dimension order to obtain the current network state vector s_t. This state vector s_t is then input into a pre-built and trained Dueling DQN decision neural network. This network first receives a 60-dimensional state vector (corresponding to 3 types of parameters for 20 gateways) through the input layer. Then, feature extraction is performed through two shared hidden layers (40 neurons per layer, using the ReLU activation function). Subsequently, the network splits into two paths: one outputs a scalar V(s_t) through a value stream fully connected layer, and the other outputs a 20-dimensional vector A(s_t, a) through a dominant stream fully connected layer. (The formula is used to...) Aggregate the data to obtain the Q-value for each potential gateway, where N is the total number of potential gateways. Next, determine the mask threshold based on the service priority of the data to be interacted with. For example, if the service priority is high, the mask threshold is 0.7; if it is low, the mask threshold is 0.9. Generate a binary mask matrix M of dimension N. For gateways with a fine-grained occupancy greater than or equal to the mask threshold, the corresponding element in M ​​is set to 0; otherwise, it is set to 1. Multiply the Q-value vector element-wise with the mask matrix M, and assign negative infinity to the Q-value of the gateways that are masked out, thereby forcibly excluding gateways that do not meet the physical layer resource constraints in the action space. Finally, from the remaining set of legal actions, a dynamic ε-greedy strategy (dynamic ε-greedy strategy) is used to select candidate gateways.

[0053] This embodiment separates state value and action advantages through the Dueling DQN network and combines it with a service priority-aware masking mechanism to achieve a refined secondary screening of optional gateways. This not only retains the intelligent decision-making capability of reinforcement learning, but also ensures the security of decision-making through physical layer constraints, avoids blind exploration, and improves convergence efficiency and decision quality.

[0054] In another embodiment of the gateway selection method provided by the present invention, step 300 specifically includes: Step 310: Based on the cache queue depth and fine-grained occupancy of the candidate gateways, and the service priority of the data to be interacted with, infer the result of any data to be interacted with through any candidate gateway to obtain the data transmission latency, gateway load data and interaction topology overhead of all interaction results. Step 320: Determine the composite reward value for each interaction result based on the data transmission latency, gateway load data, and interaction topology overhead of each interaction result; Step 330: Based on the composite reward value of each interaction result, determine the gateway selection result for each piece of data to be interacted with.

[0055] In this embodiment, the composite reward value is a comprehensive performance indicator used to evaluate the selection of a specific gateway combination for data transmission in the current state. In deep reinforcement learning, the reward value is the learning objective of the agent, and its design quality directly affects the convergence effect of the algorithm and the quality of the final strategy. Data transmission latency refers to the total time it takes for data to travel from the source (distribution network node) to the destination (transmission network node) through the selected gateway, including queuing latency, transmission latency, and propagation latency. Gateway load data is used to characterize the load balancing degree among the selected gateways. Interaction topology overhead is used to characterize the resource consumption of the routing path corresponding to the selected gateway at the network topology level.

[0056] In one specific implementation, the process of reasoning about the interaction selection and obtaining the gateway selection result is as follows: First, for each possible gateway selection scheme (i.e., each combination of selecting a preset number of k gateways from the candidate gateways), based on the current cache queue depth d_n and fine-grained occupancy o_n of each candidate gateway, and the total amount of data to be interacted, the weighted total transmission delay of the entire network under this gateway selection scheme is calculated. . Specifically, ,in, For the basic bandwidth of the link, This represents the optimal route hop count from the distribution network node to the nth selected gateway, calculated using the improved weighted Dijkstra algorithm. The propagation delay for each hop. (Through...) This method maps the physical layer resource congestion status to effective bandwidth constraints in real time. Simultaneously, it calculates the load standard deviation among selected gateways under this scheme. This is used to assess the degree of load balancing. Additionally, the average hop count of the routing path corresponding to each selected gateway is calculated. To evaluate topology overhead. Then, based on preset weighting coefficients. , , Through formula Calculate the composite reward value corresponding to this scheme. The negative sign indicates that the objective is to minimize latency, load imbalance, and topology overhead. The weighting coefficients reflect the degree of emphasis on different optimization objectives; for example, for latency-sensitive services, the weighting coefficients can be adjusted accordingly. Set it to 0.6. Set it to 0.3. Set the value to 0.1. Finally, compare the composite reward values ​​of all candidate schemes and select the gateway combination with the largest composite reward value (i.e., the smallest cost) as the final gateway selection result. If multiple schemes have the same reward value, the scheme with fewer gateways or fewer routing hops can be prioritized to save network resources.

[0057] In another implementation, if the amount of data to be interacted is extremely small (less than the cache capacity of a single gateway), the single gateway with the optimal composite reward value can be directly selected as the result to further reduce communication overhead.

[0058] This embodiment constructs a composite reward function that simultaneously includes transmission latency, load balancing, and topology overhead, thereby achieving multi-objective comprehensive optimization of gateway selection schemes. This guides the agent to not only focus on the latency of a single transmission but also to consider the balance of the load distribution across the entire network and the consumption of network resources. As a result, while improving the quality of a single service transmission, it enhances the long-term stability and robustness of the entire power communication network in the face of complex service traffic.

[0059] The gateway selection system provided by the present invention is described below. The gateway selection system described below and the gateway selection method described above can be referred to in correspondence.

[0060] Please refer to Figure 3 The present invention also provides a gateway selection system, comprising: The target gateway filtering module 301 is used to filter out the selectable gateways among all the target gateways based on the cache queue depth and fine-grained occupancy of each target gateway. The optional gateway filtering module 302 is used to filter out the candidate gateways from all the optional gateways based on the current network state vector of the power communication network and the service priority of the data to be interacted. The gateway selection result determination module 303 is used to reason about the interaction selection of the data to be interacted based on the cache queue depth and fine-grained occupancy of the candidate gateways and the service priority of the data to be interacted, and to obtain the gateway selection result.

[0061] Optionally, the gateway selection system further includes: The power communication network construction module is used to construct a power communication network, including the transmission network, target gateway, and distribution network, based on the fgOTN network. A communication topology construction module is used to construct a communication topology map that includes communication nodes and communication links; the communication nodes include the transmission network, the distribution network, and the target gateway; the communication links are composed of multiple communication nodes; The operation parameter analysis module is used to analyze the operation parameters of the communication node and the communication link to obtain the cache queue depth and fine-grained occupancy of each target gateway.

[0062] Optionally, the target gateway filtering module includes: A time slot occupancy determination unit is used to determine the time slot occupancy rate of each communication link in the power communication network based on the buffer queue depth and fine-grained occupancy rate of each target gateway. A congested gateway determination unit is used to determine the congested gateway among all the target gateways based on the time slot occupancy rate of each of the communication links. A blocking gateway screening unit is used to screen out the blocking gateways to obtain the selectable gateways from all the target gateways.

[0063] Optionally, the optional gateway filtering module includes: The current network state vector determination unit is used to determine the current network state vector of the power communication network based on the cache queue depth and fine-grained occupancy of each target gateway and the remaining bandwidth of the power communication network. The decision neural network execution unit is used to input the current network state vector into the decision neural network to obtain the value stream output scalar of the power communication network output by the decision neural network, and the advantage stream output vector of each of the optional gateways; The optional gateway filtering unit is used to filter out candidate gateways from all the optional gateways based on the value stream output scalar, the advantage stream output vector, and the business priority of the data to be interacted with.

[0064] Optionally, the gateway selection result determination module includes: The interactive reasoning unit is used to reason about the result of any interactive data being interacted with through any of the candidate gateways based on the cache queue depth and fine-grained occupancy of the candidate gateways and the service priority of the interactive data, so as to obtain the data transmission latency, gateway load data and interactive topology overhead of all interactive results. The composite reward value determination unit is used to determine the composite reward value of each interaction result based on the data transmission delay, gateway load data, and interaction topology overhead of each interaction result. The gateway selection result determination unit is used to determine the gateway selection result for each piece of data to be interacted based on the composite reward value of each interaction result.

[0065] The embodiments described above are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.

Claims

1. A gateway selection method, characterized in that, It is applied to power communication networks built on fgOTN networks, wherein the power communication network includes power communication networks for transmission networks, target gateways, and distribution networks; The transmission network and the distribution network interact with each other through the target gateway; The gateway selection method includes: Based on the cache queue depth and fine-grained occupancy of each target gateway, selectable gateways are obtained from all the target gateways. Based on the current network state vector of the power communication network and the service priority of the data to be interacted, the candidate gateways among all the available gateways are selected. Based on the cache queue depth and fine-grained occupancy of the candidate gateways, as well as the service priority of the data to be interacted with, the interaction selection of the data to be interacted with is inferred to obtain the gateway selection result.

2. The gateway selection method as described in claim 1, characterized in that, Before filtering out the selectable gateways from all the target gateways based on the cache queue depth and fine-grained occupancy of each target gateway, the following steps are included: Based on the fgOTN network, a power communication network including the transmission network, target gateway, and distribution network is constructed; A communication topology diagram containing communication nodes and communication links is constructed; the communication nodes include the transmission network, the distribution network, and the target gateway; the communication links are composed of multiple communication nodes. The operating parameters of the communication nodes and the communication links are analyzed to obtain the cache queue depth and fine-grained occupancy of each target gateway.

3. The gateway selection method as described in claim 2, characterized in that, The process of filtering out the selectable gateways from all the target gateways based on the cache queue depth and fine-grained occupancy of each target gateway includes: The cache queue depth of each target gateway is compared with a first preset threshold to obtain a first comparison result; The fine-grained occupancy of the communication link corresponding to each target gateway is compared with a second preset threshold to obtain a second comparison result; Based on the first comparison result and the second comparison result, the blocked gateway among all the target gateways is determined; By filtering out the blocked gateways, the selectable gateways from all the target gateways are obtained.

4. The gateway selection method as described in claim 1, characterized in that, The candidate gateways selected from all the available gateways, based on the current network state vector of the power communication network and the service priority of the data to be interacted with, include: Based on the cache queue depth and fine-grained occupancy of each target gateway, and the remaining bandwidth of the power communication network, the current network state vector of the power communication network is determined. The current network state vector is input into the decision neural network to obtain the value stream output scalar of the power communication network output by the decision neural network, and the advantage stream output vector of each of the optional gateways; Based on the value stream output scalar and the advantage stream output vector, as well as the business priority of the data to be interacted with, the candidate gateways are selected from all the available gateways.

5. The gateway selection method as described in claim 1, characterized in that, The reasoning process for selecting the gateway based on the cache queue depth and fine-grained occupancy of the candidate gateways, as well as the service priority of the data to be interacted with, yields the following gateway selection results: Based on the cache queue depth and fine-grained occupancy of the candidate gateways, as well as the service priority of the data to be interacted with, the result of any of the data to be interacted with through any of the candidate gateways is inferred to obtain the data transmission latency, gateway load data and interaction topology overhead of all interaction results. The composite reward value for each interaction result is determined based on the data transmission latency, gateway load data, and interaction topology overhead of each interaction result. Based on the composite reward value of each interaction result, the gateway selection result for each piece of data to be interacted with is determined.

6. A gateway selection system, characterized in that, include: The target gateway filtering module is used to filter out the selectable gateways among all the target gateways based on the cache queue depth and fine-grained occupancy of each target gateway. The optional gateway filtering module is used to filter out the candidate gateways from all the optional gateways based on the current network state vector of the power communication network and the service priority of the data to be interacted with. The gateway selection result determination module is used to reason about the interaction selection of the data to be interacted based on the cache queue depth and fine-grained occupancy of the candidate gateways and the service priority of the data to be interacted, and to obtain the gateway selection result.

7. The gateway selection system as described in claim 6, characterized in that, The gateway selection system also includes: The power communication network construction module is used to construct a power communication network, including the transmission network, target gateway, and distribution network, based on the fgOTN network. A communication topology construction module is used to construct a communication topology map that includes communication nodes and communication links; the communication nodes include the transmission network, the distribution network, and the target gateway; the communication links are composed of multiple communication nodes; The operation parameter analysis module is used to analyze the operation parameters of the communication node and the communication link to obtain the cache queue depth and fine-grained occupancy of each target gateway.

8. The gateway selection system as described in claim 7, characterized in that, The target gateway filtering module includes: The first comparison result determination unit is used to compare the cache queue depth of each target gateway with a first preset threshold to obtain a first comparison result; The second comparison result determination unit is used to compare the fine-grained occupancy of the communication link corresponding to each target gateway with a second preset threshold to obtain a second comparison result; A blocked gateway determination unit is configured to determine the blocked gateway among all the target gateways based on the first comparison result and the second comparison result; A blocking gateway screening unit is used to screen out the blocking gateways to obtain the selectable gateways from all the target gateways.

9. The gateway selection system as described in claim 6, characterized in that, The optional gateway filtering module includes: The current network state vector determination unit is used to determine the current network state vector of the power communication network based on the cache queue depth and fine-grained occupancy of each target gateway and the remaining bandwidth of the power communication network. The decision neural network execution unit is used to input the current network state vector into the decision neural network to obtain the value stream output scalar of the power communication network output by the decision neural network, and the advantage stream output vector of each of the optional gateways; The optional gateway filtering unit is used to filter out candidate gateways from all the optional gateways based on the value stream output scalar, the advantage stream output vector, and the business priority of the data to be interacted with.

10. The gateway selection system as described in claim 6, characterized in that, The gateway selection result determination module includes: The interactive inference unit is used to infer the result of any interactive data being interacted with through any of the candidate gateways based on the cache queue depth and fine-grained occupancy of the candidate gateways and the service priority of the interactive data, so as to obtain the data transmission latency, gateway load data and interactive topology overhead of all interaction results. The composite reward value determination unit is used to determine the composite reward value of each interaction result based on the data transmission delay, gateway load data, and interaction topology overhead of each interaction result. The gateway selection result determination unit is used to determine the gateway selection result for each piece of data to be interacted based on the composite reward value of each interaction result.

Images