Power communication resource allocation method and system based on reinforcement learning

Abstract

The invention discloses an electric power communication resource allocation method and system based on reinforcement learning. The method comprises the following steps: acquiring node data corresponding to each sending node in a power line communication network; determining the transmission priority of each activated node according to the transmission rate data; constructing a Markov decision process model corresponding to the power line communication networking according to the node data and a preset action space; acquiring a transmission rate of the activated node on each subcarrier, and constructing an optimization objective function with task data total throughput maximization as an objective according to the transmission rate and transmission rate data; and solving a Markov decision process model according to the transmission priority, the optimization objective function and a preset dual-depth reinforcement learning algorithm to obtain subcarriers and transmitting powers corresponding to the activation nodes, so as to improve the data throughput of power line communication networking.

Description

Technical Field

[0001] This invention relates to the field of power communication technology, specifically to a power communication resource allocation method and system based on reinforcement learning. Background Technology

[0002] Power line communication (PLC) is a communication method that uses existing power line networks as the information transmission medium. It achieves data transmission by modulating high-frequency signals onto power lines. Due to its advantages of requiring no additional wiring, wide coverage, and plug-and-play functionality, this technology has shown great application potential in fields such as smart grids, smart homes, and industrial automation. In recent years, with advancements in digital signal processing and modulation / demodulation technologies, modern PLC technology has been able to provide high-speed data transmission capabilities.

[0003] However, power lines, as a communication medium, exhibit extremely complex channel characteristics. First, power lines are not designed for high-frequency signal transmission; their impedance characteristics vary significantly and are unstable with frequency, leading to severe signal reflection. Furthermore, the load within a power line network is constantly changing, resulting in a fluctuating noise environment and highly unstable network performance, with speeds fluctuating between high and low. Moreover, nodes with communication capabilities within the network often have different communication rate requirements. For example, under the HomePlug AV2 protocol, time-sensitive data and non-time-sensitive data are transmitted using CSMA / CA (Carrier Sense Multiple Access / Collision Avoidance) and TDMA (Time Division Multiple Access) protocols, respectively. Current technologies allocate resources to different nodes using fixed resource allocation strategies, but these cannot adapt to changing channel conditions and varying node rate requirements, thus reducing the network data throughput of power lines. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention discloses a power communication resource allocation method and system based on reinforcement learning, which is used to improve the data throughput of power lines.

[0005] To achieve the above objectives, this invention discloses a power communication resource allocation method based on reinforcement learning, comprising:

[0006] Acquire node data corresponding to each transmitting node in the power line communication network; wherein, the node data includes task data, transmission rate data, and network topology position relative to the receiving node; the transmitting nodes are divided into inactive nodes and active nodes;

[0007] The transmission priority of each activated node is determined based on the transmission rate data;

[0008] Based on the node data and the preset action space, a Markov decision process model corresponding to the power line communication network is constructed; wherein, the action space includes subcarrier allocation and power adjustment;

[0009] The transmission rate of the active node on each subcarrier is obtained, and an optimization objective function is constructed based on the transmission rate and the transmission rate data to maximize the total throughput of task data.

[0010] The Markov decision process model is solved based on the transmission priority, the optimization objective function, and the preset dual deep reinforcement learning algorithm to obtain the subcarriers and transmit power corresponding to each of the activated nodes.

[0011] This invention discloses a power line communication resource allocation method based on reinforcement learning. By dynamically sensing the network status of a power line communication network and employing reinforcement learning to optimize resource allocation, it aims to improve data throughput within the network. First, by acquiring task data, transmission rates, and network topology locations from node data, the method monitors the service needs and transmission rate requirements of transmitting nodes in real time, providing a data foundation for dynamic resource allocation. Nodes are categorized into active and inactive states, allowing resource allocation to focus on active nodes with actual operational needs, avoiding resource waste. Priorities are determined based on transmission rate data, enabling resource allocation to nodes with different real-time rate requirements. A Markov decision process model incorporating subcarrier allocation and power adjustment is constructed, transforming the resource allocation problem into a dynamic decision-making process that autonomously adjusts strategies according to environmental changes. An objective function maximizing total throughput is constructed based on subcarrier transmission rates, directly linking resource allocation effectiveness to network performance indicators. A dual-deep reinforcement learning algorithm is used to solve the model, combined with transmission priority constraints, to achieve a globally optimal resource allocation scheme while ensuring the resource allocation needs of different nodes. This overcomes the limitations of traditional fixed allocation strategies and improves data throughput within the power line communication network.

[0012] As a preferred example, before acquiring the node data corresponding to each transmitting node in the power line communication network, the process includes:

[0013] The transmission bandwidth of the power line communication network is obtained, so as to determine the multiple subcarriers used by the power line communication network according to the orthogonal frequency division multiplexing technology adopted by the power line communication network;

[0014] For any one of the sending nodes:

[0015] Obtain the status data of the transmitting node within a preset beacon period;

[0016] When the status data determines that the sending node is in a working state, the sending node is classified as an active node.

[0017] When it is determined from the status data that the sending node is not in a working state, the sending node is classified as an inactive node.

[0018] The above scheme first obtains the transmission bandwidth of the power line communication network and the orthogonal frequency division multiplexing (OFDM) technology used to determine the set of subcarriers within the current network, establishing a physical layer foundation for subsequent resource allocation. Considering the dynamic changes in the power line communication network topology, the scheme uses node status data collected during beacon cycles to determine the working status in real time, classifying nodes into active and inactive categories. This dynamic node classification enables real-time adaptation of resource allocation, ensuring the matching of subcarrier allocation with the physical characteristics of the power line. Furthermore, the periodic status detection mechanism achieves dynamic awareness of the network topology, providing accurate node status input for subsequent differentiated resource allocation and improving the data throughput within the power line communication network.

[0019] As a preferred example, the step of obtaining the node data corresponding to each transmitting node in the power line communication network includes:

[0020] In the preparation frame of the beacon period, the minimum transmission rate requirement and the amount of remaining task data for each transmitting node are obtained.

[0021] Based on the preparation frame, the network topology distance from each sending node to the receiving node is obtained.

[0022] The above scheme collects data sequentially during the beacon cycle to accurately capture the dynamic communication needs of nodes in a power line communication network. First, the minimum transmission rate requirement of each node is synchronously acquired during the preparation frame phase of the beacon cycle, providing an allocation benchmark for subsequent resource allocation and avoiding the problem that traditional fixed allocation strategies cannot meet differentiated rate requirements. Second, the dynamic parameter of remaining task data volume is collected, reflecting the service load status of nodes in real time and providing a basis for optimizing data throughput in the resource allocation algorithm. Finally, the network topology distance parameter is acquired based on the preparation frame, accurately quantifying signal transmission path loss characteristics and providing spatial channel state information for subcarrier allocation and power adjustment. This coordinated acquisition mechanism of these three parameters solves the resource allocation lag problem caused by the lack of real-time dynamic parameters in traditional methods, constructing a data foundation that supports dynamic optimization using reinforcement learning algorithms to improve the data throughput of power line communication networks.

[0023] As a preferred example, determining the transmission priority of each active node based on the transmission rate data includes:

[0024] For any one of the activated nodes:

[0025] The minimum transmission rate requirement of the activated node is used to match the subcarrier interference level and transmit power level corresponding to the activated node, so as to use the subcarrier interference level and transmit power level as the transmission priority of the activated node.

[0026] The above scheme establishes a mapping relationship between transmission rate requirements and resource allocation parameters, realizing a dynamic priority adjustment mechanism to avoid ineffective resource allocation. Specifically, for each active node's minimum transmission rate requirement, the corresponding subcarrier interference level and transmit power level are matched, using these two key parameters as the basis for priority determination. This matching mechanism can dynamically adjust the resource allocation strategy according to the real-time transmission needs of nodes: for nodes requiring higher transmission rates, less interfered subcarriers and higher power levels are prioritized; for nodes with low rate requirements, resources with lower anti-interference requirements are allocated. By directly linking transmission requirements to physical layer parameters, the scheme solves the problem that traditional fixed allocation strategies cannot adapt to the differentiated needs of nodes, providing a precise priority determination basis for subsequent reinforcement learning resource optimization.

[0027] As a preferred example, the step of constructing a Markov decision process model corresponding to the power line communication network based on the node data and a preset action space includes:

[0028] Based on the minimum transmission rate requirement, the remaining task data volume, and the network topology distance, a state space corresponding to multiple sending nodes is constructed; wherein, in the state space, the minimum transmission rate requirement, the remaining task data volume, and the network topology distance of inactive nodes are all set to 0;

[0029] An action space corresponding to multiple transmitting nodes is constructed based on a pre-constructed subcarrier allocation matrix and a pre-constructed power adjustment matrix; wherein, the power adjustment matrix includes the transmit power corresponding to each transmitting node; the transmit power has multiple power levels; the subcarrier allocation matrix includes the subcarrier matched to each transmitting node;

[0030] Based on the state space and the action space, a Markov decision process model corresponding to the power line communication network is constructed.

[0031] The above scheme establishes a dynamic decision-making model by constructing a multi-dimensional state space and a composite action space, effectively solving the resource allocation problem under dynamically changing power communication environments. First, the state space is constructed by integrating three key parameters: minimum node transmission rate requirement, remaining task data volume, and network topology distance. The parameters of inactive nodes are zeroed out, accurately representing the real-time network state while avoiding interference from invalid data. Second, the action space is constructed using a subcarrier allocation matrix and a power adjustment matrix. The power adjustment matrix sets multi-level transmit power, providing a refined power control dimension for the algorithm. The subcarrier allocation matrix establishes a dynamic mapping relationship between nodes and subcarriers. By combining the state space and action space to construct a Markov decision process model, the reinforcement learning algorithm can find the optimal solution in the multi-dimensional action space, including subcarrier allocation and power adjustment, based on the real-time network state, achieving dynamic optimization of resource allocation.

[0032] As a preferred example, the step of obtaining the transmission rate of the active node on each subcarrier, and constructing an optimization objective function based on the transmission rate and the transmission rate data to maximize the total throughput of task data, includes:

[0033] Obtain the spectral distance between any two subcarriers and the preset subcarrier bandwidth, and obtain the interference coefficient between any two subcarriers through a preset interference coefficient calculation function;

[0034] For any one of the subcarriers:

[0035] Calculate the out-of-band radiation corresponding to the subcarrier based on the interference coefficient and the carrier power of the subcarrier;

[0036] The multiple out-of-band radiations corresponding to the subcarrier are aggregated to obtain the aggregated out-of-band radiation corresponding to the subcarrier.

[0037] For any one of the sending nodes:

[0038] Based on the out-of-band radiation of the aggregation, the noise power of the receiving node, and the response of the power signal to the subcarrier matched to the transmitting node, the transmission rate of the transmitting node on the subcarrier is obtained through a preset transmission rate calculation function.

[0039] Based on the minimum transmission rate requirement and the transmission rate corresponding to each of the sending nodes, an optimization objective function is constructed with the goal of maximizing the total throughput of task data and the constraint of satisfying the minimum transmission rate requirement.

[0040] The above scheme constructs an accurate transmission rate model by quantifying the interference impact between subcarriers and the dynamic noise environment, ultimately achieving optimal resource allocation. Specifically, by calculating the spectral distance and interference coefficient between subcarriers, the degree of mutual interference between different subcarriers can be accurately characterized; calculating out-of-band radiation based on subcarrier power and interference coefficients can reflect the interference intensity of a specific subcarrier to other frequency bands; and by aggregating out-of-band radiation, the overall interference level of subcarriers in complex channel environments can be comprehensively evaluated. In the transmission rate calculation, a transmission rate model that conforms to actual channel conditions is established by combining aggregated interference, receiver noise, and power signal characteristics. Finally, by constructing an optimization objective function that includes a minimum rate constraint, the basic communication needs of each node are guaranteed while maximizing the total system throughput. This scheme effectively solves the core problem that traditional fixed allocation strategies cannot adapt to the time-varying channel characteristics of power lines through multi-dimensional interference modeling and dynamic parameter calculation, thereby improving the data throughput of power line communication networks.

[0041] As a preferred example, the step of solving the Markov decision process model based on the transmission priority, the optimization objective function, and a preset dual deep reinforcement learning algorithm to obtain the subcarriers and transmit power corresponding to each of the activated nodes includes:

[0042] The reward function in the dual deep reinforcement learning algorithm is determined based on the optimization objective function;

[0043] The transmission priority and the state space in the Markov decision process model are input into a preset dual deep reinforcement learning algorithm, so as to search the action space in the Markov decision process model through the online network in the dual deep reinforcement learning algorithm and generate an action vector set.

[0044] The optimal action is selected from the action vector set by a greedy algorithm preset in the online network; wherein the optimal action includes the subcarrier and transmit power corresponding to each active node;

[0045] According to the reward function, the expected reward value of the optimal action is obtained through the target network preset in the dual deep reinforcement learning algorithm;

[0046] Based on the expected return value, the subcarrier and transmit power corresponding to each activated node in the power line communication network are output through the dual deep reinforcement learning algorithm.

[0047] The aforementioned scheme achieves adaptive allocation of power line communication resources by constructing a dynamic decision-making mechanism based on dual deep reinforcement learning. Specifically, by inputting transmission priority and state space into the dual deep reinforcement learning algorithm, a candidate action set is generated using the action space search capability of the online network. A greedy algorithm is then used to select the optimal action that balances subcarrier allocation and power adjustment. The target network evaluates the expected reward value of the optimal action through a reward function, ensuring that the resource allocation scheme meets the transmission rate requirements of each node while maximizing the total system throughput. Transmission priority, as an input parameter, guides the algorithm to prioritize the resource needs of high-priority nodes. The dual-network architecture, by separating the action generation and value evaluation processes, effectively avoids the overestimation problem that traditional reinforcement learning algorithms are prone to in complex action spaces, improving the stability and convergence efficiency of the resource allocation strategy, thereby increasing the accuracy of resource allocation and ultimately improving the data throughput of the power line communication network.

[0048] As a preferred example, the step of solving the Markov decision process model based on the transmission priority, the optimization objective function, and a preset dual deep reinforcement learning algorithm to obtain the subcarriers and transmit power corresponding to each of the activated nodes further includes:

[0049] The predicted reward value of the optimal action is obtained through the online network, and the mean square error value between the predicted reward value and the expected reward value is obtained.

[0050] Based on the mean square error value, the network parameters of the online network are updated using a preset gradient descent method.

[0051] The above scheme achieves dynamic optimization of the reinforcement learning model by introducing an error feedback mechanism between the predicted and expected rewards. The predicted reward for the optimal action is calculated through an online network and compared with the expected reward generated by the target network. The mean squared error between the two is used as the basis for adjusting model parameters, and the weight parameters of the online network are updated in real time using gradient descent. This mechanism allows the model to continuously correct policy biases, avoiding the policy rigidity problem caused by fixed network parameters in traditional reinforcement learning algorithms. By dynamically updating network parameters, the model can adapt to dynamic environments such as sudden changes in channel state and fluctuations in node rate demand in power line communication networks, ensuring that the resource allocation strategy always maintains optimal matching with the real-time network state. This improves the decision accuracy of subcarrier and power allocation, thereby increasing the data throughput of the power line communication network.

[0052] On the other hand, the present invention discloses a power communication resource allocation system based on reinforcement learning, including a network data module, a priority partitioning module, a model decision module, a function update module, and a resource allocation module;

[0053] The network data module is used to acquire node data corresponding to each transmitting node in the power line communication network; wherein, the node data includes task data, transmission rate data, and network topology position relative to the receiving node; the transmitting nodes are divided into inactive nodes and active nodes;

[0054] The priority division module is used to determine the transmission priority of each activated node based on the transmission rate data;

[0055] The model decision module is used to construct a Markov decision process model corresponding to the power line communication network based on the node data and a preset action space; wherein, the action space includes subcarrier allocation and power adjustment;

[0056] The function update module is used to obtain the transmission rate of the active node on each subcarrier, so as to construct an optimization objective function based on the transmission rate and the transmission rate data, with the goal of maximizing the total throughput of task data.

[0057] The resource allocation module is used to solve the Markov decision process model according to the transmission priority, the optimization objective function and the preset dual deep reinforcement learning algorithm, so as to obtain the subcarrier and transmit power corresponding to each of the activated nodes.

[0058] This invention discloses a power line communication resource allocation system based on reinforcement learning. It dynamically senses the network status of the power line communication network and uses reinforcement learning to optimize resource allocation, thereby improving the data throughput within the network. First, by acquiring task data, transmission rates, and network topology locations from node data, the system monitors the service needs and transmission rate requirements of transmitting nodes in real time, providing a data foundation for dynamic resource allocation. Nodes are classified into active and inactive states, allowing resource allocation to be focused on active nodes that are actually in operation, avoiding resource waste. Priorities are determined based on transmission rate data, enabling resource allocation to nodes with different real-time rate requirements. A Markov decision process model incorporating subcarrier allocation and power adjustment is constructed, transforming the resource allocation problem into a dynamic decision-making process to autonomously adjust strategies according to environmental changes. An objective function maximizing total throughput is constructed based on subcarrier transmission rates, directly linking resource allocation effectiveness to network performance indicators. A dual-deep reinforcement learning algorithm is used to solve the model, combined with transmission priority constraints, to achieve a globally optimal resource allocation scheme while ensuring the resource allocation needs of different nodes. This overcomes the limitations of traditional fixed allocation strategies and improves the data throughput within the power line communication network.

[0059] As a preferred example, the power communication resource allocation system based on reinforcement learning further includes a communication data module and a node partitioning module;

[0060] The communication data module is used to obtain the transmission bandwidth of the power line communication network, so as to determine the multiple subcarriers used by the power line communication network according to the orthogonal frequency division multiplexing technology adopted by the power line communication network;

[0061] The node partitioning module is used to: acquire the status data of the sending node within a preset beacon period for any one of the sending nodes; when the sending node is determined to be in a working state based on the status data, partition the sending node as an active node; when the sending node is determined to be in a non-working state based on the status data, partition the sending node as an inactive node.

[0062] The above scheme first obtains the transmission bandwidth of the power line communication network and the orthogonal frequency division multiplexing (OFDM) technology used to determine the set of subcarriers within the current network, establishing a physical layer foundation for subsequent resource allocation. Considering the dynamic changes in the power line communication network topology, the scheme uses node status data collected during beacon cycles to determine the working status in real time, classifying nodes into active and inactive categories. This dynamic node classification enables real-time adaptation of resource allocation, ensuring the matching of subcarrier allocation with the physical characteristics of the power line. Furthermore, the periodic status detection mechanism achieves dynamic awareness of the network topology, providing accurate node status input for subsequent differentiated resource allocation and improving the data throughput within the power line communication network. Attached Figure Description

[0063] To more clearly illustrate the technical solution of this application, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0064] Figure 1 This is a flowchart illustrating a power communication resource allocation method based on reinforcement learning disclosed in an embodiment of the present invention.

[0065] Figure 2 This is a schematic diagram of the structure of a power communication resource allocation system based on reinforcement learning disclosed in an embodiment of the present invention;

[0066] Figure 3 This is a schematic diagram of a power line communication network structure disclosed in an embodiment of the present invention;

[0067] Figure 4 This is a schematic diagram of a beacon cycle disclosed in an embodiment of the present invention;

[0068] Figure 5 This is a schematic diagram of the training process of a dual deep reinforcement learning algorithm disclosed in an embodiment of the present invention;

[0069] Figure 6 This is a schematic diagram of the network framework of a dual deep reinforcement learning algorithm disclosed in an embodiment of the present invention. Detailed Implementation

[0070] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0071] Reference Figure 1 To improve the data throughput of power line communication networks, one embodiment of the present invention provides a power communication resource allocation method based on reinforcement learning, the method mainly comprising:

[0072] Step 101: Obtain the node data corresponding to each transmitting node in the power line communication network; wherein, the node data includes task data, transmission rate data and network topology position relative to the receiving node; the transmitting nodes are divided into inactive nodes and active nodes.

[0073] In this embodiment, before acquiring the node data, the transmission bandwidth of the power line communication network is first acquired to determine the multiple subcarriers used by the power line communication network based on the orthogonal frequency division multiplexing technology adopted by the power line communication network. For any one of the transmitting nodes: the status data of the transmitting node within a preset beacon period is acquired; when the transmitting node is determined to be in a working state based on the status data, the transmitting node is classified as an active node; when the transmitting node is determined to be in a non-working state based on the status data, the transmitting node is classified as an inactive node.

[0074] Secondly, in this embodiment, this step involves obtaining the minimum transmission rate requirement and remaining task data volume corresponding to each of the sending nodes during the preparation frame of the beacon period; and obtaining the network topology distance from each of the sending nodes to the receiving nodes based on the preparation frame.

[0075] In some embodiments of this example, a central coordinator is typically used within the power line communication network to allocate subcarrier resources to multiple transmitting nodes within the network and to control the transmission power of each transmitting node by adjusting its transmission level. Specifically, the network structure of the power line communication network can be as follows: Figure 3 As shown. From Figure 3It is understood that the power line communication network consists of multiple nodes connected via AC power lines. One node is designated as the central coordinator or deployed intelligent agent (CCO), responsible for managing network operation. Each sending node (node ​​Tx) can directly transmit data to node Rx without relaying through the CCO. Figure 3 It can be seen that the power line communication network includes multiple transmitting nodes, namely R. x The nodes can be divided into multiple active nodes (such as node A, node B, and node C) and multiple inactive nodes (such as node D, node E, and node F) based on the working status of each of the sending nodes.

[0076] from Figure 3 It is understood that the central coordinator or intelligent agent (CCO) deployed in the power line communication network is responsible for allocating subcarriers and adjusting power for each active node. Preferably, when... Figure 3 The power line communication network shown employs a single-cell orthogonal frequency division multiplexing (OFDM) architecture. When the central coordinator manages multiple orthogonal subcarrier resource blocks and controls the transmission levels of each active node, it acquires the transmission bandwidth of the power line communication network. Based on the OFDM technology used in the network, it determines the multiple subcarriers employed. Simultaneously, it acquires the status data of each transmitting node within a preset beacon period to identify whether the transmitting node is an active node. It is important to note that the power line communication network divides time into beacon periods, with each period covering two AC line periods.

[0077] In some embodiments of this example, a model corresponding to the power line communication network can also be constructed based on the acquired status data of the power line communication network and the node data of each transmitting node. For example, the status data of the power line communication network might be that the network contains K nodes, with M nodes less than K active within a beacon period, and the transmission bandwidth of the network is B, using N subcarriers with Orthogonal Frequency Division Multiplexing (OFDM) technology. The node data might be the remaining task data volume, minimum transmission rate requirement, and network topology distance relative to the receiving node Rx for each transmitting node at the current time. A model corresponding to the power line communication network can then be constructed based on the node data and the status data.

[0078] In this embodiment, this step obtains the transmission bandwidth of the power line communication network and determines the subcarrier set within the current network using orthogonal frequency division multiplexing (OFDM) technology, establishing a physical layer foundation for subsequent resource allocation. Considering the dynamic nature of the power line communication network topology, the working status is determined in real time by collecting node status data during the beacon period, classifying nodes into active and inactive categories. This dynamic node classification enables real-time adaptation of resource allocation, ensuring the matching of subcarrier allocation with the physical characteristics of the power line. Furthermore, the periodic status detection mechanism achieves dynamic awareness of the network topology, providing accurate node status input for subsequent differentiated resource allocation and improving the data throughput within the power line communication network.

[0079] Step 102: Determine the transmission priority of each activated node based on the transmission rate data.

[0080] In this embodiment, the step mainly includes: for any one of the active nodes: matching the subcarrier interference level and transmit power level corresponding to the active node according to the minimum transmission rate requirement of the active node, so as to use the subcarrier interference level and the transmit power level as the transmission priority of the active node.

[0081] In some embodiments of this example, reference is made to Figure 4 As shown in the beacon period diagram, each beacon period includes a preparation frame, a control frame, and the actual transmission. Specifically, the next beacon period t+1 will only begin after the task of an active node within beacon period t has been completed.

[0082] Preferably, in order to alleviate such Figure 3 The problems of subchannel collisions and inter-subcarrier interference when multiple active nodes transmit simultaneously in a power line communication network can be addressed by implementing a channel resource allocation mechanism based on node priority within the control frame area. Specifically, in the preparation frame of the beacon period, each transmitting node reports its minimum transmission rate requirement and remaining workload to the node designated as the control center. The control center (CCO) then performs subcarrier resource scheduling and allocates transmit power to each active node within the control frame. Next, in the CSMA / CA area, through the control center's scheduling, active nodes with high transmission rate requirements are allocated subcarriers less susceptible to interference and higher transmit power. The formal transmission phase then begins until the task transmission of one active node in the active node group is completed. The next beacon period then begins. This process continues until all active nodes have completed their task transmission.

[0083] In this embodiment, this step establishes a mapping relationship between transmission rate requirements and resource allocation parameters, realizing a dynamic priority adjustment mechanism to avoid ineffective resource allocation. Specifically, for each active node's minimum transmission rate requirement, the corresponding subcarrier interference level and transmit power level are matched, using these two key parameters as the basis for priority determination. This matching mechanism can dynamically adjust the resource allocation strategy according to the real-time transmission needs of the nodes: for nodes requiring higher transmission rates, subcarriers with less interference and higher power levels are prioritized; for nodes with low rate requirements, resources with lower anti-interference requirements are allocated. By directly linking transmission requirements to physical layer parameters, the problem that traditional fixed allocation strategies cannot adapt to the differentiated needs of nodes is solved, providing a precise priority determination basis for subsequent reinforcement learning resource optimization.

[0084] Step 103: Construct a Markov decision process model corresponding to the power line communication network based on the node data and the preset action space; wherein, the action space includes subcarrier allocation and power adjustment.

[0085] In this embodiment, the step mainly includes: constructing a state space corresponding to multiple transmitting nodes based on the minimum transmission rate requirement, the remaining task data volume, and the network topology distance; wherein, the minimum transmission rate requirement, the remaining task data volume, and the network topology distance of the inactive nodes in the state space are all set to 0; constructing an action space corresponding to multiple transmitting nodes based on a pre-constructed subcarrier allocation matrix and a pre-constructed power adjustment matrix; wherein, the power adjustment matrix includes the transmit power corresponding to each transmitting node; the transmit power has multiple power levels; the subcarrier allocation matrix includes the subcarriers matched to each transmitting node; and constructing a Markov decision process model corresponding to the power line communication network based on the state space and the action space.

[0086] In some embodiments of this example, a Markov decision process model is constructed. The state space of the Markov decision process model includes the remaining task data volume of each transmitting node, the minimum transmission rate requirement, and the network topology distance. The action space of the Markov decision process model includes the allocation of subcarrier resources and the adjustment of power levels.

[0087] Specifically, the state space of the Markov decision process model is defined as follows: This represents the remaining task quantity, minimum transmission rate requirement, and network topology location for all node groups, where the task quantity, transmission rate requirement, and network topology distance for inactive nodes are all 0. The action space is defined as A(t) = {Q(t), C(t)}; where Q(t) = {V1, V2, ..., Vk} represents the transmit power of each transmitting node, and each element has three levels from low to high: P_low, P_mid, and P_high. C(t) is the subcarrier resource block allocation matrix with dimensions M×N, where an element value of 0 indicates unallocated and 1 indicates allocated. Element x i,n =1 indicates that the transmitting node i transmits on subcarrier n.

[0088] In this embodiment, the above steps establish a dynamic decision-making model by constructing a multi-dimensional state space and a composite action space, effectively solving the resource allocation problem under dynamic changes in the power communication environment. First, the state space is constructed by integrating three key parameters: minimum node transmission rate requirement, remaining task data volume, and network topology distance. The parameters of inactive nodes are zeroed out, accurately representing the real-time network state while avoiding interference from invalid data. Second, the action space is constructed using a subcarrier allocation matrix and a power adjustment matrix. The power adjustment matrix sets multiple levels of transmit power, providing a refined power control dimension for the algorithm. The subcarrier allocation matrix establishes a dynamic mapping relationship between nodes and subcarriers. By combining the state space and action space to construct a Markov decision process model, the reinforcement learning algorithm can find the optimal solution in the multi-dimensional action space, including subcarrier allocation and power adjustment, based on the real-time network state, achieving dynamic optimization of resource allocation.

[0089] Step 104: Obtain the transmission rate of the activated node on each subcarrier, and construct an optimization objective function based on the transmission rate and the transmission rate data, with the goal of maximizing the total throughput of task data.

[0090] In this embodiment, the steps mainly include: obtaining the spectral distance between any two subcarriers and a preset subcarrier bandwidth, and obtaining the interference coefficient between any two subcarriers through a preset interference coefficient calculation function; for any one subcarrier: calculating the out-of-band radiation corresponding to the subcarrier based on the interference coefficient and the carrier power of the subcarrier; aggregating the multiple out-of-band radiations corresponding to the subcarrier to obtain the aggregated out-of-band radiation corresponding to the subcarrier; for any one transmitting node: obtaining the transmission rate of the transmitting node on the subcarrier through a preset transmission rate calculation function based on the aggregated out-of-band radiation, the noise power of the receiving node, and the response of the power signal to the subcarrier matched to the transmitting node; and constructing an optimization objective function with the goal of maximizing the total throughput of task data and the constraint of satisfying the minimum transmission rate requirement, based on the minimum transmission rate requirement corresponding to each transmitting node and the transmission rate.

[0091] In some embodiments of this example, reference is made to Figure 3 The power line communication network shown in the figure represents the out-of-band radiation between different subcarriers as follows:

[0092]

[0093] in The signal power of subcarrier n1; the η n1,n2 This represents the interference coefficient between subcarriers n1 and n2.

[0094] The calculation expression for the interference coefficient is as follows:

[0095]

[0096] in The T represents the spectral distance between subcarriers n1 and n2; s The symbol interval is indicated; Δf represents the preset subcarrier bandwidth in the power line communication network.

[0097] Based on the out-of-band radiation, any subcarrier n i The extra-convergent radiation is represented as: The m-th transmitting node transmits via subcarrier n i The transmission rate can be expressed as:

[0098]

[0099] Where, N r This indicates the noise power at the receiving node; This represents the subcarrier n paired with the m-th transmitting node in the power line channel f. i The response.

[0100] Regarding the Modeling is performed to characterize the impedance characteristics of the power line and the signal attenuation and phase shift. Then, considering the minimum transmission rate requirement of the transmitting node, an optimization objective function is constructed with the goal of maximizing the overall throughput of the power line communication network; wherein the optimization objective function is:

[0101]

[0102] Where x i,n Let x represent a binary decision variable. i,n =1 indicates that the transmitting node i is transmitting on subcarrier n; otherwise, x i,n =0. This represents the minimum transmission rate required by sending node i; the "finish" represents the number of nodes in the activated node group that complete the task transmission. The C... i,n This represents the transmission rate when transmitting node i transmits on subcarrier n.

[0103] In this embodiment, this step constructs an accurate transmission rate model by quantifying the interference impact between subcarriers and the dynamic noise environment, ultimately achieving optimal resource allocation. Specifically, by calculating the spectral distance and interference coefficient between subcarriers, the mutual interference level between different subcarriers can be accurately characterized; calculating out-of-band radiation based on subcarrier power and interference coefficient can reflect the interference intensity of a specific subcarrier to other frequency bands; and by aggregating out-of-band radiation, the overall interference level of subcarriers in complex channel environments can be comprehensively evaluated. In the transmission rate calculation, a transmission rate model that conforms to actual channel conditions is established by combining aggregated interference, receiver noise, and power signal characteristics. Finally, by constructing an optimization objective function that includes a minimum rate constraint, the basic communication needs of each node are guaranteed while maximizing the total system throughput. This scheme effectively solves the core problem that traditional fixed allocation strategies cannot adapt to the time-varying channel characteristics of power lines through multi-dimensional interference modeling and dynamic parameter calculation, thereby improving the data throughput of power line communication networks.

[0104] Step 106: Solve the Markov decision process model according to the transmission priority, the optimization objective function and the preset dual deep reinforcement learning algorithm to obtain the subcarrier and transmit power corresponding to each of the activated nodes.

[0105] In this embodiment, the steps mainly include: determining the reward function in the dual deep reinforcement learning algorithm based on the optimization objective function; inputting the transmission priority and the state space in the Markov decision process model into the preset dual deep reinforcement learning algorithm to search the action space in the Markov decision process model through the online network in the dual deep reinforcement learning algorithm, generating an action vector set; selecting the optimal action from the action vector set through a greedy algorithm preset in the online network; wherein, the optimal action includes the subcarrier and transmit power corresponding to each active node; obtaining the expected reward value of the optimal action through the objective network preset in the dual deep reinforcement learning algorithm according to the reward function; and outputting the subcarrier and transmit power corresponding to each active node in the power line communication network according to the expected reward value through the dual deep reinforcement learning algorithm.

[0106] Preferably, the predicted reward value of the optimal action is obtained through the online network, and the mean square error value between the predicted reward value and the expected reward value is obtained; based on the mean square error value, the network parameters of the online network are updated using a preset gradient descent method.

[0107] In some embodiments of this example, reference is made to Figure 5 Constructed dual deep reinforcement learning algorithm and Figure 6 As shown in the network architecture of the dual deep reinforcement learning algorithm, the network architecture of the dual deep reinforcement learning algorithm includes an online network Q and a target network Q. target Action selection is performed by the online network Q, and value evaluation is performed by the target network Q. target Finish.

[0108] Among them, from Figure 5 As shown in the training process, the Q-network θ and θ' are first initialized. * The system first establishes an experience replay pool D_EM; then observes the initial state s(t), selects action a(t,k) according to the ε-greedy policy, executes the action to observe the new state s(t,k+1), calculates the reward r_t, and stores the experience in D_EM. Next, it checks whether the activated node group has met the termination condition. If it has, it samples a small batch of data from D_EM and updates θ according to gradient descent; if not, it observes the next state s(t+1) and executes the ε-greedy policy to select subsequent actions according to s(t+1). After updating θ, the target network is updated every ζ steps according to a preset step size until θ equals θ_t. * The training then ends, and the trained dual deep neural network model and its Q-value are returned. The end-of-round criterion is that the amount of data sent by the active user group is reduced to zero (Done = True).

[0109] according to Figure 6 As shown in the network architecture, the state space of the Markov decision process model is first sent to the dual deep reinforcement learning algorithm, such as sending the state s(t), the reward function r(t), and the new state s(t+1) to the dual deep reinforcement learning algorithm, so that the action a(t) can be selected by a preset greedy algorithm such as the ε-greedy algorithm.

[0110] Reference Figure 5 and Figure 6 First, a deep neural network is constructed as a function approximator to fit the Q-value function. The Q-value is the agent's expectation and judgment of long-term cumulative reward in reinforcement learning, calculated by integrating immediate rewards and estimates of potential future rewards. And according to the optimization objective function: Furthermore, a predefined single-step reward function is used, and the Q-value output by the Q-value function provides a standard for action selection, simplifying the process of finding the optimal policy to a simple, local comparison (finding the maximum value) at each state. The global cumulative reward objective (i.e., the objective function) is perfectly decomposed into optimizing the local Q-value selection at each state.

[0111] Specifically, given that the maximum number of nodes in the power grid is K, and each node is numbered from 1 to k relative to the receiver's topological position, assuming that nodes numbered 1 to m are active at the start of a certain beacon cycle, the state in the state space S(t) is defined as follows:

[0112]

[0113] To avoid the online network's Q-output entering local optima, an ε-greedy algorithm is employed. For a given network input, the environment state *s*, a random floating-point number *a* can be generated using a random number generator. If the generated *a* is less than a threshold given by the ε-greedy algorithm, an action is randomly generated in the action space, i.e., random exploration. Conversely, the online network Q-values ​​are used to obtain the Q-value vector *a* of all possible actions in the corresponding state, and the action with the largest Q-value vector, i.e., the optimal action *a*, is output. * =argmax a Q(S(t), a; θ). Both of the above cases can be obtained from... The combination of selecting a specific subcarrier allocation from 3K and selecting a specific level allocation from 3K generates the action A(t) = {Q(t), C(t)}. According to the formula... The allocated transmission rate is obtained.

[0114] Next, an experience replay buffer is initialized. Each step of experience gained by the agent during exploration of the environment, i.e., the state transition tuple [S(t), S(t+1), A(t), R(t), Done], is stored in this buffer. Done is the final indicator; Done = True indicates the end of the current training round. R(t) is the reward obtained by performing action A(t) in the corresponding state S(t) (defined in step 5). During each training iteration, a mini-batch of samples is randomly selected from the replay buffer to break the temporal correlation between data points.

[0115] Next, a target network Q with the same structure but independent parameters is introduced. target To evaluate the value of actions generated by the online network Q, S(t+1) is input into the online network Q, and Q selects the action it considers best, a′. * =argmax a Q(S(t+1),a,θ). The formula for calculating the target Q value is: Y=R(t)+γQ target (S(t+1),a′ * ;θ′), where R represents the instant reward, γ represents the discount factor, and a′ * This represents the optimal action selected by the online network Q based on state S(t+1). The selection and evaluation of the action are performed by two independent and asynchronous networks, significantly reducing the possibility of overestimation and making the target Q-value estimation more accurate and stable.

[0116] The loss function is the mean squared error (MSE) between the predicted value Q(S(t), a; θ) and the target value Y of the online network, calculated using the formula L(θ) = E[(YQ(S(t), a; θ))]. 2 The target network Q is updated only by using gradient descent to update the parameters θ of the online network Q. target The parameter θ′ is synchronized with the online network Q every certain number of steps via hard updates. Here, Q represents the reward value output by the network.

[0117] The reward is defined according to the aforementioned optimization objective function as follows:

[0118]

[0119] Where λ is a positive number, representing the penalty coefficient for failing to reach the minimum rate constraint.

[0120] In this embodiment, the above steps achieve adaptive allocation of power line communication resources by constructing a dynamic decision-making mechanism based on dual deep reinforcement learning. Specifically, by inputting transmission priority and state space into the dual deep reinforcement learning algorithm, a candidate action set is generated using the action space search capability of the online network. A greedy algorithm is then used to select the optimal action that balances subcarrier allocation and power adjustment. The target network evaluates the expected reward value of the optimal action through a reward function, ensuring that the resource allocation scheme meets the transmission rate requirements of each node while maximizing the total system throughput. Transmission priority, as an input parameter, guides the algorithm to prioritize the resource needs of high-priority nodes. The dual-network architecture, by separating the action generation and value evaluation processes, effectively avoids the overestimation problem that traditional reinforcement learning algorithms are prone to in complex action spaces, improving the stability and convergence efficiency of the resource allocation strategy, thereby increasing the accuracy of resource allocation and ultimately improving the data throughput of the power line communication network.

[0121] On the other hand, refer to Figure 2 This embodiment also discloses a power communication resource allocation system based on reinforcement learning, including a network data module 201, a priority division module 202, a model decision module 203, a function update module 204, and a resource allocation module 205.

[0122] The network data module 201 is used to acquire node data corresponding to each transmitting node in the power line communication network; wherein, the node data includes task data, transmission rate data and network topology position relative to the receiving node; the transmitting nodes are divided into inactive nodes and active nodes.

[0123] The priority division module 202 is used to determine the transmission priority of each active node based on the transmission rate data.

[0124] The model decision module 203 is used to construct a Markov decision process model corresponding to the power line communication network based on the node data and the preset action space; wherein, the action space includes subcarrier allocation and power adjustment.

[0125] The function update module 204 is used to obtain the transmission rate of the active node on each subcarrier, so as to construct an optimization objective function based on the transmission rate and the transmission rate data, with the goal of maximizing the total throughput of task data.

[0126] The resource allocation module 205 is used to solve the Markov decision process model according to the transmission priority, the optimization objective function and the preset dual deep reinforcement learning algorithm, so as to obtain the subcarrier and transmit power corresponding to each of the activated nodes.

[0127] In this embodiment, as Figure 2The power communication resource allocation system shown also includes a communication data module and a node partitioning module.

[0128] The communication data module is used to obtain the transmission bandwidth of the power line communication network, so as to determine the multiple subcarriers used by the power line communication network according to the orthogonal frequency division multiplexing technology adopted by the power line communication network.

[0129] The node partitioning module is used to: acquire the status data of the sending node within a preset beacon period for any one of the sending nodes; when the sending node is determined to be in a working state based on the status data, partition the sending node as an active node; when the sending node is determined to be in a non-working state based on the status data, partition the sending node as an inactive node.

[0130] This embodiment discloses a power line communication resource allocation method and system based on reinforcement learning, constructing a centrally managed power line network system deployed with dual deep reinforcement learning algorithms. Based on the CSMA / CA protocol, the central controller can effectively reduce the impact of inter-carrier interference on the system throughput rate by allocating different subcarriers to active nodes and configuring their power levels in real time, according to the different priorities and minimum transmission rate requirements of communication nodes within the network. Specifically, by dynamically sensing the network status of the power line communication network and using reinforcement learning to optimize resource allocation, the data throughput within the power line communication network is improved. First, by acquiring task data, transmission rate, and network topology location from node data, the service needs and transmission rate requirements of transmitting nodes are grasped in real time, providing a data foundation for dynamic resource allocation. Distinguishing nodes into active and inactive states allows for focused resource allocation to actively operating nodes, avoiding resource waste. Priority is determined based on transmission rate data, enabling resource allocation to nodes with different real-time rate requirements. By constructing a Markov decision process model that includes subcarrier allocation and power adjustment, the resource allocation problem is transformed into a dynamic decision process, allowing for autonomous strategy adjustment based on environmental changes. A target function maximizing total throughput is constructed based on subcarrier transmission rate, directly linking resource allocation effectiveness with network performance indicators. A dual deep reinforcement learning algorithm is employed to solve the model, combined with transmission priority constraints. This achieves a resource allocation scheme with optimal global throughput while ensuring the resource allocation needs of different nodes, overcoming the limitations of traditional fixed allocation strategies and improving data throughput within power line communication networks.

[0131] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific embodiments of the present invention and are not intended to limit the scope of protection of the present invention. In particular, it should be noted that any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention for those skilled in the art.

Claims

1. A power communication resource allocation method based on reinforcement learning, characterized in that, include: Acquire node data corresponding to each transmitting node in the power line communication network; wherein, the node data includes task data, transmission rate data, and network topology position relative to the receiving node; the transmitting nodes are divided into inactive nodes and active nodes; The transmission priority of each activated node is determined based on the transmission rate data; Based on the node data and the preset action space, a Markov decision process model corresponding to the power line communication network is constructed; wherein, the action space includes subcarrier allocation and power adjustment; The transmission rate of the active node on each subcarrier is obtained, and an optimization objective function is constructed based on the transmission rate and the transmission rate data to maximize the total throughput of task data. The Markov decision process model is solved based on the transmission priority, the optimization objective function, and the preset dual deep reinforcement learning algorithm to obtain the subcarriers and transmit power corresponding to each of the activated nodes.

2. The power communication resource allocation method based on reinforcement learning according to claim 1, characterized in that, Before acquiring the node data corresponding to each transmitting node in the power line communication network, the following steps are included: The transmission bandwidth of the power line communication network is obtained, so as to determine the multiple subcarriers used by the power line communication network according to the orthogonal frequency division multiplexing technology adopted by the power line communication network; For any one of the sending nodes: Obtain the status data of the transmitting node within a preset beacon period; When the status data determines that the sending node is in a working state, the sending node is classified as an active node. When it is determined from the status data that the sending node is not in a working state, the sending node is classified as an inactive node.

3. The power communication resource allocation method based on reinforcement learning according to claim 2, characterized in that, The acquisition of node data corresponding to each transmitting node within the power line communication network includes: In the preparation frame of the beacon period, the minimum transmission rate requirement and the amount of remaining task data for each transmitting node are obtained. Based on the preparation frame, the network topology distance from each sending node to the receiving node is obtained.

4. The power communication resource allocation method based on reinforcement learning according to claim 3, characterized in that, Determining the transmission priority of each activated node based on the transmission rate data includes: For any one of the activated nodes: The minimum transmission rate requirement of the activated node is used to match the subcarrier interference level and transmit power level corresponding to the activated node, so as to use the subcarrier interference level and transmit power level as the transmission priority of the activated node.

5. The power communication resource allocation method based on reinforcement learning according to claim 3, characterized in that, The step of constructing a Markov decision process model corresponding to the power line communication network based on the node data and a preset action space includes: Based on the minimum transmission rate requirement, the remaining task data volume, and the network topology distance, a state space corresponding to multiple sending nodes is constructed; wherein, in the state space, the minimum transmission rate requirement, the remaining task data volume, and the network topology distance of inactive nodes are all set to 0; An action space corresponding to multiple transmitting nodes is constructed based on a pre-constructed subcarrier allocation matrix and a pre-constructed power adjustment matrix; wherein, the power adjustment matrix includes the transmit power corresponding to each transmitting node; the transmit power has multiple power levels; the subcarrier allocation matrix includes the subcarrier matched to each transmitting node; Based on the state space and the action space, a Markov decision process model corresponding to the power line communication network is constructed.

6. The power communication resource allocation method based on reinforcement learning according to claim 1, characterized in that, The step of obtaining the transmission rate of the activated node on each subcarrier, and constructing an optimization objective function based on the transmission rate and the transmission rate data to maximize the total throughput of task data, includes: Obtain the spectral distance between any two subcarriers and the preset subcarrier bandwidth, and obtain the interference coefficient between any two subcarriers through a preset interference coefficient calculation function; For any one of the subcarriers: Calculate the out-of-band radiation corresponding to the subcarrier based on the interference coefficient and the carrier power of the subcarrier; The multiple out-of-band radiations corresponding to the subcarrier are aggregated to obtain the aggregated out-of-band radiation corresponding to the subcarrier. For any one of the sending nodes: Based on the out-of-band radiation of the aggregation, the noise power of the receiving node, and the response of the power signal to the subcarrier matched to the transmitting node, the transmission rate of the transmitting node on the subcarrier is obtained through a preset transmission rate calculation function. Based on the minimum transmission rate requirement and the transmission rate corresponding to each of the sending nodes, an optimization objective function is constructed with the goal of maximizing the total throughput of task data and the constraint of satisfying the minimum transmission rate requirement.

7. A power communication resource allocation method based on reinforcement learning according to any one of claims 4-6, characterized in that, The step of solving the Markov decision process model based on the transmission priority, the optimization objective function, and a preset dual deep reinforcement learning algorithm to obtain the subcarriers and transmit power corresponding to each of the activated nodes includes: The reward function in the dual deep reinforcement learning algorithm is determined based on the optimization objective function; The transmission priority and the state space in the Markov decision process model are input into a preset dual deep reinforcement learning algorithm, so as to search the action space in the Markov decision process model through the online network in the dual deep reinforcement learning algorithm and generate an action vector set. The optimal action is selected from the action vector set by a greedy algorithm preset in the online network; wherein the optimal action includes the subcarrier and transmit power corresponding to each active node; According to the reward function, the expected reward value of the optimal action is obtained through the target network preset in the dual deep reinforcement learning algorithm; Based on the expected return value, the subcarrier and transmit power corresponding to each activated node in the power line communication network are output through the dual deep reinforcement learning algorithm.

8. The power communication resource allocation method based on reinforcement learning according to claim 7, characterized in that, The step of solving the Markov decision process model based on the transmission priority, the optimization objective function, and a preset dual deep reinforcement learning algorithm to obtain the subcarriers and transmit power corresponding to each of the activated nodes further includes: The predicted reward value of the optimal action is obtained through the online network, and the mean square error value between the predicted reward value and the expected reward value is obtained. Based on the mean square error value, the network parameters of the online network are updated using a preset gradient descent method.

9. A power communication resource allocation system based on reinforcement learning, characterized in that, It includes a network data module, a priority allocation module, a model decision module, a function update module, and a resource allocation module; The network data module is used to acquire node data corresponding to each transmitting node in the power line communication network; wherein, the node data includes task data, transmission rate data, and network topology position relative to the receiving node; the transmitting nodes are divided into inactive nodes and active nodes; The priority division module is used to determine the transmission priority of each activated node based on the transmission rate data; The model decision module is used to construct a Markov decision process model corresponding to the power line communication network based on the node data and a preset action space; wherein, the action space includes subcarrier allocation and power adjustment; The function update module is used to obtain the transmission rate of the active node on each subcarrier, so as to construct an optimization objective function based on the transmission rate and the transmission rate data, with the goal of maximizing the total throughput of task data. The resource allocation module is used to solve the Markov decision process model according to the transmission priority, the optimization objective function and the preset dual deep reinforcement learning algorithm, so as to obtain the subcarrier and transmit power corresponding to each of the activated nodes.

10. A power communication resource allocation system based on reinforcement learning according to claim 9, characterized in that, It also includes a communication data module and a node partitioning module; The communication data module is used to obtain the transmission bandwidth of the power line communication network, so as to determine the multiple subcarriers used by the power line communication network according to the orthogonal frequency division multiplexing technology adopted by the power line communication network; The node partitioning module is used to: acquire the status data of the sending node within a preset beacon period for any one of the sending nodes; when the sending node is determined to be in a working state based on the status data, partition the sending node as an active node; when the sending node is determined to be in a non-working state based on the status data, partition the sending node as an inactive node.

Images