A multi-modal communication network networking method integrating DC power line carrier

Abstract

The present invention relates to a multimodal communication network networking method integrating DC power line carrier, and belongs to the technical field of distribution network networking. The DQN-based task admission and flow adaptation joint optimization algorithm designed by the present invention can effectively process high-dimensional and complex environmental state information, and make optimal task admission and flow adaptation decisions through deep reinforcement learning, while ensuring the high reliability and low latency communication service requirements of the DC monitoring business of the distribution network.

Description

Technical Field

[0001] The invention relates to a multi-modal communication network networking method integrating a direct current power line carrier, and belongs to the technical field of distribution network networking. Background Art

[0002] At present, with the development of new energy grid-connected technology, the grid-connected new energy such as photovoltaic, wind power, and hydropower, which are greatly affected by the environment, will change the topological structure and power distribution of the distribution network, change the size and direction of the short-circuit current, and have a great impact on the distribution network. Therefore, it is urgent to conduct comprehensive and real-time monitoring of the operating status of new energy such as photovoltaics to ensure the stability of grid connection and the safe and stable operation of the distribution network. In order to meet the low latency requirements of distribution network monitoring, the computing tasks generated by distributed photovoltaic status monitoring, distributed energy and energy storage information collection equipment can be offloaded to the edge server for processing.

[0003] In the context of high-proportion new energy access, since various distributed new energy and distributed energy storage require DC grid connection, and various electrical equipment in buildings are driven by DC, the distribution network shows a trend of AC and DC coexistence. Based on the DC characteristics of the distribution and power system, DC power line carrier communication has significant advantages. DC power line carrier communication does not require the laying of dedicated lines, and is easy to install and operate. At the same time, since there is no industrial frequency signal on its line, the DC conversion frequency is greatly reduced, there are few high-frequency pollution sources, and it has good transmission anti-interference. However, DC power line carrier has shortcomings such as high latency, poor reliability, and difficulty in crossing transformers during long-distance transmission, and is not suitable for communication between stations. Wireless communication networks have the advantages of wide coverage, ultra-low latency and high reliability, and can carry out wide-area high-reliability and low-latency communication between stations. Therefore, a multi-modal communication network of "DC power line carrier + wireless communication network" can be used to achieve full coverage communication of the distribution network. On the one hand, it supports data monitoring of distributed photovoltaics and distributed energy storage, while the construction cost is low; on the other hand, it ensures high reliability and low latency performance of wide-area data transmission of the distribution network.

[0004] In order to give full play to the advantages of different communication media, considering the differences in transmission rate, bandwidth, etc. between DC power line carrier and wireless communication network, it is necessary to include the joint optimization of task access and flow adaptation in the multimodal communication network of distribution network into the study in order to ensure the reliability of monitoring data and improve the resource utilization of multimodal communication network. Among them, task access is an application layer data control method adopted to avoid congestion problems. According to the channel status of DC power line carrier and wireless communication network, the amount of data entering the application layer is controlled; flow adaptation is a physical layer data management method in multimodal communication network that determines the segmentation ratio according to the requirements of monitoring data delay, transmission rate, etc., "breaks the whole into parts", and stores them in the DC power line carrier cache queue or the wireless communication network cache queue for parallel processing.

[0005] However, there are still many challenges in the joint optimization of task admission and traffic adaptation, which are summarized as follows:

[0006] First, the application layer task admission is coupled with the physical layer traffic adaptation decision, and the application layer task admission control affects the physical layer traffic adaptation. For example, when a large amount of application layer monitoring data is accessed, the physical layer must promptly adjust the allocation ratio of the DC power line carrier cache queue and the wireless communication network cache queue. Otherwise, if the data is accessed in a continuous and large proportion, it will cause extremely high queue delays, which will seriously threaten the safe and stable operation of the power system.

[0007] Second, due to the differences in channel characteristics, transmission rate, etc. between DC power line carriers and wireless communication networks, the state of each channel is time-varying, and the Quality of Service (QoS) requirements of various monitoring services are also different. This leads to an exponential growth in the number of traffic adaptation decision actions, resulting in the curse of dimensionality, making it difficult to make real-time dynamic decisions on multiple devices under complex environmental information.

[0008] Third, the low-latency optimization of distribution network task offloading cannot only focus on average performance indicators (such as average latency and average rate stability), but also needs to further consider the impact of extreme events (such as sudden large queue backlogs) on latency performance. Otherwise, it is difficult to guarantee the demand for high-reliability and low-latency communication services.

[0009] Therefore, it is urgent to design a joint optimization method for task admission and traffic adaptation when global information is unknown, so as to improve the performance of multimodal communication network with heterogeneous integration of DC power line carrier and wireless communication network, meet the QoS requirements of different distribution network monitoring services, effectively reduce queue delay, and realize high-quality transmission of power monitoring data. Summary of the invention

[0010] In order to solve the above technical problems, an object of the present invention is to provide a multi-modal networking method integrating a wireless communication network with a DC power line carrier.

[0011] A multi-modal communication network networking method integrating a DC power line carrier of the present invention comprises the following specific steps:

[0012] Modeling the monitoring data in the pre-built multi-modal heterogeneous networking system model as a device queue model;

[0013] According to the device queue model, determining the transmission model of the monitoring data and obtaining the device queue delay model;

[0014] The device queue delay model is respectively subjected to device queue delay period constraints and extreme event occurrence probability constraints;

[0015] Based on the device queue delay period constraints and extreme event probability constraints, the optimization problem is transformed into minimizing the device queue delay; and

[0016] Using virtual queue theory and Lyapunov theory, we designed the transformation of optimization problems and system states, actions, and reward functions, and transformed the optimization problem into solving the upper bound problem of Lyapunov's "drift + penalty";

[0017] In response to the upper bound problem, a joint optimization algorithm of task admission and traffic adaptation based on deep Q network is optimized to determine the multimodal communication network networking integrating DC power line carrier.

[0018] Furthermore, there are M distribution network monitoring devices in the pre-built multi-modal heterogeneous networking system model, and the set is Each device has two task cache queues, namely a DC power line carrier cache queue and a wireless communication network cache queue, which respectively store tasks transmitted using the DC power line carrier and the wireless communication network.

[0019] Furthermore,

[0020] The device queue model is a DC power line carrier cache queue model and a wireless communication network cache queue model, specifically:

[0021]

[0022]

[0023] in, and They are respectively represented as the queue backlog at the current moment and the next moment in the DC power line carrier cache queue; and They are respectively represented as queue backlogs at the current moment and the next moment in the wireless communication network cache queue; and They represent the amount of tasks that the device offloads to the edge server through the DC power line carrier and wireless communication network communication respectively; and They represent the data arrival amounts of the DC power line carrier cache queue and the wireless communication network communication cache queue respectively.

[0024] The monitoring data in the DC power line carrier cache queue and the wireless communication network cache queue can be transmitted via the DC power line and the wireless communication network respectively, and the transmission model is obtained according to the Shannon formula.

[0025] Furthermore, the transmission model includes a DC power line carrier transmission model and a wireless communication network transmission model, wherein:

[0026] The DC power line carrier transmission model is expressed in the following way:

[0027]

[0028]

[0029]

[0030] in, Is the device u m Signal-to-noise ratio for data transmission using DC power line subcarrier n; Is the device u m The transmission rate of data transmission using DC power line subcarrier n; Is the device u m Total data transmission rate using DC power line carrier; Is the device u m The transmission power required to transmit data using the DC power line subcarrier n; is the power gain factor of the DC power line subcarrier n; N0 is the white noise power spectrum density; B L is the total bandwidth of the DC power line carrier; N L is the total number of DC power line subcarriers; Γ L is the signal-to-noise ratio gap between different DC power line subcarriers; M L is the number of devices using DC power line carrier for task transmission;

[0031] The wireless communication network transmission model is represented by the following method:

[0032]

[0033]

[0034]

[0035] in, Is the device u m The signal-to-noise ratio of data transmission using subcarrier n of a wireless communication network; Is the device u m The transmission rate of data transmission using subcarrier n of the wireless communication network; Is the device u m The overall data transmission rate using the wireless communication network; Is the device u m The transmission power used for transmitting data using subcarrier n of the wireless communication network; is the power gain factor of subcarrier n in the wireless communication network; N0 is the white noise power spectrum density; B W is the total bandwidth of the channel transmitted in the form of a wireless communication network; e is the electromagnetic interference power; N W is the total number of subcarriers in the wireless communication network; Γ W is the signal-to-noise ratio gap between subcarriers in different wireless communication networks; M W is the number of devices using wireless communication networks for data transmission; device u m The amount of tasks unloaded to the edge server using DC power line carrier and wireless communication network communication are:

[0036]

[0037]

[0038] Where τ is the time slot length.

[0039] Furthermore, the device queue delay model is: in is the DC power line carrier buffer queue delay, is the wireless communication network cache queue delay, and the device queue delay is the maximum value of the device DC power line carrier cache queue delay and the wireless communication network cache queue delay.

[0040] Furthermore, the specific steps of the device queue long-term delay constraint and extreme event occurrence probability constraint are as follows:

[0041] Connect the device m The extreme events of the DC power line carrier buffer queue and the wireless communication network buffer queue are expressed as follows:

[0042]

[0043]

[0044] The probability of occurrence of the above two extreme events is constrained as follows:

[0045]

[0046]

[0047] in, and They represent the extreme events that occur in the DC power line carrier buffer queue and the wireless communication network buffer queue, and They are the queue backlogs of the DC power line carrier cache queue and the wireless communication network cache queue at the current moment, and are the average arrival rates of tasks in the DC power line carrier cache queue and the wireless communication network cache queue, and They represent the long-term queue delay thresholds for data transmission using DC power line carrier and wireless communication network, respectively. and They represent the thresholds of the tolerable probability of extreme events under DC power line carrier and wireless communication network communication modes respectively.

[0048] Furthermore, the specific method for designing the optimization problem based on the equipment queue time extension constraint and the extreme event occurrence probability constraint is:

[0049] The optimization problem P1 is set as:

[0050]

[0051]

[0052] C2:0≤ρ m (t)≤1

[0053]

[0054]

[0055] Where M is the number of devices, T is the total optimization time, and τ m (t) is the device queue delay, a m,min Is the device u m The minimum number of tasks allowed in each time slot, is the actual task admission amount, A m (t) is the device u m The amount of tasks generated in time slot t, ρ m (t) is the task split ratio, which indicates the ratio of tasks stored in the DC power line carrier cache queue to the actual admitted tasks. and They are the queue backlogs of the DC power line carrier cache queue and the wireless communication network cache queue at the current moment, and are the average arrival rates of tasks in the DC power line carrier cache queue and the wireless communication network cache queue, and They represent the long-term queue delay thresholds for data transmission using DC power line carrier and wireless communication network, respectively. and They represent the thresholds of the tolerable probability of extreme events under the DC power line carrier and wireless communication network communication modes respectively;

[0056] Constraint C1 is the device task admission constraint, constraint C2 is the device traffic adaptation ratio constraint, constraint C3 is the long-term queue delay constraint of the DC power line carrier cache queue and the wireless communication network cache queue, and constraint C4 is the long-term extreme event occurrence probability constraint of the DC power line carrier cache queue and the wireless communication network cache queue.

[0057] Furthermore, the specific contents of converting the optimization problem into minimizing the device queue delay and converting the optimization problem into an upper bound problem are as follows:

[0058] Transform the optimization problem P1 into the optimization problem P2:

[0059]

[0060] stC1,C2

[0061]

[0062]

[0063]

[0064] The average rate is stable, where M is the number of devices, T is the total optimization time, and τ m (t) is the device queue delay, and They represent the virtual queue backlog of the DC power line carrier at the current moment and the next moment after the long-term queue delay constraint of the DC power line carrier cache queue is transformed using the virtual queue theory, and They represent the current and next moment wireless communication network virtual queue backlogs after the long-term queue delay constraints of the wireless communication network cache queue are transformed using the virtual queue theory, and They are the queue backlogs of the DC power line carrier cache queue and the wireless communication network cache queue at the current moment, and are the average arrival rates of tasks in the DC power line carrier cache queue and the wireless communication network cache queue, and They represent the long-term queue delay thresholds for data transmission using DC power line carrier and wireless communication network, respectively. and They represent the virtual queue backlog of the DC power line carrier at the current moment and the next moment after the long-term extreme event probability constraint of the DC power line carrier cache queue is transformed using the virtual queue theory, and They represent the virtual queue backlog of the wireless communication network at the current moment and the next moment after the probability constraint of long-term extreme events in the wireless communication network cache queue is transformed using the virtual queue theory, and II{Ω L} and II{Ω W} respectively represent the indicator functions of the extreme events of the DC power line carrier cache queue and the wireless communication network cache queue, II{Ω L}=1 indicates that an extreme event occurs in the DC power line carrier buffer queue, II{Ω L}=0 means that no extreme event occurs in the DC power line carrier cache queue, II{Ω W}=1 indicates that an extreme event occurs in the wireless communication network cache queue, II{Ω W}=0 means that no extreme event occurs in the wireless communication network cache queue. and They represent the thresholds of the tolerable probability of extreme events under the DC power line carrier and wireless communication network communication modes respectively;

[0065] Constraint C1 is the equipment task access constraint, and the formula is: Constraint C2 is the device flow adaptation ratio constraint, the formula is 0≤ρ m (t)≤1.

[0066] The optimization problem P2 is transformed into minimizing the upper bound of "drift + penalty" under the premise of satisfying constraints C1 and C2, and the constraint C5 in P2 is transformed into the objective function of P3, resulting in the optimization problem P3:

[0067]

[0068] stC1,C2

[0069] in,

[0070] and They are the queue backlogs of the DC power line carrier cache queue and the wireless communication network cache queue at the current moment, and They represent the amount of tasks that the device offloads to the edge server through the DC power line carrier and the wireless communication network communication mode, respectively. and They represent the data arrival amount of the DC power line carrier cache queue and the wireless communication network communication cache queue respectively, and They represent the backlog of the DC power line carrier virtual queue and the wireless communication network cache queue at the current moment after the long-term queue delay constraints of the DC power line carrier cache queue and the wireless communication network cache queue are transformed using the virtual queue theory, and are the average arrival rates of tasks in the DC power line carrier cache queue and the wireless communication network cache queue, and They represent the long-term queue delay thresholds for data transmission using DC power line carrier and wireless communication network, respectively. and They represent the backlog of the DC power line carrier virtual queue and the wireless communication network virtual queue at the current moment after the long-term extreme event probability constraints of the DC power line carrier cache queue and the wireless communication network cache queue are transformed using the virtual queue theory, and They represent the thresholds of the probability of extreme events that can be tolerated under the DC power line carrier and wireless communication network communication modes, and II{Ω L} and II{Ω W} respectively represent the indicator functions of the extreme events of the DC power line carrier cache queue and the wireless communication network cache queue, II{Ω L}=1 indicates that an extreme event occurs in the DC power line carrier buffer queue, II{Ω L}=0 means that no extreme event occurs in the DC power line carrier cache queue, II{Ω W}=1 indicates that an extreme event occurs in the wireless communication network cache queue, II{Ω W}=0 indicates that no extreme event occurs in the wireless communication network cache queue;

[0071] Constraint C1 is the equipment task access constraint, and the formula is: Constraint C2 is the device flow adaptation ratio constraint, the formula is 0≤ρ m (t)≤1.

[0072] Furthermore, the specific contents of the system state, action and reward function are:

[0073] 1) System status S m (t) includes the status information of each queue, and the vector representation of the system status is:

[0074]

[0075] 2) System Action a m (t) is the device u m According to the system status S m (t) The task admission control and traffic adaptation decisions taken, represented by the device u in each time slot m The number of tasks allowed and flow adaptation ratio ρ m (t); and ρ m(t) are all continuously changing physical quantities. To simplify the analysis process, and ρ m (t) are discretized into K levels, expressed as ρ m (t)∈ρ, where ρ=[ρ1,ρ2,…,ρ K ]; therefore, the system action a m (t) is expressed as:

[0076]

[0077] 3) The design of the reward function should be related to the optimization problem. The goal of the optimization problem P3 is to minimize β(t), so the reward function here is R(t) = -β(t).

[0078] Furthermore, the specific steps of the joint optimization algorithm of task admission and traffic adaptation based on deep Q network are as follows:

[0079] Initialize the estimated network parameters ω eval (t), target network parameters ω target (t), loss function φ and experience pool P;

[0080] The device adopts the ε-greedy strategy, making random task admission and traffic adaptation decisions based on the estimated Q value with probability ε, and making task admission and traffic adaptation decisions based on the Q value obtained from the valuation network with probability 1-ε. Specifically: Among them, μ∈(0,1) is a random number generated by the system;

[0081] The device determines the amount of data entering the device queue, the amount of data entering the DC power line carrier cache queue, and the amount of data entering the wireless communication network cache queue according to the task admission and traffic adaptation decisions made, and unloads the data in the DC power line carrier cache queue and the wireless communication network cache queue to the edge server respectively;

[0082] After each unloading, the reward function R(t) is calculated and updated Each queue information; then, transfer to the next state S m (t+1), at the same time, S m (t), a m (t), R(t) and S m (t+1) Deposit into experience pool P;

[0083] The device randomly extracts a set of experience data from the experience pool P as training data for the valuation network, and calculates the loss function based on the above training data in, Represents the target Q value obtained by the target network;

[0084] Then, based on the loss function, the estimated network parameters ω are updated using the gradient descent method. eval , Among them, λ is the learning rate; finally, every T0 time, update the target network parameter ω target (t+1)=ω eval (t+1).

[0085] A multi-mode networking device integrating a wireless communication network and a DC power line carrier, specifically comprising:

[0086] Modeling module: used to model the monitoring data in the pre-built multi-modal heterogeneous networking system model into a device queue model, determine the transmission model of the monitoring data according to the device queue model, and obtain the device queue delay model;

[0087] Constraint module: used to impose constraints on the device queue delay period and the probability of occurrence of extreme events on the device queue delay model;

[0088] Conversion module: used to convert the optimization problem into minimizing the device queue delay based on the device queue delay period constraint and the extreme event probability constraint; and use virtual queue theory and Lyapunov theory to design the transformation of the optimization problem and the system state, action and reward function, and convert the optimization problem into the problem of solving the upper bound of Lyapunov's "drift + penalty"; for the upper bound problem, the task admission and traffic adaptation joint optimization algorithm based on the deep Q network is optimized and determined to determine the multimodal communication network networking integrating DC power line carrier.

[0089] Furthermore, there are M distribution network monitoring devices in the modeling module, and the set is Each device has two task cache queues, namely a DC power line carrier cache queue and a wireless communication network cache queue, which respectively store tasks transmitted using the DC power line carrier and the wireless communication network.

[0090] Furthermore, the constraint module is used to set the device u m The extreme events of the DC power line carrier buffer queue and the wireless communication network buffer queue are expressed as follows:

[0091]

[0092]

[0093] The probability of occurrence of the above two extreme events is constrained as follows:

[0094]

[0095]

[0096] in, and They represent the thresholds of the tolerable probability of extreme events under DC power line carrier and wireless communication network communication modes respectively.

[0097] By means of the above scheme, the present invention has at least the following advantages:

[0098] 1. Cross-layer joint optimization during decision coupling: The cross-layer optimization scheme for application layer task access and physical layer traffic adaptation designed in the present invention fully considers the impact of the differences in transmission rate, bandwidth, etc. between DC power line carrier and wireless communication network on the congestion level of the converged communication network. There is no need to decouple the decision. At the same time, the actual data access amount and traffic adaptation ratio are determined according to the channel status of the multimodal network, which effectively solves the congestion problem of the converged communication network and meets the QoS requirements of the distribution network monitoring service.

[0099] 2. Traffic adaptation under incomplete information and high dimensions: Based on the observed queue performance and deep Q network, the algorithm proposed in the present invention can avoid the high-dimensional real value solution process by using the approximate function provided by the neural network when the global information is unknown, thereby solving the dimensionality disaster problem caused by the access of a high proportion of distributed new energy, distributed energy storage and other equipment.

[0100] 3. High-reliability and low-latency communication perception: In order to meet the low-latency requirements of DC monitoring services in distribution networks, the present invention constrains long-term queue delays and the probability of occurrence of extreme events, so that the proposed algorithm can dynamically optimize task admission and flow adaptation strategies according to the performance deviation of the probability of extreme events, thereby significantly reducing the probability of extreme events, and achieving high-reliability communication of DC power line carriers and wireless communication network multimodal networks while meeting low latency.

[0101] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention and implement it according to the contents of the specification, the following is a detailed description of the preferred embodiments of the present invention in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0102] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show a certain embodiment of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0103] Figure 1 It is a schematic diagram of the system model of the present invention;

[0104] Figure 2It is a line graph showing the variation of the average local queue delay over time in the simulation experiment of the present invention;

[0105] Figure 3 is a histogram of the probability of occurrence of local extreme events of the equipment in the simulation experiment of the present invention;

[0106] Figure 4 It is a schematic diagram of the local queue backlog during the DC power line carrier and wireless communication network task unloading in the simulation experiment of the present invention;

[0107] Figure 5 It is a line graph showing how the average throughput of the simulation experiment of the present invention changes over time. DETAILED DESCRIPTION

[0108] The specific implementation of the present invention is further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0109] The entire technical solution of the present invention mainly includes five steps: (1) constructing a system model framework; (2) refining the model; (3) designing the device queue delay constraint and extreme event constraint and optimization problem; (4) transforming the optimization problem and designing the system state, action and reward function; (5) designing a joint optimization algorithm for task admission and traffic adaptation based on a deep Q-learning network (DQN). The specific introduction is as follows:

[0110] 1. Build a system model framework

[0111] The present invention considers the multi-modal distribution network communication scenario of heterogeneous networking of DC power line carrier and wireless communication network. Figure 1 As shown in Figure 1, the scenario mainly includes distribution substations, distribution network monitoring equipment (such as distributed energy storage equipment, DC energy meters, photovoltaic panels, DC monitoring equipment), DC power lines, wireless communication network base stations, DC power line carrier gateways and edge servers. There are a total of M distribution network monitoring devices in the scenario, and their set is defined as Each device includes a DC power line carrier cache queue and a wireless communication network cache queue, which respectively store tasks transmitted using DC power line carrier and wireless communication network communication. m The flow adaptation ratio is ρ m (t)∈[0,1], represents the proportion of data entering the DC power line cache queue to the admitted data. Figure 1 Taking device u1 as an example, the amount of tasks generated by device u1 is A1(t), and the actual amount of tasks admitted is Subsequently, the actual access task volume is flow-adapted, 40% (ρ1=0.4) of the tasks are stored in the DC power line carrier cache queue, and 60% (1-ρ1=0.6) of the tasks are stored in the wireless communication network cache queue; finally, the tasks in the cache queue are unloaded to the edge server for task processing by using the DC power line carrier and wireless communication network communication methods respectively. The present invention adopts a quasi-static time slot model, divides time into T equal-length time slots, the time slot length is τ, and is represented by a set as The channel state information remains unchanged within the same time slot and changes between different time slots.

[0112] 2. Model refinement

[0113] (1) Device Queue Model

[0114] The present invention considers the limited cache capacity of devices in multimodal networks and the QoS requirements of devices, and first performs task admission control. The following two constraints are met:

[0115]

[0116]

[0117] Among them, θ G ∈(0,1) represents device u m The long-term minimum average task admission ratio under the premise of ensuring its basic QoS requirements, Indicates u m The minimum task admission quantity in each time slot. Formula (1) represents the long-term average constraint of task admission quantity. However, the instantaneous task admission quantity will have a greater impact on QoS. Considering only the long-term constraint cannot guarantee the QoS requirements of the monitoring service. Therefore, the short-term task admission constraint of formula (2) is further considered.

[0118] Equipment m The traffic adaptation ratio is determined based on the queue backlog and channel status of the DC power line carrier cache queue and the wireless communication network cache queue. and the number of tasks arriving in the wireless communication network cache queue It can be expressed as

[0119]

[0120]

[0121] Therefore, the device u m DC Power Line Carrier Buffer Queue and wireless communication network cache queue They can be expressed as

[0122]

[0123]

[0124] in, and They respectively represent the amount of tasks that the device offloads to the edge server through DC power line carrier and wireless communication network communication.

[0125] (2) Transmission model

[0126] 1) DC power line carrier transmission model

[0127] The DC power line carrier adopts the modulation method of Orthogonal Frequency Division Multiplexing (OFDM), and its subcarrier set is expressed as The characteristics of each sub-channel are exactly the same, and the devices using DC power line carrier for task transmission evenly distribute sub-carrier resources. m The signal-to-noise ratio for data transmission using subcarrier n is The transmission rate is Equipment m The total data transfer rate is The specific expression is:

[0128]

[0129]

[0130]

[0131] in, Is the device u m The transmission power required to transmit data using subcarrier n; is the power gain factor of subcarrier n; N0 is the white noise power spectral density; B L is the total bandwidth of the DC power line carrier; Γ L is the signal-to-noise ratio gap between different subcarriers; M L It is the number of devices that use DC power line carrier for task transmission.

[0132] 2) Wireless communication network transmission model

[0133] The wireless communication network also uses OFDM modulation, but the total number of subcarriers is different from that of the DC power line carrier communication. The total number of subcarriers in the wireless communication network is defined as N. W, and the characteristics of each sub-channel are exactly the same, and the devices using the wireless communication network for task transmission evenly distribute sub-carrier resources. Define device u m The signal-to-noise ratio for data transmission using subcarrier n is The transmission rate is Equipment m The total data transfer rate is The specific expression is:

[0134]

[0135]

[0136]

[0137] in, Is the device u m The transmission power used to transmit data using subcarrier n; is the power gain factor of subcarrier n; B W is the total bandwidth of the channel transmitted in the form of a wireless communication network; e is the electromagnetic interference power; Γ W is the signal-to-noise ratio gap between different subcarriers; M W is the number of devices that use the wireless communication network for data transmission. m The amount of tasks unloaded to the edge server using DC power line carrier and wireless communication network communication are

[0138]

[0139]

[0140] (3) Device Queue Delay Model

[0141] Define device u m The average arrival rate of tasks in the DC power line carrier cache queue and the wireless communication network cache queue is The specific expression is:

[0142]

[0143]

[0144] According to Little's law, the device u m The delays of the DC power line carrier cache queue and the wireless communication network cache queue are

[0145]

[0146]

[0147] The device queue delay is defined as the larger value of the device DC power line carrier buffer queue delay and the wireless communication network buffer queue delay. m The queue delay is

[0148]

[0149] 3. Design of device queue delay constraints and extreme event constraints and optimization problems

[0150] Since the queuing delay has a great impact on the end-to-end delay, long-term constraints are imposed on the queuing delay to ensure the effectiveness and timeliness of the unloading task. At the same time, in order to support the real-time performance of services such as intelligent distributed distribution automation and power load demand-side response in the distribution network, the monitoring services of distributed new energy and distributed energy storage also have stringent requirements on instantaneous delay. A large backlog of tasks in the device cache queue will cause extremely high queue delay, which seriously threatens the safe and stable operation of the power system. Therefore, constraints based solely on the long-term average queue delay performance cannot meet the high reliability and low latency requirements of multi-modal heterogeneous communications in the distribution network.

[0151] (1) Long-term device queue delay constraints and extreme event probability constraints

[0152] In order to reduce the backlog of device queues and ensure the QoS requirements of devices, the DC power line carrier cache queue and the wireless communication network cache queue of the device should meet the following constraints respectively:

[0153]

[0154] in, and They represent the thresholds of the DC power line carrier buffer queue delay and the wireless communication network buffer queue delay respectively.

[0155] The present invention proposes a device extreme event constraint model. The device extreme event is defined as the device task cache queue delay exceeding the given long-term queue delay threshold. m The delays of the DC power line carrier buffer queue and the wireless communication network buffer queue when extreme events occur are defined as

[0156]

[0157]

[0158] In order to reduce the occurrence of extreme events in the distribution network and ensure the safe and stable operation of the power system, the probability of occurrence of the above two extreme events meets the following constraints:

[0159]

[0160]

[0161] in, and They represent the thresholds of the tolerable probability of extreme events under DC power line carrier and wireless communication network communication modes respectively.

[0162] (2) Design of optimization problem

[0163] The present invention considers the joint optimization problem of device task admission control and traffic adaptation. The optimization goal is to minimize the device queue delay under the premise of satisfying the long-term queue delay constraint and the long-term extreme event probability constraint. Therefore, the optimization problem P1 is defined as:

[0164]

[0165] Among them, C1 is the device task access constraint; C2 is the device traffic adaptation ratio constraint; C3 is the device long-term queue delay constraint; C4 is the device long-term extreme event occurrence probability constraint.

[0166] 4. Transformation of optimization problems and design of system states, actions and reward functions

[0167] (1) Transformation of optimization problems

[0168] First, based on the concept of virtual queues, the above constraints C3 and C4 are converted into virtual queues. The length of the virtual queue represents the deviation between the actual cumulative queue length and the delay threshold for extreme events. The formula is as follows:

[0169] For the long-term queue delay constraint C3 in P1, the following virtual queue is constructed

[0170]

[0171]

[0172] For the long-term extreme event probability constraint C4 in P1, the following virtual queue is constructed

[0173]

[0174]

[0175] Among them, II{·}={0,1} is the indicative function, II{x}=1 means that event x is true, otherwise event x is false.

[0176] After the above analysis, the optimization problem P1 can be transformed into the optimization problem P2:

[0177]

[0178] stC1,C2

[0179]

[0180]

[0181]

[0182] Average rate stability

[0183] Then, a queue set Θ(t) including cache queues and virtual queues is constructed. The specific expression is as follows

[0184]

[0185] Define the Lyapunov scalar function L(t) to characterize the backlog level of all queues in Θ(t). Its specific expression is as follows:

[0186]

[0187] When L(t) is small, the backlog of the data cache queue and the virtual queue in Θ(t) is small. Otherwise, at least one queue has a large backlog and the queue stability is poor.

[0188] Based on the Lyapunov scalar function L(t) defined above, the Lyapunov drift function ΔL(t) is further defined to characterize the queue backlog difference between two adjacent time slots. The specific expression is as follows:

[0189]

[0190] It can be seen from the above formula that when the device minimizes ΔL(t) by optimizing the task admission and traffic adaptation strategies in each time slot, all queues in Θ(t) will be pushed to a smaller backlog level, ensuring that the average rate of the queue is stable, thereby ensuring that constraint C5 is met.

[0191] To solve the optimization problem P2 under the C5 constraint, we can combine ΔL(t), which represents the stability of the average rate of the queue, with the optimization goal of P2 to obtain a new objective function Δ V L(t), the specific expression is as follows

[0192]

[0193] in, V is a non-negative weight. By adjusting the value of V, a trade-off between queue stability and queue delay can be achieved.

[0194] The objective function Δ can be obtained V Upper bound of L(t) for

[0195]

[0196] Where C is a non-negative constant.

[0197] Therefore, the optimization problem P2 can be transformed into minimizing Δ V The upper bound of L(t) gives the optimization problem P3

[0198]

[0199] in,

[0200] The present invention transforms constraint C5 in P2 into the objective function of P3, thereby ensuring the requirements of high reliability and low latency of data tasks.

[0201] (2) System state, action, and reward function design

[0202] 1) System status S m (t) includes the status information of each queue, and the vector representation of the system status is:

[0203]

[0204] 2) System Action a m (t) is defined as device u m According to the system status S m (t) The task admission control and traffic adaptation decisions taken, represented by the device u in each time slot m The number of tasks allowed and flow adaptation ratio ρ m (t). and ρ m (t) are all continuously changing physical quantities. To simplify the analysis process, the present invention will and ρ m (t) are discretized into K levels respectively ρ m (t)∈ρ. ρ=[ρ1,ρ2,…,ρ K ]. Therefore, the system action a m (t) is expressed as

[0205]

[0206] 3) The design of the reward function should be related to the optimization problem. The goal of the optimization problem P3 is to minimize β(t), so the reward function is defined here as R(t) = -β(t).

[0207] 5. DQN-based task admission and traffic adaptation algorithm

[0208] The DQN-based task admission and traffic adaptation algorithm designed in this invention is a reinforcement learning algorithm based on temporal difference learning. Its core idea is to use the deep Q network to take the action value function Q(s,a) corresponding to the action a in state s, and then make a decision based on the estimated Q value, and realize the approximation of the action value function estimate to the actual value by continuously updating the neural network. Where ω is the deep Q network estimation parameter. At the same time, the experience replay mechanism is introduced to break up the correlation between the Q-value estimation network training data, thereby improving the network training effect; at the same time, the target network is introduced to assist the estimation network training to improve the stability of the estimation network training.

[0209] The specific steps of the DQN-based task admission and traffic adaptation algorithm designed by the present invention are as follows:

[0210] (1) Initialization: Initialize the estimated network parameters ω eval (t), target network parameters ω target (t) and experience pool P.

[0211] (2) Task admission and traffic adaptation decision: The device adopts the ε-greedy strategy and makes task admission and traffic adaptation decisions based on the estimated Q value. Specifically, the device makes a random decision with a probability of ε and makes a decision based on the Q value obtained by the valuation network with a probability of 1-ε.

[0212] (3) State transfer: The device performs task admission and traffic adaptation based on the decision made, and unloads the data in the cache queue to the edge server; calculates the reward function R(t); and updates Each queue information; then, transfer to the next state S m (t+1), at the same time, S m (t), a m (t), β(t), and S m (t+1) is stored in the experience pool P.

[0213] (4) Network update: The device randomly extracts a set of experience data from the experience pool as training data for the valuation network, and calculates the loss function based on the training data. in, represents the target Q value obtained by the target network. Then, the neural network parameters are updated using the loss function Among them, λ is the learning rate. Finally, update the target network parameter ω target (t+1)=ω eval (t+1).

[0214] (5) Repeat steps (2), (3), and (4) until the optimization cycle is completed.

[0215] Through simulation verification, it is verified that the improved effect that the present invention can bring:

[0216] First, a simulation scenario is constructed. The simulation scenario considered includes 5 distribution network monitoring devices, 100 optimization cycles, and each time slot size is 100ms. To simplify the analysis process, the channel bandwidth of the two task transmission modes, DC power line carrier and wireless communication network, is set to be the same, 0.1MHz; the cache queue delay threshold is the same, 0.003ms; and the tolerable extreme event probability threshold is the same, 0.2. In addition, the amount of data arriving at the device task fluctuates between 1.0Mbits and 2.0Mbits.

[0217] The present invention conducts the following simulation comparison experiments on the joint optimization algorithm of task access and traffic adaptation designed above. Among them, method 1 is the upper confidence interval (UCB) based traffic adaptation algorithm, which selects the traffic adaptation ratio based on the traditional UCB algorithm; method 2 is the queue backlog awareness (QBA) traffic adaptation algorithm, which divides the traffic adaptation ratio according to the backlog ratio between the two queues of the DC power line carrier and the wireless communication network. In addition, UCB and QBA adopt constant data access control to meet

[0218] The simulation results are as follows:

[0219] Figure 2 The figure shows how the local queue delay of the device changes over time. Compared with other comparison algorithms, the algorithm proposed in the present invention can effectively reduce the average local queue delay.

[0220] like Figure 3 As shown in Figure 2, the proposed algorithm has the best performance in terms of the probability of extreme events. Compared with UCB and QBA, the proposed algorithm can reduce the probability of extreme events. The probability of occurrence is 58.78%, 70.81% and 70.97%, 68.05%.

[0221] like Figure 4 As shown in the figure, the algorithm proposed in the present invention can effectively reduce the backlog of two local unloading queues. Compared with UCB and QBA, the proposed algorithm can reduce the backlog of the DC power line carrier queue by 14.91% and 31.06% respectively, and the backlog of the wireless communication network queue by 23.77% and 33.72% respectively.

[0222] like Figure 5 As shown in the figure, compared with UCB and QBA, the algorithm proposed in this invention can effectively improve the average throughput by 12.34% and 24.77%. The reason is that the compared algorithms all use a constant task admission mechanism, which will not increase the amount of data admission even if the local queue backlog is small, resulting in poor throughput performance of both. Figures 2 to 5 It can be seen that the DQN-based task admission and flow adaptation joint optimization algorithm designed in the present invention can effectively process high-dimensional and complex environmental state information, and make optimal task admission and flow adaptation decisions through deep reinforcement learning, while ensuring the high-reliability and low-latency communication service requirements of the distribution network DC monitoring service.

[0223] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0224] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0225] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0226] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0227] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit its protection scope. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the field should understand that after reading the present invention, those skilled in the art can still make various changes, modifications or equivalent substitutions to the specific implementation methods of the invention, but these changes, modifications or equivalent substitutions are all within the protection scope of the pending claims of the invention.

Claims

1. A multi-modal communication network networking method integrating DC power line carrier, characterized in that: The specific steps are: Modeling the monitoring data in the pre-built multi-modal heterogeneous networking system model as a device queue model; According to the device queue model, determining the transmission model of the monitoring data and obtaining the device queue delay model; The device queue delay model is respectively subjected to device queue delay period constraints and extreme event occurrence probability constraints; Based on the device queue delay period constraints and extreme event probability constraints, the optimization problem is transformed into minimizing the device queue delay; as well as Using virtual queue theory and Lyapunov theory, we designed the transformation of optimization problems and system states, actions, and reward functions, and transformed the optimization problem into solving the upper bound problem of Lyapunov's "drift + penalty"; In view of the upper bound problem, a joint optimization algorithm of task admission and traffic adaptation based on deep Q network is optimized to determine the multimodal communication network networking integrating DC power line carrier; There are M distribution network monitoring devices in the pre-built multi-modal heterogeneous networking system model, and the set is u = {u1, ..., u m ,…,u M }, and each device has two task cache queues, namely, a DC power line carrier cache queue and a wireless communication network cache queue, which respectively store tasks transmitted by DC power line carrier and wireless communication network communication; The device queue model is a DC power line carrier cache queue model and a wireless communication network cache queue model, specifically: in, and They are respectively represented as the queue backlog at the current moment and the next moment in the DC power line carrier cache queue; and They are respectively represented as queue backlogs at the current moment and the next moment in the wireless communication network cache queue; and They represent the amount of tasks that the device offloads to the edge server through the DC power line carrier and wireless communication network communication respectively; and They represent the data arrival amount of the DC power line carrier buffer queue and the wireless communication network communication buffer queue respectively; The monitoring data in the DC power line carrier cache queue and the wireless communication network cache queue can be transmitted via the DC power line and the wireless communication network respectively, and the transmission model is obtained according to the Shannon formula; The transmission model includes a DC power line carrier transmission model and a wireless communication network transmission model, wherein: The DC power line carrier transmission model is expressed in the following way: in, Is the device u m Signal-to-noise ratio for data transmission using DC power line subcarrier n; Is the device u m The transmission rate of data transmission using DC power line subcarrier n; Is the device u m Total data transmission rate using DC power line carrier; Is the device u m The transmission power required to transmit data using the DC power line subcarrier n; is the power gain factor of the DC power line subcarrier n; N0 is the white noise power spectrum density; B L is the total bandwidth of the DC power line carrier; N L is the total number of DC power line subcarriers; Γ L is the signal-to-noise ratio gap between different DC power line subcarriers; M L is the number of devices using DC power line carrier for task transmission; The wireless communication network transmission model is represented by the following method: in, Is the device u m The signal-to-noise ratio of data transmission using subcarrier n of a wireless communication network; Is the device u m The transmission rate of data transmission using subcarrier n of the wireless communication network; Is the device u m The overall data transmission rate using the wireless communication network; Is the device u m The transmission power used for transmitting data using subcarrier n of the wireless communication network; is the power gain factor of subcarrier n in the wireless communication network; N0 is the white noise power spectrum density; B W is the total bandwidth of the channel transmitted in the form of a wireless communication network; e is the electromagnetic interference power; N W is the total number of subcarriers in the wireless communication network; Γ W is the signal-to-noise ratio gap between subcarriers in different wireless communication networks; M W is the number of devices that use wireless communication networks for data transmission; device u m The amount of tasks unloaded to the edge server using DC power line carrier and wireless communication network communication are: Where τ is the time slot length; The device queue delay model is: in is the DC power line carrier buffer queue delay, is the wireless communication network cache queue delay, and the device queue delay is the maximum value of the device DC power line carrier cache queue delay and the wireless communication network cache queue delay; The specific steps of the device queue long-term delay constraint and extreme event probability constraint are as follows: Connect the device m The extreme events of the DC power line carrier buffer queue and the wireless communication network buffer queue are expressed as: The probability of occurrence of the above two extreme events is constrained as follows: in, and They represent the extreme events that occur in the DC power line carrier buffer queue and the wireless communication network buffer queue, and They are the queue backlogs of the DC power line carrier cache queue and the wireless communication network cache queue at the current moment, and are the average arrival rates of tasks in the DC power line carrier cache queue and the wireless communication network cache queue, and They represent the long-term queue delay thresholds for data transmission using DC power line carrier and wireless communication network, respectively. and They represent the thresholds of the tolerable probability of extreme events under the DC power line carrier and wireless communication network communication modes respectively; T is the total number of time slots, that is, the total optimized time length; The specific method for designing the optimization problem based on the equipment queue time extension constraint and the extreme event occurrence probability constraint is as follows: The optimization problem P1 is set as: Where M is the number of devices, T is the total optimization time, and τ m (t) is the device queue delay, a m,min Is the device u m The minimum number of tasks allowed in each time slot, is the actual task admission, A m (t) is the device u m The amount of tasks generated in time slot t, ρ m (t) is the device traffic adaptation ratio, which indicates the ratio of tasks stored in the DC power line carrier cache queue to the actual access tasks. and They are the queue backlogs of the DC power line carrier cache queue and the wireless communication network cache queue at the current moment, and are the average arrival rates of tasks in the DC power line carrier cache queue and the wireless communication network cache queue, and They represent the long-term queue delay thresholds for data transmission using DC power line carrier and wireless communication network, respectively. and They represent the thresholds of the tolerable probability of extreme events under the DC power line carrier and wireless communication network communication modes respectively; Constraint C1 is the device task admission constraint, constraint C2 is the device traffic adaptation ratio constraint, constraint C3 is the long-term queue delay constraint of the DC power line carrier cache queue and the wireless communication network cache queue, and constraint C4 is the long-term extreme event probability constraint of the DC power line carrier cache queue and the wireless communication network cache queue; The specific content of converting the optimization problem into minimizing the device queue delay and converting the optimization problem into an upper bound problem is: Transform the optimization problem P1 into the optimization problem P2: Where M is the number of devices, T is the total optimization time, and τ m (t) is the device queue delay, and They represent the virtual queue backlog of the DC power line carrier at the current moment and the next moment after the long-term queue delay constraint of the DC power line carrier cache queue is transformed using the virtual queue theory, and They represent the current and next moment wireless communication network virtual queue backlogs after the long-term queue delay constraints of the wireless communication network cache queue are transformed using the virtual queue theory, and They are the queue backlogs of the DC power line carrier cache queue and the wireless communication network cache queue at the current moment, and are the average arrival rates of tasks in the DC power line carrier cache queue and the wireless communication network cache queue, and They represent the long-term queue delay thresholds for data transmission using DC power line carrier and wireless communication network, respectively. and They represent the virtual queue backlog of the DC power line carrier at the current moment and the next moment after the long-term extreme event probability constraint of the DC power line carrier cache queue is transformed using the virtual queue theory, and They represent the virtual queue backlog of the wireless communication network at the current moment and the next moment after the probability constraint of long-term extreme events in the wireless communication network cache queue is transformed using the virtual queue theory, and II{Ω L } and II{Ω W } respectively represent the indicator functions of the extreme events of the DC power line carrier cache queue and the wireless communication network cache queue, II{Ω L }=1 indicates that an extreme event occurs in the DC power line carrier buffer queue, II{Ω L }=0 means that no extreme event occurs in the DC power line carrier cache queue, II{Ω W }=1 indicates that an extreme event occurs in the wireless communication network cache queue, II{Ω W }=0 means that no extreme event occurs in the wireless communication network cache queue. and They represent the thresholds of the tolerable probability of extreme events under the DC power line carrier and wireless communication network communication modes respectively; Constraint C1 is the equipment task access constraint, and the formula is Constraint C2 is the device flow adaptation ratio constraint, the formula is 0≤ρ m (t)≤1; P1 is the optimization problem P1 defined in the above steps; The amount of data allowed to the device; a m,min is the minimum amount of data allowed; A m (t) is the amount of data collected by the device; ρ m (t) is the device flow adaptation ratio; The optimization problem P2 is transformed into minimizing the upper bound of "drift + penalty" under the premise of satisfying constraints C1 and C2, and the constraint C5 in P2 is transformed into the objective function of P3, resulting in the optimization problem P3: in, and They are the queue backlogs of the DC power line carrier cache queue and the wireless communication network cache queue at the current moment, and They represent the amount of tasks that the device offloads to the edge server through the DC power line carrier and the wireless communication network communication mode, respectively. and They represent the data arrival amount of the DC power line carrier cache queue and the wireless communication network communication cache queue respectively, and They represent the backlog of the DC power line carrier virtual queue and the wireless communication network cache queue at the current moment after the long-term queue delay constraints of the DC power line carrier cache queue and the wireless communication network cache queue are transformed using the virtual queue theory, and are the average arrival rates of tasks in the DC power line carrier cache queue and the wireless communication network cache queue, and They represent the long-term queue delay thresholds for data transmission using DC power line carrier and wireless communication network, respectively. and They represent the backlog of the DC power line carrier virtual queue and the wireless communication network virtual queue at the current moment after the long-term extreme event probability constraints of the DC power line carrier cache queue and the wireless communication network cache queue are transformed using the virtual queue theory, and They represent the thresholds of the probability of extreme events that can be tolerated under the DC power line carrier and wireless communication network communication modes, and II{Ω L } and II{Ω W } respectively represent the indicator functions of the extreme events of the DC power line carrier cache queue and the wireless communication network cache queue, II{Ω L }=1 indicates that an extreme event occurs in the DC power line carrier buffer queue, II{Ω L }=0 means that no extreme event occurs in the DC power line carrier cache queue, II{Ω W }=1 indicates that an extreme event occurs in the wireless communication network cache queue, II{Ω W }=0 indicates that no extreme event occurs in the wireless communication network cache queue; Constraint C1 is the equipment task access constraint, and the formula is Constraint C2 is the device flow adaptation ratio constraint, the formula is 0≤ρ m (t)≤1; The specific contents of the system state, action and reward function are: 1) System status S m (t) includes the status information of each queue, and the vector representation of the system status is: 2) System action am(t) is device u m According to the system status S m (t) The task admission control and traffic adaptation decisions taken, represented by the device u in each time slot m The number of tasks allowed and flow adaptation ratio ρ m (t); and ρ m (t) are all continuously changing physical quantities. To simplify the analysis process, and ρ m (t) are discretized into K levels, expressed as ρ m (t)∈ρ, where ρ=[ρ1,ρ2,…,ρ K ]; therefore, the system action a m (t) is expressed as: 3) The design of the reward function should be related to the optimization problem. The goal of the optimization problem P3 is to minimize β(t), so the reward function here is R(t) = -β(t); The specific steps of the joint optimization algorithm of task admission and traffic adaptation based on deep Q network are as follows: Initialize the estimated network parameters ω eval (t), target network parameters ω target (t), loss function φ and experience pool P; The device adopts the ε-greedy strategy, making random task admission and traffic adaptation decisions based on the estimated Q value with probability ε, and making task admission and traffic adaptation decisions based on the Q value obtained from the valuation network with probability 1-ε. Specifically: Among them, μ∈(0,1) is a random number generated by the system; The device determines the amount of data entering the device queue, the amount of data entering the DC power line carrier cache queue, and the amount of data entering the wireless communication network cache queue according to the task admission and traffic adaptation decisions made, and unloads the data in the DC power line carrier cache queue and the wireless communication network cache queue to the edge server respectively; After each unloading, the reward function R(t) is calculated and updated Each queue information; then, transfer to the next state S m (t+1), at the same time, S m (t), a m (t), R(t) and S m (t+1) Deposit into experience pool P; The device randomly extracts a set of experience data from the experience pool P as training data for the valuation network, and calculates the loss function based on the above training data in, Represents the target Q value obtained by the target network; Then, based on the loss function, the estimated network parameters ω are updated using the gradient descent method. eval , Among them, λ is the learning rate; finally, every T0 time, update the target network parameter ω target (t+1)=ω eval (t+1); Where S m (t) represents the state of device m, a m (t) represents the action of device m, "S(t)", "a(t)", "S(t+1)", "a(t+1)" represent the state and action of the tth time slot and the state and action of the t+1th time slot respectively, and the subscript m specifically refers to the state or action of device m; ω eval Denotes the estimated network parameter ω eval (t);ω target represents the target network parameter ω eval (t).

2. A multi-mode networking device integrating a wireless communication network and a DC power line carrier, characterized in that: Specifically include: Modeling module: used to model the monitoring data in the pre-built multi-modal heterogeneous networking system model into a device queue model, determine the transmission model of the monitoring data according to the device queue model, and obtain the device queue delay model; Constraint module: used to impose constraints on the device queue delay period and the probability of occurrence of extreme events on the device queue delay model; Conversion module: used to convert the optimization problem into minimizing the device queue delay based on the device queue delay period constraint and the extreme event probability constraint; and use the virtual queue theory and Lyapunov theory to design the transformation of the optimization problem and the system state, action and reward function, and convert the optimization problem into solving the upper bound problem of Lyapunov's "drift + penalty"; for the upper bound problem, the task admission and traffic adaptation joint optimization algorithm based on the deep Q network is optimized and determined to integrate the multimodal communication network of the DC power line carrier; There are M distribution network monitoring devices in the pre-built multi-modal heterogeneous networking system model, and the set is u = {u1, ..., u m ,…,u M }, and each device has two task cache queues, namely, a DC power line carrier cache queue and a wireless communication network cache queue, which respectively store tasks transmitted by DC power line carrier and wireless communication network communication; The device queue model is a DC power line carrier cache queue model and a wireless communication network cache queue model, specifically: in, and They are respectively represented as the queue backlog at the current moment and the next moment in the DC power line carrier cache queue; and They are respectively represented as queue backlogs at the current moment and the next moment in the wireless communication network cache queue; and They represent the amount of tasks that the device offloads to the edge server through the DC power line carrier and wireless communication network communication respectively; and They represent the data arrival amount of the DC power line carrier buffer queue and the wireless communication network communication buffer queue respectively; The monitoring data in the DC power line carrier cache queue and the wireless communication network cache queue can be transmitted via the DC power line and the wireless communication network respectively, and the transmission model is obtained according to the Shannon formula; The transmission model includes a DC power line carrier transmission model and a wireless communication network transmission model, wherein: The DC power line carrier transmission model is expressed in the following way: in, Is the device u m Signal-to-noise ratio for data transmission using DC power line subcarrier n; Is the device u m The transmission rate of data transmission using DC power line subcarrier n; Is the device u m Total data transmission rate using DC power line carrier; Is the device u m The transmission power required to transmit data using the DC power line subcarrier n; is the power gain factor of the DC power line subcarrier n; N0 is the white noise power spectrum density; B L is the total bandwidth of the DC power line carrier; N L is the total number of DC power line subcarriers; Γ L is the signal-to-noise ratio gap between different DC power line subcarriers; M L is the number of devices using DC power line carrier for task transmission; The wireless communication network transmission model is represented by the following method: in, Is the device u m The signal-to-noise ratio of data transmission using subcarrier n of a wireless communication network; Is the device u m The transmission rate of data transmission using subcarrier n of the wireless communication network; Is the device u m The overall data transmission rate using the wireless communication network; Is the device u m The transmission power used for transmitting data using subcarrier n of the wireless communication network; is the power gain factor of subcarrier n in the wireless communication network; N0 is the white noise power spectrum density; B W is the total bandwidth of the channel transmitted in the form of a wireless communication network; e is the electromagnetic interference power; N W is the total number of subcarriers in the wireless communication network; Γ W is the signal-to-noise ratio gap between subcarriers in different wireless communication networks; M W is the number of devices using wireless communication networks for data transmission; device u m The amount of tasks unloaded to the edge server using DC power line carrier and wireless communication network communication are: Where τ is the time slot length; The device queue delay model is: in is the DC power line carrier buffer queue delay, is the wireless communication network cache queue delay, and the device queue delay is the maximum value of the device DC power line carrier cache queue delay and the wireless communication network cache queue delay; The specific steps of the device queue long-term delay constraint and extreme event probability constraint are as follows: Connect the device m The extreme events of the DC power line carrier buffer queue and the wireless communication network buffer queue are expressed as: The probability of occurrence of the above two extreme events is constrained as follows: in, and They represent the extreme events that occur in the DC power line carrier buffer queue and the wireless communication network buffer queue, and They are the queue backlogs of the DC power line carrier cache queue and the wireless communication network cache queue at the current moment, and are the average arrival rates of tasks in the DC power line carrier cache queue and the wireless communication network cache queue, and They represent the long-term queue delay thresholds for data transmission using DC power line carrier and wireless communication network, respectively. and They represent the thresholds of the tolerable probability of extreme events under the DC power line carrier and wireless communication network communication modes respectively; T is the total number of time slots, that is, the total optimized time length; The specific method for designing the optimization problem based on the equipment queue time extension constraint and the extreme event occurrence probability constraint is as follows: The optimization problem P1 is set as: Where M is the number of devices, T is the total optimization time, and τ m (t) is the device queue delay, a m,min Is the device u m The minimum number of tasks allowed in each time slot, is the actual task admission, A m (t) is the device u m The amount of tasks generated in time slot t, ρ m (t) is the device traffic adaptation ratio, which indicates the ratio of tasks stored in the DC power line carrier cache queue to the actual access tasks. and They are the queue backlogs of the DC power line carrier cache queue and the wireless communication network cache queue at the current moment, and are the average arrival rates of tasks in the DC power line carrier cache queue and the wireless communication network cache queue, and They represent the long-term queue delay thresholds for data transmission using DC power line carrier and wireless communication network, respectively. and They represent the thresholds of the tolerable probability of extreme events under the DC power line carrier and wireless communication network communication modes respectively; Constraint C1 is the device task admission constraint, constraint C2 is the device traffic adaptation ratio constraint, constraint C3 is the long-term queue delay constraint of the DC power line carrier cache queue and the wireless communication network cache queue, and constraint C4 is the long-term extreme event probability constraint of the DC power line carrier cache queue and the wireless communication network cache queue; The specific content of converting the optimization problem into minimizing the device queue delay and converting the optimization problem into an upper bound problem is: Transform the optimization problem P1 into the optimization problem P2: Where M is the number of devices, T is the total optimization time, and τ m (t) is the device queue delay, and They represent the virtual queue backlog of the DC power line carrier at the current moment and the next moment after the long-term queue delay constraint of the DC power line carrier cache queue is transformed using the virtual queue theory, and They represent the current and next moment wireless communication network virtual queue backlogs after the long-term queue delay constraints of the wireless communication network cache queue are transformed using the virtual queue theory, and They are the queue backlogs of the DC power line carrier cache queue and the wireless communication network cache queue at the current moment, and are the average arrival rates of tasks in the DC power line carrier cache queue and the wireless communication network cache queue, and They represent the long-term queue delay thresholds for data transmission using DC power line carrier and wireless communication network, respectively. and They represent the virtual queue backlog of the DC power line carrier at the current moment and the next moment after the long-term extreme event probability constraint of the DC power line carrier cache queue is transformed using the virtual queue theory, and They represent the virtual queue backlog of the wireless communication network at the current moment and the next moment after the probability constraint of long-term extreme events in the wireless communication network cache queue is transformed using the virtual queue theory, and II{Ω L } and II{Ω W } respectively represent the indicator functions of the extreme events of the DC power line carrier cache queue and the wireless communication network cache queue, II{Ω L }=1 indicates that an extreme event occurs in the DC power line carrier buffer queue, II{Ω L }=0 means that no extreme event occurs in the DC power line carrier cache queue, II{Ω W }=1 indicates that an extreme event occurs in the wireless communication network cache queue, II{Ω W }=0 means that no extreme event occurs in the wireless communication network cache queue. and They represent the thresholds of the tolerable probability of extreme events under the DC power line carrier and wireless communication network communication modes respectively; Constraint C1 is the equipment task access constraint, and the formula is: Constraint C2 is the device flow adaptation ratio constraint, the formula is 0≤ρ m (t)≤1; P1 is the optimization problem P1 defined in the above steps; The amount of data allowed to the device; a m,min is the minimum amount of data allowed; A m (t) is the amount of data collected by the device; ρ m (t) is the device flow adaptation ratio; The optimization problem P2 is transformed into minimizing the upper bound of "drift + penalty" under the premise of satisfying constraints C1 and C2, and the constraint C5 in P2 is transformed into the objective function of P3, resulting in the optimization problem P3: in, and They are the queue backlogs of the DC power line carrier cache queue and the wireless communication network cache queue at the current moment, and They represent the amount of tasks that the device offloads to the edge server through the DC power line carrier and the wireless communication network communication mode, respectively. and They represent the data arrival amount of the DC power line carrier cache queue and the wireless communication network communication cache queue respectively, and They represent the backlog of the DC power line carrier virtual queue and the wireless communication network cache queue at the current moment after the long-term queue delay constraints of the DC power line carrier cache queue and the wireless communication network cache queue are transformed using the virtual queue theory, and are the average arrival rates of tasks in the DC power line carrier cache queue and the wireless communication network cache queue, and They represent the long-term queue delay thresholds for data transmission using DC power line carrier and wireless communication network, respectively. and They represent the backlog of the DC power line carrier virtual queue and the wireless communication network virtual queue at the current moment after the long-term extreme event probability constraints of the DC power line carrier cache queue and the wireless communication network cache queue are transformed using the virtual queue theory, and They represent the thresholds of the probability of extreme events that can be tolerated under the DC power line carrier and wireless communication network communication modes, and II{Ω L } and II{Ω W } respectively represent the indicator functions of the extreme events of the DC power line carrier cache queue and the wireless communication network cache queue, II{Ω L }=1 indicates that an extreme event occurs in the DC power line carrier buffer queue, II{Ω L }=0 means that no extreme event occurs in the DC power line carrier cache queue, II{Ω W }=1 indicates that an extreme event occurs in the wireless communication network cache queue, II{Ω W }=0 indicates that no extreme event occurs in the wireless communication network cache queue; Constraint C1 is the equipment task access constraint, and the formula is: Constraint C2 is the device flow adaptation ratio constraint, the formula is 0≤ρ m (t)≤1; The specific contents of the system state, action and reward function are: 1) System status S m (t) includes the status information of each queue, and the vector representation of the system status is: 2) System action am(t) is device u m According to the system status S m (t) The task admission control and traffic adaptation decisions taken, represented by the device u in each time slot m The number of tasks allowed and flow adaptation ratio ρ m (t); and ρ m (t) are all continuously changing physical quantities. To simplify the analysis process, and ρ m (t) are discretized into K levels, expressed as ρ m (t)∈ρ, where ρ=[ρ1,ρ2,…,ρ K ]; therefore, the system action am(t) is expressed as: 3) The design of the reward function should be related to the optimization problem. The goal of the optimization problem P3 is to minimize β(t), so the reward function here is R(t) = -β(t); The specific steps of the joint optimization algorithm of task admission and traffic adaptation based on deep Q network are as follows: Initialize the estimated network parameters ω eval (t), target network parameters ω target (t), loss function φ and experience pool P; The device adopts the ε-greedy strategy, making random task admission and traffic adaptation decisions based on the estimated Q value with probability ε, and making task admission and traffic adaptation decisions based on the Q value obtained from the valuation network with probability 1-ε. Specifically: Among them, μ∈(0,1) is a random number generated by the system; The device determines the amount of data entering the device queue, the amount of data entering the DC power line carrier cache queue, and the amount of data entering the wireless communication network cache queue according to the task admission and traffic adaptation decisions made, and unloads the data in the DC power line carrier cache queue and the wireless communication network cache queue to the edge server respectively; After each unloading, the reward function R(t) is calculated and updated Each queue information; then, transfer to the next state S m (t+1), at the same time, S m (t), a m (t), R(t) and S m (t+1) Deposit into experience pool P; The device randomly extracts a set of experience data from the experience pool P as training data for the valuation network, and calculates the loss function based on the above training data in, Represents the target Q value obtained by the target network; Then, based on the loss function, the estimated network parameters ω are updated using the gradient descent method. eval , Among them, λ is the learning rate; finally, every T0 time, update the target network parameter ω target (t+1)=ω eval (t+1); Where S m (t) represents the state of device m, a m (t) represents the action of device m, "S(t)", "a(t)", "S(t+1)", "a(t+1)" represent the state and action of the tth time slot and the state and action of the t+1th time slot respectively, and the subscript m specifically refers to the state or action of device m; ω eval Denotes the estimated network parameter ω eval (t);ω target represents the target network parameter ω eval (t).

3. A multi-mode networking device integrating a wireless communication network and a DC power line carrier according to claim 2, characterized in that: There are M distribution network monitoring devices in the modeling module, and their set is u={u1,…,u m ,…,u M }, and each device has two task cache queues, namely a DC power line carrier cache queue and a wireless communication network cache queue, which respectively store tasks transmitted using DC power line carrier and wireless communication network communication.

4. A multi-mode networking device integrating a wireless communication network and a DC power line carrier according to claim 3, characterized in that: The constraint module is used to set the device u m The extreme events of the DC power line carrier buffer queue and the wireless communication network buffer queue are expressed as: The probability of occurrence of the above two extreme events is constrained as follows: in, and They represent the thresholds of the tolerable probability of extreme events under DC power line carrier and wireless communication network communication modes respectively.

Images