An intelligent hierarchical time synchronization method and system for low-carbon park clusters

Abstract

The present invention relates to an intelligent hierarchical time synchronization method and system for low-carbon park clusters, and belongs to the field of communication technology. The present invention establishes a time synchronization routing selection problem model for joint optimization of synchronization delay and synchronization error under signal interference and noise ratio constraints, takes the weighted sum of synchronization delay and synchronization error as the optimization target, ensures the reliability of time synchronization data packet transmission through signal interference and noise ratio constraints, and realizes dynamic trade-off between synchronization delay and synchronization error performance by adjusting the optimization target weight, meets the time synchronization requirements of differentiated regulation services and low-carbon services, and supports the distributed resource consistency response of low-carbon park clusters.

Description

Technical Field

[0001] The invention relates to an intelligent hierarchical time synchronization method and system for a low-carbon park group, belonging to the technical field of communication. Background Art

[0002] At present, low-carbon smart park clusters cover multiple energy entities such as a high proportion of renewable energy, controllable loads and distributed energy storage, and are key pillars for achieving carbon emission reduction, carbon neutrality, and sustainable development. However, the output power of distributed resources is highly intermittent and random, which further causes problems such as voltage fluctuations and power quality degradation. At the same time, the frequent switching of distributed generators will also generate harmonic pollution, seriously affecting the stability of the power grid. Therefore, it is necessary to coordinate and regulate distributed resources in a unified manner to achieve energy supply and demand balance within and between smart parks and promote the consumption of new energy.

[0003] In order to support the intelligent and precise control of distributed resources in smart park clusters and ensure the efficient operation of low-carbon businesses such as carbon footprint monitoring, electricity-carbon fusion calculation, and carbon trading, a large number of power Internet of Things terminals need to be deployed on photovoltaic panels, air conditioners, charging piles and other equipment to collect various parameters such as voltage, current, active / reactive power, and greenhouse gas emission data such as CO2 and CH4, and transmit the data back to the control center or management platform through power line communication (PLC) for real-time data processing and in-depth analysis. In this process, the information collection, real-time control, energy trading, and carbon trading of distributed resource control and low-carbon business operation in smart park clusters all require strict time synchronization as a prerequisite, that is, the communication terminal receives the standard synchronization signal to calibrate the local clock, and further provides time for electrical equipment to ensure error-free coordinated scheduling of distributed resources, avoid the problem of inconsistent response of distributed resource control caused by out-of-step oscillation of the power grid and asynchronous operation, and realize carbon footprint monitoring and accurate carbon emission prediction of electrical equipment, support carbon trading and carbon transfer, and reduce carbon emissions. However, the research on time synchronization methods for low-carbon smart park clusters is still in its infancy, and the following challenges need to be solved:

[0004] First, distributed resources such as photovoltaics, electric vehicle charging piles, and energy storage units are widely distributed in low-carbon smart park clusters. Distributed resource control services and low-carbon services such as precise load control, carbon footprint monitoring, and energy complementarity require wide-area time synchronization coverage of massive communication terminals and electrical equipment within and between parks.

[0005] Second, time synchronization performance indicators include synchronization error, synchronization delay, and synchronization signal transmission reliability, but multiple performance indicators are mutually contradictory. For example, some timing terminals have small time synchronization errors, but the channel conditions between them and the terminals to be synchronized are poor. Using these timing terminals for timing may result in long synchronization delays, packet loss, etc.

[0006] Third, considering the large amount of signaling overhead, dynamic environmental information such as channel quality and electromagnetic interference cannot be known to the PLC gateway in real time. Therefore, the routing selection strategy for time synchronization data packet transmission needs to be optimized under the condition of uncertain information. Traditional optimization methods based on complete information or specific statistical models are no longer applicable.

[0007] The existing technology only considers a single performance indicator when optimizing the routing selection strategy for time synchronization data packet transmission, but does not consider the trade-off of multiple performance indicators, nor does it consider the high-dimensional characteristics of routing strategy optimization in the highly complex dynamic environment of low-carbon smart park clusters, and cannot guarantee the time synchronization accuracy of the terminals to be timed. Therefore, it is urgent to design an intelligent hierarchical time synchronization method and system for low-carbon park clusters to realize hierarchical time synchronization of smart park clusters from the gateway layer to the terminal layer. Summary of the invention

[0008] In order to solve the above technical problems, the purpose of the present invention is to provide an intelligent hierarchical time synchronization method and system for low-carbon park clusters.

[0009] The present invention provides an intelligent hierarchical time synchronization method for a low-carbon park group, and the specific synchronization steps are as follows:

[0010] Obtain the time synchronization signal of the clock source layer, adjust the time synchronization signal of the gateway layer, and obtain the precise time synchronization signal of the gateway layer;

[0011] Transmitting the gateway layer precise time synchronization signal to the terminal layer, establishing a synchronization data packet transmission model, a synchronization delay model, and a cumulative synchronization error model;

[0012] Based on the signal-to-interference-noise ratio, synchronization delay model, and cumulative synchronization error model in the synchronization data packet transmission model, a synchronization data packet routing strategy optimization problem is modeled to obtain a synchronization data packet routing strategy optimization problem model;

[0013] The above synchronization data packet routing strategy optimization problem model is transformed into a Markov optimization problem. A high-precision and low-latency time synchronization algorithm based on DQN is proposed. The transformed Markov optimization problem is solved, and the hierarchical time synchronization is determined according to the solution results.

[0014] Furthermore, the gateway layer precise time synchronization signal is:

[0015] The gateway receives time synchronization signals from clock source layer satellites, 5G time synchronization networks, and the power grid company's self-built time synchronization networks, and generates precise time synchronization signals on the gateway side based on multi-time source selection or comprehensive calculation strategies.

[0016] Furthermore, the synchronous data packet transmission model is specifically as follows:

[0017] At the i-th iteration d m' From d m The rate at which synchronous packets are received is:

[0018]

[0019] Among them, N s is the OFDM symbol transmission rate, is the signal-to-interference-noise ratio, expressed as:

[0020]

[0021] in, is the transmission power, It is expressed as:

[0022]

[0023] in, and δ0 are link frequency response, electromagnetic interference and Gaussian white noise power respectively; is the SINR gap, which indicates the ability to resist signal-to-interference-noise ratio fading. The calculation formula is as follows:

[0024]

[0025] Among them, P e represents the target bit error rate, Q -1 (x) indicates The inverse function of .

[0026] Furthermore, the synchronization delay model is:

[0027] Divide each iteration into several steps of length τ s time slots, at the beginning of each time slot, the synchronized terminal d m Start sending data to the connected unsynchronized terminal d m' Distribute synchronization packets; in multi-hop transmission, the i-th iteration d m' The delay in receiving the synchronization packet is given by d m The delay in receiving the synchronization packet, and d m' From d m The single-hop transmission delay of receiving synchronous data packets consists of two parts, expressed as:

[0028]

[0029] in, Indicates d m The number of time slots during which the synchronization data packet is received, is dm' From d m The delay of receiving a data packet is expressed as:

[0030]

[0031] Wherein, U(i) represents the size of the synchronization data packet of the i-th iteration.

[0032] Furthermore, the cumulative synchronization error model is:

[0033] Iteration d m' The cumulative synchronization error of receiving synchronization data packets is It is expressed as:

[0034]

[0035] in, Select indicator variables for routing, represents the i-th iteration d m' From d m Receive synchronization packets, otherwise is the i-th iteration d m' From d m The accumulated synchronization error of the received data packet is expressed as:

[0036]

[0037] in, represents the i-th iteration d m' From d m The time synchronization error formed by receiving synchronization data packets obeys the normal distribution with an error mean of α and a standard deviation of β, which can be expressed as is the i-th iteration d m The accumulated synchronization error of received synchronization packets.

[0038] Furthermore, the model for modeling the routing strategy optimization problem of the synchronization data packet is:

[0039]

[0040] Among them, χ is a weight parameter used to dynamically balance synchronization delay and synchronization error; C1 and C2 are routing constraints, indicating that the receiver d m' Can receive synchronous data packets from at most one sender; C3 means d m' With d m When not connected, d m' Unable to m Receive data packets; C4 is the SINR constraint, is the minimum SINR threshold to ensure the reliability of synchronous data packet transmission.

[0041] Furthermore, the Markov optimization problem includes a state space, an action space and a reward function, wherein:

[0042] The state space is the size of the synchronization data packet of the i-th iteration and the topological connection relationship between terminals, which can be expressed as:

[0043]

[0044] Where U(i) represents the size of the synchronization data packet of the i-th iteration; It indicates the topological connection variables between the PLC gateway and each terminal. Indicates d m With d m' Connect, otherwise

[0045] The action space is represented by a set of path selection strategies, expressed as:

[0046]

[0047] in, Select indicator variables for routing, represents the i-th iteration d m' From d m Receive synchronization packets, otherwise

[0048] The reward function is the negative value of the weighted sum of the terminal synchronization delay and synchronization error, expressed as:

[0049]

[0050] in, Iteration d m' From d m The synchronization delay of receiving synchronization data packets; represents the i-th iteration d m' From d m Cumulative synchronization error of received packets.

[0051] Furthermore, the high-precision and low-latency time synchronization algorithm based on DQN is:

[0052] (1) Initialization phase: Initialize and evaluate network weights Target network weights And experience pool

[0053] (2) Routing selection phase: In each iteration, at the beginning of each time slot, the PLC gateway makes a routing decision for the time synchronization packet based on the ε-greedy algorithm; the PLC gateway explores and randomly selects an action with probability ε, and uses the action with the maximum estimated Q value with probability 1-ε.

[0054] (3) Reward calculation and state transition phase: The unsynchronized terminal receives synchronization data packets from the synchronized terminal or PLC gateway according to the routing selection decision, and feedbacks the time synchronization delay and synchronization error; then, the PLC gateway calculates the reward function based on the information fed back by the terminal. The current state Transition to next state And the empirical data Deposit into experience pool

[0055] (4) Learning and network update phase: At the end of each iteration, the PLC gateway extracts information from the experience pool. Randomly select a sample set Calculate the loss function based on the above samples in, The target estimated Q value obtained for the target network is expressed as:

[0056]

[0057] Among them, γ is the discount factor; based on the loss function Use gradient descent to update the evaluation network weights

[0058]

[0059] Where λ represents the learning rate; then, every I Gap iterations, according to Update target network parameters And in the next I Gap -1 iteration remains unchanged;

[0060] Finally, repeat steps (2) to (4) above until the first iteration is completed.

[0061] An intelligent hierarchical time synchronization system for low-carbon park clusters, characterized by comprising a clock source layer, a gateway layer, a terminal layer and an intelligent control layer;

[0062] The clock source layer is used to obtain the clock source layer time synchronization signal, adjust the gateway layer time synchronization signal, and obtain the gateway layer accurate time synchronization signal;

[0063] The gateway layer is used to transmit the precise time synchronization signal of the gateway layer to the terminal layer, and establish a synchronization data packet transmission model, a synchronization delay model, and a cumulative synchronization error model;

[0064] The terminal layer is used to model the routing strategy optimization problem of the synchronous data packet based on the signal-to-interference-noise ratio, the synchronization delay model, and the cumulative synchronization error model in the synchronous data packet transmission model to obtain the routing strategy optimization problem model of the synchronous data packet;

[0065] The intelligent control layer is used to transform the synchronization data packet routing strategy optimization problem model into a Markov optimization problem, and solve the transformed Markov optimization problem based on the high-precision and low-latency time synchronization algorithm of DQN, and determine the hierarchical time synchronization according to the solution result. An intelligent hierarchical time synchronization device for low-carbon park clusters includes:

[0066] Modeling module: used to obtain synchronization data packet transmission model, synchronization delay model, and cumulative synchronization error model;

[0067] Optimization modeling module: used to model the optimization problem of synchronous data packet routing strategy based on the signal-to-interference-noise ratio, synchronization delay model, and cumulative synchronization error model in the synchronous data packet transmission model, and obtain the synchronous data packet routing strategy optimization problem model;

[0068] Conversion calculation module: used to convert the above-mentioned synchronization data packet routing selection strategy optimization problem model into a Markov optimization problem, propose a high-precision and low-latency time synchronization algorithm based on deep Q learning DQN, solve the converted Markov optimization problem, and determine the hierarchical time synchronization based on the solution results.

[0069] Furthermore, the model used to model the routing strategy optimization problem of the synchronous data packet in the optimization modeling module is:

[0070]

[0071] Among them, χ is a weight parameter used to dynamically balance synchronization delay and synchronization error; C1 and C2 are routing constraints, indicating that the receiver d m' Can receive synchronous data packets from at most one sender; C3 means d m' With d m When not connected, d m' Unable to m Receive data packets; C4 is the SINR constraint, is the minimum SINR threshold to ensure the reliability of synchronous data packet transmission.

[0072] Furthermore, the conversion calculation module is specifically used to convert the synchronous data packet routing strategy optimization problem model into a Markov optimization problem and calculate and solve it, wherein the Markov optimization problem includes a state space, an action space and a reward function, wherein:

[0073] The state space is the size of the synchronization data packet of the i-th iteration and the topological connection relationship between terminals, which can be expressed as:

[0074]

[0075] Where U(i) represents the size of the synchronization data packet of the i-th iteration; It indicates the topological connection variables between the PLC gateway and each terminal. Indicates d m With d m' Connect, otherwise

[0076] The action space is represented by a set of path selection strategies, expressed as:

[0077]

[0078] in, Select indicator variables for routing, represents the i-th iteration d m' From d m Receive synchronization packets, otherwise

[0079] The reward function is the negative value of the weighted sum of all terminal synchronization delays and synchronization errors, expressed as:

[0080]

[0081] in, The i-th iteration d m' From d m The total synchronization delay of receiving synchronization data packets; represents the i-th iteration d m' From d m Cumulative synchronization error of received data packets

[0082] By means of the above scheme, the present invention has at least the following advantages:

[0083] 1. The present invention establishes a time synchronization routing selection problem model for joint optimization of synchronization delay and synchronization error under signal interference and noise ratio constraints, takes the weighted sum of synchronization delay and synchronization error as the optimization target, ensures the reliability of time synchronization data packet transmission through signal interference and noise ratio constraints, and achieves dynamic trade-off between synchronization delay and synchronization error performance by adjusting the optimization target weights, thereby meeting the time synchronization requirements of differentiated regulation services and low-carbon services, and supporting the consistent response of distributed resources in low-carbon park clusters.

[0084] 2. The present invention proposes a high-precision and low-latency time synchronization algorithm based on DQN, uses DQN to optimize the time synchronization routing strategy, and estimates the value of routing actions under the condition of only local information by integrating the intelligent decision-making ability of Q learning and the complex function approximation ability of deep neural networks. Routing decisions are dynamically optimized through continuous interaction with the environment, thus realizing intelligent time synchronization routing under information uncertainty.

[0085] 3. The present invention proposes an intelligent hierarchical time synchronization system for low-carbon smart park clusters, including a clock source layer, a gateway layer, a terminal layer and an intelligent control layer. Empowered by the intelligent control layer, the PLC gateways of each smart park achieve time synchronization of the smart park cluster gateways by selecting or comprehensively calculating multiple clock sources; at the same time, based on the PLC network topology within the park, the PLC gateway executes the proposed DQN-based high-precision and low-latency time synchronization algorithm, and transmits the time synchronization data packets in layers to each terminal through routing selection optimization, ultimately achieving wide-area time synchronization hierarchical coverage of low-carbon smart park cluster gateways and terminals.

[0086] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention and implement it according to the contents of the specification, the following is a detailed description of the preferred embodiments of the present invention in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0087] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for use in the embodiments are briefly introduced below. It should be understood that the following drawings only show a certain embodiment of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without creative work.

[0088] Figure 1 It is a schematic diagram of the intelligent hierarchical time synchronization system for low-carbon park groups of the present invention;

[0089] Figure 2 It is a flow chart of the high-precision and low-latency time synchronization algorithm based on DQN of the present invention;

[0090] Figure 3It is a line graph showing the average synchronization delay of the present invention changing with the number of iterations;

[0091] Figure 4 It is a line graph showing the average synchronization error of the present invention changing with the number of iterations. DETAILED DESCRIPTION

[0092] The specific implementation of the present invention is further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.

[0093] The entire technical solution consists of two parts: the overall system and the specific method.

[0094] 1. Overall system

[0095] like Figure 1 As shown in the figure, the intelligent hierarchical time synchronization system for low-carbon park clusters includes a clock source layer, a gateway layer, a terminal layer, and an intelligent control layer. Among them, the clock source layer uses multiple clock sources such as satellites, 5G time synchronization networks, and the time synchronization network built by power grid companies to provide time synchronization signals for the gateway layer. The gateway layer is composed of multiple PLC gateways, which are responsible for receiving time synchronization signals from the clock source layer, generating gateway-side synchronization signals through multiple time source selection or comprehensive calculation strategies, realizing time synchronization between the parks in the low-carbon park cluster, and sending time synchronization information to a large number of terminals in a hierarchical manner by broadcasting and multicasting synchronous data packets, so as to realize high-precision, low-latency, wide-area time synchronization of business nodes in the park. The terminal layer deploys a large number of power Internet of Things terminals on distributed photovoltaic, charging pile and other electrical equipment to provide the gateway layer with data such as the surface temperature of solar cell modules, light intensity, wind speed, and energy storage unit charging and discharging efficiency, and feeds back the synchronization delay and synchronization error of the data packet to the gateway layer to support the optimization of the synchronous data packet routing selection strategy. At the same time, the power Internet of Things terminal can forward the dispatching instructions from the power grid dispatching center or aggregator, providing time synchronization and data transmission support for the consistency response of distributed resources. The intelligent control layer deploys deep reinforcement learning (DRL) functions applicable to the clock source layer, gateway layer, and terminal layer. At the clock source layer, DRL assists the clock source in dynamically adjusting the broadcast frequency of time synchronization signals based on link quality. At the gateway layer, DRL supports intelligent hierarchical routing of time synchronization data packets to achieve high-precision time traceability and high-performance wide-area coverage of synchronization signals. At the terminal layer, DRL assists the terminal in data access control, splitting, and unloading optimization to achieve integrated data sensing, transmission, and processing. At the same time, DRL can combine advanced technologies such as cloud-edge-end collaboration, blockchain, and digital twins to support a variety of businesses such as low-carbon park group electricity spot market, electricity-carbon fusion computing, and so on.

[0096] 2. Specific methods

[0097] The present invention designs an intelligent hierarchical time synchronization method for low-carbon park clusters, which mainly includes three stages: time synchronization model construction, optimization problem modeling, and DQN-based high-precision and low-latency time synchronization algorithm design. The specific introduction is as follows:

[0098] (1) Construction of time synchronization model

[0099] Based on the accurate time synchronization of the PLC gateway in the low-carbon smart park group, the present invention considers the scenario where the PLC gateway synchronizes the time of the power Internet of Things terminal by sending time synchronization data packets in layers. Assume that there is 1 PLC gateway and M power Internet of Things terminals, whose set is represented as d0 is the PLC gateway, and the network topology is known, that is, the link relationship between each terminal is known. In the time synchronization process, a total of I iterations are considered, and the set is expressed as Each iteration ends when all terminals in the park receive the time synchronization data packet. The length of each iteration is equal to the time when the last terminal receives the synchronization data packet. At the beginning of each iteration, the PLC gateway sends the synchronization data packet to a large number of terminals in layers. That is, after some terminals complete the reception of the synchronization data packet first, they become the sender and further distribute the data packet to the terminals in the next layer that have not received the synchronization data packet until all terminals receive the synchronization data packet. Definition d m is the synchronous data packet sender, d m' The receiving end of the synchronous data packet. The routing indicator variable is a binary variable. It indicates that, represents the i-th iteration d m' From d m Receive synchronization packets, otherwise Defining binary variables It indicates the topological connection variables between the PLC gateway and each terminal. Indicates d m With d m' Connect, otherwise

[0100] The synchronization packet transmission model, delay model and cumulative synchronization error model are considered as follows.

[0101] 1) Synchronous data packet transmission model

[0102] Considering the complex electromagnetic environment and noise interference in the park, the present invention implements synchronous data packet transmission in the PLC system based on orthogonal frequency division multiplexing (OFDM). m' From d m The rate at which synchronous packets are received is

[0103]

[0104] Among them, N s is the OFDM symbol transmission rate, is the signal-to-interference-plus-noise ratio (SINR), expressed as

[0105]

[0106] in, is the transmission power, Expressed as

[0107]

[0108] in, and δ0 are link frequency response, electromagnetic interference and Gaussian white noise power respectively. It represents the SINR gap, that is, the ability to resist signal-to-interference-noise ratio fading. The calculation formula is as follows

[0109]

[0110] Among them, P e represents the target bit error rate, Q -1 (x) indicates The inverse function of .

[0111] 2) Synchronous delay model

[0112] The present invention divides each iteration into several segments of length τ s time slots, at the beginning of each time slot, d m Start to connect to d m' Distribute synchronization packets. In multi-hop transmission, the i-th iteration d m' The delay in receiving the synchronization data packet is determined by the sender d m The delay in receiving the synchronization packet, and d m' From d m The single-hop transmission delay of receiving synchronous data packets consists of two parts, expressed as

[0113]

[0114] in, Indicates d m The number of time slots during which the synchronization data packet is received, is d m' From d m The delay of receiving a data packet is expressed as

[0115]

[0116] Wherein, U(i) represents the size of the synchronization data packet of the i-th iteration.

[0117] 3) Cumulative synchronization error model

[0118] Define the i-th iteration d m' The cumulative synchronization error of receiving synchronization data packets is Expressed as

[0119]

[0120] in, is the i-th iteration d m' From d m The cumulative synchronization error of the received data packet is

[0121]

[0122] in, represents the i-th iteration d m' From d m The time synchronization error formed by receiving synchronization data packets obeys the normal distribution with an error mean of α and a standard deviation of β, that is, is the i-th iteration d m The accumulated synchronization error of received synchronization packets.

[0123] (2) Optimization Problem Modeling

[0124] The optimization goal of the present invention is to minimize the weighted sum of the average synchronization delay and synchronization error of each terminal in each iteration by optimizing the routing selection strategy of the synchronization data packet under the SINR constraint. The optimization problem can be modeled as

[0125]

[0126] Where χ is a weight parameter used to dynamically balance synchronization delay and synchronization error. C1 and C2 are routing constraints, indicating that the receiver d m' A synchronous data packet can be received from at most one sender. C3 indicates d m' With d m When not connected, d m' Unable to m Receive data packets. C4 is the SINR constraint, is the minimum SINR threshold to ensure the reliability of synchronous data packet transmission.

[0127] (3) Design of high-precision and low-latency time synchronization algorithm based on DQN

[0128] To solve the above optimization problem, the present invention models it as a Markov decision process (MDP) optimization problem, whose key elements include state space, action space and reward function, which are specifically introduced as follows:

[0129] 1) State space: The state space is the size of the synchronization data packet of the i-th iteration and the topological connection relationship between terminals, that is,

[0130]

[0131] 2) Action space: The action space is represented by a set of path selection strategies, namely

[0132]

[0133] 3) Reward function: The reward function is defined as the negative value of the weighted sum of the terminal synchronization delay and the synchronization error, and the expression is

[0134]

[0135] The present invention proposes a high-precision, low-latency time synchronization algorithm based on DQN, which integrates the advantages of deep neural networks in solving high-dimensional problems and the decision-making ability of Q-learning, and uses DQN to estimate the Q value of the cumulative reward value of the action, and optimizes the synchronization data packet routing selection strategy based on this. For example, if the PLC gateway obtains a large reward based on the selected action in the current iteration, the Q value of the action will increase in the next iteration, thereby making the probability of selecting the action higher, effectively reducing synchronization delay and synchronization error.

[0136] like Figure 2 As shown in the figure, the proposed high-precision and low-latency time synchronization algorithm based on DQN is executed by the PLC gateway, which maintains an evaluation network and a target network for each terminal, and the weight vectors are and The evaluation network is based on Implement the mapping from "state-action" to Q value, for example Indicates in status Take action The target network assists in training the weight vector of the evaluation network to improve the convergence and stability of the learning process. At the same time, the PLC gateway maintains an experience pool with a capacity of |Ω| for each terminal. To store the experience data between two adjacent states At the end of the i-th iteration, is stored in and overwrite old experience data At the same time, an experience replay mechanism is used to randomly extract a small batch of experience data from the pool and train the neural network regularly to overcome the correlation and non-stationary distribution problems of the experience data and effectively improve the learning efficiency of time synchronization packet routing.

[0137] The specific steps of the DQN-based high-precision and low-latency time synchronization algorithm designed by the present invention are as follows:

[0138] ① Initialization phase: Initialize and evaluate network weights Target network weights And experience pool

[0139] ② Routing selection phase: In each iteration, at the beginning of each time slot, the PLC gateway makes a time synchronization packet routing decision based on the ε-greedy algorithm. Specifically, the PLC gateway explores and randomly selects an action with probability ε, and uses the action with the maximum estimated Q value with probability 1-ε.

[0140] ③ Reward calculation and state transition phase: The unsynchronized terminal receives synchronization data packets from the synchronized terminal or PLC gateway according to the routing selection decision, and feedbacks the time synchronization delay and error. Then, the PLC gateway calculates the reward function based on the information fed back by the terminal. The current state Transition to next state And the empirical data Deposit into experience pool

[0141] ④ Learning and network update phase: At the end of each iteration, the PLC gateway extracts Randomly select a sample set Calculate the loss function based on the above samples in, The target estimated Q value obtained by the target network is expressed as

[0142]

[0143] Where γ is the discount factor. Based on the loss function Use gradient descent to update the evaluation network weights Right now

[0144]

[0145] Where λ represents the learning rate. Gap iterations, according to Update target network parameters And in the next I Gap -1 iterations remain unchanged.

[0146] Finally, repeat the above steps ② to ④ until the Ith iteration is completed.

[0147] The present invention conducts the following simulation comparison on the above-mentioned DQN-based high-precision and low-latency time synchronization algorithm:

[0148] Comparison algorithm 1 is a multi-armed bandit-based time synchronization route selection algorithm (MAB-RS), and its optimization goal is to minimize the weighted sum of the average synchronization delay and synchronization error of each terminal in each iteration. Comparison algorithm 2 is a synchronization-delay-aware DQN-based time synchronization route selection algorithm (SD-DQN), which only considers the optimization of time synchronization delay performance and ignores the accumulated synchronization error in multi-hop transmission.

[0149] The simulation scenario considered in this paper includes 1 PLC gateway, 9 power IoT terminals and 100 iterations. The PLC channel and electromagnetic interference in the park are modeled using multipath fading channel model and symmetric α-stable (SαS) distribution respectively. The synchronization data packet size is randomly selected within [10,20] bits, and a multi-hop cumulative synchronization error model is adopted. The single-hop synchronization error fluctuates between 10μs and 200μs. The simulation results are as follows.

[0150] Figure 3 The variation of the average synchronization delay with the number of iterations is shown. It can be seen that compared with the MAB-RS algorithm, after 100 iterations, the average synchronization delay of the proposed algorithm is reduced by 5.43%. At the same time, the proposed algorithm can achieve convergence after 20 iterations, while the MAB-RS algorithm requires 70 iterations to converge. This is because the proposed algorithm effectively improves the synchronization delay performance and convergence speed by relying on the advantages of deep neural networks in approximating complex functions and the ability of Q-learning in intelligent decision-making. Since the SD-DQN algorithm only optimizes the single indicator of time synchronization delay, its average synchronization delay performance is the best, but it does not consider the trade-off with the time synchronization error performance, and cannot guarantee the time synchronization accuracy of the terminal to be timed. Its synchronization error performance is poor, such as Figure 4 shown.

[0151] Figure 4The variation of average synchronization error with the number of iterations is shown. It can be seen that compared with MAB-RS and SD-DQN algorithms, after 100 iterations, the average synchronization error of the proposed algorithm is reduced by 10.31% and 36.21% respectively. This is because in the complex dynamic environment of low-carbon park clusters, the proposed algorithm is better than the SD-DQN algorithm in processing high-dimensional time synchronization data packet transmission routing selection problems, and can effectively reduce the synchronization error.

[0152] Those skilled in the art should understand that the present application may be in the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes. The solutions in the embodiments of the present application may be implemented in various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0153] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0154] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0155] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0156] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0157] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. An intelligent hierarchical time synchronization method for low-carbon park clusters, characterized in that The specific synchronization steps are: Obtain the time synchronization signal of the clock source layer, adjust the time synchronization signal of the gateway layer, and obtain the precise time synchronization signal of the gateway layer; Transmitting the gateway layer precise time synchronization signal to the terminal layer, establishing a synchronization data packet transmission model, a synchronization delay model, and a cumulative synchronization error model; Based on the signal-to-interference-noise ratio, synchronization delay model, and cumulative synchronization error model in the synchronization data packet transmission model, a synchronization data packet routing strategy optimization problem is modeled to obtain a synchronization data packet routing strategy optimization problem model; The above synchronization data packet routing strategy optimization problem model is transformed into a Markov optimization problem. The high-precision and low-latency time synchronization algorithm based on DQN is used to solve the transformed Markov optimization problem. According to the solution results, the hierarchical time synchronization is determined. The synchronous data packet transmission model is specifically: At the i-th iteration d m' From d m The rate at which synchronous packets are received is: Among them, N s is the OFDM symbol transmission rate, is the signal-to-interference-noise ratio, expressed as: in, is the transmission power, It is expressed as: in, and δ0 are link frequency response, electromagnetic interference and Gaussian white noise power respectively; is the SINR gap, which indicates the ability to resist signal-to-interference-noise ratio fading. The calculation formula is as follows: Among them, P e represents the target bit error rate, Q -1 (x) indicates The inverse function of The synchronization delay model is: Divide each iteration into several steps of length τ s time slots, at the beginning of each time slot, the synchronized terminal d m Start sending data to the connected unsynchronized terminal d m' Distribute synchronization packets; in multi-hop transmission, the i-th iteration d m' The delay in receiving the synchronization packet is given by d m The delay in receiving the synchronization packet, and d m' From d m The single-hop transmission delay of receiving synchronous data packets consists of two parts, expressed as: in, Indicates d m The number of time slots during which the synchronization data packet is received, is d m' From d m The delay of receiving a data packet is expressed as: Where U(i) represents the size of the synchronization data packet of the i-th iteration; The cumulative synchronization error model is: The i-th iteration d m' The cumulative synchronization error of receiving synchronization data packets is It is expressed as: in, Select indicator variables for routing, represents the i-th iteration d m' From d m Receive synchronization packets, otherwise is the i-th iteration d m' From d m The accumulated synchronization error of the received data packet is expressed as: in, represents the i-th iteration d m' From d m The time synchronization error formed by receiving synchronization data packets obeys the normal distribution with an error mean of α and a standard deviation of β, which can be expressed as is the i-th iteration d m The accumulated synchronization error of the received synchronization data packets; The model for modeling the routing strategy optimization problem of the synchronous data packet is: Among them, χ is a weight parameter used to dynamically balance synchronization delay and synchronization error; C1 and C2 are routing constraints, indicating that the receiver d m' Can receive synchronous data packets from at most one sender; C3 means d m' With d m When not connected, d m' Unable to m Receive data packets; C4 is the SINR constraint, is the minimum SINR threshold to ensure the reliability of synchronous data packet transmission; The Markov optimization problem includes a state space, an action space, and a reward function, where: The state space is the size of the synchronization data packet of the i-th iteration and the topological connection relationship between terminals, which can be expressed as: Where U(i) represents the size of the synchronization data packet of the i-th iteration; It indicates the topological connection variables between the PLC gateway and each terminal. Indicates d m With d m' Connect, otherwise The action space is represented by a set of path selection strategies, expressed as: in, Select indicator variables for routing, represents the i-th iteration d m' From d m Receive synchronization packets, otherwise The reward function is the negative value of the weighted sum of the terminal synchronization delay and synchronization error, expressed as: in, The i-th iteration d m' From d m The synchronization delay of receiving synchronization data packets; represents the i-th iteration d m' From d m Cumulative synchronization error of received packets.

2. According to claim 1, a method for intelligent hierarchical time synchronization for low-carbon park clusters is characterized by: The gateway layer precise time synchronization signal is: The gateway receives one or more time synchronization signals sent from the clock source layer satellite, 5G time synchronization network, and the power grid company's self-built time synchronization network, and generates precise time synchronization signals on the gateway side based on multi-time source selection or comprehensive calculation strategies.

3. According to claim 1, a method for intelligent hierarchical time synchronization for low-carbon park clusters is characterized in that: The high-precision and low-latency time synchronization algorithm based on DQN is: (1) Initialize and evaluate network weights Target network weights And experience pool (2) In each iteration, at the beginning of each time slot, the PLC gateway estimates the Q value of each action in the current state based on the target network weight and makes a time synchronization packet routing decision based on the ε-greedy algorithm; the PLC gateway explores and randomly selects an action with probability ε, and uses the action with the maximum estimated Q value with probability 1-ε. (3) The unsynchronized terminal receives the synchronization data packet from the synchronized terminal or PLC gateway according to the routing decision obtained in the routing decision stage, and feedbacks the time synchronization delay and synchronization error; then, the PLC gateway calculates the reward function based on the information fed back by the terminal The current state Transition to next state And the empirical data Deposit into experience pool (4) At the end of each iteration, the PLC gateway extracts Randomly select a sample set Calculate the loss function based on the above samples in, is the target estimated Q value obtained according to the target network weight, expressed as: Among them, γ is the discount factor; based on the loss function Use gradient descent to update the evaluation network weights Where λ represents the learning rate; then, every I Gap iterations, according to Update target network parameters And in the next I Gap -1 iteration remains unchanged; (5) Repeat the above steps (2) to (4) until the Ith iteration is completed.

4. An intelligent hierarchical time synchronization system for low-carbon park clusters, characterized by: Including clock source layer, gateway layer, terminal layer and intelligent control layer; in: The clock source layer is used to obtain the clock source layer time synchronization signal, adjust the gateway layer time synchronization signal, and obtain the gateway layer accurate time synchronization signal; The gateway layer is used to transmit the precise time synchronization signal of the gateway layer to the terminal layer, and establish a synchronization data packet transmission model, a synchronization delay model, and a cumulative synchronization error model; The terminal layer is used to model the routing strategy optimization problem of the synchronous data packet based on the signal-to-interference-noise ratio, the synchronization delay model, and the cumulative synchronization error model in the synchronous data packet transmission model to obtain the routing strategy optimization problem model of the synchronous data packet; The intelligent control layer is used to transform the synchronization data packet routing strategy optimization problem model into a Markov optimization problem, and solve the transformed Markov optimization problem based on the high-precision and low-latency time synchronization algorithm of DQN. According to the solution results, the hierarchical time synchronization is determined. The synchronous data packet transmission model is specifically: At the i-th iteration d m' From d m The rate at which synchronous packets are received is: Among them, N s is the OFDM symbol transmission rate, is the signal-to-interference-noise ratio, expressed as: in, is the transmission power, It is expressed as: in, and δ0 are link frequency response, electromagnetic interference and Gaussian white noise power respectively; is the SINR gap, which indicates the ability to resist signal-to-interference-noise ratio fading. The calculation formula is as follows: Among them, P e represents the target bit error rate, Q -1 (x) indicates The inverse function of The synchronization delay model is: Divide each iteration into several steps of length τ s time slots, at the beginning of each time slot, the synchronized terminal d m Start sending data to the connected unsynchronized terminal d m' Distribute synchronization packets; in multi-hop transmission, the i-th iteration d m' The delay in receiving the synchronization packet is given by d m The delay in receiving the synchronization packet, and d m' From d m The single-hop transmission delay of receiving synchronous data packets consists of two parts, expressed as: in, Indicates d m The number of time slots during which the synchronization data packet is received, is d m' From d m The delay of receiving a data packet is expressed as: Where U(i) represents the size of the synchronization data packet of the i-th iteration; The cumulative synchronization error model is: Iteration d m' The cumulative synchronization error of receiving synchronization data packets is It is expressed as: in, Select indicator variables for routing, represents the i-th iteration d m' From d m Receive synchronization packets, otherwise is the i-th iteration d m' From d m The accumulated synchronization error of the received data packet is expressed as: in, represents the i-th iteration d m' From d m The time synchronization error formed by receiving synchronization data packets obeys the normal distribution with an error mean of α and a standard deviation of β, which can be expressed as is the i-th iteration d m The accumulated synchronization error of the received synchronization data packets; The model for modeling the routing strategy optimization problem of the synchronous data packet is: Among them, χ is a weight parameter used to dynamically balance synchronization delay and synchronization error; C1 and C2 are routing constraints, indicating that the receiver d m' Can receive synchronous data packets from at most one sender; C3 means d m' With d m When not connected, d m' Unable to m Receive data packets; C4 is the SINR constraint, is the minimum SINR threshold to ensure the reliability of synchronous data packet transmission; The Markov optimization problem includes a state space, an action space, and a reward function, where: The state space is the size of the synchronization data packet of the i-th iteration and the topological connection relationship between terminals, which can be expressed as: Where U(i) represents the size of the synchronization data packet of the i-th iteration; It indicates the topological connection variables between the PLC gateway and each terminal. Indicates d m With d m' Connect, otherwise The action space is represented by a set of path selection strategies, expressed as: in, Select indicator variables for routing, represents the i-th iteration d m' From d m Receive synchronization packets, otherwise The reward function is the negative value of the weighted sum of the terminal synchronization delay and synchronization error, expressed as: in, The i-th iteration d m' From d m The synchronization delay of receiving synchronization data packets; represents the i-th iteration d m' From d m Cumulative synchronization error of received packets.

5. An intelligent hierarchical time synchronization device for low-carbon park clusters, characterized by: include: Signal acquisition module: used to obtain the time synchronization signal of the clock source layer, adjust the time synchronization signal of the gateway layer, and obtain the precise time synchronization signal of the gateway layer; Modeling module: used to obtain synchronization data packet transmission model, synchronization delay model, and cumulative synchronization error model; Optimization modeling module: used to transmit the precise time synchronization signal of the gateway layer to the terminal layer, and establish a synchronization data packet transmission model, a synchronization delay model, and a cumulative synchronization error model; The conversion calculation module is used to model the routing strategy optimization problem of the synchronous data packet based on the signal-to-interference-noise ratio, the synchronization delay model, and the cumulative synchronization error model in the synchronous data packet transmission model, and obtain the routing strategy optimization problem model of the synchronous data packet; Solution module: used to transform the synchronization data packet routing strategy optimization problem model into a Markov optimization problem, solve the transformed Markov optimization problem based on the high-precision and low-latency time synchronization algorithm of DQN, and determine the hierarchical time synchronization according to the solution results; The synchronous data packet transmission model is specifically: At the i-th iteration d m' From d m The rate at which synchronous packets are received is: Among them, N s is the OFDM symbol transmission rate, is the signal-to-interference-noise ratio, expressed as: in, is the transmission power, It is expressed as: in, and δ0 are link frequency response, electromagnetic interference and Gaussian white noise power respectively; is the SINR gap, which indicates the ability to resist signal-to-interference-noise ratio fading. The calculation formula is as follows: Among them, P e represents the target bit error rate, Q -1 (x) indicates The inverse function of The synchronization delay model is: Divide each iteration into several steps of length τ s time slots, at the beginning of each time slot, the synchronized terminal d m Start sending data to the connected unsynchronized terminal d m' Distribute synchronization packets; in multi-hop transmission, the i-th iteration d m' The delay in receiving the synchronization packet is given by d m The delay in receiving the synchronization packet, and d m' From d m The single-hop transmission delay of receiving synchronous data packets consists of two parts, expressed as: in, Indicates d m The number of time slots during which the synchronization data packet is received, is d m' From d m The delay of receiving a data packet is expressed as: Where U(i) represents the size of the synchronization data packet of the i-th iteration; The cumulative synchronization error model is: The i-th iteration d m' The cumulative synchronization error of receiving synchronization data packets is It is expressed as: in, Select indicator variables for routing, represents the i-th iteration d m' From d m Receive synchronization packets, otherwise is the i-th iteration d m' From d m The accumulated synchronization error of the received data packet is expressed as: in, represents the i-th iteration d m' From d m The time synchronization error formed by receiving synchronization data packets obeys the normal distribution with an error mean of α and a standard deviation of β, which can be expressed as is the i-th iteration d m The accumulated synchronization error of the received synchronization data packets; The model for modeling the routing strategy optimization problem of the synchronous data packet is: Among them, χ is a weight parameter used to dynamically balance synchronization delay and synchronization error; C1 and C2 are routing constraints, indicating that the receiver d m' Can receive synchronous data packets from at most one sender; C3 means d m' With d m When not connected, d m' Unable to m Receive data packets; C4 is the SINR constraint, is the minimum SINR threshold to ensure the reliability of synchronous data packet transmission; The Markov optimization problem includes a state space, an action space, and a reward function, where: The state space is the size of the synchronization data packet of the i-th iteration and the topological connection relationship between terminals, which can be expressed as: Where U(i) represents the size of the synchronization data packet of the i-th iteration; It indicates the topological connection variables between the PLC gateway and each terminal. Indicates d m With d m' Connect, otherwise The action space is represented by a set of path selection strategies, expressed as: in, Select indicator variables for routing, represents the i-th iteration d m' From d m Receive synchronization packets, otherwise The reward function is the negative value of the weighted sum of the terminal synchronization delay and synchronization error, expressed as: in, The i-th iteration d m' From d m The synchronization delay of receiving synchronization data packets; represents the i-th iteration d m' From d m Cumulative synchronization error of received packets.

Images