Software-defined industrial wireless network resource arrangement method for guaranteeing end-to-end time delay

Abstract

The invention relates to a software-defined industrial wireless network resource arrangement method for guaranteeing end-to-end time delay. The method comprises the following steps: step 1, constructing a system model; step 2, SFC resource arrangement problem modeling; step 3, end-to-end time delay of the service in the SFC is carried out; and step 4, a VNFs resource orchestration algorithm under the delay guarantee based on the DQN. The method provided by the invention has the superior technical effects that aiming at an IIOT uplink service, the service capability of a wireless domain under short block length channel coding is obtained by classifying SINR prediction precision, time delay constraint guarantee is realized through SNC, and VNF resource arrangement is finally realized through multi-task DQN based on the time delay constraint guarantee. The E2E time delay can be analyzed more accurately, the SNC theory is adopted to analyze the end-to-end time delay characteristics, and the end-to-end time delay characteristics in the wireless transmission and wired transmission hybrid network in the industrial Internet of Things environment can be described accurately.

Description

Technical Field

[0001] The present invention relates to an industrial wireless network resource orchestration method, and more specifically to a software-defined industrial wireless network resource orchestration method that guarantees end-to-end delay. Background Art

[0002] With the evolution of factory scale and industrial production methods, as well as the development of emerging fields such as edge computing and artificial intelligence, the Industrial Internet of Things (IIoT) integrates various devices with collection, control, sensing and other functions, as well as advanced technologies such as mobile communications and intelligent analysis, into every link of the industrial production process. By collecting, transmitting, analyzing and forming intelligent feedback of various types of data, it realizes the connection of all factors, the entire industrial chain and the entire value chain of people, machines and objects, continuously promotes the formation of a new production and service system, and realizes the improvement of the efficiency of resource allocation.

[0003] Thanks to the advancement of Industry 4.0, the fifth generation (5G) mobile communication technology has been deployed in the IIoT with its highly reliable and low-latency communication performance. This enables the IIoT to support greater data throughput and its devices to quickly receive and send large amounts of data.

[0004] New network technologies, such as Network Function Virtualization (NFV) and Software-Defined Networking (SDN), offer novel solutions for managing and controlling multi-source, heterogeneous networks in the Industrial Internet of Things (IIoT). NFV is a new, future-oriented network architecture that uses virtualization and other technologies to deploy common network functions as software on uniformly formatted physical devices. It decouples services such as firewalls, resource managers, intrusion detection systems, and QoS monitors from traditional dedicated physical devices, enabling their mass deployment as software on commercial, general-purpose equipment. SDN, a form of network function virtualization, separates the control and data planes of network devices through technologies such as OpenFlow, enabling flexible control of network traffic. This allows networks to become more intelligent, like pipelines, and provides a robust platform for innovation in core networks and applications.

[0005] To ensure efficient and sustainable system operation, end-to-end (E2E) latency is a crucial metric. In addition to directly impacting system QoS, E2E latency also affects the allocation of spectrum and computing resources, further impacting the performance of virtual network function (VNF) resource orchestration. Latency calculation solutions that are closer to actual latency values help reduce redundancy in resource allocation and service scheduling, thereby fully utilizing limited resources and ultimately improving production efficiency.

[0006] In existing research, Guo Y, Hu C, Peng T, et al. published a similar method based on nonlinear regression algorithm (NLRA) (Guo Y, Hu C, Peng T, et al.

[0007] Regression-based uplink interference identification and SINR prediction for 5G ultra-dense network [C] / / ICC 2020-2020 IEEE International Conference on Communications (ICC). IEEE, 2020: 1-6.), and published interference identification schemes such as neural networks (NN) that can achieve link-level co-channel interference source identification and signal-to-interference-plus-noise ratio (SINR) prediction (Cao J, Peng T, Liu X, et al. Resource allocation for ultradense networks with machine-learning-based interference graph construction [J]. IEEE Internet of Things Journal, 2019, 7(3): 2137-2151.). Among them, NN can provide accurate modeling, but lacks interpretability and has a large computational load. NLRA enhances interpretability and computational efficiency, and helps to accurately characterize the average power of each interfering link. These methods play different roles under different business requirements or hardware device performance limitations. Due to different considerations of system overhead, different SINR prediction schemes exhibit different SINR prediction accuracy levels, which inspires us to explore how to utilize SINR information to obtain a stricter end-to-end delay upper bound.

[0008] In the research on service E2E delay, Lee KC et al. studied the application of multi-level switched Ethernet in real-time industrial networks, analyzed the delay composition of service flows in industrial networks, and conducted performance evaluation (Lee KC, Lee S, Lee M H. Worst case communication delay of real-time industrial switched Ethernet with multiple levels [J]. IEEE Transactions on Industrial Electronics, 2006, 53 (5): 1669-1676.).

[0009] Seliem M et al. studied a hierarchical architecture based on delay-sensitive networks. Using a network calculus framework, they analyzed a range of different data traffic types and evaluated factors such as traffic skew and critical links that affect end-to-end delay. In recent years, the application of SNC has provided a method for calculating delay in wireless channels. This method draws on the expression form of EC theory to analyze and obtain statistical delay bounds in fading environments, while also converting the discussion of delay from the bit domain to the SINR domain. SNC in the SINR domain has been used to solve many delay analysis problems in wireless transmission (Seliem M, Zahran A, Pesch D. Delay analysis of TSN-based industrial networks with preemptive traffic using network calculus [C] / / 2023 IFIP Networking Conference (IFIP Networking). IEEE, 2023: 1-9.).

[0010] Xiao C, Zeng J, Ni W, et al. Delay guarantee and effective capacity of downlink NOMA fading channels[J]. IEEE Journal of Selected Topics in Signal Processing, 2019, 13(3): 508-523. (Xiao C, Zeng J, Ni W, et al. Delay guarantee and effective capacity of downlink NOMA fading channels[J]. IEEE Journal of Selected Topics in Signal Processing, 2019, 13(3): 508-523.).

[0011] Mei M et al. focused on the Laplace transform of the interference received by the interested user under orthogonal frequency division multiple access (OFDMA), and used it to obtain the Mellin transform of SINR, and further obtained the delay violation probability through SNC. When solving the problem of closed expressions of logarithmic operators under Shannon capacity in SNC (Mei M, Yao M, Yang Q, et al. Delay analysis of mobile edge computing using Poisson cluster process modeling: Astochastic network calculus perspective[J]. IEEE Transactions on Communications, 2022, 70(4): 2532-2546.-6); Mei M and other authors also gave a method to use Meijer-G function to find closed expressions (Mei M, Yao M, Yang Q, et al. Delay analysis of mobile edge computing using Poisson cluster process modeling: A stochastic network calculus perspective[J]. IEEE Transactions on Communications, 2022, 70(4): 2532-2546.).

[0012] In the area of VNF resource orchestration, many studies have used intelligent algorithms to solve the problem. For example, Yao H, Chen X, Li M, and others studied how to train the orchestration process using historical data, rather than relying on any handcrafted rules. This approach allows orchestration solutions to be learned from historical requests, achieving excellent performance in terms of cost-benefit ratio and other indicators (Yao H, Chen X, Li M, et al. A novel reinforcement learning algorithm for virtual network embedding [J]. Neurocomputing, 2018, 284).

[0013] Li J, Shi W, Wu H, et al. studied the mapping and resource scheduling of VNFs in online scenarios to improve the profits of service providers. They studied the excess delays and migration costs generated by the real-time migration and re-establishment of VNFs, and transformed the dynamic VNF orchestration problem into a mixed integer linear programming problem based on a specified cost and delay model. Using a taboo search-based rescheduling and rescheduling algorithm, they obtained a suboptimal solution to the mixed integer linear programming problem (Li J, Shi W, Wu H, et al. Cost-aware dynamic SFC mapping and scheduling in SDN / NFV-enabled space-air-ground-integrated networks for Internet of Vehicles[J]. IEEE Internet of Things Journal, 2021, 9(8): 5824-5838.).

[0014] Existing research on VNF resource orchestration has not effectively integrated wireless and wired domain communications. Analysis of E2E latency for services in hybrid networks still primarily relies on the simple summation of latency results across different nodes and links, without considering the impact of each other. This leads to an overestimation of latency boundaries and, in turn, excessive resource consumption.

[0015] Specifically, for delay analysis in the wireless domain, deterministic delay bounds are difficult to obtain due to the complex and highly random nature of wireless transmission environments. While intelligent algorithms can more accurately capture wireless channel parameters and real-time status information, existing research has not fully utilized this information, relying solely on statistical methods to model SINR (such as the Nakagami-m distribution and the Rician distribution). Consequently, these derivations can be overly conservative, resulting in a less stringent upper bound on end-to-end delay. In practice, SINR cannot be simply modeled using statistical models, as the target signal and co-channel interfering signals follow fading models with different parameters.

[0016] Most studies on channel modeling have failed to consider the application of Finite Blocklength Channel Codes (FBCC) in wireless communications for the Industrial Internet of Things (IIoT), which is particularly important given the demands for low latency and high reliability. Industrial environments often require real-time transmission of control signals and sensor data, requiring communication systems to be responsive and tolerant of frequent channel variations. FBCC provides robust error correction within shorter code blocks, reducing transmission latency while improving data reliability.

[0017] In terms of resource orchestration, research has often used intelligent algorithms and focused on orchestrating resources for small-scale SFCs. However, as the number of SFCs increases, designing a separate DQN for each SFC will incur significant resource overhead. Furthermore, the initial allocation of resources to an SFC will affect the action space size of subsequent SFCs. Designing a separate DQN for each SFC in isolation cannot address this issue. Furthermore, due to overestimation in latency analysis, many resource orchestration methods that offer low resource overhead but guarantee latency constraints are not considered.

[0018] The author Fan Weixuan studied the data-driven dynamic SFC resource joint orchestration algorithm and designed DQN for each SFC to achieve resource allocation (Fan Weixuan. Research on dynamic SFC resource joint orchestration method in network slicing [D]. Beijing University of Posts and Telecommunications, 2024. DOI: 10.26969 / d.cnki.gbydu.2024.001513.).

[0019] Based on the defects in the above-mentioned background technology, the patent application of the present invention provides an E2E delay performance analysis solution in the uplink transmission scenario, and obtains an E2E delay boundary that is closer to the actual transmission. Based on this, a software-defined industrial wireless network resource orchestration mechanism across wireless and wired domains is designed to provide flexible adaptation and upgrade capabilities for industrial application services. Summary of the Invention

[0020] To better understand the software-defined industrial wireless network resource orchestration method for ensuring end-to-end latency proposed in this patent application, the following briefly introduces the Stochastic Network Calculation (SNC) theory involved:

[0021] Between time s and t, A(s,t) represents the cumulative amount of service flows arriving, and S(s,t) represents the cumulative amount of services provided by the service node. The expressions of A(s,t) and S(s,t) are shown as follows:

[0022] A(s,t)=ρ A (ts)+σ A ,

[0023] S(s,t)=ρ S (ts)+σ S ,

[0024] Among them, for the arrival process, ρ A represents the service flow arrival rate, σ A represents the burst traffic. Similarly, for dynamic servers, ρ S represents the server service rate, σ S Represents the backlog of unprocessed business.

[0025] Let X be a random variable with probability density function f X (x), the moment generating function M of the random variable X X (θ) is defined as follows:

[0026]

[0027] Among them, M X (θ) is the moment generating function of X, is the e of the random variable X θX The expectation of , θ is a real parameter;

[0028] For a random arrival process A(s,t), it is subject to the following constraints:

[0029]

[0030] Denoted as satisfying (σ A ,ρ A )-constrained arrival process;

[0031] For a random service process S(s,t), it is subject to the following constraints:

[0032]

[0033] Denoted as satisfying (σ S ,ρ S )-constrained dynamic service process;

[0034] Effective capacity (EC) defines the maximum constant arrival rate that a time-varying fading channel can support under certain QoS constraints. When the channel is block fading, it is as follows:

[0035]

[0036] Where {R[i],i=1,2,…} represents the discrete-time channel process, and the calculation formula of EC can be used to solve ρ A (θ) and ρ S (-θ):

[0037] θ describes the decay rate of the tail distribution of queue length as follows:

[0038]

[0039] In the above formula, Q(∞) is defined as the stable queue length, and x is a threshold. Specifically, a larger value of θ corresponds to a faster decay rate, which means strict QoS restrictions, while a smaller value of θ corresponds to a slower decay rate and is accompanied by loose QoS restrictions. When θ is determined, it means that in order to make the system stable, the maximum capacity allowed by the service should be E c (θ), since stricter QoS restrictions mean less allowed capacity, let θ>0, p,q>1 be Conjugate number, assuming there is a (σ A ,ρ A )-constrained arrival process and a (σ S ,ρ S )-constrained dynamic service process, given a random delay T at time t≥0 SNC The probability of default ≥ 0 is constrained to be:

[0040]

[0041] When the arrival process A and the service process S are independent, p=q=1.

[0042] Therefore, when the violation probability ω is given, the expression of the delay is as follows:

[0043]

[0044] The SNC cascade theorem is a theorem for solving the E2E delay of a service flow after it passes through multiple nodes. Compared with the general method of accumulating the transmission and processing delays of services on multiple nodes, a delay closer to the actual transmission and processing situation can be obtained. Consider N dynamic servers S i (s, t), i=1,…,N are connected together, where N>2. Assume that each server S i (s,t) is -constrained, the E2E network service process S net (s,t) is equivalent to a single dynamic server. Let θ>0, and p i ,i=1,…,N is conjugate number. Therefore -The constrained E2E network service process expression is as follows:

[0045]

[0046] in, ∈ is a minimum value added to avoid the situation where the denominator is 0.

[0047] The software-defined industrial wireless network resource orchestration method for ensuring end-to-end latency described in this method includes:

[0048] Step 1. Build a system model;

[0049] Step 2. Modeling of SFC resource scheduling problem;

[0050] Step 3. End-to-end delay of the service in SFC;

[0051] Step 4. VNFs resource orchestration algorithm with latency guarantee based on DQN.

[0052] Furthermore, the construction of the system model in step 1 includes the following specific steps:

[0053] Step 1.1: Construct a network scenario model. In a factory network model, a 4×3 grid of service devices is formed. The service devices at each corner are connected to a base station. Different types of industrial equipment access the network wirelessly through adjacent base stations. Base stations and devices with general processing functions are connected via wired connections, creating a hybrid wireless and wired network. Industrial equipment connected to different base stations may interfere with the communications of industrial equipment in adjacent cells.

[0054] Abstract the physical scene into a weighted undirected graph in Represents a set of N physical nodes, Represents a set of E links between nodes, using Represents a physical node, represents the link connecting nodes n and m. When deploying VNFs to specific nodes, there are usually overheads such as computing resources, storage space, memory, and power supply. These overheads are closely related to the specific VNF. Each node needs to be equipped with different amounts of computing resources and storage resources to run, store VNFs, or perform other operations. The computing capacity of each edge node is expressed as For each direct link The bandwidth of the link is denoted as B (n,m) , each base station has a corresponding resource block (RB), which is expressed as

[0055] Step 1.2 Wireless channel modeling:

[0056] A typical feature of the IIoT is the presence of multiple devices connected via wireless communication. These devices occupy different communication resources, generating co-channel interference. Consider a typical indoor factory environment, where various industrial user equipment (UEs), such as sensors, robotic arms, and robots, are wirelessly connected to the nearest base station (BS). These UEs are evenly distributed within the cells covered by the BS. This scenario is based on an E2E uplink transmission scenario, and considers the inter-cell interference when UEs communicate with the co-channel BS.

[0057] Assume that the base station C0 providing the target service is located at the origin of the coordinate system, and there are K interfering UEs during communication. represents the set of interfering UEs, represents the set of co-frequency base stations accessed by these interfering UEs, and the target UE is denoted as U0;

[0058] The path loss of the channel is inversely proportional to the distance, and the path loss exponent is represented by l. In addition, the small-scale fading between the UE and the BS adopts the Nakagami-m fading model, with the characteristic parameter m, the transmit power of each UE is P, and the distance from the accessed co-frequency base station is r k , when applying the partial path loss compensation factor δ∈(0,1), the kth UE (U k ) is the actual transmission power The instantaneous received SINR of U0 at the serving BS is as follows:

[0059]

[0060] in, is the signal receiving power of U0, is the variance of zero-mean additive white noise (AWGN), is the sum of the received powers of all interferences, g k Obey the Gamma distribution, d k Indicates the interference device U k The distance between the target user and the co-frequency base station C0;

[0061] SINR consists of three parts: signal, interference, and noise. Noise follows a Gaussian distribution. Based on the link layer, different scenarios have different trade-offs between resource overhead and interference perception accuracy for both signal and interference. This leads to different SINR prediction accuracies. SINR prediction accuracy is graded as follows:

[0062] Signal: In OFDMA, each UE has only one signal link with its associated base station. This application considers the following two levels of signal prediction accuracy:

[0063] Level I: Ability to accurately obtain the instantaneous power of each link;

[0064] Level D: Ability to determine the power distribution of each link;

[0065] Interference: Interference signals may exist in different units, and there may be multiple interference links. The degree of identification of these links affects modeling. The prediction accuracy of interference power is classified as follows:

[0066] Level I: Ability to accurately obtain the instantaneous power of each link;

[0067] Level D: Ability to determine the power distribution of each link;

[0068] Level A: Ability to determine the average power of each link;

[0069] Level M: only the total interference distribution received can be inferred;

[0070] Levels I, D, and A provide link-level interference awareness, while Level M does not. To focus on the impact of link-level interference information, Level M is not considered in subsequent analysis. Level I is an ideal prediction that is difficult to obtain in real experiments. The aforementioned interference identification schemes, such as NLRA, all achieve Level A predictions.

[0071] For simplicity, "signal power accuracy / interference power accuracy" is used to represent the accuracy combination of instantaneous SINR. "I / D" means that the instantaneous power of the signal link can be accurately obtained (level I), and the power distribution of each interference link can be determined (level D).

[0072] For coding blocks of finite length, the coding efficiency is usually closely related to the block error rate (BLER), which is approximately as follows:

[0073]

[0074] Where V(γ)=1-(1+γ) -2 is the channel dispersion, n is the coding block length, Q -1 (x) is the inverse Gaussian Q function. Based on this, the general expression of the effective capacity of the wireless channel during transmission under different SINR prediction accuracies is given as follows:

[0075]

[0076] in, fγ (x) is the probability density function of SINR distribution under different SINR prediction accuracy. For the above formula, J only needs to take a small number of values to obtain an approximate result;

[0077] When the target signal and interference signals When both meet the nakagami-m fading, for different SINR prediction accuracy, G J (η) and f γ (x) has different expressions, as shown below:

[0078] In the case of I / D:

[0079]

[0080] In the case of I / A:

[0081]

[0082] In the D / D case:

[0083]

[0084] in:

[0085]

[0086] In the D / A situation:

[0087]

[0088] in:

[0089]

[0090] in, T is the period, B is the bandwidth, is the shape parameter of the target signal when it fades, is the received power of the target signal, is the shape parameter of the interference signal when it fades, is the scale parameter when the interference signal fades, is the average SINR;

[0091] Step 1.3 Business Modeling:

[0092] In industrial environments, periodic updates of locations or repeated monitoring of environmental characteristics often lead to periodic traffic. Here, the target traffic is periodic traffic. In industrial ecosystems, periodic communication is a major form of communication:

[0093] Assume that a service source is at time {t=Mτ +nτ, n=0,1,2,…} generates a workload of α units, where τ represents the cycle length, M is uniformly distributed in the interval [0,1], and M τ represents the initial moment of traffic transmission. For such a source, the MGF arrival process is (σ A ,ρ A )-constrained, where:

[0094] σ A (θ)=α,

[0095] When a bursty traffic flow occurs occasionally, it indicates a service alarm and has a lower tolerance for delay and a higher priority. The arrival process of the bursty traffic flow is modeled as a Poisson arrival process. In SNC, the MGF function is used to characterize the Poisson arrival flow with an average arrival rate λ and a given packet size of 1 / v as follows:

[0096] σ A (θ)=0,

[0097] For an arriving SFC, it is necessary to determine its traffic flow characteristics λ k The attributes of SFC are used to divide the SFC into different slices for mapping, such as Figure 4 As shown, if λ k If it meets the characteristics of a periodic service flow, the SFC needs to be divided into a general network slice; if k A bursty service flow needs to be divided into bursty network slices.

[0098] Furthermore, the SFC resource scheduling problem modeling described in step 2 includes the following specific steps:

[0099] Step 2.1 SFC request modeling:

[0100] When K SFC requests arrive, use To represent the service function graph of SFC request k, for each definition Describes the properties of the SFC, where s k and d k Corresponding to the source node and destination node of SFCk, since each It consists of a given set of ordered VNFs, except for the source node and the destination node, using To represent the set of VNFs in SFCk, and are the nth and mth VNFs in SFCk, respectively. Assuming that the computing resources requested by each VNF are constant, Represents the computing resource requirements of SFCk, there is a set of virtual links connecting the source node s k , ordered VNFs and target node d k ,use To represent the link between the nth VNF and the mth VNF in SFCk, each SFC supports a feature λ k business flow, Indicates the delay that this business flow needs to meet, ω k Indicates the tolerable delay violation probability of this service;

[0101] When multiple VNF instances are deployed on the same CPU core, processing resources (e.g., CPU cores) are statistically multiplexed, so shared processing resources are not considered. When more VNFs share the same CPU, the CPU access latency experienced by each VNF increases significantly. The processing latency of already mapped VNFs will be reduced by the newly instantiated VNFs in the same CPU core, which leads to latency uncertainty in the existing SFC.

[0102] In this application, it is assumed that different SFCs cannot share the same type of VNF instances. Even if the same type of VNF instances exist in edge nodes, each VNF instance occupies dedicated processing resources and can only belong to a certain SFC. Assuming that the processing delay of a VNF instance is only related to the processing resources allocated to it, the amount of resources allocated to a VNF can be arbitrary as long as the overall processing delay of the SFC can meet the delay requirements. In addition to the allocation of CPU processing resources, the RB resources of the wireless channel are also allocated for wireless-side transmission services. The allocation of RB resources on the wireless side and the allocation of CPU processing resources on the wired side together constitute the resource orchestration of the SFC.

[0103] Step 2.2 SFC resource scheduling constraints:

[0104] For the kth SFC, use the binary variable express The deployment, when Deployed on edge nodes On time, otherwise A node can instantiate multiple VNF instances, but each VNF instance can only be deployed on one node. The deployment constraints are expressed as follows:

[0105]

[0106] Since the computing power of a node is shared by all NFV instances deployed on the node, the total CPU allocated to the VNF instances cannot exceed the total capacity of the edge node. The following formula is used:

[0107]

[0108] For SFCk, define the binary variable express Whether it is mapped to the physical link express Whether to connect the node if Mapped in Up, then are also linked, the following routing constraints are guaranteed:

[0109] if

[0110] In the above formula, the consistency of physical nodes and links is guaranteed, and the mapping of virtual links is ensured to be consistent with physical links. Then the following formula is obtained:

[0111]

[0112] In the above formula, it is ensured that the links on the path embedded in SFCk are connected to the head and tail. If the node is selected Come to SFCk To provide services, the node ensures that it is connected, that is:

[0113]

[0114] Since the bandwidth resources of the physical link (n, m) are shared by the virtual links mapped to it, the total bandwidth consumed by these virtual links cannot exceed the total bandwidth resources of the physical link (n, m). For the constraint analysis of link bandwidth resources, represents the amount of bandwidth resources required to be allocated to the virtual link between the nth and mth VNFs in SFCk, to ensure that the sum of bandwidth resources allocated to the virtual links cannot exceed the total bandwidth of the physical edge (n,m), and is defined as follows:

[0115]

[0116] Furthermore, the end-to-end delay of the service in the SFC in step 3 includes the following specific steps:

[0117] Step 3.1 Calculation method for time delay:

[0118] In URLLC, latency is a key QoS guarantee metric. The actual E2E latency of a service flow must be less than a given maximum latency under given reliability requirements. For service flows within an SFC, latency is composed of link transmission delay, propagation delay, and node processing delay. The SFC service latency is determined by the SFC's resource requirements and the amount of resources allocated to it. It is calculated as follows:

[0119] For the kth SFC, the link transmission delay is determined by the transmission delay of the wireless channel and transmission delays in wired networks The transmission delay in a wired network is composed of the size of the transmitted packet b k Divide by the bandwidth capacity allocated to the virtual link To calculate, it is given by the following formula:

[0120]

[0121] For distance propagation delay, the factor affecting it is the physical distance L that the service transmits in the link. flow and the speed of light c, that is When the link length is 3×10 5 When m, a delay of about 1ms is generated. In the IIoT scenario, the impact of resource allocation on latency is the main factor, and the propagation delay will be ignored;

[0122] Regarding data processing latency, since VNFs are typically instantiated by associating them with a certain resource combination (e.g., CPU, RAM, etc.), the service processing latency is calculated as a function of CPU frequency allocation, as given by the following formula:

[0123]

[0124] In the above formula, represents the computing capacity allocated by the node to the nth VNF in the kth SFC. The total latency is expressed as:

[0125]

[0126] Step 3.2 Improvement of the delay calculation method:

[0127] The following improvements are made to the traditional delay analysis method:

[0128] Service capability characterization: The expression of the effective capacity of the data transmission capability of the wireless access node (base station) is expressed in Indicates the service capability provided by a mapped node assigned to the nth VNF in the kth SFC; Indicates that a mapped link is given to the virtual link in the kth SFC The ability to provide services, these service capability expressions can be expressed using the random network calculus ρ S express:

[0129] E2E latency in multi-node transmission: Using the SNC latency solution formula and combining it with the characteristics of the service flow λ k The target delay at a given time can be solved Delay violation probability of the downstream service flow

[0130] because and They are similar in characterizing service capabilities and ignore the resource orchestration of VNFs. For a mapped SFCk, if the wireless channel transmission capability provided by the base station is expressed as The initial backlog is The data processing capability provided by the nodes in the mapped wired network is expressed as The business flow backlog on each mapping node is The overall service capacity and system backlog of the entire business flow E2E are expressed as follows:

[0131]

[0132] in, ∈ is to avoid the minimum value added by the denominator to be 0. When all service processes satisfy independent and identical distribution, p1, p2, ..., p N =1, when the violation probability ω is given, the expression of E2E delay is:

[0133]

[0134] Furthermore, the DQN-based VNFs resource orchestration algorithm with latency guarantee described in step 4 includes some specific steps:

[0135] Step 4.1 Algorithm design:

[0136] In order to achieve deterministic network services with minimal resource consumption while ensuring service delay, we define p and q as the resource cost coefficients in the network. For a given k-th SFC, the resource allocation required to serve its service flow is expressed as Since RB resources and CPU resources are discrete, It represents the number of resource allocations, and the resource overhead ultimately brought to the entire network is as follows:

[0137]

[0138] In order to achieve the minimum resource cost, the objective function is defined as: P:minQ,

[0139] In order to ensure strict service flow services, it is necessary to make the E2E delay violation probability of a resource allocation scheme small enough under a given E2E delay. For the SFCk service flow delay violation probability ω k As an important indicator, the reward function is defined as follows:

[0140]

[0141] When the delay violation probability requirement is met, the smaller Q is, the larger the value of r is, where The calculation method is as follows:

[0142]

[0143] The target Q network and loss function are defined as follows:

[0144]

[0145] In the above formula, s is the state, a is the action, is the target network parameter, ζ is the discount factor;

[0146] For the state space It includes the remaining resources of each base station and each physical server, as well as the characteristics of services in different SFCs;

[0147] For the action space Generate the applicable action space for each SFC each time It only includes the resource allocation actions of the associated nodes, and the service capabilities under the cascade in SNC The expression shows that its value is related to the minimum service capacity of all nodes through which the business flow flows. The business flow on each SFC flows through a base station and several physical servers. Each SFC has a resource allocation action space of different dimensions. In order to facilitate the training of DQN and unify the action space and facilitate training, it is assumed that the same computing resources are allocated to different VNFs on the SFC. In this way, for the action space Its dimension changes from (n+1) dimensions (n represents the number of VNFs) to 2 dimensions, thus compressing the action space.

[0148] This application adopts Multi-Task Reinforcement Learning (MTRL), which includes a two-layer neural network in the network design. The first layer is used to implement shared feature extraction (SFE) to learn the common state representation across SFC tasks; the second layer is task-specific output heads (TSH), which maps each task to a different action space. Realize dynamic adaptation of tasks;

[0149] Step 4.2: Training and testing the algorithm:

[0150] Based on step 4.1, we designed a VNFs resource orchestration with latency guarantee based on DQN (VROLG-DQN) algorithm. The training process of the algorithm is as follows:

[0151] The input of the algorithm is the edge network topology, the information of K SFCs and the corresponding mapping scheme. The output of the algorithm is the dynamic allocation scheme of various network resources for each SFC. Before the algorithm is executed, it needs to be initialized. Other network parameters and ζ, DQN related parameters such as replay buffer;

[0152] Then, n training sessions are started, where each time, for the kth SFC, the ∈-greedy strategy is used to select the action from the compressed action space A k Select the kth SFC resource allocation action, calculate the resource vector and allocate it to the kth SFC. For the allocated resource vector, determine whether the resource allocation action is valid and meets the VNF resource orchestration constraints. If so, calculate the service flow delay violation probability of each SFC. Each reward r k The sum of the total rewards R+=r; if not, the total reward R-=10. After calculating the reward function, the experience is stored, the estimated Q value, targetQ value and loss value are calculated, and the network state is updated from s to s'. After one training, the experience is put back. The target network is updated every 10 training times. Training is stopped after the given number of training times. The algorithm testing process is as follows. The test results are used to evaluate the algorithm performance by calculating the SFC delay requirement satisfaction rate:

[0153] The input of the algorithm is the trained DQN model, the parameters from the training phase of VROLG-DQN, the edge network topology The information and mapping scheme of K SFCs, the output is the total reward and average reward, and the SFC delay requirement satisfaction rate;

[0154] Then start n tests, update the state in each test, and for the kth SFC, from the compressed action space A k Select the SFC resource allocation action obtained by the trained DQN and calculate the probability of service flow delay violation in SFC The total reward R+=r is calculated, and the network state is updated from s to s'. At the end of each test, the SFC delay requirement satisfaction rate is calculated to evaluate the algorithm performance.

[0155] The software-defined industrial wireless network resource orchestration method for ensuring end-to-end latency described in the present invention has the following superior technical effects:

[0156] 1. The software-defined industrial wireless network resource orchestration method for ensuring end-to-end latency, described in this invention, targets IIoT uplink services. By classifying SINR prediction accuracy, it derives the service capability of the wireless domain under short block long channel coding, implements latency constraint guarantee through SNC, and ultimately implements VNF resource orchestration through a multi-task DQN.

[0157] 2. The software-defined industrial wireless network resource orchestration method for ensuring end-to-end latency described in the present invention can more accurately analyze E2E latency. The SNC theory used to analyze end-to-end latency characteristics can accurately characterize the end-to-end latency characteristics in hybrid wireless and wired transmission networks in industrial Internet of Things environments. Compared with traditional E2E latency analysis methods, the latency analysis solution of the present invention can provide the E2E latency of services in heterogeneous network environments across wireless and wired domains under short block long channel coding based on scenario requirements and different levels of SINR prediction accuracy.

[0158] 3. The software-defined industrial wireless network resource orchestration method for ensuring end-to-end latency described in this invention is a more efficient resource orchestration solution algorithm. Addressing the explosive action space, combined with the characteristics of the cascade theorem formula in SNC, it achieves action space compression. Through the multi-task DQN, it effectively learns commonalities from different SFC service flow characteristics with the simplest possible neural network structure, while also outputting task-specific results, making the algorithm widely applicable to large-scale VNF resource orchestration.

[0159] 4. The software-defined industrial wireless network resource orchestration method for ensuring end-to-end latency described in the present invention can achieve more efficient network resource utilization. Through more accurate analysis of E2E latency, resource allocation schemes that were previously excluded due to overestimation of latency can be reconsidered. The algorithm achieves a given SINR prediction accuracy level. By calculating the E2E latency under different resource allocation schemes, it can effectively select the resource allocation scheme with low resource overhead and high reliability in meeting latency requirements as the resource allocation scheme, thereby maximizing network resource efficiency.

[0160] 5. The software-defined industrial wireless network resource orchestration method for ensuring end-to-end latency described in the present invention combines a multi-task DQN with VNF resource orchestration. This allows DQN to extract shared features between different SFCs and implement universal status identification for tasks across SFCs. It also dynamically adapts the resource orchestration of VNFs within different SFCs and outputs a resource orchestration solution.

[0161] 6. The software-defined industrial wireless network resource orchestration method for ensuring end-to-end latency described in the present invention illustrates the effective channel capacity under four different SINR prediction accuracies under short block length channel coding, which helps to analyze and obtain a latency boundary that is closer to the actual transmission. BRIEF DESCRIPTION OF THE DRAWINGS

[0162] Figure 1 This is a schematic diagram of the physical scene of the present invention;

[0163] Figure 2 This is a schematic diagram of the communication process of the present invention in a co-frequency interference environment in a factory;

[0164] Figure 3 Schematic diagram of four levels of accuracy of interference according to the present invention;

[0165] Figure 4 This is a schematic diagram of allocating SFCs with different service flow types to different network slices in the present invention;

[0166] Figure 5 Schematic diagram of the change of reward value with the number of training times during the DQN training process of the present invention;

[0167] Figure 6 This is a smooth reward curve graph with a confidence interval of 95% according to the present invention;

[0168] Figure 7 Schematic diagram of the change of loss value with the number of iterations during the DQN training process of the present invention;

[0169] Figure 8 This is a smoothed benefit curve diagram of different E2E delay calculation solutions of the present invention;

[0170] Figure 9Schematic diagram of Q value distribution across SFCs of the present invention;

[0171] Figure 10 It is the success rate of satisfying the delay constraint requirement after allocating resources to different SFCs of the present invention. DETAILED DESCRIPTION

[0172] In order to more clearly understand the above-mentioned objectives, features and advantages of the present invention, the present application is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0173] Example

[0174] The software-defined industrial wireless network resource orchestration method for ensuring end-to-end latency includes:

[0175] Step 1. Build a system model:

[0176] Step 1.1: Create a scene model. Figure 1 The following figure shows a factory network model, which includes service devices connected in a 4×3 grid format. The service devices at the four corners are connected to the base station. Different types of industrial equipment access the network through wireless communication via adjacent base stations. The base stations and devices with general processing functions are connected in a wired manner, achieving a hybrid wireless and wired network. Industrial equipment connected to different base stations may interfere with the communication of industrial equipment in adjacent cells.

[0177] Abstract the physical scene into a weighted undirected graph in Represents a set of N physical nodes, Represents a set of E links between nodes, using Represents a physical node, represents the link connecting nodes n and m. When deploying VNFs to specific nodes, there are usually overheads such as computing resources, storage space, memory, and power supply. These overheads are closely related to the specific VNF. Each node needs to be equipped with different amounts of computing resources and storage resources to run, store VNFs, or perform other operations. The computing capacity of each edge node is expressed as For each direct link The bandwidth of the link is denoted as B (n,m) , each base station has a corresponding resource block (RB), which is expressed as

[0178] Step 1.2 Wireless channel modeling:

[0179] A typical feature of IIoT is that it has multiple devices connected via wireless communication. When these devices occupy different communication resources, co-channel interference will occur. Here, consider a typical indoor factory environment, that is, a factory with various industrial user equipments (UEs), such as sensors, robotic arms, and robots, and access the nearest base station (BS) via wireless means. These UEs are evenly distributed within the cells covered by the base station. Based on an E2E uplink transmission scenario, the interference between cells when UEs communicate with the co-channel base station is considered. Figure 2 As shown;

[0180] Assume that the base station C0 providing the target service is located at the origin of the coordinate system, and there are K interfering UEs during communication. represents the set of interfering UEs, represents the set of co-frequency base stations accessed by these interfering UEs, and the target UE is denoted as U0;

[0181] The path loss of the channel is inversely proportional to the distance, and the path loss exponent is represented by l. In addition, the small-scale fading between the UE and the BS adopts the Nakagami-m fading model, with the characteristic parameter m, the transmit power of each UE is P, and the distance from the accessed co-frequency base station is r k , when applying the partial path loss compensation factor δ∈(0,1), the kth UE (U k ) is the actual transmission power The instantaneous received SINR of U0 at the serving BS is as follows:

[0182]

[0183] in, is the signal receiving power of U0, is the variance of zero-mean additive white noise (AWGN), is the sum of the received powers of all interferences, g k Obey the Gamma distribution, d k Indicates the interference device U k The distance between the target user and the co-frequency base station C0;

[0184] SINR consists of three parts: signal, interference, and noise. Noise follows a Gaussian distribution. Based on the link layer, different scenarios have different trade-offs between resource overhead and interference perception accuracy for both signal and interference, resulting in different SINR prediction accuracies. Here, SINR prediction accuracy is graded.

[0185] Signal: In OFDMA, each UE has only one signal link with its associated base station. This application considers the following two levels of signal prediction accuracy:

[0186] Level I: Ability to accurately obtain the instantaneous power of each link;

[0187] Level D: Ability to determine the power distribution of each link;

[0188] Interference: Interference signals may exist in different units, and there may be multiple interference links. The degree of identification of these links affects modeling. The prediction accuracy of interference power is classified as follows:

[0189] Level I: Ability to accurately obtain the instantaneous power of each link;

[0190] Level D: Ability to determine the power distribution of each link;

[0191] Level A: Ability to determine the average power of each link;

[0192] Level M: only the total interference distribution received can be inferred;

[0193] like Figure 3 As shown in the figure, levels I, D, and A provide link-level interference awareness, while level M does not. To focus on the impact of link-level interference information, level M is not considered in subsequent analysis. Level I is an ideal prediction that is difficult to obtain in real experiments. Among the interference identification schemes mentioned above, such as NLRA, all achieve level A prediction.

[0194] For the convenience of subsequent expression, "signal power accuracy / interference power accuracy" is used to represent the accuracy combination of instantaneous SINR. For example, "I / D'" means that the instantaneous power of the signal link can be accurately obtained (level I), and the power distribution of each interference link can be determined (level D). Since the accuracy of D / I interference is higher than the accuracy of the signal, it is not considered.

[0195] For coding blocks of finite length, the coding efficiency is usually closely related to the block error rate (BLER), which is approximately as follows:

[0196]

[0197] Where V(γ)=1-(1+γ) -2 is the channel dispersion, n is the coding block length, Q -1 (x) is the inverse Gaussian Q function. Based on this, the general expression of the effective capacity of the wireless channel during transmission under different SINR prediction accuracies is given as follows:

[0198]

[0199] in, f γ (x) is the probability density function of SINR distribution under different SINR prediction accuracy. For the above formula, J only needs to take a small number of values to obtain an approximate result;

[0200] When the target signal and interference signals When both meet the nakagami-m fading, for different SINR prediction accuracy, G J (η) and f γ (x) has different expressions, as follows:

[0201] In the case of I / D:

[0202]

[0203] In the case of I / A:

[0204]

[0205] In the D / D case:

[0206]

[0207] in:

[0208]

[0209] In the D / A situation:

[0210]

[0211] in:

[0212]

[0213] in, T is the period, B is the bandwidth, is the shape parameter of the target signal when it fades, is the received power of the target signal, is the shape parameter of the interference signal when it fades, is the scale parameter when the interference signal fades, is the average SINR;

[0214] Step 1.3 Business Modeling:

[0215] In industrial environments, periodic updates of locations or repeated monitoring of environmental characteristics often lead to periodic traffic. Here, the target traffic is periodic traffic. In industrial ecosystems, periodic communication is a major form of communication:

[0216] Assume that a service source is at time {t=M τ +nτ, n=0,1,2,…} generates a workload of α units, where τ represents the cycle length, M is uniformly distributed in the interval [0,1], and M τ represents the initial moment of traffic transmission. For such a source, the MGF arrival process is (σ A ,ρ A )-constrained, where:

[0217] σ A (θ)=α,

[0218] When bursty traffic occurs occasionally but indicates a service alarm, the tolerance for delay is lower and the priority is higher. The arrival process of bursty traffic is modeled as a Poisson arrival process. In SNC, the MGF function is used to characterize the Poisson arrival flow with an average arrival rate λ and a given packet size of 1 / v as follows:

[0219] σ A (θ)=0,

[0220] For an arriving SFC, we first need to determine its traffic flow characteristics λ k The attributes of SFC are used to divide the SFC into different slices for mapping, such as Figure 4 As shown, if λ k If it meets the characteristics of a periodic service flow, the SFC needs to be divided into a general network slice; if k A bursty service flow needs to be divided into a bursty network slice;

[0221] Step 2: Modeling the SFC resource scheduling problem:

[0222] Step 2.1 SFC request modeling:

[0223] When K SFC requests arrive, use To represent the service function graph of SFC request k, for each definition Describes the properties of the SFC, where s k and d k Corresponding to the source node and destination node of SFCk, since each It consists of a given set of ordered VNFs, except for the source node and the destination node, using To represent the set of VNFs in SFCk, and are the nth and mth VNFs in SFCk, respectively. Assuming that the computing resources requested by each VNF are constant, Represents the computing resource requirements of SFCk, there is a set of virtual links connecting the source node s k , ordered VNFs and target node d k ,use To represent the link between the nth VNF and the mth VNF in SFCk, each SFC supports a feature λ k business flow, Indicates the delay that this business flow needs to meet, ω k Indicates the tolerable delay violation probability of this service;

[0224] When multiple VNF instances are deployed on the same CPU core, processing resources (e.g., CPU core) will be statistically multiplexed. Therefore, shared processing resources are not considered because it cannot ensure deterministic processing latency. When more VNFs share the same CPU, the CPU access latency experienced by each VNF will increase significantly. The processing latency of the already mapped VNFs will be reduced by the newly instantiated VNFs in the same CPU core, which leads to latency uncertainty in the existing SFC. In this application, it is assumed that different SFCs cannot share the same type of VNF instances. Even if the same type of VNF instances exist in the edge node, each VNF instance occupies exclusive processing resources and can only belong to a certain SFC. This application assumes that the processing latency of a VNF instance is only related to the processing resources allocated to it. The amount of resources allocated to a VNF can be arbitrary as long as the overall processing latency of the SFC can meet the latency requirements.

[0225] Step 2.2 VNF resource orchestration constraints:

[0226] For the kth SFC, use the binary variable express The deployment, when Deployed on edge nodes On time, otherwise A node can instantiate multiple VNF instances, but each VNF instance can only be deployed on one node. The deployment constraints are expressed as follows:

[0227]

[0228] Since the computing power of a node is shared by all NFV instances deployed on the node, the total CPU allocated to the VNF instances cannot exceed the total capacity of the edge node. The following formula is used:

[0229]

[0230] For SFCk, define the binary variable express Whether it is mapped to the physical link express Whether to connect the node if Mapped in Up, then are also linked, the following routing constraints are guaranteed:

[0231] if

[0232] In the above formula, the consistency of physical nodes and links is guaranteed, and the mapping of virtual links is ensured to be consistent with physical links. Then the following formula is obtained:

[0233]

[0234] In the above formula, it is ensured that the links on the path embedded in SFCk are connected to the head and tail. If the node is selected Come to SFCk To provide services, the node ensures that it is connected, that is:

[0235]

[0236] Since the bandwidth resources of the physical link (n, m) are shared by the virtual links mapped to it, the total bandwidth consumed by these virtual links cannot exceed the total bandwidth resources of the physical link (n, m). For the constraint analysis of link bandwidth resources, represents the amount of bandwidth resources required to be allocated to the virtual link between the nth and mth VNFs in SFCk, to ensure that the sum of bandwidth resources allocated to the virtual links cannot exceed the total bandwidth of the physical edge (n,m), and is defined as follows:

[0237]

[0238] End-to-end delay of the service in step 3SFC:

[0239] Step 3.1 Delay calculation method

[0240] In URLLC, latency is a key QoS guarantee metric. The actual E2E latency of a service flow must be less than a given maximum latency under given reliability requirements. For service flows within an SFC, latency is composed of link transmission delay, propagation delay, and node processing delay. The SFC service latency is determined by the SFC's resource requirements and the amount of resources allocated to it. It is calculated as follows:

[0241] For the kth SFC, the link transmission delay is determined by the transmission delay of the wireless channel and transmission delays in wired networks The transmission delay in a wired network is composed of the size of the transmitted packet b k Divide by the bandwidth capacity allocated to the virtual link To calculate, it is given by the following formula:

[0242]

[0243] For distance propagation delay, the factor affecting it is the physical distance L that the service transmits in the link. flow and the speed of light c, that is When the link length is 3×10 5 When m, a delay of about 1ms is generated. In the IIoT scenario, the impact of resource allocation on latency is the main factor, and the propagation delay will be ignored;

[0244] Regarding data processing latency, since VNFs are typically instantiated by associating them with a certain resource combination (e.g., CPU, RAM, etc.), the service processing latency is calculated as a function of CPU frequency allocation, as given by the following formula:

[0245]

[0246] In the above formula, represents the computing capacity allocated by the node to the nth VNF in the kth SFC. The total latency is expressed as:

[0247]

[0248] Step 3.2 Improvement of the calculation method for time delay solution:

[0249] The following improvements are made to the traditional delay analysis method:

[0250] Service capability characterization: The expression of the effective capacity of the data transmission capability of the wireless access node (base station) is expressed in Indicates the service capability provided by a mapped node assigned to the nth VNF in the kth SFC; Indicates that a mapped link is given to the virtual link in the kth SFC The ability to provide services, these service capability expressions can be expressed using the random network calculus ρ S express:

[0251] E2E latency in multi-node transmission: Using the SNC latency solution formula and combining it with the characteristics of the service flow λ k The target delay at a given time can be solved Delay violation probability of the downstream service flow

[0252] because and They are similar in characterizing service capabilities and ignore the resource orchestration of VNFs. For a mapped SFCk, if the wireless channel transmission capability provided by the base station is expressed as The initial backlog is The data processing capability provided by the nodes in the mapped wired network is expressed as The business flow backlog on each mapping node is The overall service capacity and system backlog of the entire business flow E2E are expressed as follows:

[0253]

[0254] in, ∈ is to avoid the minimum value added by the denominator to be 0. When all service processes satisfy independent and identical distribution, p1, p2, ..., p N =1, when the violation probability ω is given, the expression of E2E delay is:

[0255]

[0256] Step 4: VNFs resource orchestration algorithm with DQN-based latency guarantee:

[0257] Step 4.1 Algorithm design:

[0258] In order to achieve deterministic network services with minimal resource consumption while ensuring service delay, we define p and q as the resource cost coefficients in the network. For a given k-th SFC, the resource allocation required to serve its service flow is expressed as Since RB resources and CPU resources are discrete, It represents the number of resource allocations, and the resource overhead ultimately brought to the entire network is as follows:

[0259]

[0260] In order to achieve the minimum resource overhead, the objective function is defined as: P:minQ;

[0261] In order to guarantee the service flow, it is necessary to make the E2E delay violation probability of a resource allocation scheme small enough under a given E2E delay. For the SFCk service flow delay violation probability ω k As an important indicator, when the E2E delay of a service flow exceeds a certain value, the dB value of the delay violation probability and the delay value are approximately linearly related. The reward function is defined as follows:

[0262]

[0263] When the delay violation probability requirement is met, the smaller Q is, the larger the value of r is, where The calculation method is as follows:

[0264]

[0265] The target Q network and loss function are defined as follows:

[0266]

[0267] In the above formula, s is the state, a is the action, is the target network parameter, ζ is the discount factor;

[0268] For the state space It includes the remaining resources of each base station and each physical server, as well as the characteristics of services in different SFCs;

[0269] For the action space Generate the applicable action space for each SFC each time It only includes the resource allocation actions of the associated nodes, and the service capabilities under the cascade in SNC The expression shows that its value is related to the minimum service capacity of all nodes through which the business flow flows. The business flow on each SFC flows through a base station and several physical servers. Each SFC has a resource allocation action space of different dimensions. In order to facilitate the training of DQN and unify the action space and facilitate training, it is assumed that the same computing resources are allocated to different VNFs on the SFC. In this way, for the action space Its dimension changes from (n+1) dimensions (n represents the number of VNFs) to 2 dimensions, thus compressing the action space.

[0270] Multi-Task Reinforcement Learning (MTRL) is used here. The network design includes two layers of neural networks. The first layer is used to implement shared feature extraction (SFE) to learn the common state representation across SFC tasks; the second layer is task-specific output heads (TSH) to map each task to a different action space. Realize dynamic adaptation of tasks;

[0271] Step 4.2: Training and testing the algorithm:

[0272] Based on step 4.1, we designed a VNFs resource orchestration with latency guarantee based on DQN (VROLG-DQN) algorithm. The training process of the algorithm is as follows:

[0273] The input of the algorithm is the edge network topology, the information of K SFCs and the corresponding mapping scheme, and the output of the algorithm is the dynamic allocation scheme of various network resources for each SFC. Before the algorithm is executed, it is necessary to initialize Other network parameters and ζ, DQN related parameters such as replay buffer;

[0274] Then, n training sessions are started, where each time, for the kth SFC, the ∈-greedy strategy is used to select the action from the compressed action space Select the kth SFC resource allocation action, calculate the resource vector and allocate it to the kth SFC. For the allocated resource vector, determine whether the resource allocation action is valid and meets the VNF resource orchestration constraints. If so, calculate the service flow delay violation probability of each SFC. Each reward r k The total reward R+ = r; if not, the total reward R- = 10. After calculating the reward function, store the experience, calculate the estimated Q value, targetQ value and loss value, update the network state from s to s', and after one training, put the experience back in. Update the target network every 10 training times. Stop training after the given number of training times.

[0275] The algorithm testing process is as follows. The test results evaluate the algorithm performance by calculating the SFC delay requirement satisfaction rate:

[0276] The input of the algorithm is the trained DQN model, the parameters from the training phase of VROLG-DQN, the edge network topology The information and mapping scheme of K SFCs, the output is the total reward and average reward, and the SFC delay requirement satisfaction rate;

[0277] Then start n tests, update the state in each test, and for the kth SFC, from the compressed action space Select the SFC resource allocation action obtained by the trained DQN and calculate the probability of service flow delay violation in SFC And calculate the total reward R+=r, and finally update the network state from s to s'. At the end of each test, calculate the SFC delay requirement satisfaction rate to evaluate the algorithm performance.

[0278] Verification Example

[0279] The following verification examples exemplify the practical effects of the software-defined industrial wireless network resource orchestration method for ensuring end-to-end latency described in the present invention:

[0280] For the K=10 arriving SFCs, assume that the traffic characteristics of 8 of them are periodic and are mapped to the general network slice. The traffic characteristics of the other 2 SFCs are bursty and satisfy the Poisson distribution and are mapped to the bursty network slice. For the periodic traffic, assume that its period τ∈{10,20,30}ms and its packet size α∈{10,20,30}kb, and the required target E2E delay is Required delay violation probability ω k ∈{10 -2 ,10 -3 ,10 -4 For Poisson distributed traffic, assuming that its average arrival rate λ∈{0.1,0.2}, its packet size Required target E2E latency Required delay violation probability ω k ∈{10 -4 ,10 -5};

[0281] For the physical network, it is assumed that each base station has 30 RB resources, each RB size is 180 kHz, and each node has 30 CPU resources. The wireless channel EC model used considers the case when the SINR prediction accuracy is 1 / A. The SINR prediction accuracy is achieved with the help of the NLRA algorithm. The shape parameter m of the Nakagami-m fading is 2, the block error rate ∈ is 0.0046, and the coding block length n is 168. 80% of the resources of each physical node are allocated to the general slicing network and the remaining 20% are allocated to the bursty slicing network. The CPU clock frequency of the physical node is defined as 100 kHz.

[0282] During training, the number of neurons in both layers of the neural network was defined as 64. The resource cost coefficients p and q were set to 0.5, ζ to 0.995, and the training cycle was 2000 times. The testing process was repeated 100 times. When calculating the E2E delay, the traditional E2E delay calculation method was selected as a comparison solution. The verification results are as follows:

[0283] Figure 5 The figure shows the change of reward value during the training process of multi-task DQN under the E2E delay calculation scheme. Figure 6 The smoothed reward curve with a 95% confidence interval is shown. It can be seen that in the early exploration stage of training, the reward fluctuates greatly due to the high randomness of action selection. As the exploration rate in the ε-greedy strategy gradually decreases, the strategy gradually shifts towards utilization, which increases the mean reward. The discrete jump characteristics of resource allocation actions between different tasks lead to local optimal strategy switching. In addition, the short-term Q value overestimation caused by delayed updates of the target network causes periodic oscillations in the reward curve under the overall upward trend. However, the overall reward function gradually converges to a certain value with the increase in the number of training times.

[0284] Figure 7 The figure shows the change in loss during the training of a multi-task DQN under the E2E delay calculation scheme. It can be clearly seen that at the beginning of network training, the loss value first increases and then decreases. After 1200 times, the network loss value gradually stabilizes and approaches 0, indicating that the algorithm achieves convergence with a relatively small number of training times. The oscillation in the rising stage of the loss value is due to the high learning rate or complex action space during gradient update of the randomly initialized network parameters in the initial stage of training, which leads to a sudden increase in the Q-value estimation error. However, with the accumulation of valid samples in experience replay and the periodic synchronization of the target network, the agent gradually learns more stable value estimates. At the same time, the exponential decay of the exploration rate reduces the interference of random actions, allowing the policy network to converge to a better Q-value surface.

[0285] Figure 8 The data shows how the reward function changes during multi-task DQN training using both the SNC cascade theorem and traditional methods for calculating E2E delay. As training progresses, the E2E delay calculation scheme achieves higher rewards. Given that the reward function is a variable related to resource overhead, the lower the resource overhead, the greater the reward value. This demonstrates that the E2E delay calculation scheme can select a resource allocation solution that has low resource overhead and meets delay requirements with high reliability, thereby maximizing network efficiency.

[0286] Figure 9The Q-value distribution across SFCs is described. During the experiment, the traffic flows in SFC1 to SFC8 have periodic arrival characteristics. Among them, SFC1, SFC4, SFC5, and SFC8 have the same traffic flow size characteristics, SFC2 and SFC6 have the same traffic flow size characteristics, SFC3 and SFC7 have the same traffic flow size characteristics, and SFC9 and SFC10 are Poisson flows with different internal traffic flow characteristics. From the Q-value distribution, it is clear that although each SFC has a different mapping scheme, the Q-value distribution between SFCs with the same traffic flow size characteristics and arrival characteristics is consistent, reflecting the effective capture of common characteristics by the DQN shared layer.

[0287] Figure 10 The success rate of delay constraint satisfaction after resource allocation for different SFCs is shown. It can be seen that all SFCs achieve a delay requirement satisfaction rate of over 96%. SFC5, SFC7, SFC8, and SFC10 have some individual delay requirements not met, but overall they are met. This demonstrates that the algorithm can effectively ensure that SFCs meet delay requirements by allocating RB and CPU resources.

[0288] The present invention is not limited to the above-described embodiments. The above-described embodiments and descriptions are merely illustrative of the principles of the present invention. Various changes and modifications may be made to the present invention without departing from the spirit and scope of the present invention. Such changes and modifications are intended to fall within the scope of the present invention. The scope of protection claimed in the present invention is defined by the appended claims.

Claims

1. A software-defined industrial wireless network resource orchestration method for ensuring end-to-end latency, comprising: Step 1. Build a system model; Step 2. Modeling of SFC resource scheduling problem; Step 3. End-to-end delay of the service in SFC; Step 4. VNFs resource orchestration algorithm with latency guarantee based on DQN.

2. According to the software-defined industrial wireless network resource orchestration method for ensuring end-to-end latency according to claim 1, the step of constructing the system model in step 1 includes the following steps: Step 1.1: Construct a network scenario model. In a factory network model, a 4×3 grid of service devices is formed. The service devices at each corner are connected to a base station. Different types of industrial equipment access the network wirelessly through adjacent base stations. Base stations and devices with general processing functions are connected via wired connections, creating a hybrid wireless and wired network. Industrial equipment connected to different base stations may interfere with the communications of industrial equipment in adjacent cells. Abstract the physical scene into a weighted undirected graph in, Represents a set of N physical nodes, Represents a set of E links between nodes, using Represents a physical node, represents the link connecting nodes n and m. When deploying VNFs to specific nodes, there are usually overheads such as computing resources, storage space, memory, and power supply. These overheads are closely related to the specific VNF. Each node needs to be equipped with different amounts of computing resources and storage resources to run, store VNFs, or perform other operations. The computing capacity of each edge node is expressed as For each direct link The bandwidth of the link is denoted as B (n,m) , each base station has a corresponding resource block (RB), expressed as Step 1.2 Wireless channel modeling: A typical feature of the IIoT is the presence of multiple devices connected via wireless communication. These devices occupy different communication resources, generating co-channel interference. Consider a typical indoor factory environment, where various industrial user equipment (UEs), such as sensors, robotic arms, and robots, are wirelessly connected to the nearest base station. These UEs are evenly distributed within the base station's coverage area. This scenario is based on an E2E uplink transmission scenario, and considers the inter-cell interference when UEs communicate with the co-channel base station. Assume that the base station C0 providing the target service is located at the origin of the coordinate system, and there are K interfering UEs during communication. represents the set of interfering UEs, represents the set of co-frequency base stations accessed by these interfering UEs, and the target UE is denoted as U0; The path loss of the channel is inversely proportional to the distance, and the path loss exponent is represented by l. In addition, the small-scale fading between the UE and the BS adopts the Nakagami-m fading model, with the characteristic parameter m, the transmit power of each UE is P, and the distance from the accessed co-frequency base station is r k , when applying the partial path loss compensation factor δ∈(0,1), the kth UE, namely U k The actual transmission power is The instantaneous received SINR of U0 at the serving BS is as follows: in, is the signal receiving power of U0, is the variance of zero-mean additive white noise, is the sum of the received powers of all interferences, g k Obey the Gamma distribution, d k Indicates the interference device U k The distance between the target user and the co-frequency base station C0; SINR consists of three parts: signal, interference, and noise. Noise follows a Gaussian distribution. Based on the link layer, different scenarios have different trade-offs between resource overhead and interference perception accuracy for both signal and interference. This leads to different SINR prediction accuracies. SINR prediction accuracies are classified as follows: Signal: In OFDMA, each UE has only one signal link with its associated base station. This application considers the following two levels of signal prediction accuracy: Level I: Ability to accurately obtain the instantaneous power of each link; Level D: Ability to determine the power distribution of each link; Interference: Interference signals may exist in different units, and there may be multiple interference links. The degree of identification of these links affects modeling. The prediction accuracy of interference power is classified as follows: Level I: Ability to accurately obtain the instantaneous power of each link; Level D: Ability to determine the power distribution of each link; Level A: Ability to determine the average power of each link; Level M: only the total interference distribution received can be inferred; Levels I, D, and A provide link-level interference awareness, while Level M does not. To focus on the impact of link-level interference information, Level M is not considered in subsequent analysis. Level I is an ideal prediction that is difficult to obtain in real experiments. The interference identification schemes mentioned above, such as NLRA, all achieve Level A prediction. For simplicity, "signal power accuracy / interference power accuracy" is used to represent the accuracy combination of instantaneous SINR. "I / D" means that the instantaneous power of the signal link can be accurately obtained to level I, and the power distribution level D of each interference link can be determined. For finite-length coding blocks, the coding efficiency is usually closely related to the block error rate, which can be approximated as follows: Where V(γ)=1-(1+γ) -2 is the channel dispersion, n is the coding block length, Q -1 (x) is the inverse Gaussian Q function. Based on this, the general expression of the effective capacity of the wireless channel during transmission under different SINR prediction accuracies is given as follows: in, f γ (x) is the probability density function of SINR distribution under different SINR prediction accuracy. For the above formula, J only needs to take a small number of values to obtain an approximate result; When the target signal and interference signals When both meet the nakagami-m fading, for different SINR prediction accuracy, G J (η) and f γ (x) has different expressions, as shown below: In the I / D case: In the case of I / A: In the D / D case: in: In the D / A situation: in: in, T is the period, B is the bandwidth, is the shape parameter of the target signal when it fades, is the received power of the target signal, is the shape parameter of the interference signal when it fades, is the scale parameter when the interference signal fades, is the average SINR; Step 1.3 Business Modeling: In industrial environments, periodic updates of locations or repeated monitoring of environmental characteristics often lead to periodic traffic. Here, the target traffic is periodic traffic. In industrial ecosystems, periodic communication is a major form of communication: Assume that a service source is at time {t=M τ +nτ, n=0,1,2,…} generates a workload of α units, where τ represents the cycle length, M is uniformly distributed in the interval [0,1], and M τ represents the initial moment of traffic transmission. For such a source, the MGF arrival process is (σ A ,ρ A )-constrained, where: s A (i)=a, When a bursty service flow occurs occasionally, it indicates a service alarm, has a lower tolerance for delay, and has a higher priority. The arrival process of the bursty service flow is modeled as a Poisson arrival process. In SNC, the MGF function is used to characterize the Poisson arrival flow with an average arrival rate λ and a given packet size of 1 / ν as follows: s A (θ)=0, For an arriving SFC, it is necessary to determine its traffic flow characteristics λ k The property of SFC is used to divide it into different slices for mapping. If λ k If it meets the characteristics of a periodic service flow, the SFC needs to be divided into a general network slice; if k A bursty service flow needs to be divided into bursty network slices.

3. According to the software-defined industrial wireless network resource orchestration method for ensuring end-to-end latency as described in claim 1, the SFC resource orchestration problem modeling described in step 2 includes the following steps: Step 2.1 SFC request modeling: When K SFC requests arrive, use To represent the service function graph of SFC request k, for each definition Describes the properties of the SFC, where: s k and d k Corresponding to the source node and destination node of SFCk, since each It consists of a given set of ordered VNFs, except for the source node and the destination node, using To represent the set of VNFs in SFCk, and are the nth and mth VNFs in SFCk, respectively. Assuming that the computing resources requested by each VNF are constant, Represents the computing resource requirements of SFCk, there is a set of virtual links connecting the source node s k , ordered VNFs and target node d k ,use To represent the link between the nth VNF and the mth VNF in SFCk, each SFC supports a feature λ k business flow, Indicates the delay that this business flow needs to meet, ω k Indicates the tolerable delay violation probability of this service; When multiple VNF instances are deployed on the same CPU core, processing resources are statistically multiplexed. Sharing of processing resources is not considered here. When more VNFs share the same CPU, the CPU access latency experienced by each VNF increases significantly. The processing latency of already mapped VNFs will be reduced by the newly instantiated VNFs on the same CPU core, which leads to latency uncertainty in the existing SFC. To this end, we assume that different SFCs cannot share the same type of VNF instances. Even if VNF instances of the same type exist in edge nodes, each VNF instance occupies dedicated processing resources and can only belong to a specific SFC. Assuming that the processing latency of a VNF instance is only related to the processing resources allocated to it, the amount of resources allocated to a VNF can be arbitrary as long as the overall processing latency of the SFC can meet the latency requirements. In addition to CPU processing resources being allocated, wireless channel RB resources are also allocated for wireless-side transmission services. The allocation of wireless-side RB resources and wired-side CPU processing resources together constitute the resource orchestration of the SFC. Step 2.2 SFC resource scheduling constraints: For the kth SFC, use the binary variable express The deployment, when Deployed on edge nodes On time, otherwise A node can instantiate multiple VNF instances, but each VNF instance can only be deployed on one node. The deployment constraints are expressed as follows: Given that the computing power of a node is shared by all NFV instances deployed on that node, the total CPU allocated to VNF instances cannot exceed the total capacity of the edge node, which gives the following formula: For SFCk, define the binary variable express Whether it is mapped to the physical link express Whether to connect the node if Mapped in Up, then are also linked, the following routing constraints are guaranteed: if In the above formula, the consistency of physical nodes and links is guaranteed, and the mapping of virtual links is ensured to be consistent with physical links. Then the following formula is obtained: In the above formula, it is ensured that the links on the path embedded in SFCk are connected to the head and tail. If the node is selected Come to SFCk To provide services, the node ensures that it is connected, that is: Since the bandwidth resources of the physical link (n, m) are shared by the virtual links mapped to it, the total bandwidth consumed by these virtual links cannot exceed the total bandwidth resources of the physical link (n, m). For the constraint analysis of link bandwidth resources, represents the amount of bandwidth resources required to be allocated to the virtual link between the nth and mth VNFs in SFCk, ensuring that the sum of bandwidth resources allocated to the virtual links cannot exceed the total bandwidth of the physical edge (n,m), and is defined as follows:

4. According to the software-defined industrial wireless network resource orchestration method for ensuring end-to-end latency as described in claim 1, the end-to-end latency of the service in the SFC described in step 3 includes the following steps: Step 3.1 Calculation method for time delay: In URLLC, latency is a key QoS guarantee metric. The actual E2E latency of a service flow must be less than a given maximum latency under given reliability requirements. For service flows within an SFC, latency is composed of link transmission latency, distance propagation latency, and data processing latency in nodes. The SFC service latency is determined by the SFC's resource requirements and the amount of resources allocated to it. The calculation method is as follows: For the kth SFC, the link transmission delay is determined by the transmission delay of the wireless channel and transmission delays in wired networks The transmission delay in a wired network is composed of the size of the transmitted packet b k Divide by the bandwidth capacity allocated to the virtual link To calculate, it is given by the following formula: For distance propagation delay, the factor affecting it is the physical distance L that the service transmits in the link. flow and the speed of light c, that is When the link length is 3×10 5 When m, a delay of about 1ms is generated. In the IIoT scenario, the impact of resource allocation on latency is the main factor, and the propagation delay will be ignored; Regarding data processing latency, since VNFs are typically instantiated by associating them with a certain resource combination (e.g., CPU, RAM, etc.), the service processing latency is calculated as a function of CPU frequency allocation, as given by the following formula: In the above formula, represents the computing capacity allocated by the node to the nth VNF in the kth SFC. The total latency is expressed as: Step 3.2 Improvement of the calculation method for time delay solution: The following improvements are made to the traditional delay analysis method: Service capability characterization: The expression of the effective capacity of the data transmission capability of the wireless access node, using Indicates the service capability provided by a mapped node assigned to the nth VNF in the kth SFC; Indicates that a mapped link is given to the virtual link in the kth SFC The ability to provide services, these service capability expressions can be expressed using the random network calculus ρ S express: E2E latency in multi-node transmission: Using the SNC latency solution formula and combining it with the characteristics of the service flow λ k The target delay at a given time can be solved Delay violation probability of the downstream service flow because and They are similar in characterizing service capabilities and ignore the resource orchestration of VNFs. For a mapped SFCk, if the wireless channel transmission capability provided by the base station is expressed as The initial backlog is The data processing capability provided by the nodes in the mapped wired network is expressed as The business flow backlog on each mapping node is The overall service capacity and system backlog of the entire business flow E2E are expressed as follows: in, ∈ is to avoid the minimum value added by the denominator to be 0. When all service processes satisfy independent and identical distribution, p1, p2, ..., p N =1, when the violation probability ω is given, the expression of E2E delay is:

5. According to the software-defined industrial wireless network resource orchestration method with end-to-end latency guarantee of claim 1, the DQN-based VNFs resource orchestration algorithm with latency guarantee in step 4 comprises: Step 4.1 Algorithm design: In order to achieve deterministic network services with minimal resource consumption while ensuring service latency, we define p and q as the resource cost coefficients within the network. For a given k-th SFC, the resource allocation required to serve its service flow is expressed as Since RB resources and CPU resources are discrete, It represents the number of resource allocations, and the resource overhead ultimately brought to the entire network is as follows: In order to achieve the minimum resource cost, the objective function is defined as: P:minQ, In order to ensure strict service flow services, it is necessary to make the E2E delay violation probability of a resource allocation scheme small enough under a given E2E delay. For the SFCk service flow delay violation probability ω k As an important indicator, the reward function is defined as follows: When the delay violation probability requirement is met, the smaller Q is, the larger the value of r is, where The calculation method is as follows: The target Q network and loss function are defined as follows: In the above formula, s is the state, a is the action, is the target network parameter, ζ is the discount factor; For the state space It includes the remaining resources of each base station and each physical server, as well as the characteristics of services in different SFCs; For the action space Generate the applicable action space for each SFC each time It only includes the resource allocation actions of the associated nodes, and the service capabilities under the cascade in SNC The expression shows that its value is related to the minimum service capacity of all nodes through which the business flow flows. The business flow on each SFC flows through a base station and several physical servers. Each SFC has a resource allocation action space of different dimensions. In order to facilitate the training of DQN and unify the action space and facilitate training, it is assumed that the same computing resources are allocated to different VNFs on the SFC. In this way, for the action space Its dimension changes from (n+1) to 2, achieving the compression of the action space; This application adopts multi-task reinforcement learning. The network design includes two layers of neural networks. The first layer is used to implement shared feature extraction and learn common state representations across SFC tasks. The second layer is a task-specific output head that maps each task to a different action space. Realize dynamic adaptation of tasks; Step 4.2: Training and testing the algorithm: Based on step 4.1, design a VNFs resource orchestration algorithm with DQN latency guarantee. The algorithm training process is as follows: The input of the algorithm is the edge network topology, the information of K SFCs and the corresponding mapping scheme. The output of the algorithm is the dynamic allocation scheme of various network resources for each SFC. Before the algorithm is executed, it needs to be initialized. Other network parameters and ζ, DQN related parameters such as replay buffer; Then, n training sessions are started, where each time, for the kth SFC, the ∈-greedy strategy is used to select the action from the compressed action space A k Select the kth SFC resource allocation action, calculate the resource vector and allocate it to the kth SFC. For the allocated resource vector, determine whether the resource allocation action is valid and meets the VNF resource orchestration constraints. If so, calculate the service flow delay violation probability of each SFC. Each reward r k The sum of the total rewards R+=r; if not, the total reward R-=10. After calculating the reward function, the experience is stored, the estimated Q value, targetQ value and loss value are calculated, and the network state is updated from s to s'. After one training, the experience is put back. The target network is updated every 10 training times. Training is stopped after the given number of training times. The algorithm testing process is as follows. The test results are used to evaluate the algorithm performance by calculating the SFC delay requirement satisfaction rate: The input of the algorithm is the trained DQN model, the parameters from the training phase of VROLG-DQN, the edge network topology The information and mapping scheme of K SFCs, the output is the total reward and average reward, and the SFC delay requirement satisfaction rate; Then start n tests, update the state in each test, and for the kth SFC, from the compressed action space A k Select the SFC resource allocation action obtained by the trained DQN and calculate the probability of service flow delay violation in SFC The total reward R+=r is calculated, and the network state is updated from s to s'. At the end of each test, the SFC delay requirement satisfaction rate is calculated to evaluate the algorithm performance.

Images