A network slice control system and control method

Abstract

Embodiments of the present application provide a network slice control system and a control method. The system comprises a slice orchestrator and a plurality of domain controllers; the slice orchestrator is configured to divide the end-to-end delay guaranteed by the network slice SLA among the network domains according to the network slice SLA and the index constraint, and to issue the delay division result to the plurality of domain controllers; each domain controller is configured to perform resource allocation of the network domain by using a resource mapping mechanism according to the delay division result of the network domain issued by the slice orchestrator, and to feed back the QoS guaranteed by the resource allocation to the slice orchestrator; the slice orchestrator is further configured to determine the SLA guarantee condition according to the QoS fed back by the domain controller, and to adjust the division of the end-to-end delay guaranteed by the network slice SLA among the network domains according to the SLA guarantee condition. The present application can improve the network system capacity and guarantee the user service quality on the premise that the agreed SLA is guaranteed.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of communication technology, and in particular to a network slice control system and a control method. BACKGROUND

[0002] With the development of artificial intelligence technology, wireless communication systems are also continuously developing at a high speed. 5G (5th generation mobile networks, 5G) supports three application scenarios, including enhanced mobile broadband (eMBB), ultra-reliable low latency communications (uRLLC), and massive machine type communications (mMTC), and the future wireless communication system will evolve towards greater throughput, lower latency, higher reliability, greater number of connections, higher spectrum utilization, and the like. Existing research work shows that artificial intelligence has important application potential in many aspects, such as complex unknown environment modeling, learning, channel prediction, intelligent signal generation and processing, network state tracking and intelligent scheduling, network optimization deployment, and is expected to promote the evolution of future communication paradigm and the reform of network architecture, and has very important significance and value for next-generation network technology research. Reinforcement learning, as an important branch of machine learning, plays a huge role in decision-making and optimization of complex problems, such as defeating top masters in chess games, allocating and scheduling network resources, and intelligently recommending according to user interests. Nowadays, the structure of wireless networks is becoming increasingly complex, and the interference brought by dense network coverage cannot be ignored. Complex network environment, infinite state space, and high-dimensional optimization parameters are a challenge for traditional optimization methods.

[0003] 3GPP (3rd Generation Partnership Project, 3GPP) officially determines the name of 5G evolution as 5G-Advanced, marking the global 5G technology and standard development entering a new stage. Network slicing and edge computing, as key technologies facing the demand of the Internet of Everything era, will release vertical industry service capabilities and promote the deep integration of 5G and industry. Based on the service level agreement (SLA) signed with the same tenant, the operator creates a network slice, divides a logical network that meets specific network capabilities and characteristics, and provides mutually isolated, functionally customizable network services for different vertical industries, different customers, and different businesses. Network slicing is an end-to-end virtual network of a 5G network and is a collection of logical network functions. Network slicing is a demand-based networking approach that allows operators to separate multiple virtual end-to-end networks on a unified infrastructure. Each network slice is logically isolated from the wireless access network to the bearer network and then to the core network to adapt to various types of applications. In a network slice, at least three parts of the wireless network sub-slice, the bearer network sub-slice, and the core network sub-slice can be divided. Network slicing technology allows relatively easy configuration and reuse of network elements and functions in each network slice subnetwork instance to meet specific application requirements, realizing the transition from 'one communication subnetwork can meet all communication needs of the scene' to 'one network slice instance can meet the communication needs of the current scene by arranging and combining part of the network resources'.

[0004] eMBB and URLLC are two major scenarios of 5G communication system networks, and the business requirements are significantly different. Exploring the best resource scheduling and allocation strategy for two services with significantly different QoS (Quality of Service) requirements is a key issue.

[0005] It should be noted that the above introduction to the technical background is only to facilitate a clear and complete description of the technical solutions of the present application, and to facilitate the understanding of those skilled in the art. The above technical solutions cannot be considered as known to those skilled in the art merely because they are described in the background section of the present application. SUMMARY

[0006] The embodiment of the present application proposes a network slice control system and control method, which improves the capacity of the network system and guarantees the quality of service of the user under the premise that the agreed SLA is guaranteed.

[0007] According to a first aspect of the embodiments of the present application, a network slice control system is provided, which comprises a slice orchestrator and a plurality of domain controllers; the slice orchestrator is configured to divide an end-to-end latency guaranteed by a network slice SLA among network domains according to the network slice SLA and index constraints, and to issue a latency division result to the plurality of domain controllers; each domain controller is configured to perform resource allocation in the network domain by using a resource mapping mechanism according to the latency division result of the network domain issued by the slice orchestrator, and to feed back QoS guaranteed by the resource allocation to the slice orchestrator; the slice orchestrator is further configured to determine SLA guarantee conditions according to the QoS fed back by the domain controller, and to adjust the division of the end-to-end latency guaranteed by the network slice SLA among the network domains according to the SLA guarantee conditions.

[0008] According to a second aspect of the embodiments of the present application, a network slice control method is provided, which comprises: step 1, dividing an end-to-end latency guaranteed by a network slice SLA among network domains according to the network slice SLA and index constraints, to generate a latency division result; step 2, performing resource allocation in the network domains by using a resource mapping mechanism according to the latency division result, and feeding back QoS guaranteed by the resource allocation; and step 3, determining SLA guarantee conditions according to the fed back QoS, and returning to step 1 to adjust the division of the end-to-end latency guaranteed by the network slice SLA among the network domains according to the SLA guarantee conditions.

[0009] The network slice control system and method disclosed in the embodiments of the present application design an end-to-end network slice SLA-based cross-domain orchestration framework, take latency as a cross-domain coordination index, and take transmission rate as an intra-domain constraint index, to realize end-to-end slice resource management. The framework is used to divide a network slice end-to-end constraint latency into latency limits to be met by RAN and CN, and is easy to extend to more domains. In addition, an end-to-end constraint latency division method based on a latency balancing mechanism is proposed, which can improve network capacity (access user number) and guarantee eMBB user service quality by flexibly adjusting network domain latency limits. In order to solve the non-convex and NP-hard problems of RAN and CN resource allocation, a deep reinforcement learning algorithm is designed, which is used to manage wireless resources of users, and to allocate core network element and link resources.

[0010] Specific embodiments of the present application are disclosed in detail in the following description and claims, indicating the ways in which the principles of the present application can be employed. It should be understood that the embodiments of the present application are not limited in scope to the specific embodiments described herein. Embodiments of the present application include many alterations, modifications and equivalent arrangements within the scope of the appended claims and their equivalents.

[0011] Features described and / or illustrated with respect to one implementation can be used in the same or similar manner in one or more other implementations, in combination with or in place of features in other implementations, or in place of or in addition to features described or illustrated with respect to other implementations.

[0012] It should be emphasized that the term "comprises / comprising" when used in this specification is taken to specify the presence of stated features, integers, steps or components but does not preclude the presence or addition of one or more other features, integers, steps, components or groups thereof. BRIEF DESCRIPTION OF DRAWINGS

[0013] The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the principles of the application. It is to be understood that the drawings are solely for purposes of illustration to one of ordinary skill in the art, the present application can be implemented in many embodiments of which the following description is one example only.

[0014] Figure 1 The schematic diagram of the architecture of the network slice control system of the embodiment of the present application;

[0015] Figure 2 The schematic diagram of the SFC embedding including two stages of node mapping stage and link mapping stage in the embodiment of the present application;

[0016] Figure 3 The schematic diagram of the end-to-end network slice orchestration architecture of the DSDP strategy of the embodiment of the present application;

[0017] Figure 4 The schematic diagram of the end-to-end network slice orchestration architecture of the DTDP strategy of the embodiment of the present application;

[0018] Figure 5 The schematic diagram of the principle architecture of the pointer network of the embodiment of the present application;

[0019] Figure 6 The processing flowchart of the network slice control method of the embodiment of the present application;

[0020] Figure 7 The processing flowchart of the network slice control method of another embodiment of the present application;

[0021] Figure 8 The structural schematic diagram of the network device of the embodiment of the present application;

[0022] Figure 9 The processing flowchart of the resource allocation method of the embodiment of the present application;

[0023] Figure 10 The processing flowchart of the resource allocation method of another embodiment of the present application;

[0024] Figure 11 A flow chart of a processing method for managing end-to-end latency of a network slice of an embodiment of the present application. DETAILED DESCRIPTION

[0025] The foregoing and other features of the present application will become apparent to those skilled in the art upon consideration of the following description of specific embodiments of the application, taken in conjunction with the accompanying drawings. In the description of embodiments of the application, specific terminology is employed for the sake of clarity. However, the application is not intended to be limited to the specific embodiments described, but is to be understood to include all modifications, equivalents, and alternatives that fall within the scope of the appended claims.

[0026] In 5G mobile networks, various services are deployed on shared mobile network infrastructure, which needs to cross the Radio access Network (RAN), transport network and Core network (CN), as well as edge, aggregation and central cloud servers. The mobile network infrastructure and end-to-end (E2E) network slice logical architecture can be mainly divided into two parts: access side and wired side. The wireless access network is distributed with several Base Stations (BS), which are responsible for user admission and wireless resource allocation. After the user accesses the wired network from the base station, according to the properties of the required service (network slice), the user passes through the transport network, selects the server node and internal optical fiber link in the regional data center and cloud data center (or higher level data center, such as the district level data center), and maps between the virtual network and the physical network for different network slices. The wireless link between the user and the base station, and the wired link corresponding to the slice virtual network together form the end-to-end communication link.

[0027] In addition, eMBB and URLLC are two major scenarios of 5G communication system network, and the service requirements are obviously different. 5G New Radio proposes to puncture the ongoing eMBB data transmission to meet the strict delay requirement of URLLC service. That is, when the bandwidth is insufficient, part of the already allocated eMBB network resources will be preempted by randomly entering URLLC traffic. Therefore, eMBB users with high requirements for transmission continuity will face rate loss, thereby seriously reducing the service quality. It is a key problem to explore the best resource scheduling and allocation strategy for two services with obviously different QoS (Quality of Service) requirements.

[0028] The landing of network slices requires the enhancement of the scheduling and collaborative guarantee capabilities of the network. The main feature of multi-domain slices is that the interconnection domains are self-managed, while the E2E slice can be regarded as a cascade of subnets belonging to different network domains. When creating a network slice, multiple technologies need to be used to pull together multiple network domains to achieve it. The SLA-related indicators agreed upon by the operator and the slice tenant need to be decomposed into parts that each network domain can support. The latency requirement can be negotiated between domains in a collaborative manner without compromising the autonomy and privacy of each network domain. Therefore, how to guarantee the SLA agreed upon by the tenant and the operator is a key problem for 5G networks to move towards automation.

[0029] The embodiment of the application discloses an end-to-end network slice cross-domain management architecture, which can dynamically allocate network resources to each network slice according to network slice requests and QoS feedback under the premise of guaranteeing the agreed SLA. The end-to-end network slice orchestration architecture disclosed by the embodiment of the application is composed of an end-to-end orchestrator responsible for decomposing SLA performance indicators and a domain controller for managing each domain. In the SLA end-to-end performance indicators, the throughput is limited by the minimum link transmission rate in the transmission path, and the latency is the sum of the latencies of each link in the path. Therefore, in terms of performance indicator constraints, the embodiment of the application takes the latency as the indicator that needs to be coordinated across domains, and the throughput requirement as the constraint condition for resource allocation within the domain. On this basis, combined with the deep reinforcement learning method, two latency balancing strategies are proposed to adjust the division of the end-to-end latency of the network slice SLA guarantee in each network domain, improve the network system capacity and guarantee the user service quality.

[0030] Figure 1 The architecture diagram of the network slice control system of the embodiment of the application is shown in FIG. 1. As shown in FIG. 1, the network slice control system of the embodiment of the application is composed of an end-to-end orchestrator and a domain controller. Figure 1As shown, the system comprises a slice orchestrator (E2E NS orchestrator) 10 and a plurality of domain controllers (Domain Controller) [21, 22, …, 2n]. Since the 5G converged service has differentiated communication network performance index requirements, the slice orchestrator 10 is responsible for negotiating with the user and converting the user's service quality requirement into a signed network slice SLA, without the need to understand the network structure and resource state inside each network domain. Then, the slice orchestrator 10 divides the end-to-end service quality index (delay, throughput, reliability, etc. requirements) specified by the slice SLA among the network domains, and then corresponds to issue to each network domain. The domain controller [21, 22, …, 2n] is responsible for supervising the physical network resources of the network domain, and according to the issued index, uses the internal resource mapping mechanism (Mapping Mechanism) to perform "best effort" resource allocation to ensure the service quality of the slice (network service configuration), and completes the mapping of the SLA index to the network resource. The domain controller also supervises the resource state and service quality in the network domain. Finally, each domain controller reports to the slice orchestrator 10 the completion of the SLA related index (reports the deployment result). The slice orchestrator 10 calculates the end-to-end SLA guarantee of the network slice according to the report of the domain controller, and adjusts the proportion of the SLA index divided among the network domains. The slice orchestrator 10 also completes the connection of the network slice across domains according to the current routing path and state.

[0031] In the embodiment of the present application, the slice orchestrator 10 is responsible for negotiating with the user and converting the user's service quality requirement into a signed network slice SLA. In the end-to-end performance index agreed in the network slice SLA, the throughput is limited by the minimum link transmission rate in the transmission path, and the delay is the sum of the delays of each link on the path. Therefore, in terms of performance index constraints, the embodiment of the present application takes the delay as the index that needs to be coordinated across domains, and the throughput requirement as the constraint condition for resource allocation within the domain.

[0032] It is assumed that the slice instantiation is performed before the service is provided to the user, so once the network slice is instantiated, the network can guarantee the SLA agreed with the user. The access network adopts a wireless communication mode, and the core network adopts a wired communication mode. It is assumed that there is a direct physical link between the base station and all server nodes in the core network data center, and the problem of creating a transmission network sub-slice is not considered for the time being. Therefore, in the embodiment of the present application, the end-to-end delay constraint of the SLA guarantee of the network slice in the wireless access network and the core network is mainly considered, the slice orchestrator 10 decomposes the end-to-end delay constraint of the SLA guarantee of the network slice into an access network delay constraint and a core network delay constraint, and adjusts the allocation proportion of the access network delay constraint and the core network delay constraint in the wireless access network and the core network, respectively.

[0033] Assuming that users are randomly distributed in the base station coverage area according to a Poisson point process, taking the downlink cellular network as a wireless network environment, mainly considering the coexistence scenario of URLLC and eMBB slices, the user sets of the two types of slices are respectively denoted as U u ={1,2,......,N u}, and U e ={1,2,......,N e}. Assuming that each slice only serves one user, a specific user is denoted as u k , and the total number of users N=N u +N e . Assuming that the network has a perfect synchronization system and channel estimation function, the slice and its users have relevant QoS parameters (delay or throughput requirements), in order to meet the quality of service, wireless resources need to be flexibly allocated between slices and within slices.

[0034] Thanks to the high flexibility of the 5G system physical layer, the wireless network time-frequency resources can be flexibly divided into a plurality of physical resource blocks (PRB). Assuming that the total bandwidth of the wireless system is B, the frequency domain is evenly divided into L subchannels with a bandwidth of B l ; the time domain can be divided into a plurality of scheduling frames, each of which contains T subframes with a length of Δt. Thus, the minimum PRB size of wireless time-frequency resource allocation can be determined. The set of wireless time-frequency resources PRB is denoted as M={1,2,......,M}, and it is assumed that the base station obtains a transmission power P m for the PRB m allocated to user n, and the total number of PRBs M=L·T.

[0035] σ is the noise spectral density, g n is the channel gain between the base station transmission power and the user reception, and I is the interference from other base stations. Combined with the path loss model, if user n occupies base station PRB m, the throughput obtained is:

[0036]

[0037] Define the M×N matrix A to represent the PRB allocation result of the base station, where

[0038]

[0039] The total number of PRBs that can be allocated by the base station satisfies:

[0040]

[0041] The power limit of the base station for PRB allocation is:

[0042]

[0043] The total throughput obtained by the base station for user n is:

[0044] R n =∑ m A(m,n)RB m,n (5)

[0045] The transmission rate of the wireless network is represented by the user throughput. Considering different types of terminal services, URLLC data does not appear frequently, and when there is no URLLC data, the statically reserved spectrum resources will be wasted, resulting in low efficiency of static spectrum resource reservation. The operation delay experienced by the URLLC service traffic (the processing delay of the base station to allocate the channel, the queue delay of the data packet entering the channel, etc.) can be ignored, and for the sake of simplifying the transmission model, it is regarded as an M / M / 1 queuing model, assuming that the data packet size of the user is μ bits and the arrival rate is λ. The transmission delay experienced by the URLLC user on the wireless channel is:

[0046]

[0047] The eMBB type traffic has obvious regularity and periodicity characteristics, and requires continuous communication, and also requires the delivery of data within a certain time. Taking mobile video services as an example, the traffic grows exponentially, causing serious congestion of wireless access, core network and backhaul link, and having a negative impact on the quality of service of the cellular network. Network edge caching effectively solves this problem. Caching popular files at the network edge (such as base stations) can alleviate core and backhaul congestion and reduce content delivery delay, and improve the QoS of end users.

[0048] The cache in the wireless network is generally divided into two stages of cache placement and data transmission. In the eMBB scenario, a user requests a specific file at a certain location, and the request will be served by an associated base station, and the request will "leave" the network until the requested file is completely downloaded. In addition, under the queue model, each file has an independent virtual queue in the base station. The eMBB traffic from the base station to the mobile user will experience an edge delay, and the calculation formula is as follows:

[0049]

[0050] Wherein, υ represents the processing capacity of the base station, L q (t) represents the number of data packets in the current queue, S(t) represents the number of data packets processed in a period of time, and P K represents the packet loss rate in the queue.

[0051] It is assumed that each user requests a different file, so there will be no queue about user requests formed at the base station, and the size of the traffic processed in a period of time can be represented by Rn The edge latency experienced by the eMBB user at this time is calculated as:

[0052]

[0053] where L represents the file size.

[0054] In addition, the eMBB type slice user requires a wireless transmission rate greater than a certain threshold, in order to meet the user throughput requirement, there is:

[0055] R n ≥R rsv (9)

[0056] NFV (Network Functions Virtualization) technology can replace dedicated middleware functions by running software instances, VNF (Virtual Network Function), on general servers, and can be freely deployed at any location in the network. A variety of VNFs are combined in sequence to meet the needs of business traffic. A group of VNFs and virtual links connecting them form a VNF logical link, called a service function chain (SFC), which is used to represent the specific function sequence that the network slice needs to traverse to provide E2E services for business traffic. Embodiments of the present application take the server nodes and physical links inside the data center as the physical network (SN).

[0057] The physical network SN can be abstracted as an undirected weighted graph G s =(N s ,E s ,C s ,B s ), where N s represents the server nodes existing inside the SN data center and the routing nodes existing on the transmission links between them, E s is the optical fiber link directly connected between the nodes in N s . C s and B s represent the node resource (CPU, RAM, etc.) and link resource (bandwidth, rate, etc.) capacity, respectively. It is assumed that u,v∈N s are two directly connected SN nodes, the node capacity is C s (u), C s (v), the link directly connected between them is (u,v)∈E s , the link capacity is B s (u,v), and the transmission delay is d uv .

[0058] Similar to SN, the SFCs that the n-generated traffic needs to experience on SN can be abstracted as an undirected weighted graph G n = (N n , E n , C n , B n ), where the node set N n and the edge set E n represent the VNFs and virtual links existing in the SFCs that the slice traffic needs to experience, respectively, C n and B n represent the resource requirements of the VNFs and virtual links, respectively. It is assumed that i, j∈N n are two front and rear related VNFs, the required node processing resources are represented as C n (i), C n (j), the virtual link directly connected between them is (i,j)∈E n , and the required bandwidth is B n (i,j). Different user traffic needs to experience different SFCs.

[0059] The embodiments of the present application select CPU capacity and link bandwidth as the main resources for virtual network embedding. Network nodes and links on SN are searched to meet the resource requirements of the slice to perform SFC mapping. As shown in Figure 2 , SFC embedding includes two stages of node mapping and link mapping, and the created SFC selects the node instantiation VNF in SN and occupies the physical link resources between nodes to meet its requirements. In order to ensure the isolation between slices, VNFs are not shared between SFCs. Even if the same type of VNF is needed, different instances of the same type of VNF will be generated.

[0060] In the node mapping stage, a binary variable is defined to represent the mapping relationship between the virtual node i in the SFC and the physical node u:

[0061]

[0062] Virtual nodes can only be instantiated on SN nodes whose idle resources meet their requirements, so the virtual node i and the SN node u to which it is mapped need to meet:

[0063]

[0064] In addition, a server node of SN can at most bear one virtual node of the same slice:

[0065]

[0066] In the link mapping phase, one virtual link can be embedded on multiple SN links, and the free bandwidth of all these SN links should be greater than the requested bandwidth of the virtual link. After link mapping, several physical links are occupied by the virtual link (i, j). A binary variable is used to represent whether the virtual link (i, j) is mapped to the physical link (u, v) or not:

[0067]

[0068] The bandwidth constraint of link mapping can be defined as:

[0069]

[0070] It is assumed that the core network slice data has been generated and buffered into data frames before transmission, and the processing delay is not included in the core network delay. For simplicity, the propagation delay can be ignored. The slice exclusively obtains the resources, so there is no congestion on the link. Therefore, no queue is formed at the server node, and the queuing delay of the VNF processing data flow does not need to be considered. Therefore, the core network delay experienced by the slice data flow only considers the transmission delay between NFV nodes. After SFC mapping, the delay can be characterized as:

[0071]

[0072] When all the virtual links are successfully mapped, the SFC mapping is completed.

[0073] When the network slice user accesses the base station and the SFC mapping is successful, the end-to-end communication link is built, and at this time, the user u n The end-to-end delay τ n is:

[0074]

[0075] To meet the end-to-end delay constraint of the user's network slice SLA There are

[0076]

[0077] In the case of a fixed number of network users, if the goal is to maximize the number of network access users, the network will select to reduce the access eMBB users due to the relatively small amount of wireless resources occupied by URLLC services. In this embodiment, the slice orchestrator 10 also determines the SLA guarantee condition according to the QoS feedback of the domain controller, and adjusts the partition of the end-to-end delay of the network slice SLA guarantee in each network domain according to the SLA guarantee condition. The present application designs the eMBB user service satisfaction level (SSL) to quantify the QoS of eMBB users. At the same time, in order to balance the service quality of eMBB services and the number of network user accesses, a concern coefficient a is set, a ∈ [0, 1], and a trade-off is made in the system range. a = 1 indicates that the optimization goal only focuses on maximizing the eMBB service quality, and a = 0 indicates that the optimization goal is to maximize the user end-to-end access number.

[0078] The optimization goal of the E2E network slice model of the embodiment of the present application is:

[0079]

[0080] Wherein The eMBB user service quality in the network is represented by QoE, and the proportion of the number of successfully accessed network users to the total number of users is represented by QoE. It is difficult to directly solve the problem.

[0081] The network slice can accommodate a group of users, but different user traffic needs to traverse different SFCs, so it can be regarded as coming from different slices. The concept of SLA decomposition is introduced, and the E2E delay constraint is decomposed into air interface delay constraint and core network delay constraint by the slice orchestrator 10, and is handed over to RAN and CN for resource allocation and SFC mapping. The slice orchestrator 10 first determines the end-to-end delay of each slice The allocation proportion θ of RAN n ∈ (0, 1), so the air interface delay constraint of user n The core network delay constraint is In this way, P1 can be decomposed into a wireless network resource allocation sub-problem P2 and a core network SFC mapping sub-problem P3.

[0082] The network allocates wireless resources to users and completes slice SFC mapping, and if the user throughput and end-to-end delay limit are met, the slice user successfully accesses the network. The binary variable x n = 1 indicates that user n successfully accesses the base station and meets the air interface related constraints, and the binary variable y n = 1 indicates that the SFC mapping of the slice is successful and meets the core network delay limit, so the proportion of the number of successfully accessed network users to the total number of users is

[0083] The target of resource allocation of the RAN is consistent with P1. After the RAN obtains the end-to-end delay allocation ratio, the air interface delay limit of a certain slice at this time can be calculated, so that

[0084] P2: max a SSL + (1-a) QoE (19)

[0085]

[0086] At this time

[0087] The CN maps the SFC of each slice one by one, and assuming that the data center server resources are sufficient, they can accommodate the resource requirements of each slice SFC mapping. The overall goal of mapping is to complete the SFC mapping and minimize the hop count of each slice SFC:

[0088] P3: min Hop n (21)

[0089]

[0090] The CN obtains the core network delay limit of a certain slice at this time Assuming that the forwarding time of unit hop is the same, and is t0, the maximum hop limit (Hop Limit parameter) at this time can be obtained

[0091] P2 and P3 are solved by using reinforcement learning respectively, and the reinforcement learning outputs the RAN resource allocation and CN mapping results and the delay of each slice in the current network domain.

[0092] In the embodiment of the application, a new set T is introduced to represent the difference between the actual delay experienced by the user after obtaining the resources and the end-to-end delay constraint T:

[0093] T = tau E2E -tau 1 -tau 2 (23)

[0094] For ease of expression, the user index is omitted.

[0095] Inspired by traffic delay balancing, two heuristic algorithms are designed in the embodiment of the application to determine the allocation of end-to-end constraint delay in RAN and CN. The RAN and CN use reinforcement learning for wireless resource allocation and SFC mapping according to the ratio, and obtain the delay set tau RAN , tau CN of each user in the two network domains, and then calculate the delay margin T of each user according to (23).

[0096] Delay equalization algorithm (1): adjust the allocation proportion of end-to-end delay of network slice SLA guarantee from the perspective of a single network slice, that is, different slices, different proportions (Different slices, different proportions), which can be referred to as DSDP strategy.

[0097] Wireless resources are relatively scarce, which is a bottleneck for increasing the number of network access users. A certain constant D is set as a threshold value for iteration. If the total sum of the margin ∑T>D, it means that part of the slice not only meets the end-to-end delay, but also has a certain "surplus", and the current end-to-end delay allocation proportion has optimization space. The network access user quantity can be increased by reasonably dividing the two sides of the delay constraint proportion. In one embodiment, the step size of each adjustment proportion is defined as Δε1. In the embodiment of the present application, the constant D can be selected according to the SLA guaranteed delay, such as 1ms and 20ms required by URLLC and eMBB respectively, and the constant D can be selected as 30ms as the iteration condition.

[0098] Figure 3 It is shown how the components in the end-to-end network slice orchestration architecture communicate under the DSDP strategy.

[0099] If a certain slice, θτ E2E -τ 1 ≥τ 2 -(1-θ)τ E2E , it means that the core network side of the current slice cannot meet the end-to-end delay constraint, and the air interface delay margin is greater than the part of the core network delay exceeded. At this time, try to reduce the wireless network delay threshold, relax the restriction on the core network:

[0100] θ←θ-Δε1

[0101] If a certain slice, if (1-θ)τ E2E -τ 2 ≥τ 1 -θτ E2E , it means that the wireless network side of the current slice cannot meet the end-to-end delay constraint, and the core network delay margin is greater than the part of the wireless network delay exceeded. At this time, try to increase the wireless network delay threshold, and increase the restriction on the core network:

[0102] θ←θ+Δε1

[0103] The above two adjustment methods are for slices that exist on one side that do not meet the delay constraint, which is called "intra-slice equalization". For slices that meet the end-to-end delay constraint, if the "remaining" delay on one side is greater than a certain threshold d and greater than the "remaining" delay on the other side, adjust the allocation proportion, which is called "inter-slice equalization".

[0104] If a certain slice, θτ E2E-τ 1 ≥d,θτ E2E -τ 1 ≥(1-θ)τ E2E -τ 2 , which means that the current slice can meet the end-to-end delay constraint, and the air interface delay margin is greater than the core network delay margin. Try to reduce the wireless network delay threshold value, and relax the core network limit. Define the step size of the proportion of each slice balancing adjustment as Δε2, then:

[0105] θ←θ-Δε2

[0106] If a slice, if (1-θ)τ E2E -τ 2 ≥d,(1-θ)τ E2E -τ 2 ≥θτ E2E -τ 1 , which means that the current slice can meet the end-to-end delay constraint, and the core network delay margin is greater than the wireless network delay margin. Try to increase the wireless network delay threshold value, and increase the core network limit:

[0107] θ←θ+Δε2

[0108] Adjust the slice delay allocation proportion, record the adjusted delay allocation proportion, and perform wireless resource allocation and SFC mapping again to obtain new τ RAN ,τ CN and T, calculate the number of users successfully accessing the network. Repeat the above process until the total margin sum ∑T>D is not met. In addition, in order to ensure the stability of each slice, set the delay allocation proportion interval [lowerbound, upperbound]. In the process of adjusting the proportion, the remaining delay credit is reduced, but the final proportion is not necessarily the best. Record the changes of SSL and QoE of the system, and output the allocation proportion corresponding to the maximum value of the objective function.

[0109] The delay adjustment process under the DSDP strategy is shown in Algorithm 1:

[0110] Algorithm 1: DSDP strategy for dividing SLA guaranteed delay budget

[0111] Initialize the end-to-end delay budget allocation set θ to {0.5,....,0.5}, and the capacity to N, omit the subscript for convenience of expression;

[0112] Once the RAN, CN controller obtains the allocation proportion, solve P2, P3 to obtain the domain delay set (τ RAN ,τ CN );

[0113] Calculate the delay margin T=τ E2E-τ 1 -τ 2 ;

[0114] ●While∑T>D do

[0115] ■For each slice,

[0116] ●Ifθτ E2E -τ 1 ≥τ 2 -(1-θ)τ E2E ,

[0117] θ←max(θ-Δε1,lowerbound)

[0118] ●Else if (1-θ)τ E2E -τ 2 ≥τ 1 -θτ E2E ,

[0119] θ←min(θ+Δε1,upperbound)

[0120] ●Else ifθτ E2E -τ 1 ≥d,θτ E2E -τ 1 ≥(1-θ)τ E2E -τ 2 ,

[0121] θ←max(θ-Δε2,lowerbound)

[0122] ●Else if (1-θ)τ E2E -τ 2 ≥d,(1-θ)τ E2E -τ 2 ≥θτ E2E -τ 1 ,

[0123] θ←min(θ+Δε2,upperbound)

[0124] ■End for

[0125] ■The RAN and CN controllers obtain the updated allocation ratios and solve for P2 and P3 to obtain the local domain delay set (τ). RAN ,τ CN ) and SSL

[0126] ■ Calculate the new T, QoE, and P1 objective function values.

[0127] ●End while

[0128] • Output the maximum P1 objective function value and the corresponding θ.

[0129] Delay equalization algorithm (2): Adjust the allocation proportion of end-to-end delay of network slice SLA guarantee from the perspective of different types of network slices, that is: different types, different proportions (Different types of slices, different proportions), which can be called DTDP strategy.

[0130] Unlike the DSDP strategy, the allocation proportion of E2E delay is adjusted from the perspective of a single slice. This strategy attempts to determine the optimal E2E delay allocation proportion of eMBB and URLLC two types of network slices, and at this time the set θ = {θ1, θ2} represents the proportion of RAN allocation of E2E delay of two types of slices. Similarly, let a certain constant D, the sum of the excess ∑T > D as the condition for iteration, and define the step size of each adjustment proportion as Δδ. Figure 4 For the DTDP strategy, the working mechanism of each component of the orchestration system.

[0131] Define the eMBB slice user delay set T eMBB , the URLLC slice user delay set T URLLC If It is explained that under the current proportion, the eMBB class user obtains excess resources, and the proportion of eMBB class slice end-to-end delay in RAN delay allocation can be reduced, and the proportion of URLLC class slice end-to-end delay in RAN allocation can be increased:

[0132] θ1←max(θ1-Δδ,lowerbound)

[0133] θ2←min(θ2+Δδ,upperbound)

[0134] Similarly, if

[0135] θ1←min(θ1+Δδ,upperbound)

[0136] θ2←max(θ2-Δδ,lowerbound)

[0137] The delay adjustment process under the DTDP strategy is shown in Algorithm 2:

[0138] Algorithm 2: DTDP algorithm divides the SLA guaranteed delay budget

[0139] • Initialize the delay budget allocation set θ as {0.5, 0.5};

[0140] • Once the RAN, CN controller obtains the allocation ratio, solve P2, P3 to obtain the local domain latency set (τ RAN ,τ CN );

[0141] • Calculate the latency margin T = τ E2E -τ 1 -τ 2 ;

[0142] • While∑T>D do

[0143] If

[0144] • θ1←max(θ1-Δδ,lowerbound)

[0145] • θ2←min(θ2+Δδ,upperbound)

[0146] Else if

[0147] θ1←min(θ1+Δδ,upperbound)

[0148] θ2←max(θ2-Δδ,lowerbound)

[0149] • The RAN, CN controller obtains the updated allocation ratio, solves P2, P3 to obtain the local domain latency set (τ RAN ,τ CN ) and SSL;

[0150] • Calculate the new T, QoE and P1 target function value;

[0151] • End while

[0152] • Output the maximum P1 target function value and the corresponding θ.

[0153] Through the above two latency balancing algorithms, the corresponding θ can be obtained, that is, the optimal solution of the allocation ratio of the end-to-end constraint latency in the RAN and the CN is obtained.

[0154] The above two end-to-end constraint latency division methods based on the latency balancing mechanism, the DSDP method and the DTDP method, can improve the network capacity (the number of access users) by flexibly adjusting the network domain latency limit.

[0155] In the embodiments of the present application, the domain controller at least includes a RAN domain controller and a CN domain controller; the RAN domain controller uses a reinforcement learning algorithm to perform wireless resource allocation according to the access network delay constraint; and the CN domain controller uses a reinforcement learning algorithm to perform SFC mapping according to the core network delay constraint.

[0156] 1) Radio Access Network, RAN

[0157] The wireless resource allocation problem is NP-Hard, and the present application introduces a reinforcement learning method to solve it. Reinforcement learning is a common method for solving decision problems, which has two basic elements: state and action. A strategy is a certain action performed in a certain state. An agent needs to obtain a good strategy through continuous exploration and learning.

[0158] Q-Learning is a classic algorithm of reinforcement learning, but it has a big problem. Q-Learning uses a table form (Q-Table) to store Q values. This makes Q-Learning limited to small action spaces and sample spaces, and is generally used in discrete situations. If the number of states and actions in the model is large, the size of the Q-Table will become very large, even beyond the computer memory. Moreover, searching in a huge table every time the update is also a very time-consuming thing. However, more complex and more realistic tasks often have a large state space and action space. For the field of processing high-dimensional data, deep learning has a good performance. Deep reinforcement learning combines reinforcement learning with deep learning, and uses a neural network to replace the original table to calculate the value function.

[0159] DQN is a representative algorithm of deep reinforcement learning. Based on the original table-based Q-Learning, a neural network is used to calculate the Q value (action value function). In the decision-making process, DQN takes the state as the input of the neural network, calculates the Q value of each action through the neural network analysis, and then selects the action according to the similar principle as Q-Learning. The original Q value Q(s, a) is replaced by a new form Q(s, a; θ) with neural network parameters, where θ represents the parameters of the neural network.

[0160] To reduce the problem caused by the correlation between data, DQN introduces two key technologies of experience replay and fixed target value network, which solves the problem of model easy to appear shock and divergence, and makes the training process more stable. In order to further improve the performance of DQN, many improved schemes are proposed, such as Double DQN and Prioritized Experience Replay (PER).

[0161] DDQN solves the overestimation problem in DQN. Overestimation refers to the estimated value function is larger than the true value function, and the root cause is mainly in the maximization operation in Q-Learning. When calculating target Q, the maximum Q value of the next state is obtained. For the real policy, it is not always the action that makes the Q value maximum in a given state, because the general real policy is a random policy, so the target value directly selects the Q value of the action maximum often leads to the target value higher than the real value. Double DQN solves the overestimation problem on the basis of DQN. DDQN uses different value functions to realize the selection and evaluation of actions, while in DQN, two Q networks are proposed. Therefore, the steps of calculating target Q of DDQN can be divided into two steps. The first step is to obtain the action that makes the maximum Q value through the estimation Q network The second step is to get the action value function target Q = r + γQ(s', a max ; θ - ) of the action through the target Q network. Combining the two steps together, the loss function form of DDQN can be obtained, which realizes the unbiased action estimation in the training process.

[0162]

[0163] In addition to the change of loss function, the main process of DDQN is the same as that of DQN.

[0164] In addition, in the DQN sampling process, when the replay buffer is very large, random sampling will reduce the learning effect and efficiency. PER proposes that when sampling, some samples have a higher probability of being sampled, so as to speed up the training speed and decision-making performance. The algorithm defines the priority of the sample by calculating the TD-error, and also introduces a Sum-tree structure to store the sample and its corresponding priority. The priority calculation formula is

[0165]

[0166] where, ε is a very small positive number to ensure that even if the sample of TD-error is zero, it can also be sampled. According to the sample priority, adjusting β represents the degree of attention to priority, at this time the probability of extracting a certain sample is:

[0167]

[0168] In combination with the above two variants, the DDQN-PER algorithm is proposed in the embodiments of the present application to solve the PRB allocation and power allocation problem of the wireless network.

[0169] The reinforcement learning agent is a wireless network, and the DDQN-PER algorithm runs in the base station to manage the PRB allocation and power allocation of all users. The reinforcement learning state is the signal-to-interference-and-noise ratio (SINR) of the user, and to accelerate the convergence speed of the network, normalization is performed before inputting the neural network, denoted as

[0170] The reinforcement learning action is the channel allocation of the user and the power allocation of the user, denoted as Since the output of the DDQN-PER algorithm is a discrete value, the transmission power P m ∈[0,P max ] of the user is divided into 20 discrete power values on average, and the action space of the transmission power of the user is

[0171]

[0172] In combination with the objective function, the reward function can be defined as:

[0173] Reward=αSSL+(1-α)QoE

[0174] 2) Core network CN

[0175] The SFC mapping is an NP-Hard problem, and the pointer network is used to solve it by the reinforcement learning method in the embodiments of the present application. In combination with the combinatorial optimization idea, the PN-SFC mapping algorithm is proposed. The pointer network is selected as the agent to find the node mapping strategy, and then the Dijkstra algorithm is used to complete the link mapping. The pointer network finds the optimal mapping strategy through the search process, and then updates its own parameters. The mapping algorithm returns the mapping result of each slice SFC, and the delay is calculated according to the mapping result (hop count).

[0176] In real scenarios, large-scale network topology is complex. Heuristic algorithms such as genetic algorithm are prone to fall into local optimum and slow convergence when dealing with large-scale optimization problems. When the state set is too large, the common reinforcement learning method such as Q-learning algorithm has scalability problems in storing Qtable. With the increase of network size, the space complexity increases rapidly, and it takes a long time to traverse each state, so it is not suitable for running under large-scale network topology. Using deep neural network training to realize the fitting of behavior value function Q(s, a), the optimal action can be determined without traversal, which can significantly reduce the space complexity. Referring to the Actor-Critic structure of reinforcement learning, a SFC mapping algorithm based on pointer network is proposed.

[0177] Pointer network is a variant of sequence-to-sequence model, which can be effectively used to learn low-dimensional combination optimization problems and predict the solution of the problem with high accuracy. It maps the input to a series of pointers that point to the elements of the input sequence according to the probability. For the characteristics of SFC mapping problem, appropriate connection needs to be established between the node mapping stage and the link mapping stage, so as to automatically judge the goodness of the selected strategy. In this paper, pointer network is used as the agent to obtain the virtual node embedding strategy of the given SFC, and Dijkstra algorithm is used to embed the virtual link. Then a pointer network reward function related to the number of hops is designed to coordinate the two embedding stages of SFC.

[0178] For the input SFC request, the agent state is the current virtual node to be mapped, the state set is S = {0,..., s n}, s n = |N n |. The action of the agent at each state represents the selection of a node in the physical network, and the action set is A = {0,..., a s}, a s = |N s |.

[0179] The principle of pointer network is to map the output to a series of pointers that point to the elements of the input sequence according to the probability, which consists of an encoder and a decoder. The network architecture is shown in Figure 5 . The input of the encoder is a vector composed of node coordinates and the SFC to be mapped, and the output of the decoder is a certain ordering of node coordinates.

[0180] The embedding layer performs linear transformation on each input physical node and inputs the n-dimensional embedding information into the encoder network. The linear transformation formula is as follows

[0181] y = x * weight + bias (24)

[0182] where weight is drawn from a uniform distribution The initialized shape is a matrix of (out feature , in feature ) (in feature depends on the number of physical nodes, out feature depends on the number of nodes the current SFC is mapping to).

[0183] Encoder Network: LSTM takes as input at each time a new node and the output of the previous node, transforming into a hidden state where and gives the output of the current node. When all input nodes have been read, the encoder finally passes the state of the last encoder step as input to the first decoder step.

[0184] Decoder Network: Again, the hidden state is kept where At each decoder step, the decoding network uses an Attention mechanism to generate a probability distribution over all physical nodes and delivers the chosen node to the next decoding step. <g> is an n-dimensional zero matrix that is input to the first decoder step and is changed at each iteration.

[0185] Figure 5 The decoding process contains the decoding of the encoder input by the decoder. In the first decoding step, i.e. t = 0, the decoder uses an Attention mechanism to calculate the probability of selecting each node based on the state d1 and the states of each node obtained by the encoder The formula is shown in (25)-(26). At this time, the node with the maximum probability can be selected as the mapping selected node, which is represented by a black arrow pointing to the encoder in Figure 5 In the following decoding process, the LSTM reads in the output of the previous LSTM and the feature vector of the selected node in the previous step, and then uses an Attention mechanism to calculate the probability of each node:

[0186]

[0187]

[0188] where W e , W dand v are parameters of the neural network. In this way, nodes are selected continuously until a complete SFC mapping is constructed. In addition, after the decoder gives the selected node each time, the node is no longer selected, so as to ensure the validity of the result. At this time, the SFC mapping π is a pointer network sampling output, and it cannot be guaranteed that the word output can obtain the optimal solution. The final solution needs to be obtained through an active search process.

[0189] When the SFC completes the mapping, the agent will obtain a reward. According to the number of hops χ after the SFC mapping is completed, which is a relatively large positive number, the reward is defined as follows.

[0190]

[0191] The agent is guided to find the mapping scheme with the least number of hops by maximizing the reward. The strategy is the mapping of the state s to the action π. The strategy is usually a random strategy, that is, the probability of selecting a physical network node is modeled as

[0192]

[0193] The strategy is parameterized by the neural network parameter θ, and in the Markov process, the action probability of each step is p(π t | G t-1 | G s | G n According to the selected physical node and the virtual node to be mapped, the probability of selecting each physical node in the next step is calculated, and the final mapping strategy p θ (π| G s | G n ) is obtained by multiplying according to the chain rule. At this time, the loss function of the neural network can be defined as

[0194]

[0195] Similar to the Actor-Critic algorithm, the gradient of the loss function can be defined as

[0196]

[0197] In the formula, b is the baseline equation of the gradient, and the above formula is approximated by Monte Carlo sampling, and the following formula can be obtained

[0198]

[0199] Where B is the number of Monte Carlo sampling, and π i is the decision made in the i-th sampling.

[0200] The baseline value and the complete algorithm are as follows:

[0201] Algorithm 4 PN-SFC Mapping Algorithm Steps

[0202] • Input: Θ, G s , G n , B, K, βhlp n

[0203] • Output mapping policy π

[0204] • Randomly sample to get a solution π, record Hop π , b0= Hop π

[0205] • For i = 1 to K do

[0206] ■ For j = 1 to B do

[0207] ■ Sample π j , according to p θ (π|G s , G n )

[0208] ■ If π j is not feasible, i.e.

[0209] ■

[0210] ■ If

[0211] ■ Compute gradient

[0212] ■ Θ ← ADAM(θ, ∇J(θ))

[0213] ■ b i ← β·b i-1 +(1-β)·Hop π

[0214] ■ End for

[0215] • Return π

[0216] The two deep reinforcement learning algorithms DDQN-PER and PN-SFC proposed in the above embodiments are respectively used for managing the wireless resources of users and the core network element and link resources. Through iterative learning, the proportion of end-to-end delay allocated in RAN and CN is constantly updated, and finally the optimal eMBB user service quality and end-to-end QoE trade-off is achieved.

[0217] In summary, the network slice control system of the above embodiments is composed of a slice orchestrator responsible for decomposing the SLA guarantee delay and a domain controller managing each domain. The slice orchestrator divides the end-to-end delay guaranteed by the network slice SLA among the network domains according to the network slice SLA and index constraints, and issues the delay division result to the plurality of domain controllers. Each domain controller performs resource allocation in the network domain by using a resource mapping mechanism according to the delay division result of the network domain issued by the slice orchestrator, and feeds back the QoS guaranteed by the resource allocation to the slice orchestrator. The slice orchestrator determines the SLA guarantee condition according to the QoS feedback from the domain controller, and adjusts the division of the end-to-end delay guaranteed by the network slice SLA among the network domains according to the SLA guarantee condition. On this basis, the system architecture divides the network slice end-to-end constraint delay into delay limits that need to be met by the RAN and CN, and is easy to extend to more domains.

[0218] On the basis of the decomposition architecture, the embodiments of the present application propose two end-to-end constraint delay division methods based on delay balancing mechanism, DSDP method and DTDP method. Experiments show that by flexibly adjusting the network domain delay limit, the network capacity (access user number) can be improved.

[0219] In the scenario where eMBB and URLLC type slices coexist, the service satisfaction level of eMBB users is designed considering the service quality (throughput) of eMBB users, and the service quality of eMBB users and network capacity are balanced under the premise of meeting different needs.

[0220] In order to solve the non-convex and NP-hard problem of RAN and CN resource allocation, a deep reinforcement learning algorithm DDQN-PER and PN-SFC are respectively designed for managing the wireless resources of users and the core network element and link resources. Through iterative learning, the proportion of end-to-end delay allocated in RAN and CN is constantly updated, and finally the optimal eMBB user service quality and end-to-end QoE trade-off is achieved.

[0221] In addition, although several units of the network slice control system are mentioned in the above detailed description, such division is merely not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more units described above can be embodied in one unit. Similarly, the features and functions of one unit described above can also be further divided into embodied by multiple units.

[0222] Corresponding to the network slice control system disclosed in the above embodiment, the embodiment of the present application also discloses a network slice control method. The implementation of the embodiment method can refer to the implementation of the above-mentioned system, and the repeated parts will not be described herein. As shown in Figure 6 , comprising:

[0223] Step S601, according to the network slice SLA and the index constraint, dividing the end-to-end delay guaranteed by the network slice SLA among the network domains to generate a delay division result;

[0224] Step S602, according to the delay division result, using a resource mapping mechanism to allocate resources in the network domains, and feeding back the QoS guaranteed by the resource allocation;

[0225] Step S603, according to the feedback QoS, determining the SLA guarantee condition, and returning to step S601 to adjust the division of the end-to-end delay guaranteed by the network slice SLA among the network domains according to the SLA guarantee condition.

[0226] In an embodiment, as shown in Figure 7 , the network slice control method further comprises, before step S601: step S601', converting the service demand of a user into the network slice SLA to be signed; wherein the SLA performance index included in the network slice SLA at least includes delay and throughput; and the index constraint includes taking delay as the index for cross-domain coordination and taking throughput as the constraint condition for resource allocation within a domain.

[0227] In an embodiment, in step S601, according to the network slice SLA and the index constraint, dividing the end-to-end delay guaranteed by the network slice SLA among the network domains comprises: decomposing the end-to-end delay guaranteed by the network slice SLA into access network delay constraint and core network delay constraint, and adjusting the allocation proportion of the access network delay constraint and the core network delay constraint in the wireless access network and the core network, respectively.

[0228] In an embodiment, the adjustment of the allocation proportion of the access network delay constraint and the core network delay constraint in the wireless access network and the core network comprises: adjusting the allocation proportion of the end-to-end delay guaranteed by the network slice SLA from the perspective of a single network slice, by the following method:

[0229] Let constant D be the threshold value for iteration. If the difference between the delay experienced by the user and the end-to-end delay guaranteed by the network slice SLA ∑T>D, then the proportion of the delay constraint on both sides can be further optimized to increase the number of network access users;

[0230] wherein, for the delay constraint on the wireless access network side and the core network side of a certain network slice:

[0231] If the end-to-end delay constraint is not met due to the delay constraint of one side, and the delay margin of the other side is greater than the excess of the delay of the one side, the delay threshold value of the one side is increased;

[0232] If the end-to-end delay constraint is met, but the delay margin of one side is greater than the delay margin of the other side, the delay threshold value of the one side is reduced, and the constraint on the other side is relaxed.

[0233] In an embodiment, the adjustment of the access network delay constraint and the core network delay constraint is distributed in the radio access network and the core network, including adjusting the distribution ratio of the end-to-end delay guaranteed by the network slice SLA from the perspective of different types of network slices, by the following method:

[0234] Set a constant D as the threshold value for iteration, and the sum of the difference between the delay experienced by the user and the end-to-end delay guaranteed by the network slice SLA ∑T>D as the iteration condition;

[0235] Define the eMBB type slice user delay set as T eMBB , and the URLLC type slice user delay set as T URLLC If It is explained that under the current ratio, the eMBB type user obtains excess resources, and the ratio of the eMBB type slice end-to-end delay in the RAN delay distribution can be reduced, and the ratio of the URLLC type slice end-to-end delay in the RAN distribution can be increased.

[0236] In an embodiment, in the step S602, according to the delay division result, resource mapping mechanism is used for resource allocation in each network domain, including using reinforcement learning algorithm for wireless resource allocation in the access network network domain according to the access network delay constraint; and using reinforcement learning algorithm for SFC mapping in the core network network domain according to the core network delay constraint.

[0237] In an embodiment, in the step S602, the QoS of the resource allocation guarantee of the eMBB user network slice is quantified by the eMBB user satisfaction degree.

[0238] In an embodiment, the eMBB user satisfaction degree is characterized by the ratio of the eMBB user experience rate and the required rate; and the trade-off between the eMBB user satisfaction degree and the system capacity is realized by balancing the eMBB user quality of service and the proportion of the number of successfully accessed network users to the total number of users.

[0239] Embodiments of the application also disclose a network device, such as Figure 8As shown, the device includes a memory 81, a processor 82, and a computer program stored on the memory and executable on the processor, and the processor 81 executes the computer program to implement the method as Figure 6 - Figure 7 The network slice control method.

[0240] The embodiment of the application also discloses a computer readable storage medium, including instructions, when the instructions are executed on a computer, the computer executes the method as Figure 6 - Figure 7 The network slice control method.

[0241] The embodiment of the application also discloses a resource allocation method for allocating resources of a network domain, as Figure 9 As shown, the method includes:

[0242] Step S901, obtaining a time delay constraint of a network domain, and the time delay constraint of the network domain is determined by a network slice SLA and an index constraint;

[0243] Step S902, according to the time delay constraint, using a resource mapping mechanism to allocate resources of the network domain.

[0244] In some embodiments, the network domain includes an access network domain; for the access network domain, according to the access network time delay constraint, using a reinforcement learning algorithm to allocate wireless resources in the access network domain.

[0245] In some embodiments, the using of the reinforcement learning algorithm to allocate wireless resources in the access network domain includes: a reinforcement learning agent manages PRB allocation and power allocation of all users in the wireless network through the reinforcement learning algorithm.

[0246] In some embodiments, the network domain includes a core network domain; according to the core network time delay constraint, using a reinforcement learning algorithm to perform SFC mapping in the core network domain.

[0247] In some embodiments, the using of the reinforcement learning algorithm to perform SFC mapping in the core network domain includes: selecting a pointer network as a reinforcement learning agent, finding a node mapping strategy, and using Dijkstra algorithm to complete link mapping; wherein the pointer network finds an optimal mapping strategy through a search process.

[0248] In some embodiments, as shown in the figure, Figure 10 The resource allocation method further includes the following steps:

[0249] Step S903, obtaining a QOS guaranteed by the resource allocation according to a resource allocation result;

[0250] Step S904, determining an SLA guarantee condition of a network slice according to the QOS guaranteed by the resource allocation.

[0251] Step S905: Adjust the latency constraints of the current network domain according to the SLA guarantee.

[0252] Corresponding to the resource allocation method disclosed in the above embodiments, this embodiment of the invention also discloses a resource allocation apparatus. The implementation of the apparatus in this embodiment can refer to the implementation of the above method, and repeated details will not be elaborated further. Furthermore, although several units of the resource allocation apparatus have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of the present invention, the features and functions of two or more units described above can be embodied in one unit. Similarly, the features and functions of one unit described above can be further divided and embodied by multiple units.

[0253] The resource allocation device of this invention includes a memory and a processor; the memory is used to store computer program instructions; the processor is used to execute the computer program instructions to implement... Figure 9 and Figure 10 The resource allocation method described in the illustrated embodiment.

[0254] The resource management method and apparatus disclosed in the above embodiments divide the end-to-end constraint latency of network slices into latency limits that the RAN and CN need to meet. To address the non-convex and NP-hard problems of RAN and CN resource allocation, deep reinforcement learning algorithms—DDQN-PER and PN-SFC—are designed to manage user radio resources and core network elements and link resources, respectively. Through iterative learning, the proportion of end-to-end latency allocated in the RAN and CN is continuously updated, ultimately achieving an optimal trade-off between eMBB user service quality and system capacity.

[0255] This invention also discloses an end-to-end latency management method for network slicing, such as... Figure 11 As shown, it includes:

[0256] Step S1101: Based on the network slice SLA and indicator constraints, decompose the end-to-end delay guaranteed by the network slice SLA into access network delay constraints and core network delay constraints.

[0257] Step S1102: Adjust the allocation ratio of the end-to-end delay constraint in the radio access network and the core network, respectively.

[0258] In some embodiments, step S1102, adjusting the allocation ratio of the end-to-end delay constraint in the radio access network and the core network, includes: adjusting the allocation ratio of the end-to-end delay guaranteed by the network slice SLA from the perspective of a single network slice, through the following method:

[0259] Set constant D as the threshold value for iteration, if the sum of the difference between the delay experienced by the user and the end-to-end delay guaranteed by the network slice SLA ∑T>D, then the proportion of the two sides delay constraint can be further optimized to increase the number of network access users;

[0260] Wherein, for the delay constraint of a network slice on the side of the radio access network and the side of the core network:

[0261] If one side does not meet the end-to-end delay constraint, and the delay margin of the other side is greater than the part of the delay exceeded by the side, then the delay threshold of the side is increased;

[0262] If the current slice can meet the end-to-end delay constraint, but the delay margin of one side is greater than the delay margin of the other side, then the delay threshold of the other side is increased, and the constraint on the one side is relaxed.

[0263] In some embodiments, in step S1102, adjusting the allocation of the access network delay constraint and the core network delay constraint in the radio access network and the core network comprises: adjusting the allocation proportion of the end-to-end delay guaranteed by the network slice SLA from the perspective of different types of network slices, by the following method:

[0264] Set constant D as the threshold value for iteration, and the sum of the difference between the delay experienced by the user and the end-to-end delay guaranteed by the network slice SLA ∑T>D as the iteration condition;

[0265] Define the eMBB class slice user delay set as T eMBB , and the URLLC class slice user delay set as T URLLC If It is explained that under the current proportion, the eMBB class user obtains excess resources, and the proportion of the eMBB class slice end-to-end delay in the RAN delay allocation can be reduced, and the proportion of the URLLC class slice end-to-end delay in the RAN allocation can be increased.

[0266] Corresponding to the network slice end-to-end delay management method disclosed in the above embodiment, the embodiment of the application also discloses a network slice end-to-end delay management device. The implementation of the device of the embodiment can be referred to the implementation of the above method, and the repeated parts will not be described herein. In addition, although several units of the network slice end-to-end delay management device are mentioned in the above detailed description, such division is not mandatory. In fact, according to the embodiment of the application, the features and functions of two or more units described above can be embodied in one unit. Similarly, the features and functions of one unit described above can also be further divided into multiple units to be embodied.

[0267] The embodiment of the application discloses a network slice end-to-end delay management device, including a memory and a processor; the memory is used for storing computer program instructions; the processor is used for executing the computer program instructions to realize the network slice end-to-end delay management method as shown in the figure. Figure 11 The embodiment of the application discloses a network slice end-to-end delay management device, including a memory and a processor; the memory is used for storing computer program instructions; the processor is used for executing the computer program instructions to realize the network slice end-to-end delay management method as shown in the figure.

[0268] The embodiment of the application discloses a network slice end-to-end delay management device, including a memory and a processor; the memory is used for storing computer program instructions; the processor is used for executing the computer program instructions to realize the network slice end-to-end delay management method as shown in the figure.

[0269] The memory in the embodiment of the application may, for example, be one or more of a cache, a flash memory, a hard drive, a removable media, a volatile memory, a non-volatile memory, or other suitable device. Information related to failure may be stored, and in addition, programs for executing the information may be stored. The processor may execute the programs stored in the memory to achieve information storage or processing, etc. The functions of other components are similar to the prior art, and will not be described here. The components of other devices may be implemented by dedicated hardware, firmware, software, or a combination thereof, without departing from the scope of the application.

[0270] The embodiment of the application discloses a network slice control system and a control method, and designs an end-to-end network slice SLA-based cross-domain orchestration framework, uses delay as a cross-domain coordination index, and uses transmission rate as an intra-domain constraint index, so as to realize end-to-end slice resource management. The framework is used for dividing the network slice end-to-end constraint delay into delay limits that need to be met by RAN and CN, and is easy to extend to more domains. In addition, an end-to-end constraint delay division method based on a delay balancing mechanism is proposed, and network capacity (access user number) can be improved and eMBB user service quality can be guaranteed by flexibly adjusting network domain delay limits. In order to solve the non-convex and NP-hard problems of RAN and CN resource allocation, a deep reinforcement learning algorithm is designed, which is used for managing wireless resources of users and allocating core network element and link resources.

[0271] Those skilled in the art will appreciate that embodiments of the application can be provided as methods, systems, or computer program products. Therefore, the application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage, etc.) containing computer-usable program code.

[0272] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart Figure 1 one or more functions specified in the flowchart or multiple flows and / or blocks. Figure 1 one or more functions specified in the flowchart or multiple flows and / or blocks.

[0273] These computer program instructions can also be stored in a computer readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer readable memory produce an article of manufacture including instructions which implement the function specified in the flowchart Figure 1 one or more functions specified in the flowchart or multiple flows and / or blocks. Figure 1 one or more functions specified in the flowchart or multiple flows and / or blocks.

[0274] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart Figure 1 one or more functions specified in the flowchart or multiple flows and / or blocks. Figure 1 one or more functions specified in the flowchart or multiple flows and / or blocks.

[0275] The principles and implementation manners of the present application are described in the specific embodiments, and the above embodiment descriptions are only used to help understand the method of the present application and its core idea; meanwhile, for those skilled in the art, according to the idea of the present application, the specific implementation manners and application scopes can be changed, and the above descriptions should not be understood as the limitation of the present application.

Claims

1.A network slice control system, characterized by, The system comprises a slice orchestrator and a plurality of domain controllers; The slice orchestrator is configured to divide the end-to-end latency guaranteed by the network slice SLA among the network domains according to the network slice SLA and index constraints, and to issue the latency division result to the plurality of domain controllers; wherein the SLA is a service level agreement; Each of the domain controllers is configured to allocate resources in the network domain using a resource mapping mechanism according to the latency division result of the network domain issued by the slice orchestrator, and to feed back the QoS guaranteed by the resource allocation to the slice orchestrator; The slice orchestrator is further configured to determine the SLA guarantee condition according to the QoS fed back by the domain controllers, and to adjust the division of the end-to-end latency guaranteed by the network slice SLA among the network domains according to the SLA guarantee condition; The division of the end-to-end latency guaranteed by the network slice SLA among the network domains according to the network slice SLA and index constraints comprises: decomposing the end-to-end latency guaranteed by the network slice SLA into access network latency constraints and core network latency constraints, and adjusting the allocation proportion of the end-to-end latency in the wireless access network and the core network, respectively. 2.The network slice control system of claim 1, wherein, The slice orchestrator is further configured to: convert the service demand of a user into the signed network slice SLA; wherein The SLA performance indexes included in the network slice SLA at least include latency and throughput; The index constraints include: taking latency as the index for cross-domain coordination, and taking throughput as the constraint condition for resource allocation within a domain. 3.The network slice control system of claim 1, wherein, The adjustment of the allocation proportion of the end-to-end latency in the wireless access network and the core network, respectively, comprises: Adjusting the allocation proportion of the end-to-end latency guaranteed by the network slice SLA from the perspective of a single network slice, by the following method: Let constant As the threshold value of iteration, if the sum of the difference between the delay experienced by the user and the end-to-end delay guaranteed by the network slice SLA meets , the proportion of the two-side delay constraints can be further optimized to increase the number of network access users; wherein the represents the difference in end-to-end delay. For the latency constraints of a network slice on the wireless access network side and the core network side: If one side does not meet the side latency constraint, resulting in not meeting the end-to-end latency constraint, and the latency margin of the other side is greater than the part of the side latency exceeded, then the side latency threshold value is increased; If the current slice can meet the end-to-end latency constraint, but the latency margin of one side is greater than that of the other side, then the latency threshold value of the side is reduced, and the constraint on the other side is relaxed. 4.The network slice control system of claim 1, wherein, The adjustment of the allocation of the end-to-end latency in the wireless access network and the core network comprises: Adjusting the allocation proportion of the end-to-end latency guaranteed by the network slice SLA from the perspective of different types of network slices, by the following method: Let constant a sum of a difference between a latency experienced by a user and an end-to-end latency guaranteed by the network slice SLA as a threshold for performing iteration as a condition for iteration; The eMBB type slice user latency set is defined as The URLLC type slice user latency set is defined as If It is indicated that, under the current proportion, the eMBB type user obtains excess resources, and the proportion of the RAN latency allocation of the eMBB type slice end-to-end latency can be reduced, and the proportion of the RAN allocation of the URLLC type slice end-to-end latency can be increased; wherein, the superscript E2E represents end-to-end. 5.The network slice control system of claim 1, wherein, The domain controllers include a RAN domain controller and a CN domain controller; The RAN domain controller uses a reinforcement learning algorithm to allocate wireless resources according to the access network latency constraints; The CN domain controller uses a reinforcement learning algorithm to perform SFC mapping according to the core network latency constraints. 6.The network slice control system of claim 5, wherein, The RAN domain controller uses a reinforcement learning algorithm to allocate wireless resources in the access network network domain, comprising: The reinforcement learning agent is a wireless network, which manages the PRB allocation and power allocation of all users through a reinforcement learning algorithm. 7.The network slice control system of claim 5, wherein, The CN domain controller uses a reinforcement learning algorithm to perform SFC mapping in a core network domain, comprising: a pointer network is selected as a reinforcement learning agent to find a node mapping strategy, and dijkstra algorithm is used to complete link mapping; wherein the pointer network finds the optimal mapping strategy through a search process. 8.The network slice control system of claim 1, wherein, The QoS of the eMBB user network slice is quantified by the eMBB user satisfaction degree; The eMBB user satisfaction degree is represented by the ratio of the eMBB user experience rate to the required rate; The eMBB user satisfaction degree and the system capacity are balanced by balancing the eMBB user quality of service and the proportion of the number of successfully accessed network users to the total number of users. 9.A network slice control method, comprising: comprising: Step 1: According to the network slice SLA and index constraints, the end-to-end delay guaranteed by the network slice SLA is divided among the network domains to generate a delay division result; wherein the SLA is a service level agreement; Step 2: According to the delay division result, resource mapping mechanism is used for resource allocation of each network domain, and the QoS guaranteed by the resource allocation is fed back; Step 3: According to the feedback QoS, the SLA guarantee condition is determined, and according to the SLA guarantee condition, step 1 is returned to adjust the end-to-end delay guaranteed by the network slice SLA among the network domains; wherein the end-to-end delay guaranteed by the network slice SLA is divided among the network domains according to the network slice SLA and index constraints, comprising: the end-to-end delay guaranteed by the network slice SLA is decomposed into access network delay constraint and core network delay constraint, and the allocation proportion of the end-to-end delay in the wireless access network and the core network is adjusted respectively.

Images