Network slice allocation optimization method and device based on service drive

Abstract

The present invention relates to the field of communication network technology, and in particular to a service-driven network slice allocation optimization method and device, wherein the method comprises: constructing a slice architecture based on at least one actual requirement of each business scenario in the current network environment, constructing an intelligent agent for each slice for training a target neural network, outputting a slice allocation set from the trained target neural network for population optimization, obtaining a target slice allocation set for each business scenario, and optimizing the target slice allocation set based on a preference list of business scenarios and slices to perform slice allocation in the current network environment. The present invention can construct a network slicing framework based on the different requirements of multiple business scenarios, and perform multi-agent reinforcement learning, multi-objective optimization, and business slice preference matching based on the framework, thereby realizing a dynamically adjusted network slice optimization process according to actual business needs, thereby improving user experience and network operation and maintenance efficiency.

Description

Technical Field

[0001] The present invention relates to the field of communication network technology, and in particular to a service-driven network slice allocation optimization method and device. Background Art

[0002] With the development of 5G communications, the requirements for network service technology have increased accordingly. To meet the demand for high-quality services, network slicing technology can be used to allow operators to create multiple virtual networks on the same physical network infrastructure. Each virtual network is optimized and configured according to the needs of specific services.

[0003] In related technologies, existing network slicing methods include dynamic access control and slice allocation algorithms, which can simultaneously consider sensitive indicators of user preferences and business requests, or through the combination of Markov decision process and reinforcement learning algorithm, learn the network environment and user behavior patterns based on intelligent agents, thereby dynamically adjusting the resource configuration of slices.

[0004] However, in the related technologies, the existing slicing network technology fails to fully adapt to the dynamically changing network environment and user requirements, and cannot simultaneously meet the dynamic adjustment needs of slice resource configuration and the balanced service quality requirements of different businesses. It is difficult to make real-time dynamic adjustments according to actual business needs, which affects the user experience and reduces the service quality and operating efficiency of network technology, which needs to be solved urgently. Summary of the Invention

[0005] The present invention provides a service-driven network slice allocation optimization method and device to solve the problems in related technologies, such as the failure of existing slicing network technology to fully adapt to the dynamically changing network environment and user requirements, and the inability to simultaneously meet the dynamic adjustment requirements of slice resource configuration and the balanced service quality requirements of different services. It is difficult to make real-time dynamic adjustments according to actual business needs, which affects the user experience and reduces the service quality and operating efficiency of network technology.

[0006] The first aspect of the present invention provides a service-driven network slice allocation optimization method, comprising the following steps: obtaining all business scenarios in the current network environment, and respectively confirming at least one actual requirement of each business scenario; constructing a slice architecture for each business scenario based on the common attributes of each business scenario and the at least one actual requirement, and confirming all slices corresponding to each business scenario according to the slice architecture; respectively constructing a corresponding intelligent agent for each slice in all slices, generating a target neural network using the reward function of the intelligent agent, and training the target neural network based on the transfer samples of the intelligent agent to obtain a trained target neural network; the trained target neural network outputs a slice allocation set for the corresponding slice, and performs population optimization on all slice allocation sets of each business scenario to obtain a target slice allocation set for each business scenario; based on the utility function of each business scenario, obtaining a preference list of all business scenarios and all slices, optimizing the target slice allocation set according to the preference list, and using the optimized target slice allocation set to perform slice allocation in the current network environment.

[0007] Optionally, in one embodiment of the present invention, before obtaining the preference list of all business scenarios and all slices based on the utility function of each business scenario, it also includes: generating all indicators of the current business scenario based on all actual needs of the current business scenario; judging whether there is at least one indicator among all the indicators that does not meet the preset quantitative requirements; if there is at least one indicator among all the indicators that does not meet the preset quantitative requirements, constructing a corresponding indicator system for the indicators that do not meet the preset quantitative requirements, confirming the weight coefficient according to the indicator system, and generating the utility function of the current business scenario according to all the indicators and the weight coefficient; otherwise, generating the utility function based on all indicators that meet the preset quantitative requirements.

[0008] Optionally, in one embodiment of the present invention, constructing the slice architecture of each business scenario based on the common attributes of each business scenario and the at least one actual requirement includes: matching the actual category of each business scenario based on the common attributes of each business scenario, and confirming the first-level slice corresponding to each business scenario by the actual category; performing clustering calculation on the first-level slice based on the at least one actual requirement to obtain multiple second-level slices within the range of the first-level slice, so as to construct the slice architecture based on all first-level slices and all second-level slices.

[0009] Optionally, in one embodiment of the present invention, the target neural network is trained based on the transfer samples of the intelligent agent to obtain the trained target neural network, including: sampling the transfer samples to obtain sampling results, and calculating the corresponding target expected value from the sampling results; based on a preset loss function, updating the network parameters of the target neural network using the target expected value and the actual expected value of the sampling result to obtain an updated target neural network; iteratively calculating the target neural network according to a preset time step interval until the updated target neural network meets a preset convergence condition, thereby obtaining the trained target neural network.

[0010] Optionally, in one embodiment of the present invention, the population optimization is performed on all slice allocation sets of each business scenario respectively to obtain the target slice allocation set of each business scenario, including: confirming the optimal population corresponding to each slice allocation set in the current business scenario; and matching the target slice allocation set of the current business scenario in the non-dominated solution set of all optimal populations based on the at least one actual demand.

[0011] Optionally, in one embodiment of the present invention, before using the optimized target slice allocation set to perform slice allocation in the current network environment, it also includes: obtaining all mapping combinations of the optimized target slice allocation set, and detecting whether the actual preferences of all mapping combinations meet the preset stability requirements; if the actual preferences of all mapping combinations do not meet the preset stability requirements, all mapping combinations that do not meet the preset stability requirements are recombined to obtain an adjusted target slice allocation set, so as to use the adjusted target slice allocation set for network allocation.

[0012] In a second aspect, an embodiment of the present invention provides a business-driven network slice allocation optimization device, comprising: an acquisition module for acquiring all business scenarios in the current network environment and respectively confirming at least one actual requirement of each business scenario; a construction module for constructing a slice architecture for each business scenario based on the common attributes of each business scenario and the at least one actual requirement, and confirming all slices corresponding to each business scenario according to the slice architecture; a training module for respectively constructing a corresponding intelligent agent for each slice in all slices, generating a target neural network using the reward function of the intelligent agent, and training the target neural network based on the transfer samples of the intelligent agent to obtain a trained target neural network; an allocation module for outputting a slice allocation set of the corresponding slice from the trained target neural network, performing population optimization on all slice allocation sets of each business scenario respectively, and obtaining a target slice allocation set for each business scenario; an optimization module for obtaining a preference list of all business scenarios and all slices based on the utility function of each business scenario, optimizing the target slice allocation set according to the preference list, and using the optimized target slice allocation set to perform slice allocation in the current network environment.

[0013] Optionally, in one embodiment of the present invention, the device further includes: a first generation module, used to generate all indicators of the current business scenario based on all actual needs of the current business scenario before obtaining the preference list of all business scenarios and all slices based on the utility function of each business scenario; a judgment module, used to judge whether there is at least one indicator among all the indicators that does not meet the preset quantitative requirements; a second generation module, used to construct a corresponding indicator system for the indicators that do not meet the preset quantitative requirements if there is at least one indicator among all the indicators that does not meet the preset quantitative requirements, confirm the weight coefficient according to the indicator system, and generate the utility function of the current business scenario based on all the indicators and the weight coefficient; otherwise, generate the utility function based on all indicators that meet the preset quantitative requirements.

[0014] Optionally, in one embodiment of the present invention, the construction module includes: a first matching unit, used to match the actual category of each business scenario based on the common attributes of each business scenario, and confirm the first-level slice corresponding to each business scenario by the actual category; a construction unit, used to perform clustering calculation on the first-level slice based on the at least one actual requirement, to obtain multiple second-level slices within the range of the first-level slice, so as to construct the slice architecture based on all first-level slices and all second-level slices.

[0015] Optionally, in one embodiment of the present invention, the training module includes: a sampling unit, used to sample the transfer sample to obtain a sampling result, and calculate the corresponding target expected value from the sampling result; an updating unit, used to update the network parameters of the target neural network based on a preset loss function using the target expected value and the actual expected value of the sampling result to obtain an updated target neural network; an iteration unit, used to iteratively calculate the target neural network according to a preset time step interval until the updated target neural network meets a preset convergence condition, thereby obtaining the trained target neural network.

[0016] Optionally, in one embodiment of the present invention, the allocation module includes: a confirmation unit, used to confirm the optimal population corresponding to each slice allocation set in the current business scenario; and a second matching unit, used to match the target slice allocation set of the current business scenario in the non-dominated solution set of all optimal populations based on the at least one actual requirement.

[0017] Optionally, in one embodiment of the present invention, it further includes: a detection module, which is used to obtain all mapping combinations of the optimized target slice allocation set before using the optimized target slice allocation set to perform slice allocation in the current network environment, and detect whether the actual preferences of all mapping combinations meet the preset stability requirements; an adjustment module, which is used to recombine all mapping combinations that do not meet the preset stability requirements when the actual preferences of all mapping combinations do not meet the preset stability requirements, to obtain an adjusted target slice allocation set, so as to use the adjusted target slice allocation set for network allocation.

[0018] An embodiment of the third aspect of the present invention provides an electronic device, comprising: a memory, a processor, and a computer program stored on the memory and runnable on the processor, wherein the processor executes the program to implement the service-driven network slice allocation optimization method as described in the above embodiment.

[0019] A fourth aspect of the present invention provides a computer-readable storage medium, which stores a computer program that, when executed by a processor, implements the above-mentioned service-driven network slice allocation optimization method.

[0020] A fifth aspect of the present invention provides a computer program, which, when executed, implements the above-mentioned service-driven network slice allocation optimization method.

[0021] The embodiments of the present invention can construct a network slicing framework based on the different needs of multiple business scenarios, and perform multi-agent reinforcement learning, multi-objective optimization, and business slice preference matching based on the framework, thereby realizing a dynamically adjusted network slicing optimization process based on actual business needs, ensuring the target requirements of network resource utilization efficiency and service quality, further improving user experience, and improving network operation and maintenance efficiency. This solves the problem in the related art that existing slicing network technology fails to fully adapt to the dynamically changing network environment and user requirements, and cannot simultaneously meet the dynamic adjustment requirements of slice resource configuration and the balanced service quality requirements of different businesses. It is difficult to make real-time dynamic adjustments according to actual business needs, which affects the user experience and reduces the service quality and operational efficiency of network technology.

[0022] Additional aspects and advantages of the present invention will be set forth in part in the description which follows and, in part, will be obvious from the description which follows, or may be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0024] Figure 1 A flowchart of a service-driven network slice allocation optimization method provided according to an embodiment of the present invention;

[0025] Figure 2 A logical diagram of adaptive service-driven network slicing optimization according to an embodiment of the present invention;

[0026] Figure 3 This is a structural diagram of a service-driven network slice allocation optimization device according to an embodiment of the present invention;

[0027] Figure 4 FIG. 1 is a schematic structural diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0028] The following describes embodiments of the present invention in detail, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and are not to be construed as limiting the present invention.

[0029] The following describes a service-driven network slice allocation optimization method and apparatus according to an embodiment of the present invention with reference to the accompanying drawings. In view of the problems mentioned in the above background technology, existing slicing network technologies fail to fully adapt to dynamically changing network environments and user requirements, and fail to simultaneously meet the dynamic adjustment requirements of slice resource configuration and the balanced service quality requirements of different services. This makes it difficult to make real-time dynamic adjustments based on actual service needs, affecting the user experience and reducing the service quality and operational efficiency of the network technology. The present invention provides a service-driven network slice allocation optimization method. In this method, a network slicing framework can be constructed based on the different requirements of multiple service scenarios. Multi-agent reinforcement learning, multi-objective optimization, and service slice preference matching are performed on this framework to achieve a dynamically adjusted network slicing optimization process based on actual service needs. This ensures the target requirements of network resource utilization efficiency and service quality, further enhances the user experience, and improves network operation and maintenance efficiency. This solves the problems in the related art that existing slicing network technologies fail to fully adapt to dynamically changing network environments and user requirements, and fail to simultaneously meet the dynamic adjustment requirements of slice resource configuration and the balanced service quality requirements of different services. This makes it difficult to make real-time dynamic adjustments based on actual service needs, affecting the user experience and reducing the service quality and operational efficiency of the network technology.

[0030] Specifically, Figure 1 A flowchart of a service-driven network slice allocation optimization method provided in an embodiment of the present invention.

[0031] like Figure 1 As shown, the service-driven network slice allocation optimization method includes the following steps:

[0032] In step S101, all business scenarios in the current network environment are acquired, and at least one actual requirement of each business scenario is confirmed respectively.

[0033] It is understandable that in the embodiments of the present invention, personalized needs of different business scenarios (such as high-definition video, telemedicine, industrial control, etc.) can be collected through questionnaires, customer interviews, etc., including indicators such as bandwidth, latency, and reliability as the actual needs of the business scenarios. For example, video services require high bandwidth and low latency, while IoT services require low power consumption and high reliability.

[0034] In step S102, based on the common attributes of each business scenario and at least one actual requirement, a slice architecture for each business scenario is constructed, and all slices corresponding to each business scenario are confirmed according to the slice architecture.

[0035] It can be understood that in an embodiment of the present invention, the utility function constructed according to the actual needs of the business scenario is used to obtain the common attributes of the business scenario (such as delay sensitivity), and each business scenario is divided into multiple slices to construct a slice architecture for each business scenario.

[0036] Optionally, in one embodiment of the present invention, a slice architecture for each business scenario is constructed based on the common attributes of each business scenario and at least one actual requirement, including: matching the actual category of each business scenario based on the common attributes of each business scenario, and confirming the first-level slice corresponding to each business scenario by the actual category; performing clustering calculation on the first-level slices based on at least one actual requirement to obtain multiple second-level slices within the first-level slice range, so as to construct a slice architecture based on all first-level slices and all second-level slices.

[0037] In the actual implementation process, the services are divided into several major categories, corresponding to coarse-grained first-level slices, such as eMBB (Enhanced Mobile Broadband), URLLC (Ultra-Reliable Low Latency Communications), mMTC (massive Machine Type of Communication), and other major 5G application scenarios, as well as new application scenarios expected in 6G, such as holographic communications and integrated aerospace networks.

[0038] Within each first-level slice, fine-grained second-level slices can be further divided based on individual needs (such as bandwidth requirements). For example, the eMBB first-level slice can be divided into eMBB-UHD Video Transmission, eMBB-Immersive Experience, etc.; the holographic communication first-level slice can be divided into holographic communication-Holographic Video Conferencing, Holographic Communication-Remote Holographic Education, etc. Clustering algorithms such as K-means clustering can be used to group services and automatically complete the multi-level slice division, forming a hierarchical slice architecture, with each slice corresponding to a specific set of Service Level Agreements (SLAs).

[0039] Specifically, the K-means algorithm steps include: inputting a service set S = [s1, s2, ..., s n ], each business s i By its attribute vector x i Indicates that the number of clusters is K; output the slice division result, that is, the slice label to which each business belongs. Randomly select K businesses as the initial cluster centers [μ1, μ2, ..., μ K ], for each business s i∈S, calculate its distance from each cluster center as:

[0040] d ij =∥x i -μ j ∥ 2 , j = 1, 2, ..., K,

[0041] Then the business i Assign to the slice corresponding to the nearest cluster center:

[0042] l(s i )=arg min j d ij ,

[0043] For each slice j = 1, 2, ..., K, recalculate its cluster center Among them C j Represents the business set assigned to slice j. Repeat the iteration until the clustering result converges or the maximum number of iterations is reached, and return the final slice result [C1, C2, ..., C K ], and get the slice architecture.

[0044] In step S103, a corresponding intelligent agent is constructed for each slice in all slices, a target neural network is generated using the reward function of the intelligent agent, and the target neural network is trained based on the transfer samples of the intelligent agent to obtain a trained target neural network.

[0045] It can be understood that in the embodiment of the present invention, each slice is modeled as an agent, and the state space and action space of the agent are defined to obtain the state space S of each agent, including factors such as the resource occupancy of the slice and business satisfaction, which can be expressed as a vector: S = [s1, s2, ..., s n ],s i Represents the i-th state feature, such as the resource occupancy rate of a specific slice, the current latency, etc. The action space A of the agent includes actions such as applying for or releasing resources, adjusting slice strategies, etc., which can be expressed as: A = [a1, a2, ..., a m ], a j Represents the jth possible action. The agent’s reward function Q represents the expected value of the long-term cumulative reward for taking action a in state s, that is:

[0046] Q(s,a)=E[R t ∣∣s t =s,a t =a],

[0047] Among them, R tis the reward value of the agent. The Q function can be approximated using the neural network Q(s, a, θ), where θ is the network parameter, to obtain the target neural network. The target neural network is then trained based on the agent's transfer samples to obtain the trained target neural network.

[0048] Optionally, in one embodiment of the present invention, a target neural network is trained based on the transfer samples of the intelligent agent to obtain a trained target neural network, including: sampling the transfer samples to obtain sampling results, and calculating the corresponding target expected value from the sampling results; based on a preset loss function, updating the network parameters of the target neural network using the target expected value and the actual expected value of the sampling result to obtain an updated target neural network; iteratively calculating the target neural network according to a preset time step interval until the updated target neural network meets a preset convergence condition, thereby obtaining a trained target neural network.

[0049] It should be noted that the preset time step interval and the preset convergence condition can be set by those skilled in the art according to actual conditions and are not specifically limited here.

[0050] In the actual execution process, an Experience Replay buffer D can be defined to store the transfer samples (s t , a t , r t , s t+1 ). At each time step, the agent randomly selects an action with probability ε and selects an action according to the current Q function with probability 1-ε (the actual value of the Q function), that is, a t =arg max a Q(s t , a, θ). The transferred sample (s t , a t , r t , s t+1 ) is stored in the experience replay buffer D.

[0051] Specifically, the DNQ (Dueling Network Q-Network) reinforcement learning algorithm can be used to randomly sample and transfer samples from the experience replay buffer D to obtain the sampling result (s, a, r, s′). If s′ is the terminal state, then y = r, otherwise it can be expressed as:

[0052] y=r+γmax a‘ Q(s′, a′, θ - ),

[0053] Among them, θ - is the parameter of the target network, γ is the discount factor, y represents the target Q value, Q(s′, a′, θ -) represents the maximum Q value of the target network in the next state s', that is, the target expected value.

[0054] The target expected value and the actual expected value are input into the preset loss function, which is expressed as:

[0055] L(θ)=E (s，a，r′)～D [(yQ(s,a,θ) 2 )],

[0056] Use the gradient descent method to update the parameters θ of the Q network, that is, Where α is the learning rate, and the updated network parameters are obtained. Every certain number of steps, the parameters θ of the Q network are copied to the parameters θ of the target network. - , that is, θ - ←θ.

[0057] For each slice, according to the current state s t , select the action a with the largest Q value t , perform action a t , observe the reward r t and the next state s t+1 , and transfer samples (s t , a t , r t , s t+1 ) is stored in the experience replay buffer D. Repeat the above steps for iterative calculation to achieve online learning and dynamic adjustment until the updated target neural network meets the preset convergence conditions and the trained target neural network is obtained under a fixed number of iterations.

[0058] Specifically, for example, as shown in the following Algorithm 1, which is the pseudo code of the multi-agent network slicing optimization process, the target neural network can be trained based on the transfer samples of the agent to obtain the trained target neural network:

[0059]

[0060]

[0061] In step S104, the trained target neural network outputs a slice allocation set corresponding to the slice, and population optimization is performed on all slice allocation sets of each business scenario to obtain a target slice allocation set for each business scenario.

[0062] It can be understood that in an embodiment of the present invention, the slice allocation set can be a slice strategy, and the slice strategies of multiple intelligent agents are regarded as individuals in the population. Each individual corresponds to a set of slice parameters (such as bandwidth, latency, etc.), and the fitness function of the population includes business satisfaction, cost, and resource utilization, forming a multi-objective optimization problem of the population, and finally obtaining the target slice allocation set for each business scenario.

[0063] Optionally, in one embodiment of the present invention, population optimization is performed on all slice allocation sets of each business scenario to obtain a target slice allocation set for each business scenario, including: confirming the optimal population corresponding to each slice allocation set in the current business scenario; and matching the target slice allocation set of the current business scenario in the non-dominated solution set of all optimal populations based on at least one actual requirement.

[0064] In the actual execution process, an initial population P0 of size N can be randomly generated, each individual represents a set of slice parameters, and the population P0 is non-dominated sorted to obtain the rank and crowding distance of each individual. t , using the binary tournament selection algorithm, from P t Select N individuals from the parent population Q t ; for Q t The individuals in the crossover operation are performed to obtain the offspring population R t ; for R t Perform mutation operations on individuals in P to introduce new diversity; t and R t Merge to get a population S of size 2N t ; for S t Perform non-dominated sorting to obtain the rank of each individual; for each rank of individuals, calculate its crowding distance; select S according to the rank and crowding distance t The first N individuals in the new population P are obtained t+1 , repeat the above steps until the convergence condition is met or the number of iterations is reached.

[0065] Specifically, the binary tournament selection algorithm initializes the parent population Q t is an empty set, from the current population P t Randomly select two individuals a and b from Q. If a’s non-dominated rank is lower than b’s, or if a and b have the same non-dominated rank but a’s crowding distance is greater than b’s, then a is added to Q. t Otherwise, add b to Q t , returns the parent population Q t , and repeat the above process.

[0066] For non-dominated sorting, in multi-objective optimization, if a solution is considered to be "non-dominated" by another solution, and in all objective functions, the solution is better than the other solution in some objectives, but not inferior to the other solution in other objectives, then the solution is considered to "dominate" the other solution. In non-dominated sorting, the dominating set S is initialized for each individual in the population P. p and the dominated number n p Empty; for each individual p, compare it with other individuals in the population, and if p dominates another individual q, add q to S p If p is dominated by another individual q, then n p Add 1; put all individuals with 0 dominated number into the first non-dominated level F1; for each individual p in F1, access its dominating set S p For each individual q in , reduce the dominance number of q by 1. If the dominance number of q becomes 0, put it into the next non-dominated level F2, and repeat the above steps until all individuals are assigned to their respective non-dominated levels.

[0067] The crowding distance is a measure of the degree of crowding of individuals in a population. It represents the distance between an individual and other individuals in the target space and reflects the density of the space surrounding the individual. A larger crowding distance indicates a sparser space around the individual and a higher diversity of that individual. Conversely, a smaller crowding distance indicates a more crowded space around the individual and a lower diversity of that individual. It is used to measure the differences and diversity between different slicing strategies. The crowding distance can be calculated by sorting individuals according to the target value for each objective function, calculating the distance between two adjacent individuals, and adding this distance to the individual's crowding distance. For boundary individuals (the first and last individuals after sorting), their crowding distances are set to infinity.

[0068] Finally, according to at least one actual demand of the vehicle, the optimal slicing strategy for the current business scenario is selected from the non-dominated solution set of the last generation population, that is, the optimal population, to obtain the target slice allocation set.

[0069] In step S105, based on the utility function of each business scenario, a preference list of all business scenarios and all slices is obtained, and the target slice allocation set is optimized according to the preference list to use the optimized target slice allocation set to perform slice allocation in the current network environment.

[0070] It can be understood that in an embodiment of the present invention, a utility function is constructed based on the actual needs of each business scenario, and the utility function is used to sort all business scenarios and all slices to obtain preference lists of business scenarios and slices respectively, so that the target slice allocation set is adjusted secondary according to the preference list to utilize the optimized target slice allocation set to allocate each business to the corresponding slice, and dynamically adjust the matching relationship during operation according to changes in business needs and network status.

[0071] The service set can be expressed as U = [u1, u2, ..., u n ], and the slice set is represented as V = [v1, v2, ..., v m ], for each business u i , sort all slices according to their utility function and get the preference list P(u i )=[u i1 ,u i2 ,…,u im ], for each slice v j Sort all services according to resource utilization and other indicators to obtain the preference list P(v j )=[v j1 , v j2 ,…,v jn ].

[0072] Initialize the matching state of each service to free and the matching state of each slice to idle. i , according to u im Preference list P(u im ), to its most preferred slice v j Initiate an application, if v j Idle, then (u i , v j ) join the matching set M; if v j Already with another business u k Matches, but u i In v j is ranked higher in the preference list, then release (u k , v j ) matches, and (u i , v j ) is added to the matching set M, and u k Mark as free. Repeat the above steps until all businesses are matched with slices or no new matches can be found, and obtain the optimized target slice allocation set.

[0073] Optionally, in one embodiment of the present invention, before obtaining the preference list of all business scenarios and all slices based on the utility function of each business scenario, it also includes: generating all indicators of the current business scenario based on all actual needs of the current business scenario; judging whether there is at least one indicator among all indicators that does not meet the preset quantitative requirements; if there is at least one indicator among all indicators that does not meet the preset quantitative requirements, constructing a corresponding indicator system for the indicators that do not meet the preset quantitative requirements, confirming the weight coefficient according to the indicator system, and generating the utility function of the current business scenario based on all indicators and weight coefficients; otherwise, generating a utility function based on all indicators that meet the preset quantitative requirements.

[0074] It should be noted that the preset quantization requirements can be set by those skilled in the art according to actual conditions and are not specifically limited here.

[0075] During the actual implementation process, the qualitative requirements in the actual needs can be first converted into quantitative indicators. For example, "HD video" can be converted into "throughput ≥ 10Mbps, latency ≤ 50ms", etc., to obtain all indicators. For example, the bandwidth requirement of the video service is 100Mbps, and the latency requirement is 20ms; the power consumption requirement of the IoT service is 0.1W, and the reliability requirement is 99.999%.

[0076] For indicators that don't meet pre-defined quantitative requirements—that is, requirements that can't be directly quantified, such as user experience—we use the Analytic Hierarchy Process (AHP) to construct an indicator system, with weight coefficients determined through expert scoring. By integrating various indicators and weights, we construct a business demand model, expressed as a utility function U(b, ​​d, r), where b, d, and r represent bandwidth, latency, and reliability, respectively.

[0077] Optionally, in one embodiment of the present invention, before using the optimized target slice allocation set to perform slice allocation in the current network environment, it also includes: obtaining all mapping combinations of the optimized target slice allocation set, and detecting whether the actual preferences of all mapping combinations meet the preset stability requirements; if the actual preferences of all mapping combinations do not meet the preset stability requirements, all mapping combinations that do not meet the preset stability requirements are recombined to obtain an adjusted target slice allocation set, so as to use the adjusted target slice allocation set for network allocation.

[0078] It should be noted that the preset stability requirement can be set by those skilled in the art according to actual conditions and is not specifically limited here.

[0079] In the actual execution process, for each match (u i , v j ), check if there is another service uk and slice v l Satisfy the conditions: u i Preference v l Better than v j , and v l Preference i Outperforms its current matching business; j Preference k Better than u i , and u k Preference v j If the above conditions are met, the actual preferences of the two groups of combinations are considered to not meet the preset stability requirements, and they are readjusted to obtain the adjusted target slice allocation set for network allocation.

[0080] The working content of the embodiment of the present invention is described in detail below with a specific embodiment. Figure 2 The figure shows a logical diagram of adaptive business-driven network slicing optimization according to an embodiment of the present invention.

[0081] Specifically, logic ①: obtain personalized needs of different business scenarios, convert the needs into measurable indicators, and detect whether there are needs that are difficult to quantify. If so, use methods such as AHP to convert them into weight coefficients. Otherwise, establish a business demand model.

[0082] Logic ②: Divide the first-level slices according to the common attributes of the business demand model, divide the second-level slices according to personalized needs within the first-level slices, and check whether the personalized needs of all businesses are met. If not, divide the second-level slices again according to personalized needs within the first-level slices until entering the next step.

[0083] Logic ③: Perform multi-agent reinforcement learning, model each slice as an agent, define the agent's state space, action space, and reward function, deploy and train the agent's corresponding neural network, and the agent completes the training phase by learning the optimal slicing strategy through interaction with the environment. The trained neural network is then deployed, allowing the agent to dynamically adjust the slicing strategy based on real-time network status and business needs.

[0084] Logic ④: Take the slicing strategies of multiple agents as individuals in the population, define the fitness function in the population, use NSGA-II (Non-dominated Sorting Genetic Algorithms) to generate the Pareto front, and detect whether the number of iterations or convergence conditions are reached. If not, optimize the population through genetic operations to regenerate the Pareto front until the number of iterations or convergence conditions are reached, and select the most appropriate slicing strategy from the Pareto front.

[0085] Logic ⑤: Treat services as suitors and slices as acceptors. A bipartite graph is constructed, and services and slices are sorted according to their respective metrics. A delayed acceptance algorithm is used to achieve stable service-slice matching. The matching result is then tested to see if it meets the service's QoS (Quality of Service) requirements. If not, the service-slice mapping relationship is adjusted until the QoS requirements are met, completing the matching.

[0086] The service-driven network slice allocation optimization method proposed in the embodiment of the present invention can achieve a dynamically adjusted network slice optimization process based on actual business needs by constructing a network slicing framework based on the different needs of multiple business scenarios, and performing multi-agent reinforcement learning, multi-objective optimization, and business slice preference matching based on the framework, thereby ensuring the target requirements of network resource utilization efficiency and service quality, further improving user experience, and improving network operation and maintenance efficiency. This solves the problem in the related art that existing slicing network technology fails to fully adapt to the dynamically changing network environment and user requirements, and cannot simultaneously meet the dynamic adjustment requirements of slice resource configuration and the balanced service quality requirements of different businesses. It is difficult to make real-time dynamic adjustments according to actual business needs, which affects the user experience and reduces the service quality and operating efficiency of network technology.

[0087] Next, a service-driven network slice allocation optimization device proposed according to an embodiment of the present invention will be described with reference to the accompanying drawings.

[0088] Figure 3 It is a structural diagram of a service-driven network slice allocation optimization device according to an embodiment of the present invention.

[0089] like Figure 3 As shown, the service-driven network slice allocation optimization device 10 includes: an acquisition module 100, a construction module 200, a training module 300, an allocation module 400 and an optimization module 500.

[0090] The acquisition module 100 is used to acquire all business scenarios in the current network environment and confirm at least one actual requirement of each business scenario.

[0091] Construction module 200 is used to build a slice architecture for each business scenario based on the common attributes of each business scenario and at least one actual requirement, and confirm all slices corresponding to each business scenario according to the slice architecture.

[0092] The training module 300 is used to construct the corresponding intelligent agent for each slice in all slices respectively, generate the target neural network using the reward function of the intelligent agent, and train the target neural network based on the transfer sample of the intelligent agent to obtain the trained target neural network.

[0093] The allocation module 400 is used to output the slice allocation set corresponding to the slice by the trained target neural network, and perform population optimization on all slice allocation sets of each business scenario to obtain the target slice allocation set of each business scenario.

[0094] The optimization module 500 is used to obtain a preference list of all business scenarios and all slices based on the utility function of each business scenario, and optimize the target slice allocation set according to the preference list to use the optimized target slice allocation set to perform slice allocation in the current network environment.

[0095] Optionally, in one embodiment of the present invention, the service-driven network slice allocation optimization device 10 further includes: a first generation module, a judgment module and a second generation module.

[0096] Among them, the first generation module is used to generate all indicators of the current business scenario based on all actual needs of the current business scenario before obtaining the preference list of all business scenarios and all slices based on the utility function of each business scenario.

[0097] The judgment module is used to judge whether there is at least one indicator among all the indicators that does not meet the preset quantitative requirements.

[0098] The second generation module is used to construct a corresponding indicator system for the indicators that do not meet the preset quantitative requirements if there is at least one indicator among all the indicators that does not meet the preset quantitative requirements, confirm the weight coefficient according to the indicator system, and generate the utility function of the current business scenario based on all indicators and weight coefficients. Otherwise, a utility function is generated based on all indicators that meet the preset quantitative requirements.

[0099] Optionally, in one embodiment of the present invention, the construction module 200 includes: a first matching unit and a construction unit.

[0100] Among them, the first matching unit is used to match the actual category of each business scenario based on the common attributes of each business scenario, and the first-level slice corresponding to each business scenario is confirmed by the actual category.

[0101] The construction unit is used to perform cluster calculation on the first-level slices based on at least one actual requirement to obtain multiple second-level slices within the first-level slice range, so as to construct a slice architecture according to all the first-level slices and all the second-level slices.

[0102] Optionally, in one embodiment of the present invention, the training module 300 includes: a sampling unit, an updating unit and an iteration unit.

[0103] The sampling unit is used to sample the transfer sample to obtain a sampling result, and calculate the corresponding target expected value based on the sampling result.

[0104] The updating unit is used to update the network parameters of the target neural network based on the preset loss function using the target expected value and the actual expected value of the sampling result to obtain an updated target neural network.

[0105] The iterative unit is used to iteratively calculate the target neural network according to a preset time step interval until the updated target neural network meets the preset convergence condition, thereby obtaining the trained target neural network.

[0106] Optionally, in one embodiment of the present invention, the allocation module 400 includes: a confirmation unit and a second matching unit.

[0107] Among them, the confirmation unit is used to confirm the optimal population corresponding to each slice allocation set in the current business scenario.

[0108] The second matching unit is configured to match a target slice allocation set of the current service scenario in the non-dominated solution sets of all optimal populations based on at least one actual requirement.

[0109] Optionally, in one embodiment of the present invention, the service-driven network slice allocation optimization device 10 further includes: a detection module and an adjustment module.

[0110] Among them, the detection module is used to obtain all mapping combinations of the optimized target slice allocation set before using the optimized target slice allocation set to allocate slices in the current network environment, and detect whether the actual preferences of all mapping combinations meet the preset stability requirements.

[0111] The adjustment module is used to recombine all mapping combinations that do not meet the preset stability requirements when the actual preferences of all mapping combinations do not meet the preset stability requirements, so as to obtain an adjusted target slice allocation set, so as to use the adjusted target slice allocation set for network allocation.

[0112] It should be noted that the above explanation of the embodiment of the service-driven network slice allocation optimization method is also applicable to the service-driven network slice allocation optimization device of this embodiment, and will not be repeated here.

[0113] The service-driven network slice allocation optimization device proposed in an embodiment of the present invention can construct a network slicing framework based on the different needs of multiple business scenarios, and perform multi-agent reinforcement learning, multi-objective optimization, and business slice preference matching based on the framework, thereby realizing a dynamically adjusted network slicing optimization process according to actual business needs, ensuring the target requirements of network resource utilization efficiency and service quality, further improving user experience, and improving network operation and maintenance efficiency. This solves the problem in the related art that existing slicing network technology fails to fully adapt to the dynamically changing network environment and user requirements, and cannot simultaneously meet the dynamic adjustment requirements of slice resource configuration and the balanced service quality requirements of different businesses. It is difficult to make real-time dynamic adjustments according to actual business needs, which affects the user experience and reduces the service quality and operating efficiency of network technology.

[0114] Figure 4 This is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. The electronic device may include:

[0115] Memory 401 , processor 402 , and computer programs stored in the memory 401 and executable on the processor 402 .

[0116] When the processor 402 executes the program, the service-driven network slice allocation optimization method provided in the above embodiment is implemented.

[0117] Furthermore, the electronic device further includes:

[0118] The communication interface 403 is used for communication between the memory 401 and the processor 402 .

[0119] The memory 401 is used to store computer programs that can be run on the processor 402 .

[0120] The memory 401 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0121] If the memory 401, the processor 402, and the communication interface 403 are implemented independently, the communication interface 403, the memory 401, and the processor 402 can be connected to each other via a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0122] Optionally, in a specific implementation, if the memory 401 , the processor 402 and the communication interface 403 are integrated on a chip, the memory 401 , the processor 402 and the communication interface 403 can communicate with each other through an internal interface.

[0123] The processor 402 may be a central processing unit (CPU), an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention.

[0124] This embodiment also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, it implements the above-mentioned service-driven network slice allocation optimization method.

[0125] This embodiment also provides a computer program, which, when executed, implements the above-mentioned service-driven network slice allocation optimization method.

[0126] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples without contradiction.

[0127] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be understood to indicate or imply relative importance or implicitly specify the number of technical features indicated. Thus, a feature specified as "first" or "second" may explicitly or implicitly include at least one such feature. In the description of the present invention, "N" means at least two, such as two, three, etc., unless otherwise specifically defined.

[0128] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or N executable instructions for implementing a custom logical function or step of a process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may be performed out of the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present invention pertain.

[0129] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processor, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include the following: an electrical connection with one or N wires (electronic devices), a portable computer disk cartridge (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and a portable compact disc read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program can be obtained electronically by optically scanning the paper or other medium and then editing, interpreting or processing it in other suitable ways as necessary, and then storing it in a computer memory.

[0130] It should be understood that various parts of the present invention can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiment, the N steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used: a discrete logic circuit having logic gate circuits for implementing logic functions on data signals, an application-specific integrated circuit having suitable combinational logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0131] Those skilled in the art will understand that all or part of the steps in the method of the above embodiment can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0132] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing module, or each unit may exist physically separately, or two or more units may be integrated into a single module. The aforementioned integrated modules may be implemented in the form of hardware or in the form of software functional modules. If the integrated modules are implemented in the form of software functional modules and sold or used as independent products, they may also be stored in a computer-readable storage medium.

[0133] The storage medium mentioned above may be a read-only memory, a magnetic disk, or an optical disk, etc. Although the embodiments of the present invention have been shown and described above, it should be understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Persons skilled in the art may make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present invention.

Claims

1. A service-driven network slice allocation optimization method, characterized in that: The following steps are involved: Obtain all business scenarios in the current network environment and confirm at least one actual requirement for each business scenario; Building a slice architecture for each business scenario based on the common attributes of each business scenario and the at least one actual requirement, and confirming all slices corresponding to each business scenario according to the slice architecture; Constructing a corresponding intelligent agent for each of all slices respectively, generating a target neural network using the reward function of the intelligent agent, and training the target neural network based on the transfer samples of the intelligent agent to obtain a trained target neural network; The trained target neural network outputs a slice allocation set corresponding to the slice, and population optimization is performed on all slice allocation sets of each business scenario to obtain a target slice allocation set for each business scenario; Based on the utility function of each business scenario, a preference list of all business scenarios and all slices is obtained, and the target slice allocation set is optimized according to the preference list to use the optimized target slice allocation set to perform slice allocation in the current network environment.

2. The service-driven network slice allocation optimization method according to claim 1, characterized in that: Before obtaining the preference lists of all the business scenarios and all the slices based on the utility function of each business scenario, the method further includes: Generate all indicators of the current business scenario based on all actual requirements of the current business scenario; Determining whether at least one of all the indicators does not meet the preset quantitative requirements; If at least one of all the indicators does not meet the preset quantitative requirements, a corresponding indicator system is constructed for the indicators that do not meet the preset quantitative requirements, the weight coefficient is confirmed according to the indicator system, and the utility function of the current business scenario is generated based on all the indicators and the weight coefficient; otherwise, the utility function is generated based on all the indicators that meet the preset quantitative requirements.

3. The service-driven network slice allocation optimization method according to claim 1, characterized in that: The constructing a slice architecture for each business scenario based on the common attributes of each business scenario and the at least one actual requirement includes: Matching the actual category of each business scenario based on the common attributes of each business scenario, and determining the first-level slice corresponding to each business scenario according to the actual category; Cluster calculation is performed on the first-level slice based on the at least one actual requirement to obtain multiple second-level slices within the first-level slice range, so as to construct the slice architecture according to all the first-level slices and all the second-level slices.

4. The service-driven network slice allocation optimization method according to claim 1, characterized in that: The step of training the target neural network based on the transfer sample of the agent to obtain the trained target neural network includes: Sampling the transferred sample to obtain a sampling result, and calculating a corresponding target expected value based on the sampling result; Based on a preset loss function, the network parameters of the target neural network are updated using the target expected value and the actual expected value of the sampling result to obtain an updated target neural network; The target neural network is iteratively calculated according to a preset time step interval until the updated target neural network meets a preset convergence condition, thereby obtaining the trained target neural network.

5. The service-driven network slice allocation optimization method according to claim 1, characterized in that: The performing population optimization on all slice allocation sets of each business scenario to obtain a target slice allocation set for each business scenario includes: Confirm the optimal population corresponding to each slice allocation set in the current business scenario; Based on the at least one actual demand, a target slice allocation set of the current service scenario is matched in non-dominated solution sets of all optimal populations.

6. The service-driven network slice allocation optimization method according to claim 1, characterized in that: Before using the optimized target slice allocation set to allocate slices in the current network environment, the method further includes: Obtaining all mapping combinations of the optimized target slice allocation set, and detecting whether actual preferences of all mapping combinations meet preset stability requirements; If the actual preferences of all mapping combinations do not meet the preset stability requirement, all mapping combinations that do not meet the preset stability requirement are recombined to obtain an adjusted target slice allocation set, so as to use the adjusted target slice allocation set for network allocation.

7. A service-driven network slice allocation optimization device, characterized in that: include: The acquisition module is used to obtain all business scenarios in the current network environment and confirm at least one actual requirement for each business scenario; A construction module, configured to construct a slice architecture for each business scenario based on the common attributes of each business scenario and the at least one actual requirement, and confirm all slices corresponding to each business scenario according to the slice architecture; A training module is used to construct a corresponding intelligent agent for each of all slices, generate a target neural network using the reward function of the intelligent agent, and train the target neural network based on the transfer samples of the intelligent agent to obtain a trained target neural network; An allocation module, configured to output a slice allocation set corresponding to the slices from the trained target neural network, and perform population optimization on all slice allocation sets for each business scenario to obtain a target slice allocation set for each business scenario; An optimization module is used to obtain a preference list of all the business scenarios and all the slices based on the utility function of each business scenario, and optimize the target slice allocation set according to the preference list to use the optimized target slice allocation set to perform slice allocation in the current network environment.

8. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor executes the program to implement the service-driven network slice allocation optimization method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the service-driven network slice allocation optimization method as described in any one of claims 1 to 6.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed, it implements the service-driven network slice allocation optimization method as described in any one of claims 1 to 6.

Images