Network slice distribution optimization method and device based on service driving

Abstract

The invention relates to the technical field of communication networks, in particular to a network slice allocation optimization method and device based on service driving, and the method comprises the steps: constructing a slice architecture according to at least one actual demand of each service scene in a current network environment, and constructing an intelligent agent of each slice for training a target neural network, and outputting a slice allocation set by the trained target neural network to perform population optimization to obtain a target slice allocation set of each service scene, and optimizing the target slice allocation set based on the service scene and the preference list of the slices to perform slice allocation of the current network environment. According to the invention, the network slice framework can be constructed according to different requirements of a plurality of service scenes, and multi-agent reinforcement learning, multi-objective optimization and service slice preference matching are carried out on the basis of the framework, so that a dynamically adjusted network slice optimization process is realized according to actual service requirements, the user experience is improved, and the user experience is improved. And the network operation and maintenance efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of communication network technology, and in particular to a service-driven network slice allocation optimization method and device. Background Art

[0002] With the development of 5G communications, the requirements for network service technology have increased accordingly. In order to meet the demand for high-quality services, network slicing technology can be used to allow operators to create multiple virtual networks on the same physical network infrastructure. Each virtual network is optimized and configured according to the needs of specific services.

[0003] In the related technology, existing network slicing methods include dynamic access control and slice allocation algorithms, which can simultaneously consider sensitive indicators of user preferences and business requests, or through the combination of Markov decision process and reinforcement learning algorithm, learn the network environment and user behavior patterns based on intelligent agents, thereby dynamically adjusting the resource configuration of the slice.

[0004] However, in the related technologies, the existing slicing network technology fails to fully adapt to the dynamically changing network environment and user requirements, and cannot simultaneously meet the dynamic adjustment needs of slice resource configuration and the balanced service quality requirements of different businesses. It is difficult to make real-time dynamic adjustments according to actual business needs, which affects the user experience and reduces the service quality and operating efficiency of network technology, which needs to be solved urgently. Summary of the invention

[0005] The present invention provides a service-driven network slice allocation optimization method and device to solve the problem in the related technology that the existing slicing network technology fails to fully adapt to the dynamically changing network environment and user requirements, and cannot simultaneously meet the dynamic adjustment requirements of slice resource configuration and the balanced service quality requirements of different services, and is difficult to make real-time dynamic adjustments according to actual business needs, which affects the user experience and reduces the service quality and operation efficiency of the network technology.

[0006] The first aspect of the present invention provides a service-driven network slice allocation optimization method, comprising the following steps: obtaining all business scenarios in the current network environment, and respectively confirming at least one actual requirement of each business scenario; constructing a slice architecture for each business scenario based on the common attributes of each business scenario and the at least one actual requirement, and confirming all slices corresponding to each business scenario according to the slice architecture; respectively constructing a corresponding intelligent agent for each slice in all slices, generating a target neural network using the reward function of the intelligent agent, and training the target neural network based on the transfer samples of the intelligent agent to obtain a trained target neural network; outputting a slice allocation set of the corresponding slice by the trained target neural network, and performing population optimization on all slice allocation sets of each business scenario to obtain a target slice allocation set for each business scenario; based on the utility function of each business scenario, obtaining a preference list of all business scenarios and all slices, optimizing the target slice allocation set according to the preference list, and using the optimized target slice allocation set to perform slice allocation in the current network environment.

[0007] Optionally, in one embodiment of the present invention, before obtaining the preference list of all the business scenarios and all the slices based on the utility function of each business scenario, it also includes: generating all indicators of the current business scenario based on all actual needs of the current business scenario; judging whether there is at least one indicator among all the indicators that does not meet the preset quantitative requirements; if there is at least one indicator among all the indicators that does not meet the preset quantitative requirements, constructing a corresponding indicator system for the indicators that do not meet the preset quantitative requirements, confirming the weight coefficient according to the indicator system, and generating the utility function of the current business scenario according to all the indicators and the weight coefficient; otherwise, generating the utility function based on all indicators that meet the preset quantitative requirements.

[0008] Optionally, in one embodiment of the present invention, constructing a slice architecture for each business scenario based on the common attributes of each business scenario and the at least one actual requirement includes: matching the actual category of each business scenario based on the common attributes of each business scenario, and confirming the first-level slice corresponding to each business scenario by the actual category; performing clustering calculation on the first-level slice based on the at least one actual requirement to obtain multiple second-level slices within the range of the first-level slice, so as to construct the slice architecture based on all the first-level slices and all the second-level slices.

[0009] Optionally, in one embodiment of the present invention, the target neural network is trained based on the transfer samples of the agent to obtain the trained target neural network, including: sampling the transfer samples to obtain sampling results, and calculating the corresponding target expected value from the sampling results; based on a preset loss function, updating the network parameters of the target neural network using the target expected value and the actual expected value of the sampling result to obtain an updated target neural network; iteratively calculating the target neural network according to a preset time step interval until the updated target neural network meets a preset convergence condition to obtain the trained target neural network.

[0010] Optionally, in one embodiment of the present invention, the population optimization is performed on all slice allocation sets of each business scenario respectively to obtain the target slice allocation set of each business scenario, including: confirming the optimal population corresponding to each slice allocation set in the current business scenario; based on the at least one actual requirement, matching the target slice allocation set of the current business scenario in the non-dominated solution set of all optimal populations.

[0011] Optionally, in one embodiment of the present invention, before using the optimized target slice allocation set to perform slice allocation in the current network environment, it also includes: obtaining all mapping combinations of the optimized target slice allocation set, and detecting whether the actual preferences of all mapping combinations meet the preset stability requirements; if the actual preferences of all mapping combinations do not meet the preset stability requirements, all mapping combinations that do not meet the preset stability requirements are recombined to obtain an adjusted target slice allocation set, so as to use the adjusted target slice allocation set for network allocation.

[0012] The second aspect of the present invention provides a service-driven network slice allocation optimization device, including: an acquisition module, used to acquire all business scenarios in the current network environment, and respectively confirm at least one actual requirement of each business scenario; a construction module, used to construct a slice architecture for each business scenario based on the common attributes of each business scenario and the at least one actual requirement, and confirm all slices corresponding to each business scenario according to the slice architecture; a training module, used to respectively construct a corresponding intelligent agent for each slice in all slices, generate a target neural network using the reward function of the intelligent agent, and train the target neural network based on the transfer sample of the intelligent agent to obtain a trained target neural network; an allocation module, used to output a slice allocation set of the corresponding slice by the trained target neural network, and perform population optimization on all slice allocation sets of each business scenario to obtain a target slice allocation set for each business scenario; an optimization module, used to obtain a preference list of all business scenarios and all slices based on the utility function of each business scenario, and optimize the target slice allocation set according to the preference list to use the optimized target slice allocation set to perform slice allocation of the current network environment.

[0013] Optionally, in one embodiment of the present invention, the device also includes: a first generation module, which is used to generate all indicators of the current business scenario based on all actual needs of the current business scenario before obtaining the preference list of all business scenarios and all slices based on the utility function of each business scenario; a judgment module, which is used to judge whether there is at least one indicator among all the indicators that does not meet the preset quantitative requirements; a second generation module, which is used to construct a corresponding indicator system for the indicators that do not meet the preset quantitative requirements if there is at least one indicator among all the indicators that does not meet the preset quantitative requirements, confirm the weight coefficient according to the indicator system, and generate the utility function of the current business scenario according to all the indicators and the weight coefficient; otherwise, generate the utility function based on all indicators that meet the preset quantitative requirements.

[0014] Optionally, in one embodiment of the present invention, the construction module includes: a first matching unit, used to match the actual category of each business scenario based on the common attributes of each business scenario, and confirm the first-level slice corresponding to each business scenario by the actual category; a construction unit, used to perform clustering calculation on the first-level slice based on the at least one actual demand to obtain multiple second-level slices within the range of the first-level slice, so as to construct the slice architecture based on all the first-level slices and all the second-level slices.

[0015] Optionally, in one embodiment of the present invention, the training module includes: a sampling unit, used to sample the transfer samples to obtain sampling results, and calculate the corresponding target expected value from the sampling results; an updating unit, used to update the network parameters of the target neural network based on a preset loss function using the target expected value and the actual expected value of the sampling result to obtain an updated target neural network; an iteration unit, used to iteratively calculate the target neural network according to a preset time step interval until the updated target neural network meets a preset convergence condition to obtain the trained target neural network.

[0016] Optionally, in one embodiment of the present invention, the allocation module includes: a confirmation unit, used to confirm the optimal population corresponding to each slice allocation set in the current business scenario; and a second matching unit, used to match the target slice allocation set of the current business scenario in the non-dominated solution set of all optimal populations based on the at least one actual requirement.

[0017] Optionally, in one embodiment of the present invention, it also includes: a detection module, which is used to obtain all mapping combinations of the optimized target slice allocation set before using the optimized target slice allocation set to perform slice allocation in the current network environment, and detect whether the actual preferences of all mapping combinations meet the preset stability requirements; an adjustment module, which is used to recombine all mapping combinations that do not meet the preset stability requirements when the actual preferences of all mapping combinations do not meet the preset stability requirements, so as to obtain an adjusted target slice allocation set, so as to use the adjusted target slice allocation set for network allocation.

[0018] An embodiment of the third aspect of the present invention provides an electronic device, comprising: a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor executes the program to implement the service-driven network slice allocation optimization method as described in the above embodiment.

[0019] A fourth aspect of the present invention provides a computer-readable storage medium, which stores a computer program that, when executed by a processor, implements the above-mentioned service-driven network slice allocation optimization method.

[0020] A fifth aspect of the present invention provides a computer program, which, when executed, implements the above-mentioned service-driven network slice allocation optimization method.

[0021] The embodiments of the present invention can construct a network slicing framework according to the different requirements of multiple business scenarios, and perform multi-agent reinforcement learning, multi-objective optimization and business slice preference matching based on the framework, thereby realizing a dynamically adjusted network slicing optimization process according to actual business needs, ensuring the target requirements of network resource utilization efficiency and service quality, further improving user experience, and improving network operation and maintenance efficiency. Thus, the problems in the related technology that the existing slicing network technology fails to fully adapt to the dynamically changing network environment and user requirements, and cannot simultaneously meet the dynamic adjustment requirements of slice resource configuration and the balanced requirements of different businesses for service quality, and it is difficult to make real-time dynamic adjustments according to actual business needs, which affects the user experience and reduces the service quality and operation efficiency of network technology.

[0022] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0024] Figure 1 A flowchart of a service-driven network slice allocation optimization method according to an embodiment of the present invention;

[0025] Figure 2 A logical schematic diagram of adaptive service-driven network slicing optimization according to an embodiment of the present invention;

[0026] Figure 3 This is a schematic diagram of the structure of a service-driven network slice allocation optimization device according to an embodiment of the present invention;

[0027] Figure 4 FIG. 4 is a schematic diagram of the structure of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0028] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0029] The following describes the service-driven network slice allocation optimization method and device of the embodiment of the present invention with reference to the accompanying drawings. In view of the related technologies mentioned in the above background technology, the existing slicing network technology fails to fully adapt to the dynamically changing network environment and user requirements, and cannot simultaneously meet the dynamic adjustment requirements of slice resource configuration and the balanced requirements of different services for service quality. It is difficult to make real-time dynamic adjustments according to actual business needs, which affects the user experience and reduces the service quality and operation efficiency of network technology. The present invention provides a service-driven network slice allocation optimization method, in which a network slicing framework can be constructed according to the different requirements of multiple business scenarios, and multi-agent reinforcement learning, multi-objective optimization and business slice preference matching are performed on the basis of the framework, so as to realize the dynamic adjustment of the network slice optimization process according to the actual business needs, ensure the utilization efficiency of network resources and the target requirements of service quality, further improve the user experience, and improve the network operation and maintenance efficiency. Therefore, the existing slicing network technology in the related technology fails to fully adapt to the dynamically changing network environment and user requirements, and cannot simultaneously meet the dynamic adjustment requirements of slice resource configuration and the balanced requirements of different services for service quality. It is difficult to make real-time dynamic adjustments according to actual business needs, which affects the user experience and reduces the service quality and operation efficiency of network technology.

[0030] Specifically, Figure 1 A flow chart of a service-driven network slice allocation optimization method provided in an embodiment of the present invention.

[0031] like Figure 1 As shown, the service-driven network slice allocation optimization method includes the following steps:

[0032] In step S101, all business scenarios in the current network environment are acquired, and at least one actual requirement of each business scenario is confirmed respectively.

[0033] It is understandable that in the embodiments of the present invention, personalized needs of different business scenarios (such as high-definition video, telemedicine, industrial control, etc.) can be collected through questionnaires, customer interviews, etc., including indicators such as bandwidth, latency, and reliability as the actual needs of the business scenarios. For example, video services require high bandwidth and low latency, while IoT services require low power consumption and high reliability.

[0034] In step S102, based on the common attributes of each business scenario and at least one actual requirement, a slice architecture for each business scenario is constructed, and all slices corresponding to each business scenario are confirmed according to the slice architecture.

[0035] It can be understood that in an embodiment of the present invention, the utility function constructed according to the actual needs of the business scenario is used to obtain the common attributes of the business scenario (such as delay sensitivity), and each business scenario is divided into multiple slices to construct a slice architecture for each business scenario.

[0036] Optionally, in one embodiment of the present invention, a slice architecture for each business scenario is constructed based on the common attributes of each business scenario and at least one actual requirement, including: matching the actual category of each business scenario based on the common attributes of each business scenario, and confirming the first-level slice corresponding to each business scenario by the actual category; performing cluster calculation on the first-level slices based on at least one actual requirement to obtain multiple second-level slices within the range of the first-level slices, so as to construct a slice architecture based on all first-level slices and all second-level slices.

[0037] In the actual implementation process, the services are divided into several major categories, corresponding to coarse-grained first-level slices, such as eMBB (Enhanced Mobile Broadband), URLLC (Ultra-Reliable Low Latency Communications), mMTC (massive Machine Type of Communication), and other major 5G application scenarios, as well as new application scenarios expected in 6G, such as holographic communications and integrated space-air networks.

[0038] In each first-level slice, fine-grained second-level slices can be further divided according to the differences in personalized needs (such as bandwidth requirements). For example, under the eMBB first-level slice, it can be divided into eMBB-ultra-high-definition video transmission, eMBB-immersive experience, etc.; under the holographic communication first-level slice, it can be divided into holographic communication-holographic video conferencing, holographic communication-remote holographic teaching, etc. Among them, clustering algorithms such as K-means clustering algorithm can be used to group services, automatically complete the division of multi-level slices, and form a hierarchical slice architecture, each slice corresponds to a specific set of service level agreements (SLA, Service Level Agreement).

[0039] Specifically, the K-means algorithm steps include: inputting a service set S = [s1, s2, ..., s n ], each business i By its attribute vector x i Indicates that the number of clusters is K; output the slice division result, that is, the slice label to which each service belongs. Randomly select K services as the initial cluster centers [μ1, μ2, …, μ K ], for each business s i∈S, and calculate its distance from each cluster center as:

[0040] d ij =∥x i -μ j ∥ 2 , j = 1, 2, ..., K,

[0041] Then the business i Assign to the slice corresponding to the nearest cluster center:

[0042] l(s i ) = arg min j d ij ,

[0043] For each slice j=1, 2, ..., K, recalculate its cluster center Among them C j Represents the business set assigned to slice j. Repeat the iteration until the clustering result converges or the maximum number of iterations is reached, and return the final slice result [C1, C2, ..., C K ], and get the slice architecture.

[0044] In step S103, a corresponding intelligent agent is constructed for each slice in all slices, a target neural network is generated using the reward function of the intelligent agent, and the target neural network is trained based on the transfer samples of the intelligent agent to obtain a trained target neural network.

[0045] It can be understood that in the embodiment of the present invention, each slice is modeled as an agent, and the state space and action space of the agent are defined to obtain the state space S of each agent, including factors such as the resource occupancy of the slice and business satisfaction, which can be expressed as a vector: S = [s1, s2, ..., s n ],s i represents the i-th state feature, such as the resource occupancy rate of a specific slice, the current latency, etc. The action space A of the agent includes actions such as applying for or releasing resources and adjusting slice strategies, which can be expressed as: A = [a1, a2, ..., a m ], a j Represents the jth possible action. The agent's reward function Q represents the expected value of the long-term cumulative reward for taking action a in state s, that is:

[0046] Q(s,a)=E[R t ∣∣s t =s,a t =a],

[0047] Among them, R tis the reward value of the agent. The neural network Q(s, a, θ) can be used to approximate the Q function, θ is the network parameter, and the target neural network is obtained. The target neural network is trained based on the transfer sample of the agent to obtain the trained target neural network.

[0048] Optionally, in one embodiment of the present invention, a target neural network is trained based on the transfer samples of the intelligent agent to obtain a trained target neural network, including: sampling the transfer samples to obtain sampling results, and calculating the corresponding target expected value from the sampling results; based on a preset loss function, updating the network parameters of the target neural network using the target expected value and the actual expected value of the sampling result to obtain an updated target neural network; iteratively calculating the target neural network according to a preset time step interval until the updated target neural network meets a preset convergence condition to obtain a trained target neural network.

[0049] It should be noted that the preset time step interval and the preset convergence condition can be set by those skilled in the art according to actual conditions and are not specifically limited here.

[0050] In the actual execution process, an experience replay buffer D can be defined to store the transfer samples (s t , a t , r t ,s t+1 ). At each time step, the agent randomly selects an action with probability ε and selects an action according to the current Q function (the actual value of the Q function) with probability 1-ε, that is, a t = arg max a Q(s t , a, θ). The transferred sample (s t , a t , r t ,s t+1 ) is stored in the experience replay buffer D.

[0051] Specifically, the DNQ (Dueling Network Q-Network) reinforcement learning algorithm can be used to randomly sample transfer samples from the experience replay buffer D to obtain the sampling result (s, a, r, s′). If s′ is the terminal state, y=r, otherwise it is expressed as:

[0052] y=r+γmax a‘ Q(s′, a′, θ - ),

[0053] Among them, θ - is the parameter of the target network, γ is the discount factor, y represents the target Q value, Q(s′, a′, θ -) represents the maximum Q value of the target network in the next state s', that is, the target expected value.

[0054] The target expected value and the actual expected value are input into the preset loss function, which is expressed as:

[0055] L(θ)=E (s，a，r′)～D [(yQ(s,a,θ) 2 )],

[0056] Use the gradient descent method to update the parameters θ of the Q network, that is, Where α is the learning rate, and the updated network parameters are obtained. Every certain number of steps, the parameters θ of the Q network are copied to the parameters θ of the target network. - , that is, θ - ←θ.

[0057] For each slice, according to the current state s t , select the action a with the largest Q value t , perform action a t , observe the reward r t and the next state s t+1 , and transfer samples (s t , a t , r t ,s t+1 ) is stored in the experience playback buffer D. Repeat the above steps for iterative calculation, to achieve online learning and dynamic adjustment, until the updated target neural network meets the preset convergence conditions, and the trained target neural network is obtained under a fixed number of iterations.

[0058] Specifically, for example, as shown in the following Algorithm 1, which is a pseudo code for the multi-agent network slicing optimization process, the target neural network can be trained based on the transfer samples of the agent to obtain the trained target neural network:

[0059]

[0060]

[0061] In step S104, the trained target neural network outputs a slice allocation set corresponding to the slice, and population optimization is performed on all slice allocation sets of each business scenario to obtain a target slice allocation set for each business scenario.

[0062] It can be understood that in an embodiment of the present invention, the slice allocation set can be a slice strategy, and the slice strategies of multiple intelligent agents are used as individuals in the population. Each individual corresponds to a set of slice parameters (such as bandwidth, latency, etc.), and the fitness function of the population includes business satisfaction, cost, and resource utilization, forming a multi-objective optimization problem of the population, and finally obtaining the target slice allocation set for each business scenario.

[0063] Optionally, in one embodiment of the present invention, population optimization is performed on all slice allocation sets of each business scenario to obtain a target slice allocation set for each business scenario, including: confirming the optimal population corresponding to each slice allocation set in the current business scenario; based on at least one actual requirement, matching the target slice allocation set of the current business scenario in the non-dominated solution set of all optimal populations.

[0064] In the actual execution process, an initial population P0 of size N can be randomly generated, each individual represents a set of slice parameters, and the population P0 is non-dominated sorted to obtain the rank and crowding distance of each individual. t , using the binary tournament selection algorithm, from P t Select N individuals from the parent population Q t ; for Q t The individuals in the crossover operation are carried out to obtain the offspring population R t ; for R t The individuals in P are mutated to introduce new diversity; t and R t Merge to get a population S of size 2N t ; for S t Perform non-dominated sorting to obtain the rank of each individual; for each rank of individuals, calculate its crowding distance; select S according to the rank and crowding distance t The first N individuals in the new population P are obtained. t+1 , repeat the above steps until the convergence condition is met or the number of iterations is reached.

[0065] Specifically, the binary tournament selection algorithm initializes the parent population Q t is an empty set, from the current population P t Randomly select two individuals a and b from Q. If a’s non-dominated rank is lower than b’s, or if a and b have the same non-dominated rank but a’s crowding distance is greater than b’s, then add a to Q. t Otherwise, add b to Q t , returns the parent population Q t , and repeat the above process.

[0066] For non-dominated sorting, in multi-objective optimization, if a solution is considered to "non-dominate" another solution, and in all objective functions, the solution is better than another solution in some objectives, but not inferior to the other solution in other objectives, then the solution is considered to "dominate" another solution. In non-dominated sorting, the dominating set S is initialized for each individual in the population P. p and the number of dominated p is empty; for each individual p, compare it with other individuals in the population, and if p dominates another individual q, add q to S p , if p is dominated by another individual q, then n p Add 1; put all individuals with 0 dominated number into the first non-dominated level F1; for each individual p in F1, access its dominating set S p For each individual q in , reduce the dominated number of q by 1. If the dominated number of q becomes 0, put it into the next non-dominated level F2, and repeat the above steps until all individuals are assigned to their respective non-dominated levels.

[0067] The crowding distance is an indicator to measure the degree of crowding of individuals in a population. It indicates the distance between an individual and other individuals in the target space and reflects the density of the space around the individual. The larger the crowding distance, the sparser the space around the individual and the higher the diversity of the individual. On the contrary, the smaller the crowding distance, the more crowded the space around the individual and the lower the diversity of the individual. It is used to measure the difference and diversity between different slicing strategies. The crowding distance can be calculated by sorting the individuals according to the target value for each objective function, calculating the distance between two adjacent individuals, and adding it to the crowding distance of the individual; for the boundary individuals (the first and last after sorting), their crowding distance is set to infinity.

[0068] Finally, according to at least one actual demand of the vehicle, the optimal slicing strategy for the current business scenario is selected from the non-dominated solution set of the last generation population, that is, the optimal population, to obtain the target slice allocation set.

[0069] In step S105, based on the utility function of each business scenario, a preference list of all business scenarios and all slices is obtained, and the target slice allocation set is optimized according to the preference list to use the optimized target slice allocation set to perform slice allocation in the current network environment.

[0070] It can be understood that in an embodiment of the present invention, a utility function is constructed based on the actual needs of each business scenario, and all business scenarios and all slices are sorted using the utility function to obtain preference lists of business scenarios and slices, respectively, so that the target slice allocation set is adjusted secondary according to the preference list, so as to utilize the optimized target slice allocation set to allocate each business to the corresponding slice, and dynamically adjust the matching relationship during operation according to changes in business needs and network status.

[0071] The service set can be expressed as U = [u1, u2, ..., u n ], and the slice set is represented as V = [v1, v2, ..., v m ], for each business u i , sort all slices according to their utility functions and obtain the preference list P(u i )=[u i1 ,u i2 ,…,u im ], for each slice v j Sort all services according to resource utilization and other indicators to obtain the preference list P(v j )=[v j1 , v j2 , …, v jn ].

[0072] Initialize the matching state of each service to free and the matching state of each slice to idle. i , according to u im The preference list P(u im ), to its most preferred slice v j Initiate an application, if v j Idle, then (u i , v j ) is added to the matching set M; if v j Already with another business k matches, but u i In v j rank higher in the preference list, then release (u k , v j ) matches, and (u i , v j ) is added to the matching set M, and u k Mark as free. Repeat the above steps until all businesses are matched to slices or no new matches can be found, and obtain the optimized target slice allocation set.

[0073] Optionally, in one embodiment of the present invention, before obtaining a preference list of all business scenarios and all slices based on the utility function of each business scenario, it also includes: generating all indicators of the current business scenario based on all actual needs of the current business scenario; judging whether there is at least one indicator among all indicators that does not meet the preset quantitative requirements; if there is at least one indicator among all indicators that does not meet the preset quantitative requirements, constructing a corresponding indicator system for the indicators that do not meet the preset quantitative requirements, confirming the weight coefficient according to the indicator system, and generating the utility function of the current business scenario according to all indicators and the weight coefficients; otherwise, generating a utility function based on all indicators that meet the preset quantitative requirements.

[0074] It should be noted that the preset quantization requirement can be set by those skilled in the art according to actual conditions and is not specifically limited here.

[0075] In the actual implementation process, we can first convert the qualitative demand description in the actual demand into quantitative indicators, such as converting "high-definition video" into "throughput ≥ 10Mbps, latency ≤ 50ms", etc., to obtain all indicators. For example, the bandwidth requirement of the video service is 100Mbps, and the latency requirement is 20ms; the power consumption requirement of the IoT service is 0.1W, and the reliability requirement is 99.999%.

[0076] For indicators that do not meet the preset quantitative requirements, that is, requirements that cannot be directly quantified, such as user experience, the Analytic Hierarchy Process (AHP) is used to build an indicator system, and the weight coefficient is determined by expert scoring. The business demand model is constructed by combining various indicators and weights, expressed as a utility function U (b, d, r), where b, d, and r represent indicators such as bandwidth, latency, and reliability, respectively.

[0077] Optionally, in one embodiment of the present invention, before using the optimized target slice allocation set to perform slice allocation in the current network environment, it also includes: obtaining all mapping combinations of the optimized target slice allocation set, and detecting whether the actual preferences of all mapping combinations meet the preset stability requirements; if the actual preferences of all mapping combinations do not meet the preset stability requirements, all mapping combinations that do not meet the preset stability requirements are recombined to obtain an adjusted target slice allocation set, so as to use the adjusted target slice allocation set for network allocation.

[0078] It should be noted that the preset stability requirement can be set by those skilled in the art according to actual conditions and is not specifically limited here.

[0079] In the actual execution process, for each match (u i , v j ), check if there is another service uk and slice v l Satisfy the conditions: i Preference l Better than v j , and v l Preference i Outperforms its current matching business; j Preference k Better than u i , and u k Preference j If the above conditions are met, the actual preference of the two groups of combinations is considered to not meet the preset stability requirements, and they are readjusted to obtain the adjusted target slice allocation set for network allocation.

[0080] The working content of the embodiment of the present invention is described in detail below with a specific embodiment. Figure 2 The figure shows a logical diagram of adaptive business-driven network slicing optimization according to an embodiment of the present invention.

[0081] Specifically, logic ①: obtain personalized needs of different business scenarios, convert the needs into measurable indicators, and detect whether there are needs that are difficult to quantify. If so, use methods such as AHP to convert them into weight coefficients. Otherwise, establish a business demand model.

[0082] Logic ②: Divide the first-level slices according to the common attributes of the business demand model, divide the second-level slices according to personalized needs within the first-level slices, and check whether the personalized needs of all businesses are met. If not, divide the second-level slices within the first-level slices according to personalized needs until entering the next step.

[0083] Logic ③: Perform multi-agent reinforcement learning, model each slice as an agent, define the agent's state space, action space, and reward function, deploy and train the neural network corresponding to the agent, and the agent completes the training phase by learning the optimal slicing strategy through interaction with the environment. The trained neural network is then deployed, allowing the agent to dynamically adjust the slicing strategy based on the real-time network status and business needs.

[0084] Logic ④: Take the slicing strategies of multiple agents as individuals in the population, define the fitness function in the population, use NSGA-II (Non-dominated Sorting Genetic Algorithms) to generate the Pareto frontier, and detect whether the number of iterations or convergence conditions are reached. If not, optimize the population through genetic operations to regenerate the Pareto frontier until the number of iterations or convergence conditions are reached, and select the most appropriate slicing strategy from the Pareto frontier.

[0085] Logic ⑤: The business is regarded as a suitor and the slice is regarded as an acceptor. A bipartite graph is constructed. The business and slice are ranked according to their respective indicators. A delayed acceptance algorithm is used to achieve stable matching of business and slice. The matching result is tested to see whether it meets the QoS (Quality of Service) requirement of the business. If not, the mapping relationship between the business and slice is adjusted until the QoS requirement is met and the matching is completed.

[0086] The service-driven network slice allocation optimization method proposed in the embodiment of the present invention can construct a network slice framework according to the different requirements of multiple business scenarios, and perform multi-agent reinforcement learning, multi-objective optimization and business slice preference matching based on the framework, thereby realizing a dynamically adjusted network slice optimization process according to actual business needs, ensuring the target requirements of network resource utilization efficiency and service quality, further improving user experience, and improving network operation and maintenance efficiency. As a result, the problem in the related technology that the existing slicing network technology fails to fully adapt to the dynamically changing network environment and user requirements, and cannot simultaneously meet the dynamic adjustment requirements of slice resource configuration and the balanced requirements of different services for service quality, and is difficult to make real-time dynamic adjustments according to actual business needs, affecting the user experience, and reducing the service quality and operation efficiency of network technology.

[0087] Secondly, a service-driven network slice allocation optimization device proposed in an embodiment of the present invention is described with reference to the accompanying drawings.

[0088] Figure 3 It is a structural diagram of a service-driven network slice allocation optimization device according to an embodiment of the present invention.

[0089] like Figure 3 As shown, the service-driven network slice allocation optimization device 10 includes: an acquisition module 100, a construction module 200, a training module 300, an allocation module 400 and an optimization module 500.

[0090] The acquisition module 100 is used to acquire all business scenarios in the current network environment and confirm at least one actual requirement of each business scenario.

[0091] Construction module 200 is used to build a slice architecture for each business scenario based on the common attributes of each business scenario and at least one actual requirement, and confirm all slices corresponding to each business scenario according to the slice architecture.

[0092] The training module 300 is used to construct the corresponding intelligent agent for each slice in all the slices, generate the target neural network using the reward function of the intelligent agent, and train the target neural network based on the transfer samples of the intelligent agent to obtain the trained target neural network.

[0093] The allocation module 400 is used to output the slice allocation set corresponding to the slice by the trained target neural network, and perform population optimization on all slice allocation sets of each business scenario to obtain the target slice allocation set of each business scenario.

[0094] The optimization module 500 is used to obtain a preference list of all business scenarios and all slices based on the utility function of each business scenario, and optimize the target slice allocation set according to the preference list to use the optimized target slice allocation set to perform slice allocation in the current network environment.

[0095] Optionally, in one embodiment of the present invention, the service-driven network slice allocation optimization device 10 also includes: a first generation module, a judgment module and a second generation module.

[0096] Among them, the first generation module is used to generate all indicators of the current business scenario based on all actual needs of the current business scenario before obtaining the preference list of all business scenarios and all slices based on the utility function of each business scenario.

[0097] The judgment module is used to judge whether there is at least one indicator among all the indicators that does not meet the preset quantitative requirements.

[0098] The second generation module is used to construct a corresponding indicator system for the indicators that do not meet the preset quantitative requirements if there is at least one indicator among all the indicators that does not meet the preset quantitative requirements, confirm the weight coefficient according to the indicator system, and generate the utility function of the current business scenario according to all the indicators and the weight coefficients; otherwise, generate the utility function based on all the indicators that meet the preset quantitative requirements.

[0099] Optionally, in one embodiment of the present invention, the construction module 200 includes: a first matching unit and a construction unit.

[0100] Among them, the first matching unit is used to match the actual category of each business scenario based on the common attributes of each business scenario, and the first-level slice corresponding to each business scenario is confirmed by the actual category.

[0101] The construction unit is used to perform cluster calculation on the first-level slices based on at least one actual demand to obtain multiple second-level slices within the first-level slice range, so as to construct a slice architecture according to all the first-level slices and all the second-level slices.

[0102] Optionally, in one embodiment of the present invention, the training module 300 includes: a sampling unit, an updating unit and an iteration unit.

[0103] The sampling unit is used to sample the transfer sample to obtain a sampling result, and calculate the corresponding target expected value based on the sampling result.

[0104] The updating unit is used to update the network parameters of the target neural network based on the preset loss function using the target expected value and the actual expected value of the sampling result to obtain an updated target neural network.

[0105] The iterative unit is used to iteratively calculate the target neural network according to a preset time step interval until the updated target neural network meets the preset convergence condition, thereby obtaining the trained target neural network.

[0106] Optionally, in one embodiment of the present invention, the allocation module 400 includes: a confirmation unit and a second matching unit.

[0107] Among them, the confirmation unit is used to confirm the optimal population corresponding to each slice allocation set in the current business scenario.

[0108] The second matching unit is used to match the target slice allocation set of the current business scenario in the non-dominated solution set of all optimal populations based on at least one actual demand.

[0109] Optionally, in one embodiment of the present invention, the service-driven network slice allocation optimization device 10 also includes: a detection module and an adjustment module.

[0110] Among them, the detection module is used to obtain all mapping combinations of the optimized target slice allocation set before using the optimized target slice allocation set to allocate slices in the current network environment, and detect whether the actual preferences of all mapping combinations meet the preset stability requirements.

[0111] The adjustment module is used to re-combine all mapping combinations that do not meet the preset stability requirements when the actual preferences of all mapping combinations do not meet the preset stability requirements, so as to obtain an adjusted target slice allocation set, so as to use the adjusted target slice allocation set for network allocation.

[0112] It should be noted that the aforementioned explanation of the embodiment of the service-driven network slice allocation optimization method is also applicable to the service-driven network slice allocation optimization device of this embodiment, and will not be repeated here.

[0113] According to the service-driven network slice allocation optimization device proposed in the embodiment of the present invention, a network slice framework can be constructed according to the different requirements of multiple business scenarios, and multi-agent reinforcement learning, multi-objective optimization and business slice preference matching can be performed on the basis of the framework, thereby realizing a dynamically adjusted network slice optimization process according to actual business needs, ensuring the target requirements of network resource utilization efficiency and service quality, further improving user experience, and improving network operation and maintenance efficiency. Thus, the problem in the related technology that the existing slicing network technology fails to fully adapt to the dynamically changing network environment and user requirements, and cannot simultaneously meet the dynamic adjustment requirements of slice resource configuration and the balanced requirements of different services for service quality, and it is difficult to make real-time dynamic adjustments according to actual business needs, which affects the user experience and reduces the service quality and operation efficiency of network technology.

[0114] Figure 4 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. The electronic device may include:

[0115] Memory 401 , processor 402 , and a computer program stored in the memory 401 and executable on the processor 402 .

[0116] When the processor 402 executes the program, the service-driven network slice allocation optimization method provided in the above embodiment is implemented.

[0117] Furthermore, the electronic device further comprises:

[0118] The communication interface 403 is used for communication between the memory 401 and the processor 402 .

[0119] The memory 401 is used to store computer programs that can be executed on the processor 402 .

[0120] The memory 401 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0121] If the memory 401, the processor 402 and the communication interface 403 are implemented independently, the communication interface 403, the memory 401 and the processor 402 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0122] Optionally, in a specific implementation, if the memory 401, the processor 402 and the communication interface 403 are integrated on a chip, the memory 401, the processor 402 and the communication interface 403 can communicate with each other through an internal interface.

[0123] The processor 402 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention.

[0124] This embodiment also provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, it implements the above-mentioned service-driven network slice allocation optimization method.

[0125] This embodiment also provides a computer program, which, when executed, implements the above-mentioned service-driven network slice allocation optimization method.

[0126] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or N embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0127] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "N" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0128] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or N executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present invention belong.

[0129] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or N wirings (electronic devices), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways as necessary and then storing it in a computer memory.

[0130] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above embodiment, the N steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0131] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0132] In addition, each functional unit in each embodiment of the present invention may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0133] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present invention have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present invention. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present invention.

Claims

1. A service-driven network slice allocation optimization method, characterized in that: The following steps are involved: Obtain all business scenarios in the current network environment and confirm at least one actual requirement for each business scenario; Based on the common attributes of each business scenario and the at least one actual requirement, construct a slice architecture for each business scenario, and confirm all slices corresponding to each business scenario according to the slice architecture; Constructing a corresponding intelligent agent for each of all the slices respectively, generating a target neural network using a reward function of the intelligent agent, and training the target neural network based on the transfer samples of the intelligent agent to obtain a trained target neural network; The trained target neural network outputs a slice allocation set corresponding to the slice, and performs population optimization on all slice allocation sets of each business scenario to obtain a target slice allocation set for each business scenario; Based on the utility function of each business scenario, a preference list of all the business scenarios and all the slices is obtained, and the target slice allocation set is optimized according to the preference list to use the optimized target slice allocation set to perform slice allocation in the current network environment.

2. The service-driven network slice allocation optimization method according to claim 1, characterized in that: Before obtaining the preference lists of all the business scenarios and all the slices based on the utility function of each business scenario, the method further includes: Based on all actual requirements of the current business scenario, generate all indicators of the current business scenario; Determine whether there is at least one indicator among all the indicators that does not meet the preset quantitative requirements; If at least one of all the indicators does not meet the preset quantitative requirements, a corresponding indicator system is constructed for the indicators that do not meet the preset quantitative requirements, the weight coefficients are confirmed according to the indicator system, and the utility function of the current business scenario is generated according to all the indicators and the weight coefficients; otherwise, the utility function is generated based on all the indicators that meet the preset quantitative requirements.

3. The service-driven network slice allocation optimization method according to claim 1, characterized in that: The constructing a slice architecture for each business scenario based on the common attributes of each business scenario and the at least one actual requirement includes: Matching the actual category of each business scenario based on the common attributes of each business scenario, and confirming the first-level slice corresponding to each business scenario according to the actual category; Based on the at least one actual requirement, cluster calculation is performed on the first-level slice to obtain multiple second-level slices within the first-level slice range, so as to construct the slice architecture according to all the first-level slices and all the second-level slices.

4. The service-driven network slice allocation optimization method according to claim 1, characterized in that: The step of training the target neural network based on the transfer sample of the agent to obtain the trained target neural network includes: Sampling the transfer sample to obtain a sampling result, and calculating a corresponding target expected value based on the sampling result; Based on a preset loss function, the network parameters of the target neural network are updated using the target expected value and the actual expected value of the sampling result to obtain an updated target neural network; The target neural network is iteratively calculated according to a preset time step interval until the updated target neural network meets a preset convergence condition, thereby obtaining the trained target neural network.

5. The service-driven network slice allocation optimization method according to claim 1, characterized in that: The performing population optimization on all slice allocation sets of each business scenario to obtain a target slice allocation set of each business scenario includes: Confirm the optimal population corresponding to each slice allocation set in the current business scenario; Based on the at least one actual demand, a target slice allocation set of the current service scenario is matched in a non-dominated solution set of all optimal populations.

6. The service-driven network slice allocation optimization method according to claim 1, characterized in that: Before using the optimized target slice allocation set to allocate slices in the current network environment, the method further includes: Acquire all mapping combinations of the optimized target slice allocation set, and detect whether actual preferences of all mapping combinations meet preset stability requirements; If the actual preferences of all mapping combinations do not meet the preset stability requirement, all mapping combinations that do not meet the preset stability requirement are recombined to obtain an adjusted target slice allocation set, so as to use the adjusted target slice allocation set for network allocation.

7. A service-driven network slice allocation optimization device, characterized in that: include: An acquisition module is used to acquire all business scenarios in the current network environment and confirm at least one actual requirement of each business scenario; A construction module, configured to construct a slice architecture for each business scenario based on the common attributes of each business scenario and the at least one actual requirement, and confirm all slices corresponding to each business scenario according to the slice architecture; A training module is used to construct a corresponding intelligent agent for each of all the slices, generate a target neural network using a reward function of the intelligent agent, and train the target neural network based on the transfer sample of the intelligent agent to obtain a trained target neural network; An allocation module, configured to output a slice allocation set corresponding to the slices from the trained target neural network, and perform population optimization on all slice allocation sets of each business scenario to obtain a target slice allocation set for each business scenario; An optimization module is used to obtain a preference list of all the business scenarios and all the slices based on the utility function of each business scenario, and optimize the target slice allocation set according to the preference list to use the optimized target slice allocation set to perform slice allocation in the current network environment.

8. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the service-driven network slice allocation optimization method as described in any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the service-driven network slice allocation optimization method as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed, the service-driven network slice allocation optimization method as described in any one of claims 1 to 6 is implemented.

Images