A hybrid model-based optimization method and system for allocating edge cloud resources

Abstract

This invention discloses a method for optimizing edge cloud resource allocation based on a hybrid model, comprising the following steps: collecting characteristic data of edge cloud devices and edge cloud services and converting the characteristic data; inputting the converted characteristic data into a multi-layer perceptron for training, and simultaneously optimizing the training using a gradient descent-based BP algorithm to obtain a bandwidth usage prediction model; calculating average bandwidth utilization, gross profit, and gross profit margin based on the bandwidth usage predicted by the bandwidth usage prediction model; and using the NSGA-II genetic algorithm and the DQN algorithm to solve the objective optimization problem of maximizing average bandwidth utilization, gross profit, and gross profit margin to obtain an optimal resource allocation solution. This invention solves the hybrid deployment problem and optimizes comprehensive resource utilization.

Description

Technical Field

[0001] The present invention belongs to the field of edge cloud computing technology, and specifically relates to a method and system for optimizing the allocation of edge cloud resources based on a hybrid model. The method can not only be applied to edge cloud scenarios, but can also be reused in multiple scenarios such as central cloud, IDC, CDN, terminal devices, etc. to optimize cloud resource allocation and deployment. Background Art

[0002] The ultra-low latency, massive data volumes, and edge intelligence of the edge cloud have fueled its rapid growth, making edge cloud computing a crucial component of the network between the central cloud and endpoints. In edge cloud computing scenarios, fully utilizing computing resources and providing as many services as possible within limited resources is a core competency of edge cloud computing services. A well-designed edge cloud resource allocation approach, by co-locating various types of services (computing, storage, and traffic), leverages differences in service resource and time preferences, improving resource utilization and ultimately generating more revenue.

[0003] Existing cloud resource allocation methods only focus on the performance of homogeneous central cloud resources when allocating them to different services. They cannot address the complex heterogeneity of edge cloud devices and the volatile nature of services in actual edge cloud scenarios. Specifically, they have the following shortcomings: they only address relatively limited scenarios, and edge cloud device resources are extremely heterogeneous. The performance of services on edge cloud devices (such as bandwidth usage and revenue) is affected by a variety of complex factors, including server attributes, deployment region, service characteristics, population, time and space. Due to rigid business demand, existing allocation schemes are based on a single optimization algorithm or scoring model. When allocating resources, they assume that business demand is fixed or follows the patterns of the previous time period, resulting in demand deviation, reducing the effectiveness of allocation schemes in actual applications and causing lags. The optimization objectives are relatively simple, and the effectiveness of edge cloud resource allocation requires multi-dimensional measurement, such as resource utilization, revenue (revenue), gross profit, and rate of return. Due to the heterogeneous performance of edge clouds, some edge cloud devices can provide more resources than a single service requires. This creates the need to deploy multiple services on the same equipment vendor, which greatly increases the difficulty of designing allocation methods. Summary of the Invention

[0004] Due to the heterogeneous and complex nature of edge cloud devices and the volatile nature of their services, traditional cloud resource allocation solutions are unable to adapt to existing heterogeneous devices and cannot meet the needs of mixed deployment of services on edge clouds. This invention proposes a method and system for optimizing the allocation of edge cloud resources based on a hybrid model, which solves the lag caused by the conventional cloud resource allocation solution mainly based on the allocation of known business needs, the heterogeneity brought about by edge cloud device resources, and the mixed deployment of services on edge clouds, i.e., mixed deployment. To solve the above technical problems, the technical solutions adopted by this invention are as follows:

[0005] A method for optimizing the allocation of edge cloud resources based on a hybrid model includes the following steps:

[0006] S1, collects feature data of edge cloud devices and edge cloud services, and converts the feature data;

[0007] S2, input the converted feature data into the multi-layer perceptron for training, and optimize the training based on the gradient descent BP algorithm to obtain the bandwidth running volume prediction model;

[0008] S3 calculates the average bandwidth utilization, gross profit, and gross profit margin based on the bandwidth usage predicted by the bandwidth usage prediction model. The NSGA-II genetic algorithm and the DQN algorithm are used to solve the objective optimization problem of maximizing the average bandwidth utilization, gross profit, and gross profit margin to obtain the optimal resource allocation solution.

[0009] The step S1 comprises the following steps:

[0010] S1.1, collect feature data of edge cloud devices and edge cloud services, and preprocess the collected feature data;

[0011] The feature data includes task ID, bandwidth type, NAT type, operator type, billing rules, bandwidth, number of CPU threads, memory, bandwidth stress test satisfaction, packet loss stress test satisfaction and historical bandwidth, among which task ID, bandwidth type, NAT type, ISP operator and billing rules are discrete feature data, and the rest of the data are continuous feature data;

[0012] S1.2, using the Min-Max normalization method to transform the preprocessed continuous feature data into dimensionless feature sequences;

[0013] S1.3, use the one-hot encoding method to encode discrete feature data.

[0014] In step S2, the bandwidth usage prediction model is expressed as:

[0015]

[0016] Where g is the activation function, h is the output of the hidden layer, N is the total number of edge cloud devices, and W is the ik represents the weighted weight of the MLP model for the input element i of the kth layer, X ik represents the hidden vector, when k=1, X ik Represents the input feature data. When k≠1, X ik Represents the output data of the previous layer, and y represents the predicted bandwidth.

[0017] The step S3 comprises the following steps:

[0018] S3.1, collect the hardware requirements of the business for edge cloud devices and convert the hardware requirements into constraints using the rule set;

[0019] S3.2, traverse the constraints established in step S3.1 in sequence to determine whether each service can be deployed on the edge cloud device. If so, use the bandwidth usage predicted by the bandwidth usage prediction model to calculate the average bandwidth utilization, gross profit, and gross profit margin as the revenue of the edge cloud device. If not, set the revenue of the edge cloud device to -∞;

[0020] S3.3, generate a resource allocation plan based on the services deployed on the edge cloud devices, use the resource allocation plan to build a population, iteratively optimize the population based on the NSGA-II genetic algorithm and the DQN algorithm to solve the Pareto optimal solution, and take the individual that maximizes the comprehensive benefits of the edge cloud devices in the Pareto optimal solution as the optimal resource allocation plan.

[0021] In step S3.2, the calculation formula of the average bandwidth utilization is:

[0022]

[0023] Where U represents the average bandwidth utilization, represents the set of edge cloud devices, B i Indicates the predicted bandwidth of edge cloud device i, upbandwidth i represents the upload bandwidth of edge cloud device i;

[0024] The formula for calculating the gross profit is:

[0025]

[0026] In the formula, G represents gross profit, I i，j represents the unit bandwidth benefit of running type j service on edge cloud device i, C i represents the unit bandwidth cost of edge cloud device i;

[0027] The formula for calculating the gross profit margin is:

[0028]

[0029] In the formula, GR represents gross profit margin.

[0030] The step S3.3 includes the following steps:

[0031] S3.3.1, preset the total number of iterations GEN and initialize the number of iteration rounds gen = 1;

[0032] S3.3.2, using a random distribution generation algorithm to generate an initial population including several resource allocation schemes;

[0033] S3.3.3, add a cyclic activation algorithm to the genetic operations of selection, crossover, and mutation to recombine the primary population to generate a progeny population, the crossover operation being a binary crossover algorithm and the mutation operation being a polynomial mutation algorithm;

[0034] S3.3.4: Combine the primary population and the offspring population into population P. Input population P into the Deep Q-network. Generate hybrid services in the Deep Q-network based on a probability threshold. Train the Deep Q-network to maximize the proportion of hybrid services. Output the optimal hybrid resource allocation plan, and add the optimal hybrid resource allocation plan to population P.

[0035] S3.3.5, calculating the comprehensive benefit based on the benefits of the edge cloud devices in step S3.2, and determining the Pareto rank of each allocation scheme in population P based on the comprehensive benefit;

[0036] S3.3.6, obtain different Pareto rank sets by fast non-dominated sorting;

[0037] S3.3.7, calculate the crowding degree of each allocation scheme in population P, and execute the elite retention strategy to generate a new parent population based on the crowding degree and Pareto rank set;

[0038] S3.3.8, determine if gen < GEN. If so, execute gen = gen + 1, use the new parent population to update the initial population and execute step S3.3.3. Otherwise, output the individual in the parent population that maximizes the comprehensive benefit as the final optimal resource allocation plan.

[0039] Step S3.3.3 includes the following steps:

[0040] a. Initialize the action network Q with random network parameters ω a Parameters;

[0041] b, the action network Q a The parameters are copied to the target network Q t Parameters;

[0042] c. Initialize the experience replay buffer R and the DQN algorithm iteration number t = 1;

[0043] d. Input all allocation schemes in population P as initial state s1 into action network Q a , using the activation function to calculate the probability of the convolutional neural network output value, generating hybrid services based on the probability threshold and selecting action a based on the greedy strategy t ;

[0044] e, execute action a t , according to action a t Calculate reward r t , the state changes to s t+1 ;

[0045] f,will (s t , a t , r t , s t+1 ) Stored in the experience replay buffer R;

[0046] g, if the data in the experience playback buffer R is greater than the preset sampling number Sampling from the experience replay buffer R Data(s i , a i , r i , s i+1 );

[0047] h, for each data sampled in step g, use the target network Q t Calculate y i =r i +γmax a Q t (s, a), γ represents the new reward weight, y i Represents the median value of the objective function calculated for the corresponding sample;

[0048] i, minimize the loss function And update the action network Q according to the gradient descent method a ;

[0049] j, according to the action network Q a The network parameters of the target network Q are updated t , execute t=t+1, and determine whether the number of iterations reaches the maximum value. If not, return to step d. Otherwise, output the optimal colocation resource allocation solution and add it to the population P.

[0050] A hybrid model-based allocation system for optimizing edge cloud resources, comprising:

[0051] Feature data collection module: used to collect feature data of edge cloud devices and edge cloud services, and convert the feature data;

[0052] Bandwidth usage prediction model generation module: This module inputs the converted feature data output by the feature data collection module into a multi-layer perceptron for training, and adjusts the loss function based on the gradient descent BP algorithm to obtain a bandwidth usage prediction model.

[0053] Optimal resource allocation solution generation module: This module calculates the comprehensive benefits of edge cloud devices based on the bandwidth usage predicted by the bandwidth usage prediction model generation module, and uses the NSGA-II genetic algorithm and the DQN algorithm to solve the optimal resource allocation solution with the goal of maximizing the comprehensive benefits.

[0054] The comprehensive revenue of the edge cloud device is obtained by summing the average bandwidth utilization, gross profit and gross profit margin.

[0055] Beneficial effects of the present invention:

[0056] A combined model of the MLP neural network model, the NSGA-II genetic learning model, and the DQN deep reinforcement learning model was designed. This model effectively predicts the bandwidth usage of edge cloud devices in services using a multi-feature MLP model instead of directly using historical data, thus avoiding model lag. Average bandwidth utilization, gross profit, and gross profit margin are used as the dominance level calculation criteria for the NSGA-II. The NSGA-II genetic learning model is used to generate and optimally screen allocation plans. Furthermore, DQN is incorporated into the NSGA-II iteration process to generate colocation plans. By comprehensively weighing multiple benefit indicators, an optimal allocation plan that meets service deployment constraints is obtained, ensuring balanced and relatively optimal benefits for all aspects.

[0057] The innovative approach combines the iterations of NSGA-II with those of DQN. The NSGA-II allocation plan output serves as the initial state of the DQN model. DQN then generates a colocation solution based on the existing allocation plan. The DQN results are then embedded in the population generation and screening phase of the NSGA-II model, participating in the optimization iteration of the entire allocation solution. The two models iterate together to ultimately output the optimal resource allocation solution for both non-colocation and colocation, resolving the colocation issue and optimizing overall resource utilization. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0059] Figure 1 Schematic diagram of the NSGA-II algorithm flow.

[0060] Figure 2 is an example of feature data.

[0061] Figure 3 This is a diagram of one-hot encoding of the ISP type.

[0062] Figure 4 Schematic diagram of the structure of the MLP model.

[0063] Figure 5 Schematic diagram of the structure of three hidden layers.

[0064] Figure 6 A schematic diagram of the bandwidth usage of an edge cloud device.

[0065] Figure 7 Hardware deployment requirements for Kuaishou's dedicated line services.

[0066] Figure 8 This is a diagram showing the training of the bandwidth usage prediction model.

[0067] Figure 9 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0068] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without creative work are within the scope of protection of the present invention.

[0069] The Multilayer Perceptron (MLP) model, also known as a fully connected feedforward artificial neural network (ANN), consists of at least three layers of nodes: an input layer, a hidden layer, and an output layer. Except for the input node, each node is a neuron using a nonlinear activation function. The MLP is trained using a supervised learning technique called backpropagation. Its multiple layers and nonlinear activations distinguish it from a linear perceptron. It can distinguish data that is not linearly separable.

[0070] NSGA-II algorithm, namely the fast non-dominated multi-objective optimization algorithm with elite retention strategy, is a multi-objective optimization algorithm based on Pareto optimal solution. The basic idea of ​​NSGA-II algorithm is: Figure 1 As shown in the figure, 1. An initial population of size N (i.e., N allocation schemes) is randomly generated. After non-dominated sorting, the first generation offspring population is obtained through the three basic operations of genetic algorithm: selection, crossover, and mutation. 2. Starting from the second generation, the parent population and the offspring population are merged and fast non-dominated sorting is performed. At the same time, the crowding degree of individuals in each non-dominated layer is calculated, and suitable individuals are selected to form a new parent population based on the non-dominated relationship and the crowding degree of the individuals. 3. A new offspring population is generated through the basic operations of the genetic algorithm. This process is deduced and repeated until the conditions for the program to end are met.

[0071] The DQN algorithm's main algorithmic process combines neural networks with the Q-learning algorithm. Leveraging the powerful representational capabilities of neural networks, it uses high-dimensional input data as the state in reinforcement learning and as the input to the neural network model (agent). The neural network model then outputs the value (Q value) corresponding to each action, resulting in the action to be executed (Actor). The goal of reinforcement learning is to maximize rewards through learning.

[0072] Example 1: A method for optimizing the allocation of edge cloud resources based on a hybrid model, such as Figure 9 As shown, the following steps are included:

[0073] S1, collects feature data of edge cloud devices and edge cloud services, and converts the feature data;

[0074] S1.1, collect feature data of edge cloud devices and edge cloud services, and preprocess the collected feature data;

[0075] Since the hardware configuration is the key factor that determines the performance of the edge cloud device, and the real-time bandwidth running volume is the target value that needs to be predicted in step S2 of this application, the hardware configuration and real-time bandwidth running volume data of the edge cloud device are collected based on the real large-scale data set accumulated by the edge cloud service platform to make a more accurate prediction of the bandwidth running volume of the edge cloud. The characteristic data include task id, the province where the device is located, bandwidth type, NAT type, operator type, billing rules, the maximum available bandwidth of the edge cloud, number of cpu threads, memory, bandwidth stress test satisfaction, packet loss stress test satisfaction and historical bandwidth, among which task id, province, bandwidth type, NAT type, operator type and billing rules are discrete characteristic data, and other data are continuous characteristic data. Task id and billing rules are business data, the province where the device is located, bandwidth type, NAT type, operator type, the maximum available bandwidth of the device, number of cpu threads, memory, bandwidth stress test satisfaction and packet loss stress test satisfaction are all hardware data that characterize the edge cloud device, and historical bandwidth refers to the actual running bandwidth of the device in the past unit time (1 day or 1 week, depending on the specific time length to be predicted by the model). As Figure 2 The following is a data example. All feature data can be queried and downloaded from the Alibaba Cloud database using SQL commands:

[0076] SELECT machine_id, bw_upload_95, bw_real_upload2_95, dt

[0077] FROM t_ads_machine_95net_info

[0078] WHERE dt in {};

[0079] For continuous data, the following steps can be used to preprocess the data: a) Clean abnormal data, such as formatting or logical errors, and fill the continuous data using mean substitution. b) Use the boxplot method to remove outliers in the data. The boxplot method determines whether the feature value in a record is abnormal based on the upper and lower quartiles of the data. This is a prior art and will not be described in detail in this embodiment.

[0080] S1.2, using the Min-Max normalization method to transform the preprocessed continuous feature data into dimensionless feature sequences;

[0081] Since the data in each field spans a large range and has data unit restrictions, the data needs to be standardized, that is, the data needs to be scaled proportionally so that it falls into a very small specific interval and converted into a dimensionless pure numerical value, so that indicators of different units can be weighted.

[0082] S1.3, use one-hot encoding method to encode discrete feature data;

[0083] One-hot encoding uses an N-bit state register to encode N states, each state has its own independent register bit, and at any time, only one of them is valid. For example: Figure 3 As shown in the figure, the ISP (Internet Service Provider) of all edge cloud devices, that is, the operator type, mainly includes China Telecom, China Mobile and China Unicom. The one-hot encoding of the ISP type of any device is as follows: First, the ISP type feature is split into isp_China Telecom, isp_China Mobile and isp_China Unicom. If the ISP type of the device is China Telecom, the corresponding code is 100.

[0084] S2, such as Figure 8 As shown in the figure, the converted feature data is input into the Multilayer Perceptron (MLP) model for training, and the BP algorithm based on gradient descent is used for optimization training to obtain the bandwidth running amount prediction model;

[0085] Since the nonlinear model based on neural network has stronger expressive power, the effect of the model will be more significantly improved when the amount of data and the number of features increase. Especially when it is uncertain which feature has a greater impact on the service bandwidth of the server, the model based on neural network can reduce the manual feature screening process due to its ability to fit any function, while making the prediction accuracy higher. Figure 4As shown, the MLP model includes an input layer, several hidden layers and an output layer, as shown in Figure 5 As shown, the network structure of the hidden layer can be set as a parallel structure, a diamond structure, or a pyramid structure. Specifically, the parallel structure means that the dimensions of each hidden layer in the network structure are the same, the diamond structure means that the dimensions of the hidden layers on both sides of the network structure with the middle hidden layer as the symmetry axis are the same and are both smaller than the dimension of the middle hidden layer, and the pyramid structure means that the dimension of the hidden layer close to the input layer in the hidden layer network structure is the largest, and the dimension of the hidden layer farther away from the input layer is smaller. Due to the large amount of data and complex features of edge cloud devices, the initial input needs to process more information, but the final output only has one bandwidth running amount, which requires the information to be gradually converged and simplified. Therefore, the hidden layer of the bandwidth running amount prediction model is designed as a pyramid structure, and the effect of different network layers on the effect is experimented. The experiment found that the network effect of 3-6 layers is better, and the deeper network layers do not have additional accuracy improvement, and the online response time will be slower. In this application, the input layer dimension is set to 128*1024, and the three hidden layers are set to 1024*512, 512*256, and 256*1.

[0086] The bandwidth usage prediction model can be used to predict the bandwidth usage of edge cloud devices per unit time, i.e., upload traffic. The bandwidth usage prediction model is expressed as follows:

[0087]

[0088] Where g is the activation function, h is the output of the hidden layer, N is the total number of edge cloud devices, and W is the ik represents the weighted weight of the MLP model for the input element of the k-th layer edge cloud device i, X ik represents the hidden vector, and k∈(1, 2, 3) represents the number of hidden layers of the current bandwidth running volume prediction model. When k=1, X ik Represents the input feature data. When k≠1, X ik represents the output data of the previous layer, and y represents the predicted bandwidth. Since bandwidth can only be a non-negative real number, this application uses the ReLU function as the activation function. The activation function gives the MLP model the ability to fit nonlinear relationships. The characteristic of the ReLU function is that when the input signal is less than 0, the output is 0; when the input signal is greater than 0, the output is equal to the input, which meets the bandwidth prediction goal.

[0089] The mean absolute error (MAE) is used as the loss function during optimization. The formula of the loss function is:

[0090]

[0091] In the formula, loss represents the loss function, Indicates the predicted bandwidth running amount, B i represents the actual bandwidth usage per unit time for edge cloud device i. Using a gradient descent-based BP algorithm, we can minimize the loss by adjusting model parameters, enabling the model to more accurately predict the bandwidth usage after deploying the corresponding service based on feature data.

[0092] The actual bandwidth running amount B i The calculation formula is:

[0093]

[0094] Where, represents the edge cloud device i Bandwidth sampling data, Z represents the total number of bandwidth samples per unit time. Figure 6 As shown, the bandwidth fluctuation of an edge cloud device in one day, B i In order to sort the bandwidth usage of each sampling point in a day from large to small, the value ranked 95% is selected as the bandwidth usage value of that day.

[0095] Traditional machine learning methods, such as those based on boosting, have limited complexity and expressiveness in predictive models. When the number of features and data increases beyond a certain point, the model's accuracy also reaches a ceiling. MLP, on the other hand, is a nonlinear deep learning model whose complexity and expressiveness increase with the number of layers and width. It is therefore more suitable for model building for long-term prediction tasks. Its higher expressiveness also leads to higher prediction accuracy. Traditional machine learning methods use least squares or residual learning for parameter training. New data requires complete model retraining, resulting in low iteration efficiency. MLP, as a deep learning model, supports incremental training, requiring only the trained model to fit new data, resulting in more efficient model iteration. Unlike utilization prediction, bandwidth usage prediction for services on edge cloud devices does not rely on the machine's bandwidth as the denominator in the problem fraction. Therefore, it is affected by different features and is more complex. MLP models, through cross-parameter calculations in neural networks, can better capture the influence of different features without overly relying on server bandwidth, unlike boosting models, making them more suitable for this task.

[0096] S3 calculates the average bandwidth utilization, gross profit, and gross profit margin based on the bandwidth usage predicted by the bandwidth usage prediction model. The NSGA-II genetic algorithm and the DQN algorithm are used to solve the multi-objective optimization problem of maximizing the average bandwidth utilization, gross profit, and gross profit margin, and the optimal resource allocation solution is obtained.

[0097] S3.1, collect the hardware requirements of the business for edge cloud devices and convert the hardware requirements into constraints based on the rule set;

[0098] Different businesses have certain hardware requirements for deployable edge cloud devices, including operator type, NAT type, number of CPU threads, memory size, disk type, and disk size, etc. Figure 7 As shown in the figure, Kuaishou's dedicated line service has clear constraints on the edge cloud device's operator, public network, network type, reported single-line bandwidth, network quality, CPU, memory size, disk type, and disk size. These constraints are formalized using a rule set to facilitate their use as hard constraints in the resource allocation process. For example:

[0099] {'Kuaishou': {'upbandwidth': [900, 1501, 2500, 3500, 5500], 'cpu_num': [16, 24, 32, 32, 48], ' memory_size′: [20, 28, 40, 40, 60], ′disk_upband′: [0.7, 0.7, 0.6, 0.6, 0.5], ′avg_iops _line′: [3000, 3000, 3000, 5000, 10000]}, 'B site broadcast': {'upbandwidth': [2000, 5000], 'cpu_num': [32, 32], 'memory_size': [64, 64], 'ssd': [3.5, 8], 'disk_upband': [1.5, 1.5]}}.

[0100] S3.2, traverse the constraints established in step S3.1 in sequence to determine whether each service can be deployed on the edge cloud device. If so, use the bandwidth usage predicted by the bandwidth usage prediction model to calculate the average bandwidth utilization, gross profit, and gross profit margin as the revenue of the edge cloud device. If not, set the revenue of the edge cloud device to -∞;

[0101] Based on the hardware requirements of the business for edge cloud devices and the corresponding hardware data of the edge cloud devices, it is possible to judge one by one whether each business can run on each edge cloud device, which can improve allocation efficiency.

[0102] The calculation formula of the average bandwidth utilization is:

[0103]

[0104] Where U represents the average bandwidth utilization, represents the set of edge cloud devices, B i Indicates the predicted bandwidth of edge cloud device i, upbandwidth iIndicates the upload bandwidth of edge cloud device i.

[0105] The formula for calculating the gross profit is:

[0106]

[0107] In the formula, G represents gross profit, I i，j represents the unit bandwidth benefit of running type j service on edge cloud device i, C i represents the unit bandwidth cost of edge cloud device i.

[0108] The formula for calculating the gross profit margin is:

[0109]

[0110] Where GR represents gross profit margin.

[0111] S3.3 generates a resource allocation plan based on the services deployed on the edge cloud devices, uses the resource allocation plan to build a population, and iteratively optimizes the population using the NSGA-II genetic algorithm and the DQN algorithm to solve the Pareto optimal solution. The individual that maximizes the comprehensive benefits of the edge cloud devices in the Pareto optimal solution is used as the optimal resource allocation plan, including the following steps:

[0112] S3.3.1, preset the total number of iterations GEN and initialize the number of iteration rounds gen = 1;

[0113] S3.3.2, using a random distribution generation algorithm to generate an initial population including several resource allocation schemes;

[0114] A random distribution generation algorithm is used to generate a specified number of uniformly distributed coding allocation schemes as the initial population. Specifically, a random integer generation algorithm is used to generate L numbers that satisfy a uniform distribution in the range of [1, N*K] as the position number of 1 in the allocation scheme code, and the remaining unselected numbers are set to 0. This is repeated θ times to generate θ allocation schemes as the initial population, where K represents the total number of services and N represents the total number of edge cloud devices.

[0115] According to the services deployed on the edge cloud devices, a K-bit code is calculated for each edge cloud device. The K-bit code of each edge cloud device is used as the resource allocation plan for the edge cloud device. The service bit to be deployed is set to 1, and the remaining service bits are set to 0. Then, the codes of all edge cloud devices are spliced ​​together to obtain a complete code allocation plan, as shown in the following table. a , X b represents two different resource allocation schemes, mac1, mac2, and mac3 represent different edge cloud devices, Kuaishou, ByteDance, and Baidu Netdisk represent different businesses, and K=3.

[0116]

[0117] S3.3.3. Add a cyclic activation function (CAF) to the genetic operations of selection, crossover, and mutation to recombine the primary population to achieve intergenerational inheritance and generate a progeny population. The crossover operation is a binary crossover algorithm, and the mutation operation is a polynomial mutation algorithm.

[0118] Binary crossover algorithm: First, randomly select two individuals X from the population obtained by random selection operation. a and X b , then generate a distributed random number 0≤μa≤1, and calculate the parameter β through the multinomial probability distribution q :

[0119]

[0120] Among them, η is the distribution index of the crossover operation, η is a non-negative number, and the larger the value, the closer the offspring is to the parent.

[0121] Using parameter β q Calculate the offspring X′ after crossover a and offspring X′ b , the expression is:

[0122] X′ a =0.5[(X a +X b )-β q |X b -X a |];

[0123] X′ b =0.5[(X a +X b )+β q |X b -X a |];

[0124] Since each bit in the allocation scheme is an integer (representing whether each type of service is allocated), this application proposes to introduce a cyclic activation algorithm in the inheritance and mutation stages of the population, and use the Softmax function in the multi-classification problem to activate the scores in the code into a one-hot form (one bit is 1 and the rest are 0).

[0125] After adding the cyclic activation algorithm, the expressions of offspring and offspring generated by each crossover are:

[0126] X″ a =caf(X′ a );

[0127] X″ b =caf(X′ b );

[0128] In order to increase the diversity of allocation schemes, a polynomial mutation algorithm is introduced to generate new allocation schemes from the parent population. Its expression is:

[0129] X′ a =X a +Δ a ;

[0130] Where X′ a represents the new allocation scheme generated by the polynomial mutation algorithm, Where 0≤u a ≤1, η is the mutation distribution index. The larger η is, the closer the offspring is to the parent. The allocation scheme after adding the cyclic activation algorithm in the mutation stage is expressed as X″ a =caf(X a +Δ a ).

[0131] S3.3.4: Combine the initial population and the offspring population into population P. Input population P into a deep Q-network (DQN). Generate colocation services in the DQN based on a probability threshold. Train the DQN to maximize the proportion of colocation services. Output the optimal colocation resource allocation plan, and add the optimal colocation resource allocation plan to population P.

[0132] The DQN algorithm maintains two sets of Q networks (linear layer combinations), including the action network Q for training and generating hybrid service combinations. a The target network Q is used to calculate the sum of the proportions of different types of resources occupied by hybrid services on edge cloud devices. t , the action network Q a Convolutional Neural Networks (CNNs) implement actions through matrix convolution. The population P is input into a deep Q-network, where hybrid services are generated based on probability thresholds. The deep Q-network is trained to maximize the proportion of hybrid services and output an optimal hybrid resource allocation solution, including the following steps:

[0133] a. Initialize the action network Q with random network parameters ω a Parameters;

[0134] b, the action network Q a The parameters are copied to the target network Q t Parameters;

[0135] c. Initialize the experience replay buffer R and the DQN algorithm iteration number t = 1;

[0136] d. Input all allocation schemes in population P as initial state s1 into action network Q a , using the activation function to calculate the probability of the convolutional neural network output value, generating hybrid services based on the probability threshold and selecting action a based on ∈-greedy t ;

[0137] The action a t The expression is:

[0138]

[0139] Where caf(·) represents the Softmax activation function for calculating probability, Represents the output value of the last layer of the convolutional neural network.

[0140] The output of the convolutional neural network is expressed as:

[0141]

[0142] Where, represents the output of the lth layer of the convolutional neural network at the tth round, represents the output of the l-1th layer of the convolutional neural network in the tth round, J represents the set of convolution kernels, It represents the jth convolution kernel of the lth layer of the convolutional neural network in the tth round, that is, the parameter matrix. Represents the bias of the jth convolution kernel of the lth network layer. When l = 1, s t Indicates the state at round t.

[0143] In this embodiment, the corresponding values ​​of the coding allocation schemes with probability values ​​greater than 0.5 are set to 1, and the others are set to 0. The action is defined as adding deployment services to a certain number of independent servers to change to mixed operation, such as changing from 010 to 011.

[0144] e, execute action a t , according to action a t Calculate reward r t , the state changes to s t+1 ;

[0145] The reward r, i.e., the calculation formula for maximizing the proportion of colocation services, is:

[0146] r=μ cpu +μ upbandwidth +μ disk ;

[0147] Where μ cpu Indicates the CPU utilization ratio, μ upbandwidth Indicates the bandwidth utilization ratio, μ disk Indicates the hard disk utilization ratio.

[0148] f,will (s t , a t , r t , s t+1 ) Stored in the experience replay buffer R;

[0149] g, if the data in the experience playback buffer R is greater than the preset sampling number Sampling from the experience replay buffer R Data(s i , a i , r i , s i+1 ), where s i represents the i-th state in the experience replay buffer R;

[0150] h, for each data sampled in step g, use the target network to calculate y i =r i +γmax a Q t (s, a), γ represents the new reward weight, y i Represents the intermediate value of the objective function calculated for the corresponding sample, which is used to calculate the loss function and update the network;

[0151] In this application, γ=0.5.

[0152] i, minimize the loss function And update the action network Q according to the gradient descent method a ;

[0153] Among them, Q a (s i , a i ) represents the action network Q a The benefit generated by the selected action, i.e., the reward, is due to the action network Q a Updated, the reward here is not equal to the sample data r i .

[0154] j, according to the action network Q a The network parameters of the target network Q are updated t , execute t=t+1, and determine whether the number of iterations reaches the maximum value. If not, return to step d. Otherwise, output the optimal colocation resource allocation solution and add it to the population P.

[0155] The DQN model can be embedded in the NSGA-II model's population generation and screening process, participating in the iterative optimization of the allocation plan, thereby providing a co-location solution for the service mix. The DQN model is an action-feedback based deep learning model. Compared to traditional reinforcement learning models, it can understand the current allocation plan generation state and take further actions. Therefore, it can be jointly optimized with the NSGA-II algorithm. Through neural network training, a highly feasible cross-location co-location solution for services can be obtained, solving the high computational complexity problem caused by NSGA-II's limited random inheritance from parent generations.

[0156] S3.3.5, calculating the comprehensive benefit based on the benefits of the edge cloud devices in step S3.2, and determining the Pareto rank of each allocation scheme in population P based on the comprehensive benefit;

[0157] If no other allocation solution can dominate a particular allocation solution, then the allocation solution is called a non-dominated solution. In a set of solutions, the Pareto rank of the optimal allocation solution (non-dominated solution) is defined as 1. If non-dominated solutions are removed from the solution set, the Pareto rank of the remaining solutions is defined as 2. This process is repeated to calculate the Pareto rank of all solutions in the allocation solution set.

[0158] Specifically, if Z l >Z m and G, GR, so that F(X l )<F(X m ) holds true, then X l DominateX m Among them, X l and X m All are allocation schemes, Z l =U l +G l +GR l , Z m =U m +G m +GR m , Z l and Z m are the comprehensive benefits of edge cloud devices under the corresponding allocation scheme, U l For allocation plan X l Average bandwidth utilization, G l For allocation plan X l Gross profit, GR l For allocation plan X l Gross profit margin, U m For allocation plan X m Average bandwidth utilization, G m For allocation plan X mGross profit, GR m For allocation plan X m gross profit margin.

[0159] S3.3.6, obtain different Pareto rank sets by fast non-dominated sorting;

[0160] In each iteration of the NSGA-II algorithm, An allocation plan that meets the constraints. The allocation schemes are called a population, and each allocation scheme is an individual. Specifically, step S3.3.5 includes the following steps:

[0161] a. Calculate the two parameters of the number of dominated individuals and the set of solutions dominated by each individual in the population P, and initialize the sets F1 and F2 to empty sets;

[0162] b. Put the individuals with zero dominated number in population P into set F1;

[0163] c, traverse each individual in the set F1 in turn, if individual l∈S τ , then let n l =n l -1, otherwise n is unchanged l , S τ represents the set of solutions dominated by individual τ, n l represents the number of dominated individuals of individual l;

[0164] d. Put the individuals with zero dominated number in set F1 into set F2 to obtain individuals with Pareto level 2;

[0165] d. Repeat this algorithm for set F2 according to the method of steps c and d, and so on until the population levels are all divided into various sets.

[0166] S3.3.7, calculate the crowding degree of each allocation scheme, and execute the elite retention strategy to generate the parent population based on the crowding degree and Pareto rank set;

[0167] Allocation Plan X a The congestion degree is defined as X a The minimum distance from other allocation schemes in the benefit space U, G, GR is introduced to select the individuals with the strongest differences to retain and pass on to the offspring.

[0168] The calculation formula for congestion is:

[0169]

[0170] Where n d represents the congestion degree, fm (i+1) indicates that individual i has m The objective function value of the individual ranked last in , represents the maximum value of the individual objective function, Represents the minimum value of the individual objective function.

[0171] The purpose of the elite retention strategy is to select the most suitable individuals for genetic mutation as parents for algorithm iteration based on the Pareto level and crowding degree of the individual's comprehensive benefits, including the following steps:

[0172] a, the parent population C i and the offspring population D i Synthetic population R i ;

[0173] b. From the population R according to the following rules i Generate a new parent population S i+1 ;

[0174] (1) Place the entire population into the parent population S according to the order of Pareto level from low to high. i+1 , until all individuals in a certain layer cannot be put into the parent population S i+1 ;

[0175] (2) Sort the individuals in this layer from large to small according to the degree of crowding and put them into the parent population S in sequence. i+1 Until the parent population S i+1 The number of individuals in reaches the specified value δ.

[0176] S3.3.8, determine if gen < GEN. If so, execute gen = gen + 1, use the parent population to update the initial population and execute step S3.3.3. Otherwise, output the individual in the parent population that maximizes the comprehensive benefit as the optimal resource allocation plan.

[0177] Each iteration of the above steps is considered an iteration. When the combined payoff of an individual reaches the set value or the current iteration limit is reached, NSGA-II stops and selects the best individual from the current generational population as the final allocation solution. Based on an elitist strategy, the NSGA-II algorithm merges the current allocation solution with the subpopulations generated through selection, crossover, and mutation. These two subpopulations compete to produce the next set of allocation solutions, ensuring that individuals with superior characteristics remain in the population, thereby increasing the diversity and optimality of allocation solutions. Furthermore, the NSGA-II algorithm reduces computational complexity and improves the efficiency of allocation solution generation through non-dominated sorting.

[0178] However, the NSGA-II algorithm suffers from two drawbacks: Each iteration has high computational complexity. In each NSGA-II iteration, the comprehensive benefit, Pareto level, and Pareto order of each individual (allocation solution) must be calculated. Therefore, the more individuals involved in each iteration, the more iterations required, and the longer the run time. NSGA-II's allocation solution iterations are based on the parent population and can only proceed in the direction of optimizing the benefit, resulting in low flexibility. These two drawbacks prevent NSGA-II from generating allocation solutions for multiple hybrid services. Increasing the number of hybrid services requires a significant increase in the number of individuals generated in each iteration to accommodate a sufficient number of hybrid solutions. Furthermore, since hybrid solutions cannot be optimized independently and must be cross-generated with standalone solutions to generate a new population, the effectiveness of the hybrid solutions cannot be gradually improved with each iteration, requiring more iterations for algorithm convergence, significantly increasing the overall runtime of NSGA-II. To address the issue of generating hybrid resource allocation solutions in NSGA-II, this application introduces the DQN algorithm. This algorithm uses DQN to add a hybrid resource allocation solution to the population generated in each iteration of NSGA-II, and then optimizes the iterations of this hybrid resource allocation solution separately. By using the aforementioned MLP, NSGA-II, and DQN algorithms in edge resource allocation scenarios, we can provide a reasonable, multi-objective optimized allocation solution for edge cloud resources that supports hybrid deployment. This solution can also address issues that conventional algorithms cannot address, such as fixed business requirements and single-objective optimization.

[0179] Embodiment 2: A hybrid model-based system for optimizing edge cloud resource allocation, comprising:

[0180] Feature data collection module: used to collect feature data of edge cloud devices and edge cloud services, and convert the feature data;

[0181] Bandwidth usage prediction model generation module: This module inputs the converted feature data output by the feature data collection module into a multi-layer perceptron for training, and adjusts the loss function based on the gradient descent BP algorithm to obtain a bandwidth usage prediction model.

[0182] Optimal resource allocation solution generation module: This module calculates the comprehensive benefits of edge cloud devices based on the bandwidth usage predicted by the bandwidth usage prediction model generation module, and uses the NSGA-II genetic algorithm and the DQN algorithm to solve the optimal resource allocation solution with the goal of maximizing the comprehensive benefits.

[0183] The comprehensive revenue of the edge cloud device is obtained by summing the average bandwidth utilization, gross profit and gross profit margin.

[0184] In this embodiment, the bandwidth usage prediction model and the method for generating the optimal resource allocation solution and the method for converting the feature data are as described in Example 1 and will not be repeated in this embodiment.

[0185] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for optimizing the allocation of edge cloud resources based on a hybrid model, characterized in that: The steps include: S1, collects feature data of edge cloud devices and edge cloud services, and converts the feature data; S2, input the converted feature data into the multi-layer perceptron for training, and optimize the training based on the gradient descent BP algorithm to obtain the bandwidth running volume prediction model; S3: Calculate the average bandwidth utilization, gross profit, and gross profit margin based on the bandwidth usage predicted by the bandwidth usage prediction model. Use the NSGA-II genetic algorithm and the DQN algorithm to solve the objective optimization problem of maximizing the average bandwidth utilization, gross profit, and gross profit margin to obtain the optimal resource allocation solution. The feature data includes task ID, bandwidth type, NAT type, operator type, billing rules, bandwidth, number of CPU threads, memory, bandwidth stress test satisfaction, packet loss stress test satisfaction and historical bandwidth; In step S2, the bandwidth usage prediction model is expressed as: Where g is the activation function, h is the output of the hidden layer, N is the total number of edge cloud devices, and W is the ik represents the weighted weight of the MLP model for the input element i of the kth layer, X ik represents the hidden vector, when k=1, X ik Represents the input feature data. When k≠1, X ik Represents the output data of the previous layer, and y represents the predicted bandwidth; The step S3 comprises the following steps: S3.1, collect the hardware requirements of the business for edge cloud devices and convert the hardware requirements into constraints using the rule set; S3.2, traverse the constraints established in step S3.1 in sequence to determine whether each service can be deployed on the edge cloud device. If so, use the bandwidth usage predicted by the bandwidth usage prediction model to calculate the average bandwidth utilization, gross profit, and gross profit margin as the revenue of the edge cloud device. If not, set the revenue of the edge cloud device to -∞; S3.3, generate a resource allocation plan based on the services deployed on the edge cloud devices, use the resource allocation plan to build a population, iteratively optimize the population based on the NSGA-II genetic algorithm and the DQN algorithm to solve the Pareto optimal solution, and take the individual that maximizes the comprehensive benefits of the edge cloud devices in the Pareto optimal solution as the optimal resource allocation plan.

2. The method for optimizing edge cloud resource allocation based on a hybrid model according to claim 1, characterized in that: The step S1 includes the following steps: S1.1, collect feature data of edge cloud devices and edge cloud services, and preprocess the collected feature data; The task ID, bandwidth type, NAT type, ISP operator and billing rules are discrete feature data, and the remaining data are continuous feature data; S1.2, using the Min-Max normalization method to transform the preprocessed continuous feature data into dimensionless feature sequences; S1.3, use the one-hot encoding method to encode discrete feature data.

3. The method for optimizing edge cloud resource allocation based on a hybrid model according to claim 1, characterized in that: In step S3.2, the calculation formula of the average bandwidth utilization is: Where U represents the average bandwidth utilization, represents the set of edge cloud devices, B i Indicates the predicted bandwidth of edge cloud device i, upbandwidth i represents the upload bandwidth of edge cloud device i; The formula for calculating the gross profit is: In the formula, G represents gross profit, I i,4 represents the unit bandwidth benefit of running type j service on edge cloud device i, C i represents the unit bandwidth cost of edge cloud device i; The formula for calculating the gross profit margin is: Where GR represents gross profit margin.

4. The method for optimizing edge cloud resource allocation based on a hybrid model according to claim 1, characterized in that: The step S3.3 includes the following steps: S3.3.1, preset the total number of iterations GEN and initialize the number of iteration rounds gen = 1; S3.3.2, using a random distribution generation algorithm to generate an initial population including several resource allocation schemes; S3.3.3, add a cyclic activation algorithm to the genetic operations of selection, crossover, and mutation to recombine the primary population to generate a progeny population, the crossover operation being a binary crossover algorithm and the mutation operation being a polynomial mutation algorithm; S3.3.

4. Combine the initial population and the offspring population to form population P. Input population P into the deep Q-network, generate co-located services in the deep Q-network based on a probability threshold, train the deep Q-network with the goal of maximizing the proportion of co-located services, and output the optimal co-located resource allocation scheme, then add the optimal co-located resource allocation scheme to population P; S3.3.

5. Calculate the comprehensive revenue based on the revenue of the edge cloud devices in step S3.2, and determine the Pareto rank of each allocation scheme in population P according to the comprehensive revenue; S3.3.

6. Obtain different Pareto rank sets through fast non-dominated sorting; S3.3.

7. Calculate the crowding degree of each allocation scheme in population P, and execute the elitist retention strategy to generate a new parent population according to the crowding degree and the Pareto rank sets; S3.3.

8. Judge whether gen < GEN. If so, execute gen = gen + 1, update the initial population with the new parent population and execute step S3.3.

3. Otherwise, output the individual with the maximum comprehensive revenue in the parent population as the final optimal resource allocation scheme.

5. The method for optimizing edge cloud resource allocation based on a hybrid model according to claim 4, characterized in that: The said step S3.3.3 includes the following steps: a. Initialize the action network Q with random network parameters ω a Parameters; b, the action network Q a The parameters are copied to the target network Q ％ Parameters; c. Initialize the experience replay buffer R, and initialize the DQN algorithm iteration round t = 1; d. Input all allocation schemes in population P as initial state s1 into action network Q a , using the activation function to calculate the probability of the convolutional neural network output value, generating hybrid services based on the probability threshold and selecting action a based on the greedy strategy ％ ; e, execute action a ％ , according to action a ％ Calculate reward r ％ , the state changes to s ％G1 ; f,will (s ％ ,a ％ ,r ％ ,s ％G1 ) Stored in the experience replay buffer R; g, if the data in the experience playback buffer R is greater than the preset sampling number Sampling from the experience replay buffer R Data(s i ,a i ,r i ,s iG1 ); h, for each data sampled in step g, use the target network Q ％ Calculate y i =r i +γmax a Q ％ (s,a), γ represents the new reward weight, y i Represents the median value of the objective function calculated for the corresponding sample; i, minimize the loss function And update the action network Q according to the gradient descent method a ; j, according to the action network Q a The network parameters of the target network Q are updated ％ , execute t=t+1, and determine whether the number of iterations reaches the maximum value. If not, return to step d. Otherwise, output the optimal co-location resource allocation solution and add it to the population P.

6. A hybrid model-based optimization edge cloud resource allocation system, characterized in that: including: Feature data collection module: used to collect the feature data of edge cloud devices and edge cloud services, and convert the feature data; Bandwidth traffic prediction model generation module: input the converted feature data output by the feature data collection module into a multi-layer perceptron for training, and adjust the loss function based on the gradient descent BP algorithm to obtain the bandwidth traffic prediction model; Optimal resource allocation scheme generation module: used to calculate the comprehensive revenue of edge cloud devices according to the bandwidth traffic predicted by the bandwidth traffic prediction model generation module, and use the NSGA-II genetic algorithm and the DQN algorithm to solve the optimal resource allocation scheme with the goal of maximizing the comprehensive revenue; The comprehensive revenue of the said edge cloud device is obtained by summing the average bandwidth utilization rate, gross profit and gross profit margin; The said feature data includes task id, bandwidth type, NAT type, operator type, billing rule, bandwidth, cpu thread count, memory, bandwidth stress test satisfaction, packet loss stress test satisfaction and historical bandwidth; The expression of the said bandwidth traffic prediction model is: Where g is the activation function, h is the output of the hidden layer, N is the total number of edge cloud devices, and W is the ik represents the weighted weight of the MLP model for the input element i of the kth layer, X ik represents the hidden vector, when k=1, X ik Represents the input feature data. When k≠1, X ik Represents the output data of the previous layer, and y represents the predicted bandwidth; The solution method of the said optimal resource allocation scheme includes the following steps: S3.

1. Collect the hardware requirements of services for edge cloud devices, and convert the hardware requirements into constraint conditions using a rule set; S3.

2. Traverse the constraint conditions established in step S3.1 in turn, judge whether each service can be deployed on the edge cloud device. If so, calculate the average bandwidth utilization rate, gross profit and gross profit margin as the revenue of the edge cloud device respectively using the bandwidth traffic predicted by the bandwidth traffic prediction model. If not, set the revenue of the edge cloud device to -∞; S3.3, generate a resource allocation plan based on the services deployed on the edge cloud devices, use the resource allocation plan to build a population, iteratively optimize the population based on the NSGA-II genetic algorithm and the DQN algorithm to solve the Pareto optimal solution, and take the individual that maximizes the comprehensive benefits of the edge cloud devices in the Pareto optimal solution as the optimal resource allocation plan.

Images