Content delivery network resource allocation method and device, equipment and medium

Abstract

The embodiment of the invention provides a content distribution network resource allocation method and device, equipment and a medium. The method comprises the following steps: acquiring network demand data and network resource data of a user network node; carrying out feature extraction on the network demand data and the network resource data to obtain network data feature values; then, a resource allocation scheme utility function is determined, and a first resource allocation scheme is generated through a preset deep reinforcement learning algorithm model under the condition that the resource allocation scheme utility function is used as a reward function; and generating a second resource allocation scheme according to the first resource allocation scheme and a preset genetic algorithm model, and determining a mode of a target resource allocation scheme according to the first resource allocation scheme and the second resource allocation scheme. The optimal resource allocation scheme is determined by comprehensively considering the network demand data and the network resource data of the user, the deployment effect is improved while the actual network demand of the user is met, and the actual network experience of the user is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of content distribution network, and in particular to a content distribution network resource allocation method, device, equipment and medium. Background Art

[0002] Content Delivery Network (CDN) aims to optimize content distribution by deploying nodes in multiple geographical locations, reduce the load on source servers, and reduce the latency of user access to content, thereby providing a better user experience. In the Internet era, content delivery networks have become an important application technology, which has a profound impact on the network service quality and user experience of multiple users.

[0003] The content distribution network resource allocation scheme (also known as CDN deployment scheme) in the related technology usually relies on a centralized control system. When deploying, it generally focuses on considering the number, location, capacity, caching strategy and other parameter characteristics of edge servers to determine the corresponding CDN deployment scheme.

[0004] However, focusing only on the parameter characteristics of edge servers means that it is difficult to fully consider the actual network needs and resource conditions of multiple users in the content distribution network. This not only easily leads to inefficient deployment results, but also easily leads to resource waste and unbalanced services, seriously affecting the user's network experience. Summary of the invention

[0005] In view of the above problems, a content distribution network resource allocation method, device, equipment, and medium are proposed to overcome the above problems or at least partially solve the above problems, including:

[0006] A content distribution network resource allocation method, the method comprising:

[0007] Acquiring network demand data and network resource data of user network nodes in the content distribution network;

[0008] Extracting features from the network demand data and the network resource data to obtain network data feature values ​​of the user network node;

[0009] Determine a resource allocation scheme utility function according to the network data characteristic value, and generate a first resource allocation scheme through a preset deep reinforcement learning algorithm model under the condition that the resource allocation scheme utility function is used as a reward function;

[0010] Generate a second resource allocation scheme according to the first resource allocation scheme and a preset genetic algorithm model, and determine a target resource allocation scheme according to the first resource allocation scheme and the second resource allocation scheme;

[0011] According to the target resource allocation scheme, resources are allocated to the content distribution network.

[0012] Optionally, the extracting features of the network demand data and the network resource data to obtain the network data feature value of the user network node includes:

[0013] Generate an implicit vector eigenvalue according to a preset dimensionality reduction and reconstruction operator, the network demand data, and the network resource data;

[0014] The implicit vector eigenvalue is subjected to noise enhancement processing according to a preset adaptive noise operator, and the network data eigenvalue of the user network node is obtained according to the output data of the preset deep reinforcement learning-genetic algorithm operator and the noise enhancement processing.

[0015] Optionally, the formula of the preset dimensionality reduction reconstruction operator is:

[0016]

[0017] Among them, x i is the i-th eigenvalue, N represents the number of input features, x is the feature of the current input, δ is the coding factor used to prevent the input of zero vectors, and L(x) is the implicit vector eigenvalue;

[0018] The formula of the preset adaptive noise operator is:

[0019]

[0020] Among them, x i is the input feature, N is the input feature dimension, α is the adaptive noise adjustment factor, and θ is the output data of the noise enhancement process;

[0021] The formula of the preset deep reinforcement learning-genetic algorithm operator is:

[0022] x t+1 =ωRL(x t ,y t )

[0023] y t+1 =θGA(x t+1 ,y t )

[0024] Among them, x t is the decision variable for the tth iteration, y t is the objective function value of the tth iteration, ω, θ are both iteration factors, RL() is the deep reinforcement learning algorithm, and GA() is the genetic algorithm.

[0025] Optionally, before determining the resource allocation scheme utility function according to the network data characteristic value, the method further includes:

[0026] The network data characteristic values ​​are clustered to obtain a user network database.

[0027] Optionally, determining a resource allocation scheme utility function according to the network data characteristic value includes:

[0028] The resource allocation scheme utility function is determined according to the network data feature values ​​in the user network database.

[0029] Optionally, before obtaining a target resource allocation scheme according to the first resource allocation scheme and the second resource allocation scheme, the method further includes:

[0030] Acquire an original resource allocation scheme, and determine whether it is necessary to update a weight coefficient in a utility function of the resource allocation scheme according to the original resource allocation scheme and the second resource allocation scheme;

[0031] In the case where it is necessary to update the weight coefficient in the utility function of the resource allocation scheme, the weight coefficient is updated, and the step of determining the utility function of the resource allocation scheme based on the network data characteristic value is re-executed according to the updated weight coefficient, and under the condition of using the resource allocation scheme utility function as a reward function, generating a first resource allocation scheme through a preset deep reinforcement learning algorithm model.

[0032] Optionally, generating a second resource allocation scheme according to the first resource allocation scheme and a preset genetic algorithm model includes:

[0033] The first resource allocation scheme is subjected to one or more of cross processing, mutation processing and selection processing through the preset genetic algorithm model to generate the second resource allocation scheme.

[0034] A content distribution network resource allocation device, the device comprising:

[0035] A network data acquisition module, used to acquire network demand data and network resource data of user network nodes in the content distribution network;

[0036] A feature extraction module, used to extract features from the network demand data and network resource data to obtain network data feature values ​​of the user network node;

[0037] A first resource allocation scheme generating module, used to determine a resource allocation scheme utility function, and generate a first resource allocation scheme according to the resource allocation scheme utility function, the network data feature value and a preset deep reinforcement learning algorithm model;

[0038] a target resource allocation scheme determining module, configured to generate a second resource allocation scheme according to the first resource allocation scheme and a preset genetic algorithm model, and determine a target resource allocation scheme according to the first resource allocation scheme and the second resource allocation scheme;

[0039] The resource allocation module is used to allocate resources to the content distribution network according to the target resource allocation scheme.

[0040] An electronic device comprises a processor, a memory and a computer program stored in the memory and capable of running on the processor, wherein the computer program implements the above-mentioned content distribution network resource allocation method when executed by the processor.

[0041] A computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the content distribution network resource allocation method as described above is implemented.

[0042] A computer program product includes a computer program, and when the computer program is executed by a processor, the computer program implements the content distribution network resource allocation method as described above.

[0043] The embodiments of the present invention have the following advantages:

[0044] The present invention provides a method for allocating resources in a content distribution network, which obtains network demand data and network resource data of user network nodes in a content distribution network; extracts features of the network demand data and network resource data to obtain network data feature values ​​of user network nodes; then determines the resource allocation scheme utility function, and generates a first resource allocation scheme by using a preset deep reinforcement learning algorithm model under the condition that the resource allocation scheme utility function is used as a reward function; thereby generating a second resource allocation scheme according to the first resource allocation scheme and a preset genetic algorithm model, and then determining a target resource allocation scheme according to the first resource allocation scheme and the second resource allocation scheme; and allocates resources to the content distribution network according to the target resource allocation scheme. The method realizes determining the resource allocation scheme of the content distribution network by comprehensively considering the user network demand data and the network resource data, and comprehensively determining the optimal resource allocation scheme by using the resource allocation scheme utility function, the deep reinforcement learning algorithm and the genetic algorithm, thereby improving the deployment effect while meeting the actual network needs of the user, avoiding the waste of resources and the imbalance of services, and improving the actual network experience of the user. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solution of the present invention, the accompanying drawings required for use in the description of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0046] Figure 1 is a flowchart of a method for allocating resources in a content distribution network provided by some embodiments of the present invention;

[0047] Figure 2 is a data processing flow chart of network data characteristic values ​​provided by some embodiments of the present invention;

[0048] Figure 3 is an example diagram of a user network database provided by some embodiments of the present invention;

[0049] Figure 4 It is a diagram of an actual algorithm architecture for obtaining a target resource allocation solution provided by some embodiments of the present invention;

[0050] Figure 5 It is a schematic diagram of the overall execution flow of the content distribution network resource allocation method provided by some embodiments of the present invention;

[0051] Figure 6 It is a structural diagram of a content distribution network resource allocation device provided by some embodiments of the present invention. DETAILED DESCRIPTION

[0052] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0053] In the embodiments of the present application, words such as "first" and "second" are used to distinguish between identical or similar items with basically the same functions and effects. This is only for the purpose of clearly describing the technical solutions of the embodiments of the present application, and shall not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features.

[0054] In the embodiments of the present application, the terms "upper", "lower", etc. indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present application.

[0055] For the sake of convenience, the following explanations are given for the relevant terms involved in the embodiments of the present invention:

[0056] CDN (Content Delivery Network): The purpose of content distribution network is to publish the content of the website to the "edge" of the network closest to the user (edge ​​server) by adding a new network architecture to the existing Internet, so that users can obtain the required content nearby, solve the problem of Internet network congestion, and improve the response speed of users visiting the website.

[0057] RL (Reinforcement Learning, RL), reinforcement learning, refers to a class of problems that continuously learn from interactions (with the environment) and methods to solve such problems. Reinforcement learning problems can be described as an intelligent agent that continuously learns from interactions with the environment to achieve specific goals (such as obtaining the maximum reward value). Similar to deep learning, the key problem in reinforcement learning is also the contribution distribution problem. Each action cannot directly obtain supervision information, but needs to be obtained through the final supervision information (reward) of the entire model, and there is a certain delay.

[0058] Genetic Algorithm (GA): It was first proposed by John Holland in the United States in the 1970s. The algorithm is designed and proposed based on the laws of evolution of organisms in nature. It is a computational model of biological evolution that simulates the natural selection and genetic mechanism of Darwin's theory of biological evolution. It is a method of searching for the optimal solution by simulating the natural evolution process.

[0059] In the Internet era, content distribution network (CDN) has become an important application technology, which has a profound impact on the network service quality and user experience of multiple users. Traditional CDN deployment technology has obvious defects in the face of this challenge, and an innovative solution is urgently needed. First, traditional CDN deployment methods usually rely on centralized control systems, which have limited coverage and are difficult to fully consider the network needs and resource conditions of multiple users. This leads to inefficient deployment results, prone to resource waste and service imbalance. Due to the centralization of the control system, it is impossible to coordinate and optimize the CDN deployment among multiple users, which means that the effect and cost of CDN deployment may not be optimized, and the network service quality and user experience of multiple users are affected. Secondly, traditional methods have problems in the dynamic adjustment of CDN deployment. Due to the lack of real-time network data and user feedback, traditional technologies are often difficult to adapt to different network environments and user needs. This reduces the flexibility and reliability of CDN deployment because the parameters and strategies of CDN deployment cannot be adjusted in time. In addition, there is a delay in data processing and transmission in traditional methods, which cannot meet real-time requirements. CDN deployment requires instant response and updates to improve network service quality and user experience, while the delays of traditional methods may cause delays in deployment actions, further exacerbating the problems of CDN deployment.

[0060] In order to solve the above problems, the present invention is based on the core technical concept of determining the resource allocation scheme of the content distribution network by comprehensively considering the user network demand data and the network resource data, and improves the content distribution network resource allocation method in the related art. The present invention will be described in detail in conjunction with the accompanying drawings below:

[0061] Reference Figure 1 , shows a flow chart of a method for allocating resources in a content distribution network provided by some embodiments of the present invention, which may specifically include the following steps:

[0062] Step 101, obtaining network demand data and network resource data of user network nodes in the content distribution network;

[0063] In a specific implementation, it can be obtained by deploying a network demand and resource monitoring device at the user end. Specifically, the network demand and resource monitoring device can be deployed according to the network characteristics and user behaviors of multiple users. It is worth noting that the specific deployment location can be determined according to the dynamic changes of the network demand and resources of multiple users. That is, first of all, it is necessary to ensure that the monitoring coverage of the network demand and resources of multiple users, such as network traffic, network delay, network bandwidth, number of users, user satisfaction, etc., and the corresponding network resource allocation, such as the number, location, capacity, cache strategy, etc. of edge servers, can be accurately obtained and fed back by the network demand and resource monitoring device. Secondly, the computing power and storage space of the network demand and resource monitoring device should be considered to avoid data loss or error caused by the network demand and resource monitoring device. Finally, the data communication module should be configured, and data compression and encryption should be performed on it to ensure that the data generated by the network demand and resource monitoring device can be safely transmitted to the network management system. Finally, a network management system can be established to monitor the status and data reception of the network demand and resource monitoring device, and set up data analysis and processing modules to provide effective input data for subsequent processes.

[0064] Step 102, extracting features from the network demand data and network resource data to obtain network data feature values ​​of the user network node;

[0065] In a specific implementation, corresponding operators may be pre-set to perform feature extraction on network demand data and network resource data to obtain network data feature values ​​of user network nodes.

[0066] In some embodiments of the present invention, the extracting features of the network demand data and the network resource data to obtain the network data feature value of the user network node includes:

[0067] Generate an implicit vector eigenvalue according to a preset dimensionality reduction and reconstruction operator, the network demand data, and the network resource data;

[0068] The implicit vector eigenvalue is subjected to noise enhancement processing according to a preset adaptive noise operator, and the network data eigenvalue of the user network node is obtained according to the output data of the preset deep reinforcement learning-genetic algorithm operator and the noise enhancement processing.

[0069] In the specific implementation, Figure 2 The figure is a data processing flow chart of obtaining network data characteristic values ​​of user network nodes according to an embodiment of the present invention;

[0070] In this step, a cloud storage system can be established to receive and store the network demand of multiple users and the network data stream of the resource monitoring device so that the algorithm module can call it later. The original network data of each user in the cloud storage system can be processed to eliminate noise and outliers. For network traffic data, a sliding window mechanism can be used, that is, the network traffic data is divided into time series of fixed length, each time series is added with a position code, and then the numerical range of the time series is scaled to between [0,1] by normalization to reduce the scale difference of the data, and then it is input into the encoder to improve the data operation efficiency and enhance the robustness of the whole process. A new type of DAE operator (preset dimensionality reduction and reconstruction operator) can be used as the core of CDN deployment. The overall task of the operator is to compress the network data into a low-dimensional implicit vector and restore the vector to the original data during subsequent decoding. For the pre-training process, an operator with adaptive noise (preset adaptive noise operator) can be used. The operator dynamically adjusts the intensity and type of noise to add different noises to the input data to enhance the diversity and robustness of the data. At the same time, an RL-GA operator (preset deep reinforcement learning-genetic algorithm operator) can be used to optimize the input data (output data of noise enhancement processing) to obtain network data feature values ​​by combining the advantages of deep reinforcement learning (RL) and genetic algorithm (GA) to improve the effect of CDN deployment. In addition, for all missing values, in order to increase the response speed of the overall module, the interpolation method can be directly used, that is, the adjacent valid values ​​are used to fill the missing values. In addition, the corresponding location code is added to each network data source to facilitate the rapid positioning of subsequent CDN deployment.

[0071] In addition, in this step, it is necessary to maintain the comparability and consistency of network data. Therefore, the stored data can be converted to units first, that is, all byte data is converted to bits, and the sampling frequency and sampling bit number are unified. In addition, standard deviation scaling operations are performed on all data to ensure the consistency of data in scale. The characteristics of the overall network data can include network traffic, network delay, network bandwidth, number of users, user satisfaction, number of edge servers, location, capacity and caching strategy, etc. Among them, the five characteristics of network traffic, network delay, network bandwidth, number of users and user satisfaction can be directly used as eigenvalues ​​x1, x2, x3, x4, x5. The aforementioned DAE operator is the core of CDN deployment. The overall task of this operator is to compress network data into a low-dimensional implicit vector and restore the vector to the original data during subsequent decoding.

[0072] In some embodiments of the present invention, the formula of the preset dimensionality reduction reconstruction operator is:

[0073]

[0074] Among them, x i is the i-th eigenvalue, N represents the number of input features, x is the feature of the current input, δ is the coding factor used to prevent the input of zero vectors, and L(x) is the implicit vector eigenvalue;

[0075] The formula of the preset adaptive noise operator is:

[0076]

[0077] Among them, x i is the input feature, N is the input feature dimension, α is the adaptive noise adjustment factor, and θ is the output data of the noise enhancement process;

[0078] The formula of the preset deep reinforcement learning-genetic algorithm operator is:

[0079] x t+1 =ωRL(x t ,y t )

[0080] y t+1 =θGA(x t+1 ,y t )

[0081] Among them, x t is the decision variable for the tth iteration, y t is the objective function value of the tth iteration, ω, θ are both iteration factors, RL() is the deep reinforcement learning algorithm, and GA() is the genetic algorithm.

[0082] In practical applications, the formula of the new DAE operator (i.e., the preset dimensionality reduction and reconstruction operator) can be expressed as:

[0083]

[0084] Among them, x i is the i-th eigenvalue, N represents the number of input features, x is the feature of the current input, δ is the coding factor used to prevent the input of zero vectors, and L(x) is the eigenvalue of the latent vector. Through this formula, the input network data can be compressed into a low-dimensional latent vector, which is then restored to the original data during subsequent decoding.

[0085] Furthermore, the above operator with adaptive noise adds different noises to the input data by dynamically adjusting the intensity and type of noise to enhance the diversity and robustness of the data.

[0086] Among them, the preset adaptive noise operator formula is:

[0087]

[0088] Among them, x i is the input feature, N is the input feature dimension, α is the adaptive noise adjustment factor, and θ is the output data of the noise enhancement processing. Through this operator, the intensity and type of noise can be dynamically adjusted to add different noises to the input data to enhance the diversity and robustness of the data.

[0089] Furthermore, for the above-mentioned RL-GA operator, the operator can optimize the input data by combining the advantages of deep reinforcement learning (RL) and genetic algorithm (GA) to improve the effect of CDN deployment.

[0090] Among them, the formula of RL-GA operator (i.e., preset deep reinforcement learning-genetic algorithm operator) is:

[0091] x t+1 =ωRL(x t ,y t ) (3)

[0092] y t+1 =θGA(x t+1 ,y t ) (4)

[0093] Among them, x t is the decision variable for the tth iteration, y t is the objective function value of the tth iteration, ω and θ are iteration factors, RL() is a deep reinforcement learning algorithm, which continuously updates the decision variables through feedback from the learning environment to achieve the goal of maximizing rewards, and GA() is a genetic algorithm, which continuously updates the objective function value by simulating the process of natural selection and genetic variation to achieve the goal of minimizing costs. This operator utilizes the exploration ability of deep reinforcement learning and the convergence ability of genetic algorithms to achieve the synergy of data optimization and CDN deployment.

[0094] Step 103, determining a resource allocation scheme utility function according to the network data characteristic value, and generating a first resource allocation scheme by using a preset deep reinforcement learning algorithm model under the condition that the resource allocation scheme utility function is used as a reward function;

[0095] In a specific implementation, the utility function of the resource allocation scheme can be determined according to the characteristic value of the network data, and then the CDN deployment decision between multiple users can be optimized by improving the deep reinforcement learning (RL) and genetic algorithm (GA) network architecture. For example, the CDN deployment utility function (resource allocation scheme utility function) can be used to give the reward of the algorithm network to generate a first resource allocation scheme through a preset deep reinforcement learning algorithm model; in addition, the CDN deployment effect and cost of the corresponding resource allocation scheme on the user side can be further evaluated, and the weight coefficient in the utility function can be adaptively adjusted based on the specific evaluation results using the dynamic weight adjustment principle to help determine the optimal resource allocation scheme later.

[0096] In some embodiments of the present invention, before determining the resource allocation scheme utility function according to the network data characteristic value, the method further includes:

[0097] The network data characteristic values ​​are clustered to obtain a user network database.

[0098] In practical applications, such as Figure 3 The figure shows an example diagram of the user network database constructed by the present invention. In the foregoing content, the deployment of the network demand and resource monitoring device and the feature extraction of the network data are completed, so the construction of the network database of multiple users can be further completed to more flexibly coordinate and balance the CDN deployment effect and cost among multiple users. It should be noted that in addition to constructing the user network database for the network demand and resource monitoring devices of multiple users, classification can also be performed, that is, the network database stores the characteristic data of the network demand and resources of each user, including network traffic, network delay, network bandwidth, number of users, user satisfaction, number of edge servers, location, capacity, caching strategy, etc.

[0099] Furthermore, for CDN deployment characteristic data, a CDN deployment database can also be constructed based on the network requirements and resource monitoring devices of the above-mentioned multiple users, that is, two network databases can be constructed. The data in these two network databases are the data of the network requirements and resource monitoring devices of each user, but the first network database only contains the characteristic data of network requirements and resources, while the second network database only contains the CDN deployment area characteristic data of each user.

[0100] Specifically, a clustering algorithm may be used to construct a system-wide network database based on network requirements and distributed nodes of resource monitoring devices, as well as regional characteristics.

[0101] Among them, the clustering algorithm can be expressed as follows:

[0102]

[0103] Among them, xi represents the i-th eigenvalue, N represents the number of input features, C i is the center of the ith cluster, M represents the number of clusters, and the smaller the S value, the better the clustering effect.

[0104] In the above algorithm, we can first select features of network data, and select nine features such as network traffic, network delay, network bandwidth, number of users, user satisfaction, number of edge servers, location, capacity, and cache strategy as clustering inputs. Then, we can use formula (5) to cluster each data point and generate a vector with a length of the number of clusters, where each element indicates which cluster the data point belongs to. Then, we can minimize the distance within each cluster. Then, we can use formula (6) to update the center of each cluster and calculate the average value of the data points within each cluster as the new cluster center. And we can repeat the above steps until the cluster no longer changes or the maximum number of iterations is reached.

[0105] In general, the network database can be constructed by formula (6). For example, when a network demand and resource monitoring device detects data with high network traffic, high network latency, low network bandwidth, high number of users, low user satisfaction, few edge servers, far location, small capacity, and poor cache strategy, the data may belong to the cluster of network congestion. When a network demand and resource monitoring device detects data with low network traffic, low network latency, high network bandwidth, low number of users, high user satisfaction, multiple edge servers, near location, large capacity, and good cache strategy, the data may belong to the cluster of network smoothness.

[0106] In some embodiments of the present invention, determining the resource allocation scheme utility function according to the network data characteristic value includes:

[0107] The resource allocation scheme utility function is determined according to the network data feature values ​​in the user network database.

[0108] In the specific implementation, Figure 4 As shown is the actual algorithm architecture diagram for obtaining the target resource allocation solution (optimal CDN deployment solution) provided by the present invention, the user network database constructed in the aforementioned steps can be used to analyze and optimize the network needs and resources of different users, that is, the characteristic data of the network needs and resources of different users in the user network database can be used as the input feature items of the preset deep reinforcement learning algorithm model, including network traffic, network delay, network bandwidth, number of users, user satisfaction, number of edge servers, location, capacity, caching strategy, etc., and its output item is a CDN deployment plan, indicating how many edge servers should be deployed for each user, as well as the location, capacity, caching strategy, etc. of the servers.

[0109] The core models involved in the embodiments of the present invention are a preset deep reinforcement learning algorithm model and a preset genetic algorithm model, which are combined to form an improved deep reinforcement learning (RL) and genetic algorithm (GA) network architecture. Based on deep reinforcement learning (RL) and genetic algorithm (GA), the present invention introduces a multi-objective optimization and decentralized control mechanism to enhance the efficiency and reliability of the model while reducing the complexity and risk of centralized control.

[0110] The main idea of ​​improving the deep reinforcement learning (RL) and genetic algorithm (GA) network architecture is to first pre-process the network demand and resource data of multiple users, convert them into numerical feature vectors, and then input them into the decentralized controller with a preset deep reinforcement learning algorithm model. The decentralized controller distributes the feature vectors to different sub-controllers, each of which is responsible for the CDN deployment decision of one user and collaborates and coordinates with other sub-controllers. Each sub-controller uses the deep reinforcement learning (RL) method to select an action, i.e. a CDN deployment plan, according to the current network status and user needs, and then controls the sub-controller's own CDN deployment input and output according to the plan, and obtains feedback from multiple user network databases, i.e. the effect and cost of the plan, as well as the CDN deployment of other users.

[0111] The goal of each sub-controller is to maximize its own CDN deployment utility function, which can take into account multiple indicators such as content distribution efficiency, network load balancing, user satisfaction and other network data characteristic values, as well as the cost of CDN deployment. It can also consider the impact of other users' CDN deployment utility functions to achieve a balance between CDN deployment effects and costs among multiple users. The CDN deployment plan of each sub-controller can then be input into the preset genetic algorithm model. The genetic algorithm (GA) can treat the CDN deployment plans of multiple sub-controllers as an individual, and then use operations such as crossover, mutation, and selection to generate new individuals, that is, new CDN deployment plans. At the same time, according to the principle of multi-objective optimization, the optimal individual, that is, the optimal CDN deployment plan, is selected as the final output.

[0112] Finally, the output of the genetic algorithm (GA) can be compared with the original input to calculate the improvement of the CDN deployment utility function as the reward function of the model. Through optimization algorithms such as back propagation and gradient descent, the parameters of the model are updated to maximize the CDN deployment utility function.

[0113] Among them, the CDN deployment utility function (resource allocation scheme utility function) can be expressed as:

[0114] E i =σU i +λP i +μMi +νK i (7)

[0115] Among them, E i represents the CDN deployment utility function of the i-th user, U i represents the content distribution efficiency of the i-th user, P i represents the network load balancing of the i-th user, M i represents the user satisfaction of the i-th user, K i represents the CDN deployment cost of the i-th user, σ, λ, μ, ν are weight coefficients, which are adjusted according to the network requirements of different users and the dynamic changes of resources.

[0116] Formula (7) can be used to evaluate the pros and cons of each user's CDN deployment solution, while also considering the impact of other users' CDN deployment utility functions to achieve a balance between CDN deployment effects and costs among multiple users.

[0117] Step 104, generating a second resource allocation scheme according to the first resource allocation scheme and a preset genetic algorithm model, and determining a target resource allocation scheme according to the first resource allocation scheme and the second resource allocation scheme;

[0118] In practical applications, the core models involved in the embodiments of the present invention are a preset deep reinforcement learning algorithm model and a preset genetic algorithm model, which are combined to form an improved deep reinforcement learning (RL) and genetic algorithm (GA) network architecture. Based on deep reinforcement learning (RL) and genetic algorithm (GA), the present invention introduces a multi-objective optimization and decentralized control mechanism to enhance the efficiency and reliability of the model while reducing the complexity and risk of centralized control.

[0119] The main idea of ​​improving the deep reinforcement learning (RL) and genetic algorithm (GA) network architecture is to first pre-process the network demand and resource data of multiple users, convert them into numerical feature vectors, and then input them into the decentralized controller with a preset deep reinforcement learning algorithm model. The decentralized controller distributes the feature vectors to different sub-controllers, each of which is responsible for the CDN deployment decision of one user and collaborates and coordinates with other sub-controllers. Each sub-controller uses the deep reinforcement learning (RL) method to select an action, i.e. a CDN deployment plan, according to the current network status and user needs, and then controls the sub-controller's own CDN deployment input and output according to the plan, and obtains feedback from multiple user network databases, i.e. the effect and cost of the plan, as well as the CDN deployment of other users.

[0120] The goal of each sub-controller is to maximize its own CDN deployment utility function, which can take into account multiple indicators such as content distribution efficiency, network load balancing, user satisfaction and other network data characteristic values, as well as the cost of CDN deployment. It can also consider the impact of other users' CDN deployment utility functions to achieve a balance between CDN deployment effects and costs among multiple users. The CDN deployment plan of each sub-controller can then be input into a preset genetic algorithm model. The genetic algorithm (GA) can treat the CDN deployment plans of multiple sub-controllers as an individual, and then use operations such as crossover, mutation, and selection to generate new individuals, i.e., new CDN deployment plans (second resource allocation plans). At the same time, the optimal individual, i.e., the optimal CDN deployment plan (target resource allocation plan), can be selected as the final output based on the principle of multi-objective optimization.

[0121] In some embodiments of the present invention, before obtaining the target resource allocation scheme according to the first resource allocation scheme and the second resource allocation scheme, the method further includes:

[0122] Acquire an original resource allocation scheme, and determine whether it is necessary to update a weight coefficient in a utility function of the resource allocation scheme according to the original resource allocation scheme and the second resource allocation scheme;

[0123] In the case where it is necessary to update the weight coefficient in the utility function of the resource allocation scheme, the weight coefficient is updated, and the step of determining the utility function of the resource allocation scheme based on the network data characteristic value is re-executed according to the updated weight coefficient, and under the condition of using the resource allocation scheme utility function as a reward function, generating a first resource allocation scheme through a preset deep reinforcement learning algorithm model.

[0124] In the specific implementation, a principle of dynamically adjusting weight coefficients can be implemented in the multi-objective optimization process. The specific process is to compare the CDN deployment plan (second resource allocation plan) predicted by the improved deep reinforcement learning (RL) and genetic algorithm (GA) network architecture with the original CDN deployment plan (original resource allocation plan). When the update formula is satisfied, the weight coefficient is updated to make the CDN deployment utility function larger. When the update formula is not satisfied, the weight coefficient is kept unchanged.

[0125] Among them, the weight update formula can be expressed as follows:

[0126]

[0127] Among them, Δw i is the change in the weight of the i-th CDN deployment target, ζ is the adjustment factor, N represents the number of eigenvalues, and x 2i and Xi Represent the decision vector and the value of network demand and resource data on the i-th feature respectively. If Δw i ≥0.5, the weight of the i-th CDN deployment target is updated. After the weight of the CDN deployment utility function is updated, the network demand and resource data of different users are input into the improved deep reinforcement learning (RL) and genetic algorithm (GA) network architecture again, and the above process is repeated until the preset number of iterations or convergence condition is reached.

[0128] In some embodiments of the present invention, generating a second resource allocation scheme according to the first resource allocation scheme and a preset genetic algorithm model includes:

[0129] The first resource allocation scheme is subjected to one or more of cross processing, mutation processing and selection processing through the preset genetic algorithm model to generate the second resource allocation scheme.

[0130] In the specific implementation, as mentioned above, the genetic algorithm (GA) can treat the CDN deployment plans of multiple sub-controllers as an individual, and then use operations such as crossover, mutation, and selection to generate new individuals, that is, new CDN deployment plans (second resource allocation plans). At the same time, according to the principle of multi-objective optimization, the optimal individual, that is, the optimal CDN deployment plan (target resource allocation plan) can be selected as the final output.

[0131] Step 105: Allocate resources to the content distribution network according to the target resource allocation scheme.

[0132] In actual applications, after the optimal CDN deployment plan is determined, CDN deployment can be performed according to the optimal CDN deployment plan to achieve resource allocation for the content distribution network that can meet the actual needs of the user end.

[0133] The following will be combined Figure 5 The embodiments of the present invention are further described as follows:

[0134] Reference Figure 5 The overall execution process of the content distribution network resource allocation method provided by the present invention can be summarized as follows:

[0135] 1) Deployment of network demand and resource monitoring devices:

[0136] By deploying network demand and resource monitoring devices to network nodes of multiple users, configuring data communication modules, and performing data compression and encryption processing, it is ensured that the data generated by the network demand and resource monitoring devices can be securely transmitted to the network management system, and by establishing a network management system, the device status and data transmission can be monitored.

[0137] 2) Data collection and processing:

[0138] By establishing a cloud storage system, the network demands of multiple users and the network data streams of resource monitoring devices are received and stored, thereby generating network data eigenvalues. The characteristics are given by the normalization and standard deviation scaling methods. In addition, the method uses a new DAE operator to reduce the dimension and reconstruct the network data to generate implicit vector eigenvalues. This application proposes an adaptive noise operator for the first time to enhance the diversity and robustness of the data. At the same time, it proposes an RL-GA operator for the first time to optimize the input data and improve the effect of CDN deployment.

[0139] 3) Construction of multi-user network database:

[0140] The network data feature vector of each user is obtained through the network demand and resource monitoring device, and then the network database of each user is constructed. The present invention proposes to use a clustering algorithm to generate a network database of the whole system based on the distributed nodes and regional characteristics of the network demand and resource monitoring device, so as to realize the integration of network data among multiple users.

[0141] 4) Improve the construction of deep reinforcement learning (RL) and genetic algorithm (GA) network architecture:

[0142] By improving the deep reinforcement learning (RL) and genetic algorithm (GA) network architecture, the CDN deployment decision between multiple users is optimized, and the CDN deployment utility function is used to give the algorithm network rewards. In addition, the CDN deployment effect and cost are evaluated according to the decision vector. The specific evaluation results are adjusted using the dynamic weight adjustment principle, and the CDN deployment optimization work is finally completed. At the same time, in the present invention, a principle of dynamically adjusting the weight coefficient is proposed. When the update formula is satisfied, the weight coefficient is updated to make the CDN deployment utility function larger.

[0143] The content distribution network resource allocation method provided by the present invention can cover the network nodes of multiple users by deploying network demand and resource monitoring devices, and with the help of cloud storage and dimensionality reduction reconstruction technology, it realizes the ability to fully consider the network needs and resources of multiple users. This enables the network management departments of multiple users to obtain the network data of the entire system in a timely manner, and can optimize the CDN deployment in real time, thereby better improving the network service quality and user experience. It is also possible to more flexibly coordinate and balance the CDN deployment effects and costs among multiple users by building a multi-user network database and introducing a decentralized control structure. This is crucial for achieving network resource sharing and maximizing benefits among multiple users, and helps the network management departments of multiple users to quickly adjust and optimize CDN deployment strategies.

[0144] In addition, network traffic, network delay, network bandwidth, number of users, user satisfaction, etc. are introduced for the first time as one of the network data features, and the DAE operator method is used to give its characteristics. Different from traditional network data features that usually focus on parameters such as the number, location, capacity, and caching strategy of edge servers, network traffic, network delay, network bandwidth, number of users, user satisfaction, etc. are used as a new quantitative feature to characterize network demand and resources, which can provide more information for CDN deployment optimization and help to more accurately evaluate the utility function of CDN deployment. Furthermore, the present invention also proposes the principle of dynamically adjusting weights. This architecture allows the network data collected in real time to be input into the algorithm network, and the CDN deployment decisions are optimized through utility functions and weight adjustments. The principle of dynamically adjusting weights further improves the adaptability of the utility function and helps to adapt to different network environments and user needs.

[0145] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0146] Reference Figure 6 , shows a schematic diagram of the structure of a content distribution network resource allocation device provided by some embodiments of the present invention; specifically, it may include the following modules:

[0147] The network data acquisition module 601 is used to acquire the network demand data and network resource data of the user network nodes in the content distribution network;

[0148] A feature extraction module 602 is used to extract features from the network demand data and the network resource data to obtain network data feature values ​​of the user network node;

[0149] A first resource allocation scheme generating module 603 is used to determine a resource allocation scheme utility function, and generate a first resource allocation scheme according to the resource allocation scheme utility function, the network data feature value and a preset deep reinforcement learning algorithm model;

[0150] A target resource allocation scheme determining module 604 is used to generate a second resource allocation scheme according to the first resource allocation scheme and a preset genetic algorithm model, and determine a target resource allocation scheme according to the first resource allocation scheme and the second resource allocation scheme;

[0151] The resource allocation module 605 is configured to allocate resources to the content distribution network according to the target resource allocation scheme.

[0152] In some embodiments of the present invention, the feature extraction module 602 includes:

[0153] An implicit vector eigenvalue generation submodule, used to generate an implicit vector eigenvalue according to a preset dimensionality reduction and reconstruction operator, the network demand data and the network resource data;

[0154] The network data eigenvalue determination submodule is used to perform noise enhancement processing on the implicit vector eigenvalue according to a preset adaptive noise operator, and obtain the network data eigenvalue of the user network node according to the output data of the preset deep reinforcement learning-genetic algorithm operator and the noise enhancement processing.

[0155] In some embodiments of the present invention, the device further comprises:

[0156] The user network database construction module is used to cluster the network data feature values ​​to obtain the user network database.

[0157] In some embodiments of the present invention, the first resource allocation scheme generating module 603 includes:

[0158] The resource allocation scheme utility function determination submodule is used to determine the resource allocation scheme utility function according to the network data feature value in the user network database.

[0159] In some embodiments of the present invention, the device further comprises:

[0160] A weight coefficient judgment module, used for obtaining an original resource allocation scheme, and judging whether it is necessary to update the weight coefficient in the utility function of the resource allocation scheme according to the original resource allocation scheme and the second resource allocation scheme;

[0161] A weight coefficient updating module is used to update the weight coefficient in the resource allocation scheme utility function when it is necessary to update the weight coefficient, and re-execute the step of determining the resource allocation scheme utility function according to the network data feature value according to the updated weight coefficient, and generate a first resource allocation scheme through a preset deep reinforcement learning algorithm model under the condition that the resource allocation scheme utility function is used as a reward function.

[0162] In some embodiments of the present invention, the target resource allocation scheme determining module 604 includes:

[0163] The second resource allocation scheme generating module is used to perform one or more of cross processing, mutation processing and selection processing on the first resource allocation scheme through the preset genetic algorithm model to generate the second resource allocation scheme.

[0164] Some embodiments of the present invention further provide an electronic device, which may include a processor, a memory, and a computer program stored in the memory and capable of running on the processor, and when the computer program is executed by the processor, the above content distribution network resource allocation method is implemented.

[0165] Some embodiments of the present invention further provide a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the above content distribution network resource allocation method is implemented.

[0166] Some embodiments of the present invention further provide a computer program product, including a computer program, which implements the above content distribution network resource allocation method when executed by a processor.

[0167] As for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0168] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0169] Those skilled in the art will appreciate that the embodiments of the present invention may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0170] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0171] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0172] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0173] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present invention.

[0174] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the above elements.

[0175] The above is a detailed introduction to a content distribution network resource allocation method, device, equipment, and medium provided. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for general technicians in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A content distribution network resource allocation method, characterized in that: The method comprises: Acquiring network demand data and network resource data of user network nodes in the content distribution network; Extracting features from the network demand data and the network resource data to obtain network data feature values ​​of the user network node; Determine a resource allocation scheme utility function according to the network data characteristic value, and generate a first resource allocation scheme through a preset deep reinforcement learning algorithm model under the condition that the resource allocation scheme utility function is used as a reward function; Generate a second resource allocation scheme according to the first resource allocation scheme and a preset genetic algorithm model, and determine a target resource allocation scheme according to the first resource allocation scheme and the second resource allocation scheme; According to the target resource allocation scheme, resources are allocated to the content distribution network.

2. The method according to claim 1, characterized in that The extracting features of the network demand data and the network resource data to obtain the network data feature value of the user network node includes: Generate an implicit vector eigenvalue according to a preset dimensionality reduction and reconstruction operator, the network demand data, and the network resource data; The implicit vector eigenvalue is subjected to noise enhancement processing according to a preset adaptive noise operator, and the network data eigenvalue of the user network node is obtained according to the output data of the preset deep reinforcement learning-genetic algorithm operator and the noise enhancement processing.

3. The method according to claim 2, characterized in that The formula of the preset dimensionality reduction reconstruction operator is: Among them, x i is the i-th eigenvalue, N represents the number of input features, x is the feature of the current input, δ is the coding factor used to prevent the input of zero vectors, and L(x) is the implicit vector eigenvalue; The formula of the preset adaptive noise operator is: Among them, x i is the input feature, N is the input feature dimension, α is the adaptive noise adjustment factor, and θ is the output data of the noise enhancement process; The formula of the preset deep reinforcement learning-genetic algorithm operator is: x t+1 =ωRL(x t ,and t ) and t+1 =θGA(x t+1 ,and t ) Among them, x t is the decision variable for the tth iteration, y t is the objective function value of the tth iteration, ω, θ are both iteration factors, RL() is the deep reinforcement learning algorithm, and GA() is the genetic algorithm.

4. The method according to claim 1, characterized in that: Before determining the resource allocation scheme utility function according to the network data characteristic value, the method further includes: The network data characteristic values ​​are clustered to obtain a user network database.

5. The method according to claim 4, characterized in that Determining the resource allocation scheme utility function according to the network data characteristic value includes: The resource allocation scheme utility function is determined according to the network data feature values ​​in the user network database.

6. The method according to claim 1, characterized in that Before obtaining a target resource allocation scheme according to the first resource allocation scheme and the second resource allocation scheme, the method further includes: Acquire an original resource allocation scheme, and determine whether it is necessary to update a weight coefficient in a utility function of the resource allocation scheme according to the original resource allocation scheme and the second resource allocation scheme; In the case where it is necessary to update the weight coefficient in the utility function of the resource allocation scheme, the weight coefficient is updated, and the step of determining the utility function of the resource allocation scheme based on the network data characteristic value is re-executed according to the updated weight coefficient, and under the condition of using the resource allocation scheme utility function as a reward function, generating a first resource allocation scheme through a preset deep reinforcement learning algorithm model.

7. The method according to claim 1, characterized in that The step of generating a second resource allocation scheme according to the first resource allocation scheme and a preset genetic algorithm model includes: The first resource allocation scheme is subjected to one or more of cross processing, mutation processing and selection processing through the preset genetic algorithm model to generate the second resource allocation scheme.

8. A content distribution network resource allocation device, characterized in that: The device comprises: A network data acquisition module, used to acquire network demand data and network resource data of user network nodes in the content distribution network; A feature extraction module, used to extract features from the network demand data and network resource data to obtain network data feature values ​​of the user network node; A first resource allocation scheme generating module, used to determine a resource allocation scheme utility function, and generate a first resource allocation scheme according to the resource allocation scheme utility function, the network data feature value and a preset deep reinforcement learning algorithm model; a target resource allocation scheme determining module, configured to generate a second resource allocation scheme according to the first resource allocation scheme and a preset genetic algorithm model, and determine a target resource allocation scheme according to the first resource allocation scheme and the second resource allocation scheme; The resource allocation module is used to allocate resources to the content distribution network according to the target resource allocation scheme.

9. An electronic device, characterized in that: The method comprises a processor, a memory and a computer program stored in the memory and capable of running on the processor, wherein when the computer program is executed by the processor, the content distribution network resource allocation method according to any one of claims 1 to 7 is implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the content distribution network resource allocation method according to any one of claims 1 to 7 is implemented.

Images