A content distribution network resource allocation method and device, equipment and medium

Abstract

The embodiment of the application provides a content distribution network resource allocation method and device, equipment and medium, network demand data and network resource data of a user network node are acquired, feature extraction is performed on the network demand data and the network resource data to obtain network data feature values, then a resource allocation scheme utility function is determined, and a first resource allocation scheme is generated through a preset deep reinforcement learning algorithm model under the condition that the resource allocation scheme utility function is used as a reward function, so that a second resource allocation scheme is generated according to the first resource allocation scheme and a preset genetic algorithm model, and then a target resource allocation scheme is determined according to the first resource allocation scheme and the second resource allocation scheme. The optimal resource allocation scheme is determined by comprehensively considering the user network demand data and the network resource data, the deployment effect is improved while the actual network demand of the user is met, and the actual network experience of the user is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of content delivery network, and particularly relates to a content delivery network resource allocation method and device, equipment and medium. BACKGROUND

[0002] The content delivery network (CDN, Content Delivery Network) aims to optimize the distribution of content by deploying nodes in multiple geographic locations, reduce the load of the source server, and reduce the delay of user access to content, thereby providing better user experience, and in the Internet era, the content delivery network has become an important application technology, which has a profound impact on the network service quality and user experience of multiple users.

[0003] The content delivery network resource allocation scheme (also referred to as CDN deployment scheme) in the related art generally relies on a centralized control system, and generally focuses on considering the number, location, capacity, cache strategy and other parameter characteristics of the edge server to determine the corresponding CDN deployment scheme when deploying.

[0004] However, focusing only on the parameter characteristics of the edge server means that it is difficult to fully consider the actual network demand and resource situation of multiple users in the content delivery network, which not only easily leads to the inefficiency of the deployment result, but also easily leads to resource waste and service imbalance, which seriously affects the network experience of users. SUMMARY

[0005] In view of the above problems, a content delivery network resource allocation method and device, equipment and medium are provided to overcome the above problems or at least partially solve the above problems, comprising:

[0006] A content delivery network resource allocation method, the method comprising:

[0007] Obtaining network demand data and network resource data of a user network node in the content delivery network;

[0008] Feature extraction is performed on the network demand data and network resource data to obtain network data feature values of the user network node;

[0009] According to the network data feature values, a resource allocation scheme utility function is determined, and a first resource allocation scheme is generated by a preset deep reinforcement learning algorithm model under the condition that the resource allocation scheme utility function is used as a reward function;

[0010] According to the first resource allocation scheme and a preset genetic algorithm model, a second resource allocation scheme is generated, and a target resource allocation scheme is determined according to the first resource allocation scheme and the second resource allocation scheme;

[0011] According to the target resource allocation scheme, resource allocation is performed on the content distribution network.

[0012] Optionally, the feature extraction on the network demand data and the network resource data to obtain the network data feature value of the user network node comprises:

[0013] According to the preset dimensionality reduction reconstruction operator, the network demand data and the network resource data, an implicit vector feature value is generated;

[0014] According to a preset adaptive noise operator, noise enhancement processing is performed on the implicit vector feature value, and according to a preset deep reinforcement learning-genetic algorithm operator and output data of the noise enhancement processing, the network data feature value of the user network node is obtained.

[0015] Optionally, a formula of the preset dimensionality reduction reconstruction operator is:

[0016]

[0017] wherein, x i is the i th feature value, N represents the number of input features, x is the current input feature, δ is an encoding factor for preventing zero vector input, and L(x) is the implicit vector feature value;

[0018] A formula of the preset adaptive noise operator is:

[0019]

[0020] wherein, x i is an input feature, N is an input feature dimension, α is an adaptive noise adjustment factor, and θ is output data of the noise enhancement processing;

[0021] A formula of the preset deep reinforcement learning-genetic algorithm operator is:

[0022] x t+1 =ωRL(x t ,y t )

[0023] y t+1 =θGA(x t+1 ,y t )

[0024] wherein, x t is a decision variable of the t th iteration, y t is a target function value of the t th iteration, ω and θ are iteration factors, RL() is a deep reinforcement learning algorithm, and GA() is a genetic algorithm.

[0025] Optionally, before the determining the resource allocation scheme utility function according to the network data feature value, the method further comprises:

[0026] performing clustering on the network data feature value to obtain a user network database.

[0027] Optionally, the determining the resource allocation scheme utility function according to the network data feature value comprises:

[0028] determining the resource allocation scheme utility function according to the network data feature value in the user network database.

[0029] Optionally, before the obtaining the target resource allocation scheme according to the first resource allocation scheme and the second resource allocation scheme, the method further comprises:

[0030] obtaining an original resource allocation scheme, and determining whether the weight coefficient in the resource allocation scheme utility function needs to be updated according to the original resource allocation scheme and the second resource allocation scheme;

[0031] in a case where the weight coefficient in the resource allocation scheme utility function needs to be updated, updating the weight coefficient, and re-executing the determining the resource allocation scheme utility function according to the network data feature value and the generating the first resource allocation scheme by the preset deep reinforcement learning algorithm model under the condition that the resource allocation scheme utility function is used as a reward function according to the updated weight coefficient.

[0032] Optionally, the generating the second resource allocation scheme according to the first resource allocation scheme and the preset genetic algorithm model comprises:

[0033] generating the second resource allocation scheme by performing one or more of cross processing, mutation processing and selection processing on the first resource allocation scheme by the preset genetic algorithm model.

[0034] A content distribution network resource allocation apparatus, the apparatus comprising:

[0035] a network data acquisition module configured to acquire network demand data and network resource data of a user network node in the content distribution network;

[0036] a feature extraction module configured to perform feature extraction on the network demand data and the network resource data to obtain a network data feature value of the user network node;

[0037] a first resource allocation scheme generation module configured to determine a resource allocation scheme utility function, and generate a first resource allocation scheme according to the resource allocation scheme utility function, the network data feature value and a preset deep reinforcement learning algorithm model.

[0038] a target resource allocation scheme determination module configured to generate a second resource allocation scheme according to the first resource allocation scheme and a preset genetic algorithm model, and determine a target resource allocation scheme according to the first resource allocation scheme and the second resource allocation scheme;

[0039] a resource allocation module configured to allocate resources to the content distribution network according to the target resource allocation scheme.

[0040] An electronic device includes a processor, a memory, and a computer program stored on the memory and capable of running on the processor, and the computer program, when executed by the processor, implements the content distribution network resource allocation method as described above.

[0041] A computer readable storage medium, characterized in that the computer readable storage medium stores a computer program, and the computer program, when executed by a processor, implements the content distribution network resource allocation method as described above.

[0042] A computer program product, comprising a computer program, which, when executed by a processor, implements the content distribution network resource allocation method as described above.

[0043] Embodiments of the present application have the following advantages:

[0044] The present application provides a content distribution network resource allocation method, by obtaining network demand data and network resource data of user network nodes in a content distribution network; and extracting features of the network demand data and the network resource data to obtain network data feature values of the user network nodes; then determining a resource allocation scheme utility function, and generating a first resource allocation scheme by a preset deep reinforcement learning algorithm model under the condition that the resource allocation scheme utility function is used as a reward function; thereby generating a second resource allocation scheme according to the first resource allocation scheme and a preset genetic algorithm model, and further determining a target resource allocation scheme according to the first resource allocation scheme and the second resource allocation scheme; and allocating resources to the content distribution network according to the target resource allocation scheme. The method realizes the determination of the resource allocation scheme of the content distribution network by comprehensively considering the user network demand data and the network resource data, and determines the optimal resource allocation scheme by comprehensively using the resource allocation scheme utility function, the deep reinforcement learning algorithm and the genetic algorithm, thereby meeting the actual network demand of users, improving the deployment effect, avoiding the waste of resources and the imbalance of services, and improving the actual network experience of users. BRIEF DESCRIPTION OF DRAWINGS

[0045] In order to more clearly illustrate the technical solutions of the present application, the drawings needed to be used in the description of the present application will be briefly introduced as follows. Obviously, the drawings in the following description are only some embodiments of the present application, and all other drawings obtained by those skilled in the art without creative labor on the basis of these drawings also belong to the protection scope of the present application.

[0046] Figure 1 is a step flow chart of a content distribution network resource allocation method provided by some embodiments of the present application;

[0047] Figure 2 is a data processing flow chart of network data characteristic values provided by some embodiments of the present application;

[0048] Figure 3 is an example diagram of a user network database provided by some embodiments of the present application;

[0049] Figure 4 is an actual algorithm architecture diagram for obtaining a target resource allocation scheme provided by some embodiments of the present application;

[0050] Figure 5 is a whole execution flow schematic diagram of a content distribution network resource allocation method provided by some embodiments of the present application;

[0051] Figure 6 is a structure schematic diagram of a content distribution network resource allocation device provided by some embodiments of the present application. DETAILED DESCRIPTION

[0052] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application will be further described in detail below in combination with the drawings and specific embodiments. Obviously, the described embodiments are some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor also belong to the protection scope of the present application.

[0053] In the embodiments of the present application, the same items or similar items with basically the same functions and effects are distinguished by using "first", "second", etc. only for the purpose of clearly describing the technical solutions of the embodiments of the present application, and cannot be understood as indicating or implying the relative importance of the indicated technical features or implicitly indicating the number of the indicated technical features.

[0054] In the embodiments of the present application, the positions or location relationships indicated by the terms "upper", "lower", etc. are based on the positions or location relationships shown in the drawings, and are only for the purpose of facilitating the description of the present application and simplifying the description, and therefore cannot be understood as indicating or implying that the indicated devices or elements must have a specific position, be constructed and operated in a specific position, and therefore cannot be understood as a limitation on the present application.

[0055] For the convenience of explanation, the following will give corresponding explanations of related terms involved in the embodiments of the present application:

[0056] CDN (Content Delivery Network): content delivery network, its purpose is to add a new network architecture in the existing Internet, the content of the website is published to the nearest network "edge" (edge server), so that users can get the required content, solve the Internet network congestion, improve the user response speed of accessing the website.

[0057] RL (Reinforcement Learning, RL), reinforcement learning, refers to a class of problems learned from interaction with the environment and methods for solving such problems. Reinforcement learning problems can be described as an agent learning from interaction with the environment to achieve a specific goal (such as obtaining the maximum reward value). Similar to deep learning, the key problem in reinforcement learning is also the contribution allocation problem. Each action cannot directly obtain supervised information, and needs to obtain the final supervised information (reward) of the entire model, and has a certain delay.

[0058] Genetic Algorithm (Genetic Algorithm, GA): It was first proposed by John Holland of the United States in the 1970s. The algorithm is designed according to the evolution law of biological organisms in nature. It is a computational model simulating the natural selection and genetic mechanism of Darwin's biological evolution process. It is a method of searching for optimal solutions by simulating the natural evolution process.

[0059] In the Internet era, Content Delivery Network (CDN) has become an important application technology, which has a profound impact on the network service quality and user experience of multiple users. Traditional CDN deployment techniques have obvious defects in the face of this challenge, and an innovative solution is urgently needed. First, the traditional CDN deployment method usually relies on centralized control systems, which have limited coverage and are difficult to fully consider the network needs and resource conditions of multiple users. This leads to the inefficiency of the deployment results, and is prone to resource waste and service imbalance. Due to the centralization of the control system, it is impossible to realize the coordination and optimization of CDN deployment among multiple users, which means that the effectiveness and cost of CDN deployment may not be optimized, and the network service quality and user experience of multiple users are affected. Second, the traditional method has problems in the dynamic adjustment of CDN deployment. Due to the lack of real-time network data and user feedback, the traditional technology is often difficult to adapt to different network environments and user needs. This makes the flexibility and reliability of CDN deployment decrease, because it is impossible to adjust the parameters and strategies of CDN deployment in time. In addition, the data processing and transmission of the traditional method have delay, which cannot meet the real-time requirements. CDN deployment needs immediate response and update to improve the network service quality and user experience, while the delay of the traditional method may lead to the lag of deployment action, further exacerbating the problem of CDN deployment.

[0060] In order to solve the above problems, the core technical concept of the present application is to determine the resource allocation scheme of the content delivery network based on the comprehensive consideration of user network demand data and network resource data, which improves the content delivery network resource allocation method in the related art. The present application will be described in detail below with reference to the accompanying drawings:

[0061] Referring to Figure 1 , a step flow chart of a content delivery network resource allocation method provided by some embodiments of the present application is shown, which can specifically include the following steps:

[0062] Step 101, obtaining network demand data and network resource data of user network nodes in the content delivery network;

[0063] In a specific implementation, the network demand and resource monitoring device can be deployed at the user end to obtain the network demand and resource data. Specifically, the network demand and resource monitoring device can be deployed according to the network characteristics and user behaviors of multiple users. It is worth noting that the specific deployment location can be determined according to the dynamic changes of the network demand and resources of multiple users. That is, the monitoring coverage of the network demand and resources of multiple users, such as network traffic, network delay, network bandwidth, user quantity, user satisfaction, etc., and the allocation of corresponding network resources, such as the number, location, capacity, and cache strategy of edge servers, can be accurately obtained and fed back by the network demand and resource monitoring device. Secondly, the computing power and storage space of the network demand and resource monitoring device should be considered to avoid data loss or errors caused by the network demand and resource monitoring device. Finally, the data communication module should be configured and processed by data compression and encryption to ensure that the data generated by the network demand and resource monitoring device can be safely transmitted to the network management system. Finally, the network management system can be established to monitor the status and data reception of the network demand and resource monitoring device, and set up modules for data analysis and processing to provide effective input data for subsequent processes.

[0064] Step 102, feature extraction is performed on the network demand data and network resource data to obtain network data feature values of the user network node.

[0065] In a specific implementation, the corresponding operator can be pre-set to perform feature extraction on the network demand data and network resource data to obtain network data feature values of the user network node.

[0066] In some embodiments of the present application, the feature extraction on the network demand data and network resource data to obtain the network data feature values of the user network node includes:

[0067] According to the preset dimensionality reduction reconstruction operator, the network demand data and the network resource data, an implicit vector feature value is generated.

[0068] According to the preset adaptive noise operator, noise enhancement processing is performed on the implicit vector feature value, and according to the preset deep reinforcement learning-genetic algorithm operator and the output data of the noise enhancement processing, the network data feature values of the user network node are obtained.

[0069] In a specific implementation, as shown in Figure 2 FIG. 2 shows a data processing flowchart for obtaining network data feature values of a user network node according to an embodiment of the present application.

[0070] In this step, a cloud storage system can be established to receive and store network data streams of multiple users' network requirements and resource monitoring devices for subsequent calling by an algorithm module. The original network data of each user in the cloud storage system can be processed to eliminate noise and outliers. For network traffic data, a sliding window mechanism can be used, i.e., the network traffic data is divided into fixed-length time series, a position code is added to each time series, and then the time series is scaled to [0, 1] by normalization to reduce the scale difference of the data, and then input into the encoder to improve the operation efficiency of the data and enhance the robustness of the whole process. A new DAE operator (preset dimensionality reduction reconstruction operator) can be used as the core of CDN deployment, which overall task is to compress the network data into a low-dimensional hidden vector, and restore the vector to the original data in subsequent decoding. For the pre-training process, an operator with adaptive noise (preset adaptive noise operator) can be used, which adds different noises to the input data by dynamically adjusting the intensity and type of the noise to enhance the diversity and robustness of the data. At the same time, an RL-GA operator (preset deep reinforcement learning-genetic algorithm operator) can be used to combine the advantages of deep reinforcement learning (RL) and genetic algorithm (GA) to optimize the input data (output data of noise enhancement processing) to obtain network data feature values, thereby improving the effect of CDN deployment. In addition, for all missing values, an interpolation method can be used to fill the missing values with adjacent valid values to increase the reaction speed of the overall module. In addition, a corresponding position code is added to each network data source to facilitate subsequent CDN deployment to quickly locate.

[0071] In addition, in this step, the comparability and consistency of the network data need to be maintained, so the stored data can be first converted into units, i.e., all byte data is converted into bits, and the sampling frequency and sampling bit number are unified, and a standard deviation scaling operation is performed on all data to ensure the consistency of the data in scale. The features of the overall network data can include network traffic, network delay, network bandwidth, number of users, user satisfaction, number of edge servers, location, capacity, and cache strategy. Among them, the network traffic, network delay, network bandwidth, number of users, and user satisfaction can be directly used as feature values x1, x2, x3, x4, and x5. The aforementioned DAE operator is the core of CDN deployment, which overall task is to compress the network data into a low-dimensional hidden vector, and restore the vector to the original data in subsequent decoding.

[0072] In some embodiments of the present application, the formula of the preset dimensionality reduction reconstruction operator is:

[0073]

[0074] Where, x i Let be the i-th feature value, N represent the number of input features, x be the current input feature, δ be the encoding factor used to prevent zero vector input, and L(x) be the latent vector feature value.

[0075] The formula for the preset adaptive noise operator is:

[0076]

[0077] Where, x i α is the input feature, N is the dimension of the input feature, α is the adaptive noise adjustment factor, and θ is the output data of the noise enhancement process;

[0078] The formula for the preset deep reinforcement learning-genetic algorithm operator is:

[0079] x t+1 =ωRL(x t ,y t )

[0080] y t+1 =θGA(x t+1 ,y t )

[0081] Where, x t Let y be the decision variable for the t-th iteration. t Let ω and θ be the objective function value for the t-th iteration, where ω and θ are iteration factors, RL() is a deep reinforcement learning algorithm, and GA() is a genetic algorithm.

[0082] In practical applications, the formula for the novel DAE operator (i.e., the predefined dimensionality reduction and reconstruction operator) can be expressed as:

[0083]

[0084] Where, x i Let be the i-th feature value, N represent the number of input features, x be the current input feature, δ be the encoding factor used to prevent zero vector input, and L(x) be the latent vector feature value. This formula can compress the input network data into a low-dimensional latent vector, which is then restored to the original data during subsequent decoding.

[0085] Furthermore, the aforementioned operator with adaptive noise dynamically adjusts the intensity and type of noise to add different noises to the input data, thereby enhancing the diversity and robustness of the data.

[0086] The preset adaptive noise operator formula is as follows:

[0087]

[0088] Where, x i Let N be the input feature, α be the adaptive noise adjustment factor, and θ be the output data of the noise enhancement process. Through this operator, the intensity and type of noise can be dynamically adjusted to add different noises to the input data, thereby enhancing the diversity and robustness of the data.

[0089] Furthermore, the RL-GA operator mentioned above can optimize the input data by combining the advantages of deep reinforcement learning (RL) and genetic algorithm (GA) to improve the effectiveness of CDN deployment.

[0090] The formula for the RL-GA operator (i.e., the pre-defined deep reinforcement learning-genetic algorithm operator) is as follows:

[0091] x t+1 =ωRL(x t ,y t (3)

[0092] y t+1 =θGA(x t+1 ,y t (4)

[0093] Where, x t Let y be the decision variable for the t-th iteration. t Let ω and θ be the objective function value for the t-th iteration. RL() is a deep reinforcement learning algorithm that continuously updates decision variables based on feedback from the learning environment to maximize the reward. GA() is a genetic algorithm that continuously updates the objective function value by simulating natural selection and genetic mutation to minimize the cost. This operator leverages the exploratory capabilities of deep reinforcement learning and the convergence capabilities of genetic algorithms to achieve a synergistic effect between data optimization and CDN deployment.

[0094] Step 103: Determine the utility function of the resource allocation scheme based on the network data feature values, and generate a first resource allocation scheme by using the utility function of the resource allocation scheme as the reward function through a preset deep reinforcement learning algorithm model.

[0095] In practical implementation, the utility function of resource allocation schemes can be determined based on network data feature values. Then, by improving the network architecture of deep reinforcement learning (RL) and genetic algorithm (GA), the CDN deployment decision among multiple users can be optimized. For example, the CDN deployment utility function (resource allocation scheme utility function) can be used to give the reward of the algorithm network to generate the first resource allocation scheme through a preset deep reinforcement learning algorithm model. In addition, the CDN deployment effect and cost of the corresponding resource allocation scheme at the user end can be further evaluated. Based on the specific evaluation results, the weight coefficients in the utility function can be adaptively adjusted using the principle of dynamic weight adjustment to help determine the optimal resource allocation scheme in the future.

[0096] In some embodiments of the present invention, before determining the utility function of the resource allocation scheme based on the network data feature values, the method further includes:

[0097] Clustering is performed on the network data feature values ​​to obtain a user network database.

[0098] In practical applications, such as Figure 3 The diagram shown is an example of the user network database constructed according to this invention. Having completed the deployment of network demand and resource monitoring devices and the extraction of network data features, we can further construct network databases for multiple users to more flexibly coordinate and balance the CDN deployment effects and costs among multiple users. It is worth noting that, in addition to constructing user network databases for network demand and resource monitoring devices for multiple users, classification is also possible. That is, the network database can store characteristic data of each user's network demands and resources, including network traffic, network latency, network bandwidth, number of users, user satisfaction, number of edge servers, location, capacity, caching strategies, etc.

[0099] Furthermore, for CDN deployment characteristic data, a CDN deployment database can be constructed based on the network requirements of the aforementioned multiple users and resource monitoring devices. That is, two network databases can be constructed. The data in both network databases are the network requirements and resource monitoring device data of each user. However, the first network database only contains characteristic data of network requirements and resources, while the second network database only contains characteristic data of CDN deployment areas of each user.

[0100] Specifically, clustering algorithms can be used to construct a system-wide network database based on network demand, distributed nodes of resource monitoring devices, and regional characteristics.

[0101] The clustering algorithm can be represented as follows:

[0102]

[0103] Where, xi Let C represent the i-th feature value, N represent the number of input features, and C represent the number of input features. i S represents the center of the i-th cluster, M represents the number of clusters, and the smaller the value of S, the better the clustering effect.

[0104] In the above algorithm, feature selection can be performed on the network data first. Nine features are selected as input for clustering: network traffic, network latency, network bandwidth, number of users, user satisfaction, number of edge servers, location, capacity, and caching strategy. Then, formula (5) can be used to cluster each data point, generating a vector of length equal to the number of clusters, where each element represents which cluster the data point belongs to. Subsequently, the distance within each cluster can be minimized. Then, formula (6) can be used to update the center of each cluster, calculating the average value of the data points within each cluster as the new cluster center. The above steps can be repeated until the clustering no longer changes or the maximum number of iterations is reached.

[0105] In general, the network database can be constructed using formula (6). For example, when a network demand and resource monitoring device detects data with high network traffic, high network latency, low network bandwidth, high number of users, low user satisfaction, few edge servers, remote location, small capacity, and poor caching strategy, the data may belong to the cluster of network congestion. On the other hand, when a network demand and resource monitoring device detects data with low network traffic, low network latency, high network bandwidth, low number of users, high user satisfaction, many edge servers, near location, large capacity, and good caching strategy, the data may belong to the cluster of network smoothness.

[0106] In some embodiments of the present invention, determining the utility function of the resource allocation scheme based on the network data feature values ​​includes:

[0107] The utility function of the resource allocation scheme is determined based on the network data feature values ​​in the user network database.

[0108] In specific implementations, such as Figure 4 The diagram shows the actual algorithm architecture of the target resource allocation scheme (optimal CDN deployment scheme) provided by this invention. It can utilize the user network database constructed in the preceding steps to analyze and optimize the network needs and resources of different users. Specifically, the feature data of the network needs and resources of different users in the user network database can be used as input features of a preset deep reinforcement learning algorithm model, including network traffic, network latency, network bandwidth, number of users, user satisfaction, number, location, capacity, caching strategy, etc. The output is a CDN deployment scheme, indicating how many edge servers should be deployed for each user, as well as the location, capacity, caching strategy, etc.

[0109] The core model involved in the embodiments of the present invention is a preset deep reinforcement learning algorithm model and a preset genetic algorithm model, which are combined to form an improved deep reinforcement learning (RL) and genetic algorithm (GA) network architecture. Based on deep reinforcement learning (RL) and genetic algorithm (GA), the present invention introduces a multi-objective optimization and decentralized control mechanism to enhance the efficiency and reliability of the model, while reducing the complexity and risk of centralized control.

[0110] The main idea behind improving the deep reinforcement learning (RL) and genetic algorithm (GA) network architecture is to first preprocess the network demand and resource data of multiple users, converting them into numerical feature vectors. These vectors are then input into a distributed controller based on a pre-defined deep reinforcement learning algorithm model. The distributed controller assigns the feature vectors to different sub-controllers, each responsible for the CDN deployment decision for one user, while collaborating and coordinating with other sub-controllers. Each sub-controller employs deep reinforcement learning (RL) to select an action—a CDN deployment scheme—based on the current network state and user needs. It then controls its own CDN deployment inputs and outputs according to this scheme, obtaining feedback from multiple user network databases, including the effectiveness and cost of the scheme, and the CDN deployment status of other users.

[0111] Each sub-controller aims to maximize its own CDN deployment utility function. This function considers network data characteristics such as content distribution efficiency, network load balancing, and user satisfaction, as well as multiple metrics including CDN deployment cost. It also considers the impact of other users' CDN deployment utility functions to achieve a balance between CDN deployment effectiveness and cost across multiple users. Subsequently, each sub-controller's CDN deployment plan is input into a pre-defined genetic algorithm model. The genetic algorithm (GA) treats the CDN deployment plans of multiple sub-controllers as individuals and uses operations such as crossover, mutation, and selection to generate new individuals, i.e., new CDN deployment plans. Based on the principle of multi-objective optimization, it selects the optimal individual, i.e., the optimal CDN deployment plan, as the final output.

[0112] Finally, the output of the genetic algorithm (GA) can be compared with the original input to calculate the improvement in the CDN deployment utility function, which is used as the model's reward function. The model's parameters are then updated using optimization algorithms such as backpropagation and gradient descent to maximize the CDN deployment utility function.

[0113] The CDN deployment utility function (resource allocation scheme utility function) can be expressed as:

[0114] E i =σU i +λP i +μMi +νK i (7)

[0115] Among them, E i U represents the CDN deployment utility function for the i-th user. i P represents the content distribution efficiency for the i-th user. i M represents the network load balancing for the i-th user. i K represents the user satisfaction of the i-th user. i Let denot represent the CDN deployment cost for the i-th user, and σ, λ, μ, ν are weighting coefficients that are adjusted according to the dynamic changes in network needs and resources of different users.

[0116] Formula (7) can be used to evaluate the merits of each user's CDN deployment scheme, and the impact of other users' CDN deployment utility functions can also be considered to achieve a balance between CDN deployment effectiveness and cost among multiple users.

[0117] Step 104: Generate a second resource allocation scheme based on the first resource allocation scheme and the preset genetic algorithm model, and determine the target resource allocation scheme based on the first resource allocation scheme and the second resource allocation scheme.

[0118] In practical applications, the core models involved in the embodiments of this invention are a preset deep reinforcement learning algorithm model and a preset genetic algorithm model, which are combined to form an improved deep reinforcement learning (RL) and genetic algorithm (GA) network architecture. Based on deep reinforcement learning (RL) and genetic algorithm (GA), this invention introduces a multi-objective optimization and decentralized control mechanism to enhance the efficiency and reliability of the model, while reducing the complexity and risk of centralized control.

[0119] The main idea behind improving the deep reinforcement learning (RL) and genetic algorithm (GA) network architecture is to first preprocess the network demand and resource data of multiple users, converting them into numerical feature vectors. These vectors are then input into a distributed controller based on a pre-defined deep reinforcement learning algorithm model. The distributed controller assigns the feature vectors to different sub-controllers, each responsible for the CDN deployment decision for one user, while collaborating and coordinating with other sub-controllers. Each sub-controller employs deep reinforcement learning (RL) to select an action—a CDN deployment scheme—based on the current network state and user needs. It then controls its own CDN deployment inputs and outputs according to this scheme, obtaining feedback from multiple user network databases, including the effectiveness and cost of the scheme, and the CDN deployment status of other users.

[0120] Each sub-controller aims to maximize its own CDN deployment utility function. This function considers network data characteristics such as content distribution efficiency, network load balancing, and user satisfaction, as well as multiple metrics including CDN deployment cost. It also considers the impact of other users' CDN deployment utility functions to achieve a balance between CDN deployment effectiveness and cost across multiple users. Subsequently, each sub-controller's CDN deployment plan is input into a pre-defined genetic algorithm model. The genetic algorithm (GA) treats the CDN deployment plans of multiple sub-controllers as individuals and uses operations such as crossover, mutation, and selection to generate new individuals, i.e., new CDN deployment plans (secondary resource allocation plans). Simultaneously, based on the principle of multi-objective optimization, it selects the optimal individual, i.e., the optimal CDN deployment plan (target resource allocation plan), as the final output.

[0121] In some embodiments of the present invention, before obtaining the target resource allocation scheme based on the first resource allocation scheme and the second resource allocation scheme, the method further includes:

[0122] Obtain the original resource allocation scheme, and based on the original resource allocation scheme and the second resource allocation scheme, determine whether it is necessary to update the weight coefficients in the utility function of the resource allocation scheme;

[0123] If it is necessary to update the weight coefficients in the utility function of the resource allocation scheme, update the weight coefficients, and re-execute the steps of determining the utility function of the resource allocation scheme based on the network data feature values, and generating the first resource allocation scheme through a preset deep reinforcement learning algorithm model under the condition that the utility function of the resource allocation scheme is used as the reward function.

[0124] In practical implementation, a principle of dynamically adjusting weight coefficients can be implemented during multi-objective optimization. Specifically, the CDN deployment scheme (second resource allocation scheme) predicted by the improved deep reinforcement learning (RL) and genetic algorithm (GA) network architecture is compared with the original CDN deployment scheme (original resource allocation scheme). When the update formula is satisfied, the weight coefficients are updated to maximize the CDN deployment utility function. When the update formula is not satisfied, the weight coefficients remain unchanged.

[0125] The weight update formula can be expressed as follows:

[0126]

[0127] Where, Δw i Let ζ be the change in weight of the i-th CDN deployment target, ζ be the adjustment factor, N be the number of eigenvalues, and x be the weight of the i-th CDN deployment target. 2i and Xi Let Δw and Δw represent the values ​​of the decision vector and network demand and resource data on the i-th feature, respectively. i If the value is ≥0.5, the weight of the i-th CDN deployment target is updated. After the weight of the CDN deployment utility function is updated, the network requirements and resource data of different users are input into the improved deep reinforcement learning (RL) and genetic algorithm (GA) network architecture again, and the above process is repeated until the preset number of iterations or convergence conditions are reached.

[0128] In some embodiments of the present invention, generating a second resource allocation scheme based on the first resource allocation scheme and a preset genetic algorithm model includes:

[0129] The first resource allocation scheme is subjected to crossover, mutation, and selection processing or one or more of these processes using the preset genetic algorithm model to generate the second resource allocation scheme.

[0130] In its implementation, as mentioned earlier, the Genetic Algorithm (GA) can treat the CDN deployment schemes of multiple sub-controllers as an individual, and then use operations such as crossover, mutation, and selection to generate new individuals, i.e., new CDN deployment schemes (second resource allocation schemes). At the same time, based on the principle of multi-objective optimization, the optimal individual, i.e. the optimal CDN deployment scheme (target resource allocation scheme), is selected as the final output.

[0131] Step 105: Allocate resources to the content delivery network according to the target resource allocation scheme.

[0132] In practical applications, once the optimal CDN deployment scheme is determined, CDN deployment can be carried out according to the optimal CDN deployment scheme to achieve resource allocation of the content delivery network that can meet the actual needs of users.

[0133] The following will combine Figure 5 The embodiments of the present invention will be further described as follows:

[0134] Reference Figure 5 The overall execution flow of the content delivery network resource allocation method provided by this invention can be summarized as follows:

[0135] 1) Deployment of network demand and resource monitoring devices:

[0136] By deploying network demand and resource monitoring devices to network nodes of multiple users, configuring data communication modules, and performing data compression and encryption, the data generated by the network demand and resource monitoring devices can be securely transmitted to the network management system. Furthermore, by establishing a network management system, the device status and data transmission can be monitored.

[0137] 2) Data collection and processing:

[0138] By establishing a cloud storage system, the network data streams of multiple users' network demands and resource monitoring devices are received and stored, thereby generating network data feature values. Normalization and standard deviation scaling methods are used to derive these features. Furthermore, this method utilizes a novel DAE operator to reduce the dimensionality and reconstruct the network data, generating latent vector feature values. This application proposes for the first time an adaptive noise operator to enhance data diversity and robustness, and also proposes for the first time an RL-GA operator to optimize the input data, improving the effectiveness of CDN deployment.

[0139] 3) Construction of a multi-user network database:

[0140] By acquiring network data feature vectors of each user through network demand and resource monitoring devices, and then constructing a network database for each user, this invention proposes to use a clustering algorithm to generate a system-wide network database based on the characteristics of dispersed nodes and regions of network demand and resource monitoring devices, so as to achieve the integration of network data among multiple users.

[0141] 4) Improve the construction of deep reinforcement learning (RL) and genetic algorithm (GA) network architectures:

[0142] This invention optimizes CDN deployment decisions among multiple users by improving the network architecture of deep reinforcement learning (RL) and genetic algorithm (GA). The CDN deployment utility function is used to determine the reward of the algorithm network. Furthermore, the CDN deployment effect and cost are evaluated based on this decision vector. The evaluation results are adjusted using a dynamic weight adjustment principle, ultimately completing the CDN deployment optimization. This invention also proposes a principle for dynamically adjusting weight coefficients; when the update formula is satisfied, the weight coefficients are updated to maximize the CDN deployment utility function.

[0143] The content delivery network (CDN) resource allocation method provided by this invention can cover the network nodes of multiple users by deploying network demand and resource monitoring devices, and achieve the ability to comprehensively consider the network needs and resources of multiple users by leveraging cloud storage and dimensionality reduction reconstruction technologies. This allows network management departments of multiple users to obtain network data of the entire system in a timely manner and optimize CDN deployment in real time, thereby improving network service quality and user experience. Furthermore, by constructing a multi-user network database and introducing a decentralized control structure, it is possible to more flexibly coordinate and balance the CDN deployment effects and costs among multiple users. This is crucial for achieving network resource sharing and maximizing benefits among multiple users, and helps network management departments of multiple users to quickly adjust and optimize CDN deployment strategies.

[0144] Furthermore, this invention introduces network traffic, network latency, network bandwidth, number of users, and user satisfaction as network data features for the first time, and uses the DAE operator method to derive these features. Unlike traditional network data features that typically focus on parameters such as the number, location, capacity, and caching strategies of edge servers, network traffic, network latency, network bandwidth, number of users, and user satisfaction serve as new quantitative characteristics representing network demand and resources. This provides more information for CDN deployment optimization and helps to more accurately evaluate the utility function of CDN deployment. Further, this invention proposes a dynamic weight adjustment principle. This architecture allows real-time collected network data to be input into the algorithm network, optimizing CDN deployment decisions through utility function and weight adjustment. The dynamic weight adjustment principle further improves the adaptability of the utility function, helping to adapt to different network environments and user needs.

[0145] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of the present invention are not limited to the described order of actions, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to the embodiments of the present invention.

[0146] Reference Figure 6 The diagram illustrates a structural schematic of a content delivery network resource allocation device according to some embodiments of the present invention; specifically, it may include the following modules:

[0147] Network data acquisition module 601 is used to acquire network demand data and network resource data of user network nodes in the content delivery network;

[0148] Feature extraction module 602 is used to extract features from the network demand data and network resource data to obtain network data feature values ​​of the user network nodes;

[0149] The first resource allocation scheme generation module 603 is used to determine the utility function of the resource allocation scheme, and generate a first resource allocation scheme based on the utility function of the resource allocation scheme, the network data feature values ​​and the preset deep reinforcement learning algorithm model.

[0150] The target resource allocation scheme determination module 604 is used to generate a second resource allocation scheme based on the first resource allocation scheme and the preset genetic algorithm model, and to determine the target resource allocation scheme based on the first resource allocation scheme and the second resource allocation scheme.

[0151] The resource allocation module 605 is used to allocate resources to the content delivery network according to the target resource allocation scheme.

[0152] In some embodiments of the present invention, the feature extraction module 602 includes:

[0153] The latent vector feature value generation submodule is used to generate latent vector feature values ​​based on the preset dimensionality reduction and reconstruction operator, the network requirement data, and the network resource data.

[0154] The network data feature value determination submodule is used to perform noise enhancement processing on the latent vector feature values ​​according to a preset adaptive noise operator, and obtain the network data feature values ​​of the user network node according to the preset deep reinforcement learning-genetic algorithm operator and the output data of the noise enhancement processing.

[0155] In some embodiments of the present invention, the apparatus further includes:

[0156] The user network database construction module is used to cluster the feature values ​​of the network data to obtain the user network database.

[0157] In some embodiments of the present invention, the first resource allocation scheme generation module 603 includes:

[0158] The resource allocation scheme utility function determination submodule is used to determine the resource allocation scheme utility function based on the network data feature values ​​in the user network database.

[0159] In some embodiments of the present invention, the apparatus further includes:

[0160] The weight coefficient determination module is used to obtain the original resource allocation scheme and determine whether the weight coefficients in the utility function of the resource allocation scheme need to be updated based on the original resource allocation scheme and the second resource allocation scheme.

[0161] The weight coefficient update module is used to update the weight coefficients when it is necessary to update the weight coefficients in the utility function of the resource allocation scheme, and to re-execute the steps of determining the utility function of the resource allocation scheme based on the network data feature values ​​and generating the first resource allocation scheme through a preset deep reinforcement learning algorithm model under the condition that the utility function of the resource allocation scheme is used as the reward function.

[0162] In some embodiments of the present invention, the target resource allocation scheme determination module 604 includes:

[0163] The second resource allocation scheme generation module is used to perform one or more of crossover, mutation and selection processing on the first resource allocation scheme through the preset genetic algorithm model to generate the second resource allocation scheme.

[0164] Some embodiments of the present invention also provide an electronic device, which may include a processor, a memory, and a computer program stored in the memory and capable of running on the processor. When the computer program is executed by the processor, it implements the content delivery network resource allocation method described above.

[0165] Some embodiments of the present invention also provide a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, it implements the above-described content distribution network resource allocation method.

[0166] Some embodiments of the present invention also provide a computer program product, including a computer program that, when executed by a processor, implements the above-described content delivery network resource allocation method.

[0167] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0168] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0169] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, apparatus, or computer program products. Therefore, embodiments of the present invention can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of the present invention can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0170] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxesFigure 1 A device that provides the functions specified in one or more boxes.

[0171] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0172] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0173] Although preferred embodiments of the present invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present invention.

[0174] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes the aforementioned element.

[0175] The above provides a detailed description of a content delivery network resource allocation method, apparatus, device, and medium. Specific examples have been used to illustrate the principles and implementation methods of the invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the invention. At the same time, those skilled in the art will recognize that, based on the ideas of the invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the invention.

Claims

1. A method for allocating resources in a content delivery network, characterized in that, The method includes: Obtain network demand data and network resource data of user network nodes in the content delivery network; Feature extraction is performed on the network demand data and network resource data to obtain the network data feature values ​​of the user network nodes; Based on the network data feature values, a utility function for the resource allocation scheme is determined, and under the condition that the utility function for the resource allocation scheme is used as a reward function, a first resource allocation scheme is generated through a preset deep reinforcement learning algorithm model. Based on the first resource allocation scheme and the preset genetic algorithm model, a second resource allocation scheme is generated, and based on the first resource allocation scheme and the second resource allocation scheme, a target resource allocation scheme is determined. According to the target resource allocation scheme, the content delivery network is allocated resources; The step of extracting features from the network demand data and network resource data to obtain the network data feature values ​​of the user network nodes includes: Based on the preset dimensionality reduction and reconstruction operator, the network demand data, and the network resource data, latent vector feature values ​​are generated. The latent vector feature values ​​are subjected to noise enhancement processing according to a preset adaptive noise operator, and the network data feature values ​​of the user network node are obtained according to the preset deep reinforcement learning-genetic algorithm operator and the output data of the noise enhancement processing. The formula for the preset dimensionality reduction and reconstruction operator is: in, Let N be the value of the i-th feature component, and N represent the number of input features. The current input feature value, The encoding factor used to prevent zero vector input, The latent vector eigenvalues; The formula for the preset adaptive noise operator is: in, Let N be the value of the i-th feature component, and N be the number of input features. The current input feature value, This is an adaptive noise adjustment factor. The output data of the noise enhancement process; The formula for the preset deep reinforcement learning-genetic algorithm operator is: in, Let be the decision variable for the t-th iteration. Let be the objective function value in the t-th iteration. , Both are iteration factors; RL() is a deep reinforcement learning algorithm, and GA() is a genetic algorithm.

2. The method according to claim 1, characterized in that, Before determining the utility function of the resource allocation scheme based on the network data feature values, the method further includes: Clustering is performed on the network data feature values ​​to obtain a user network database.

3. The method according to claim 2, characterized in that, The step of determining the utility function of the resource allocation scheme based on the network data feature values ​​includes: The utility function of the resource allocation scheme is determined based on the network data feature values ​​in the user network database.

4. The method according to claim 1, characterized in that, Before obtaining the target resource allocation scheme based on the first resource allocation scheme and the second resource allocation scheme, the method further includes: Obtain the original resource allocation scheme, and based on the original resource allocation scheme and the second resource allocation scheme, determine whether it is necessary to update the weight coefficients in the utility function of the resource allocation scheme; If it is necessary to update the weight coefficients in the utility function of the resource allocation scheme, update the weight coefficients, and re-execute the steps of determining the utility function of the resource allocation scheme based on the network data feature values, and generating the first resource allocation scheme through a preset deep reinforcement learning algorithm model under the condition that the utility function of the resource allocation scheme is used as the reward function.

5. The method according to claim 1, characterized in that, The step of generating a second resource allocation scheme based on the first resource allocation scheme and a preset genetic algorithm model includes: The first resource allocation scheme is subjected to crossover, mutation, and selection processing or one or more of these processes using the preset genetic algorithm model to generate the second resource allocation scheme.

6. A content delivery network resource allocation device, characterized in that, The device includes: The network data acquisition module is used to acquire network demand data and network resource data of user network nodes in the content delivery network; The feature extraction module is used to extract features from the network demand data and network resource data to obtain the network data feature values ​​of the user network nodes; The first resource allocation scheme generation module is used to determine the utility function of the resource allocation scheme, and generate the first resource allocation scheme based on the utility function of the resource allocation scheme, the network data feature values ​​and the preset deep reinforcement learning algorithm model. The target resource allocation scheme determination module is used to generate a second resource allocation scheme based on the first resource allocation scheme and a preset genetic algorithm model, and to determine the target resource allocation scheme based on the first resource allocation scheme and the second resource allocation scheme. The resource allocation module is used to allocate resources to the content delivery network according to the target resource allocation scheme; The feature extraction module includes: The latent vector feature value generation submodule is used to generate latent vector feature values ​​based on the preset dimensionality reduction and reconstruction operator, the network requirement data, and the network resource data. The network data feature value determination submodule is used to perform noise enhancement processing on the latent vector feature values ​​according to a preset adaptive noise operator, and obtain the network data feature values ​​of the user network node according to a preset deep reinforcement learning-genetic algorithm operator and the output data of the noise enhancement processing. The formula for the preset dimensionality reduction and reconstruction operator is: in, Let N be the value of the i-th feature component, and N represent the number of input features. The current input feature value, The encoding factor used to prevent zero vector input, The latent vector eigenvalues; The formula for the preset adaptive noise operator is: in, Let N be the value of the i-th feature component, and N be the number of input features. The current input feature value, This is an adaptive noise adjustment factor. The output data of the noise enhancement process; The formula for the preset deep reinforcement learning-genetic algorithm operator is: in, Let be the decision variable for the t-th iteration. Let be the objective function value in the t-th iteration. , Both are iteration factors; RL() is a deep reinforcement learning algorithm, and GA() is a genetic algorithm.

7. An electronic device, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program, when executed by the processor, implements the content delivery network resource allocation method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, which, when executed by a processor, implements the content delivery network resource allocation method as described in any one of claims 1 to 5.

Images