Container resource dynamic configuration system based on reinforcement learning

Abstract

The application discloses a container resource dynamic configuration system based on reinforcement learning and relates to the technical field of resource configuration and scheduling.The container resource data in each resource node of a container cloud platform is collected in real time, the collected container resource data is standardized through container resource constraint conditions, the standardized container resource data is processed through a data processing mode after the standardization is completed, the processed container resource data is analyzed through a data analysis mode, a container resource load prediction model is constructed through a model construction mode based on the analyzed container resource data after the analysis is completed, finally, a container resource configuration mechanism is set, and the container resource data in each resource node of the container cloud platform is dynamically configured through the constructed container resource load prediction model, so that the accuracy of container resource data configuration is improved.

Description

Technical Field

[0001] This invention relates to the field of resource allocation and scheduling technology, specifically to a container resource dynamic allocation system based on reinforcement learning. Background Technology

[0002] Containers, with their flexibility, efficiency, and speed, have spurred the rapid development of container clouds, alleviating the conflict between limited computing resources, limited expansion budgets, and surging application demands, and making the high elasticity, high availability, and high utilization characteristics of cloud computing even more prominent. However, due to the relatively short development time and low maturity of container cloud technology, applying containers in large-scale clusters often faces complex resource management issues. Summary of the Invention

[0003] The purpose of this invention is to provide a container resource dynamic configuration system based on reinforcement learning, which solves the problems existing in the background technology.

[0004] To solve the above technical problems, the present invention adopts the following technical solution: The present invention provides a container resource dynamic configuration system based on reinforcement learning, the system including: a container cloud platform, a data acquisition module, a data processing module, a data analysis module, a container resource load prediction module, and a container resource dynamic configuration module; The container cloud platform is used to store container resource data in each resource node; The data acquisition module is used to collect container resource data from each resource node of the container cloud platform in real time. The data processing module is used to process the container resource data collected from each resource node to obtain the processed container resource data. The data analysis module is used to analyze the processed container resource data to obtain the analyzed container resource data; The container resource load prediction module is used to construct a container resource load prediction model through model building. The container resource dynamic configuration module is used to set the container resource configuration mechanism and dynamically configure the container resource data in each resource node of the container cloud platform by constructing a container resource load prediction model.

[0005] Preferably, the data acquisition module is used to collect container resource data from each resource node of the container cloud platform in real time, including the following steps: The container resource data in each resource node of the container cloud platform includes: physical hosts, virtual machines, container resources, and container resource configuration status data; The container resources include: bandwidth, disk resources, CPU, and memory; The container resource constraints include: when a virtual machine is configured on a physical host, the resources provided by the physical host must be higher than the bandwidth, disk resources, CPU, and memory of the virtual machine; when a container is configured on a virtual machine, the resources provided by the virtual machine must be higher than the sum of the container's disk, memory, and bandwidth; when a container is configured on a virtual machine, the virtual machine's CPU resources must be higher than the sum of the CPU workload of all containers; multiple physical hosts cannot be configured with one virtual machine; multiple virtual machines cannot be configured with one virtual machine. The container resource configuration status data includes: configuration status data of each resource node of the container cloud platform; set up Configure the container resource status for the physical host. Configure the container resource status for the virtual machine; in, Indicates the number of virtual machines. Indicates the number of container resources. Represents the i-th physical host. This represents the j-th virtual machine; Standardized container resource data is obtained by aggregating container resource data that meets the container resource constraints.

[0006] Preferably, the data processing module is used to process the container resource data collected from each resource node to obtain the processed container resource data, including the following steps: S21. Filter the standardized container resource data to obtain filtered container resource data; Filtering container resource data based on the Bloom filter algorithm: Create a length of Select the array Each hash function iterates through each piece of data in the container resource data and stores the results in an array; During the traversal using the hash function, if two data entries have the same traversal result, each bit in the two data entries is compared. When the comparison results are consistent, the two data points are set to be the same, and the container resource data that arrives later is deleted based on the arrival time of the task resource. After the traversal is complete, the container resource data stored in the array during the traversal is summarized to obtain the filtered container resource data; S22. Calculate the energy consumption data of the container resource data based on the filtered container resource data; S23. Summarize the energy consumption data of container resource data to obtain the processed container resource data.

[0007] Preferably, the step of calculating the energy consumption data of container resource data based on the filtered container resource data includes the following steps: The energy consumption data mainly includes: bandwidth, hard disk resources, CPU and memory utilization, wherein the utilization of hard disk resources is obtained directly through container resource configuration status data; The formula for calculating CPU utilization is shown below: ; in, This indicates the CPU's idle time. This indicates the total CPU runtime. Indicates CPU utilization; The formula for calculating bandwidth utilization is as follows: Calculate the maximum data transmission rate under the current bandwidth of the data transmission line using Shannon's theorem; ; in, This indicates the maximum data transmission rate for the current road segment. This indicates the bandwidth of the current data transmission line being collected. Indicates data symbols; Record the data transmission rate of resource nodes under the current data transmission line bandwidth, and determine the bandwidth utilization rate based on the calculated data limit transmission rate; ; in, Indicates bandwidth utilization. This indicates the data transmission rate of the first resource node under the corresponding bandwidth of the current data transmission line; Memory utilization is calculated separately for independent tasks and partially reused tasks using distributed computing, yielding the memory processing time for any independent task or partially reused task.

[0008] Preferably, the data analysis module is used to analyze the processed container resource data, and obtaining the analyzed container resource data includes the following steps: S31. Perform stationarity analysis on the processed container resource data using stationarity analysis. S32. Analyze the processed container resource data using load analysis. S33. Summarize the results of load analysis and stability analysis to obtain the analyzed container resource data.

[0009] Preferably, the stationarity analysis of the processed container resource data includes the following steps: The processed container resource data is sorted in chronological order to obtain a time series of container resource data. Based on the time series of container resource data, curves of bandwidth, disk resources, CPU and memory utilization are plotted using image plotting methods. Based on the plotted curves, calculate the mean, maximum, minimum, and inflection points of bandwidth, hard disk resources, CPU, and memory utilization, respectively. The time series characteristics are obtained by summarizing the mean, maximum, minimum, and inflection points of bandwidth, hard disk resources, CPU, and memory utilization. The obtained time series features are tested using a stationarity test to determine whether the obtained time series features are stable. The time series features are calculated using an autoregressive process to determine whether the eigenvalues ​​of the time series data are in the unit cell. When the eigenvalues ​​are in the unit cell, it indicates that the corresponding time series... Stable, otherwise unstable; If the corresponding time series For non-stationary time series, perform d-order differencing to output a stationary time series. By summarizing the container resource data corresponding to the stationary time series, we obtain the container resource data after stationarity analysis.

[0010] Preferably, the analysis of the processed container resource data through load analysis includes the following steps: The container resource data load constraints are as follows: Set container resource data status ; in, =0 indicates that the first virtual machine in the i-th physical host is in an abnormal state. =1 indicates that the first virtual machine in the physical host is configured normally. =0 indicates that the i-th virtual machine has not been configured with container resources. =1 indicates that the i-th virtual machine has been configured with container resources; Calculate the start and end times of the task set executed by the configured container resource nodes; The end time of the task set is estimated based on a distributed computing approach, and a load constraint threshold is set based on the difference between the estimated and actual end times. When the difference between the estimated and actual end time exceeds the set load constraint threshold, it indicates that the currently configured container resource nodes are overloaded; otherwise, they are not overloaded.

[0011] Preferably, the container resource load prediction module is used to construct a container resource load prediction model through model building, including the following steps: S41. Initialize the analyzed container resource data, and set chromosome encoding for the analyzed container resource data based on domain-related knowledge and experience; Construct a population set, and based on the size of the analyzed container resource data, set the population size, crossover rate, mutation rate, and fitness function. And the maximum number of iterations; S42. Based on the rules set in step S41, decode, generate, and determine the decimal values ​​of the three parameters—time step, unit, and prediction step—in the analyzed container resource data. The time step represents the time interval between two adjacent sets of container resource data, and the prediction step represents the time interval between two adjacent sets of predicted container resource data. S43. Determine the neural network structure, and process the analyzed container resource data again using the three parameters of time step, unit and prediction step in the analyzed container resource data to convert the analyzed container resource data into a supervised time series. Based on the three parameters of time step, unit, and prediction step from the analyzed container resource data, the current neural network structure is set, and the weights and biases in the current neural network are initialized. Supervised time series data are divided into training and validation sets, and the container resource data of the corresponding time series in the training set are input into the neural network for training. S44. Initialize the LSTM network; LSTM networks include input gates, output gates, and forget gates; The container resource data corresponding to the time series in the training set is used as the input gate to input the LSTM network; S45. Train the LSTM network using forward computation and backpropagation of error. S46. Input the container resource data of the corresponding time series in the test set into the trained LSTM network, calculate the predicted value of each group of corresponding time series container resource data in the trained LSTM network, and perform differential and normalization restoration operations on it. Then, calculate the mean error based on the real data and the predicted restored data. S47. Use the mean error calculated in step S46 as the fitness function. And based on the fitness function Calculate the fitness value of each individual in the population set; S48. Determine whether the maximum number of iterations has been reached based on the number of iterations of the algorithm. If the maximum number of iterations has been reached, output the container resource load prediction model. If the maximum number of iterations has not been reached, continue to execute steps S41-S47. The output container resource load prediction model is set to the trained LSTM network.

[0012] Preferably, the mean error calculated in step S46 is used as the fitness function. And based on the fitness function Calculating the fitness value of each individual in the population includes the following steps: The optimal chromosome coding is selected from the chromosome coding by fitness value ratio selection; A new population is generated by sequentially crossing over selected superior chromosomes according to a set crossover rate. ; A new population is generated after crossover according to the set mutation rate. Mutation is performed to generate a mutated population. ; Compare the fitness difference between the initial population and the population after crossover mutation using the genetic algorithm. When the fitness difference is less than 0, it means that the fitness of the mutated population is higher than that of the initial population, and the population is accepted. When the fitness difference is greater than or equal to 0, it means that the fitness of the mutated population is lower than that of the initial population, and the population is rejected.

[0013] Preferably, the container resource dynamic configuration module is used to set the container resource configuration mechanism and dynamically configure the container resource data in each resource node of the container cloud platform by constructing a container resource load prediction model, including the following steps: When r different execution tasks are received, the execution tasks will be sorted according to the container resource configuration status of the current resource node; The ways to sort r distinct execution tasks include: Configure the container resource configuration status information matrix Ava[ ,…, ], Maximum Container Resource Matrix for Task Execution (Max[) ,…, ]、All allocated container resource matrix All[ ,…, ], Execution task requirement container resource matrix Need[ ,…, ]; Among them, the container resource configuration status information matrix Ava[ ,…, ]: Each element represents the number of container resources of a certain type; Maximum Container Resource Requirement Matrix for Executing Tasks (Max[) ,…, ]: Indicates the maximum resource requirement for each type of container during task execution; All allocated container resource matrix All[ ,…, ]: Indicates the number of container resources of each type currently allocated to each executing task; Execution task requirement container resource matrix Need[ ,…, ]: Indicates the number of various container resources still needed for each task to be executed; When Ava[ ,…, ] <Need[ ,…, When the message "[]" appears, it indicates that the remaining container resources are insufficient to complete the task. A container resource load prediction model is constructed to dynamically configure the container resource data in each resource node of the container cloud platform. When Ava[ ,…, ]≥Need[ ,…, When [the task is executed], the execution order is calculated to complete the various required tasks.

[0014] The beneficial effects of this invention are as follows: (1) This invention collects container resource data from each resource node of the container cloud platform in real time, and standardizes the collected container resource data through container resource constraints. After standardization, the standardized container resource data is processed through data processing and analyzed through data analysis. After analysis, a container resource load prediction model is built based on the analyzed container resource data. Finally, a container resource configuration mechanism is set, and the container resource data in each resource node of the container cloud platform is dynamically configured by building the container resource load prediction model, thereby improving the accuracy of container resource data configuration.

[0015] (2) The present invention analyzes the processed container resource data through stability analysis and load analysis, which ensures the stability of the processed container resource data, determines the load of the processed container resource data, and improves the rationality of container resource data configuration. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a schematic diagram of the container resource dynamic configuration method of the present invention.

[0018] Figure 2 This is a schematic diagram of the container resource dynamic configuration system module of the present invention. Detailed Implementation

[0019] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the drawings, not all structures. Moreover, unless otherwise specified, the embodiments and features described herein can be combined with each other.

[0020] It should also be noted that, for ease of description, the accompanying drawings show only the parts relevant to the invention and not all of them. Before discussing exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the operations (or steps) as sequential processes, many of the operations can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but it may also have additional steps not included in the drawings. The process may correspond to a method, function, procedure, subroutine, subroutine, etc.

[0021] Example 1 A container resource dynamic allocation system based on reinforcement learning, in its specific implementation, such as... Figure 1 The flowchart illustrates the reinforcement learning-based dynamic configuration method for container resources according to the present invention, which includes the following steps: S1. Collect container resource data from each resource node of the container cloud platform in real time, and standardize the collected container resource data through container resource constraints to obtain standardized container resource data. S2. Process the standardized container resource data using data processing methods to obtain processed container resource data; S3. Analyze the processed container resource data using data analysis methods to obtain the analyzed container resource data; S4. Based on the analyzed container resource data, construct a container resource load prediction model using model building methods; S5. Set up a container resource configuration mechanism and dynamically configure container resource data in each resource node of the container cloud platform by building a container resource load prediction model; Furthermore, referring to Figure 1 As shown, container resource data is collected in real time from each resource node of the container cloud platform, and the collected container resource data is standardized by container resource constraints to obtain standardized container resource data. The process includes the following steps: The container resource data in each resource node of the container cloud platform includes: physical hosts, virtual machines, container resources, and container resource configuration status data; The container resources include: bandwidth, disk resources, CPU, and memory; The container resource constraints include: when a virtual machine is configured on a physical host, the resources provided by the physical host must be higher than the bandwidth, disk resources, CPU, and memory of the virtual machine; when a container is configured on a virtual machine, the resources provided by the virtual machine must be higher than the sum of the container's disk, memory, and bandwidth; when a container is configured on a virtual machine, the virtual machine's CPU resources must be higher than the sum of the CPU workload of all containers; multiple physical hosts cannot be configured with one virtual machine; multiple virtual machines cannot be configured with one virtual machine. The container resource configuration status data includes: configuration status data of each resource node of the container cloud platform; set up Configure the container resource status for the physical host. Configure the container resource status for the virtual machine; in, Indicates the number of virtual machines. Indicates the number of container resources. Represents the i-th physical host. This represents the j-th virtual machine; Furthermore, standardized container resource data is obtained by aggregating container resource data that meets container resource constraints. Furthermore, referring to Figure 1 As shown, the standardized container resource data is processed using data processing methods to obtain processed container resource data, including the following steps: S21. Filter the standardized container resource data to obtain filtered container resource data; Filtering container resource data based on the Bloom filter algorithm: Create a length of Select the array Each hash function iterates through each piece of data in the container resource data and stores the results in an array; During the traversal using the hash function, if two data entries have the same traversal result, each bit in the two data entries is compared. When the comparison results are consistent, the two data points are set to be the same, and the container resource data that arrives later is deleted based on the arrival time of the task resource. After the traversal is complete, the container resource data stored in the array during the traversal is summarized to obtain the filtered container resource data; S22. Calculate the energy consumption data of the container resource data based on the filtered container resource data; The energy consumption data mainly includes: bandwidth, hard disk resources, CPU and memory utilization, wherein the utilization of hard disk resources is obtained directly through container resource configuration status data; The formula for calculating CPU utilization is shown below: ; in, This indicates the CPU's idle time. This indicates the total CPU runtime. Indicates CPU utilization; The formula for calculating bandwidth utilization is as follows: Calculate the maximum data transmission rate under the current bandwidth of the data transmission line using Shannon's theorem; ; in, This indicates the maximum data transmission rate for the current road segment. This indicates the bandwidth of the current data transmission line being collected. Indicates data symbols; Furthermore, the data transmission rate of resource nodes under the current data transmission line bandwidth is recorded, and the bandwidth utilization rate is determined based on the calculated data limit transmission rate; ; in, Indicates bandwidth utilization. This indicates the data transmission rate of the first resource node under the corresponding bandwidth of the current data transmission line; The formula for calculating memory utilization is as follows: The memory processing time of any independent task or partially reused task is obtained by performing calculations on independent tasks and partially reused tasks separately using distributed computing. The distributed computing approach is as follows: ; in, This indicates the processing time, where t represents time. This indicates a function related to processing time. This represents the proportion of reusable computations to the total computational resources in memory. Indicates memory lookup latency. Calculation duration; S23. Summarize the energy consumption data of container resource data to obtain the processed container resource data; Furthermore, referring to Figure 1 As shown, the processed container resource data is analyzed using data analysis methods. The analyzed container resource data includes the following steps: S31. Perform stationarity analysis on the processed container resource data using stationarity analysis. The processed container resource data is sorted in chronological order to obtain a time series of container resource data. Based on the time series of container resource data, curves of bandwidth, disk resources, CPU and memory utilization are plotted using image plotting methods. Furthermore, based on the plotted curves, the mean, maximum, minimum, and inflection points of bandwidth, hard disk resources, CPU, and memory utilization are calculated respectively. Furthermore, the mean, maximum, minimum, and inflection points of bandwidth, hard disk resources, CPU, and memory utilization are summarized to obtain time series characteristics; Furthermore, the obtained time series features are tested using a stationarity test to determine whether the obtained time series features are stable. The time series features obtained are calculated through an autoregressive process to determine whether the eigenvalues ​​of the time series data are in the unit cell. The formula for the autoregressive process is shown below: ; in, This represents the time series data at time t. Represents the autoregressive coefficient. This represents time series data at time tp. Represents a random disturbance term; The formula for the characteristic equation of a time series is shown below: ; in, Let p be the p-th eigenvalue, when This indicates the existence of a unit root autoregressive coefficient sum equal to 1, corresponding to the time series. It is non-stationary, and conversely, it corresponds to a time series. It is stable; If the corresponding time series For non-stationary time series, perform d-order differencing to output a stationary time series. The formula for calculating the d-th order difference is as follows: ; in, This represents a d-order difference operation; Furthermore, the container resource data corresponding to the stationary time series are summarized to obtain the container resource data after stationarity analysis; S32. Analyze the processed container resource data using load analysis. The container resource data load constraints are as follows: Set container resource data status ; in, =0 indicates that the first virtual machine in the i-th physical host is in an abnormal state. =1 indicates that the first virtual machine in the physical host is configured normally. =0 indicates that the i-th virtual machine has not been configured with container resources. =1 indicates that the i-th virtual machine has been configured with container resources; Calculate the start and end times of the task set executed by the configured container resource nodes; Furthermore, the end time of the task set is estimated based on the distributed computing method, and the load constraint threshold is set based on the difference between the estimated and actual end time. When the difference between the estimated and actual end time exceeds the set load constraint threshold, it indicates that the currently configured container resource nodes are overloaded; otherwise, they are not overloaded. S33. Summarize the results of load analysis and stability analysis to obtain the analyzed container resource data; Furthermore, referring to Figure 1 As shown, based on the analyzed container resource data, the construction of a container resource load prediction model through model building includes the following steps: S41. Initialize the analyzed container resource data, and set chromosome encoding for the analyzed container resource data based on domain-related knowledge and experience; Set the individual chromosome encoding length, use binary code, the individual represents a set of analyzed container resource data, and set three bit string intervals to represent the three variables to be optimized respectively; Furthermore, a population set is constructed, and based on the analyzed container resource data scale, the population size, crossover rate, mutation rate, and fitness function are set. And the maximum number of iterations; S42. Based on the rules set in step S41, decode, generate, and determine the decimal values ​​of the three parameters—time step, unit, and prediction step—in the analyzed container resource data. The time step represents the time interval between two adjacent sets of container resource data, and the prediction step represents the time interval between two adjacent sets of predicted container resource data. S43. Determine the neural network structure, and process the analyzed container resource data again using the three parameters of time step, unit and prediction step in the analyzed container resource data to convert the analyzed container resource data into a supervised time series. Furthermore, based on the three parameters of time step, unit, and prediction step in the analyzed container resource data, the current neural network structure is set, and the weights and bias terms in the current neural network are initialized; Furthermore, the supervised time series is divided into a training set and a validation set, and the container resource data of the corresponding time series in the training set is input into the neural network for training; S44. Initialize the LSTM network; LSTM networks include input gates, output gates, and forget gates; The container resource data corresponding to the time series in the training set is used as the input gate to input the LSTM network; S45. Train the LSTM network using forward computation and backpropagation of error. The forward computation means that the container resource data of the corresponding time series is input from the input gate, processed by the forget gate for the input at the current time and the input at the previous time, and finally reaches the output gate. The output gate outputs the container resource data of the corresponding time series after the forget gate is processed, and obtains the predicted container resource data for the next time. The error backpropagation is achieved by setting an error threshold between the predicted container resource data and the collected container resource data. When the error between the predicted container resource data and the collected container resource data exceeds the set error threshold, the weights and biases in the current neural network are adjusted in reverse until the error between the predicted container resource data and the collected container resource data is within the set error threshold range. The iteration stops, and the trained LSTM network is determined and obtained. S46. Input the container resource data of the corresponding time series in the test set into the trained LSTM network, calculate the predicted value of each group of corresponding time series container resource data in the trained LSTM network, and perform differential and normalization restoration operations on it. Then, calculate the mean error based on the real data and the predicted restored data. S47. Use the mean error calculated in step S46 as the fitness function. And based on the fitness function Calculate the fitness value of each individual in the population set; Furthermore, superior chromosome codes are selected from chromosome codes using a fitness value ratio selection method; Furthermore, by sequential crossover according to a set crossover rate, selected superior chromosome codes are crossovered, resulting in a new population. ; Furthermore, a new population is generated after crossover according to the set mutation rate. Mutation is performed to generate a mutated population. ; Furthermore, the fitness difference between the initial population and the population after crossover mutation using the genetic algorithm is compared. When the fitness difference is less than 0, it means that the fitness of the mutated population is higher than that of the initial population, and the population is accepted. When the fitness difference is greater than or equal to 0, it means that the fitness of the mutated population is lower than that of the initial population, and the population is rejected. S48. Determine whether the maximum number of iterations has been reached based on the number of iterations of the algorithm. If the maximum number of iterations has been reached, output the container resource load prediction model. If the maximum number of iterations has not been reached, continue to execute steps S41-S47. The output container resource load prediction model is set to the trained LSTM network; Furthermore, referring to Figure 1 As shown, the process of setting up a container resource configuration mechanism and dynamically configuring container resource data in each resource node of the container cloud platform by building a container resource load prediction model includes the following steps: When r different execution tasks are received, the execution tasks will be sorted according to the container resource configuration status of the current resource node; The ways to sort r distinct execution tasks include: Configure the container resource configuration status information matrix Ava[ ,…, ], Maximum Container Resource Matrix for Task Execution (Max[) ,…, ]、All allocated container resource matrix All[ ,…, ], Execution task requirement container resource matrix Need[ ,…, ]; Among them, the container resource configuration status information matrix Ava[ ,…, ]: Each element represents the number of container resources of a certain type; Maximum Container Resource Requirement Matrix for Executing Tasks (Max[) ,…, ]: Indicates the maximum resource requirement for each type of container during task execution; All allocated container resource matrix All[ ,…, ]: Indicates the number of container resources of each type currently allocated to each executing task; Execution task requirement container resource matrix Need[ ,…, ]: Indicates the number of various container resources still needed for each task to be executed; When Ava[ ,…, ] <Need[ ,…, When the message "[]" appears, it indicates that the remaining container resources are insufficient to complete the task. A container resource load prediction model is constructed to dynamically configure the container resource data in each resource node of the container cloud platform. When Ava[ ,…, ]≥Need[ ,…, When [the task is executed], the execution order is calculated to complete the various required tasks. Example 2 This invention also provides a container resource dynamic configuration system based on reinforcement learning, used to implement a container resource dynamic configuration method based on reinforcement learning. The system includes: a container cloud platform, a data acquisition module, a data processing module, a data analysis module, a container resource load prediction module, and a container resource dynamic configuration module. The container cloud platform is used to store container resource data in each resource node; The data acquisition module is used to collect container resource data from each resource node of the container cloud platform in real time. The data processing module is used to process the container resource data collected from each resource node to obtain the processed container resource data. The data analysis module is used to analyze the processed container resource data to obtain the analyzed container resource data; The container resource load prediction module is used to construct a container resource load prediction model through model building. The container resource dynamic configuration module is used to set the container resource configuration mechanism and dynamically configure the container resource data in each resource node of the container cloud platform by constructing a container resource load prediction model.

[0022] It should be noted that, The above content is merely an example and illustration of the concept of the present invention. Those skilled in the art can make various modifications or additions to the specific embodiments described, or use similar methods to replace them, as long as they do not deviate from the concept of the invention or exceed the scope defined by the present invention, and all such modifications and additions should fall within the protection scope of the present invention.

Claims

1. A container resource dynamic allocation system based on reinforcement learning, characterized in that, include: Container cloud platform, data acquisition module, data processing module, data analysis module, container resource load prediction module, and container resource dynamic configuration module; The container cloud platform is used to store container resource data in each resource node; The data acquisition module is used to collect container resource data from each resource node of the container cloud platform in real time. The data processing module is used to process the container resource data collected from each resource node to obtain the processed container resource data. The data analysis module is used to analyze the processed container resource data to obtain the analyzed container resource data; The container resource load prediction module is used to construct a container resource load prediction model through a model building method. The container resource dynamic configuration module is used to set the container resource configuration mechanism and dynamically configure the container resource data in each resource node of the container cloud platform by constructing a container resource load prediction model.

2. The container resource dynamic allocation system based on reinforcement learning according to claim 1, characterized in that, The data acquisition module is used to collect container resource data from each resource node of the container cloud platform in real time, including the following steps: Container resource data is collected in real time from each resource node of the container cloud platform, and the collected container resource data is standardized by container resource constraints to obtain standardized container resource data. The container resource data in each resource node of the container cloud platform includes: physical hosts, virtual machines, container resources, and container resource configuration status data; The container resources include: bandwidth, disk resources, CPU, and memory; The container resource constraints include: when a virtual machine is configured on a physical host, the resources provided by the physical host must be higher than the bandwidth, disk resources, CPU, and memory of the virtual machine; when a container is configured on a virtual machine, the resources provided by the virtual machine must be higher than the sum of the container's disk, memory, and bandwidth; when a container is configured on a virtual machine, the virtual machine's CPU resources must be higher than the sum of the CPU workload of all containers; multiple physical hosts cannot be configured with one virtual machine; multiple virtual machines cannot be configured with one virtual machine. The container resource configuration status data includes: configuration status data of each resource node of the container cloud platform; set up Configure the container resource status for the physical host. Configure the container resource status for the virtual machine; in, Indicates the number of virtual machines. Indicates the number of container resources. Represents the i-th physical host. This represents the j-th virtual machine; Standardized container resource data is obtained by aggregating container resource data that meets container resource constraints.

3. The container resource dynamic allocation system based on reinforcement learning according to claim 1, characterized in that, The data processing module is used to process the container resource data collected from each resource node, and the processed container resource data includes the following steps: S21. Filter the standardized container resource data to obtain filtered container resource data; Filtering container resource data based on the Bloom filter algorithm: Create a length of Select the array Each hash function iterates through each piece of data in the container resource data and stores the results in an array; During the traversal using the hash function, if two data entries have the same traversal result, each bit in the two data entries is compared. When the comparison results are consistent, the two data points are set to be the same, and the container resource data that arrives later is deleted based on the arrival time of the task resource. After the traversal is complete, the container resource data stored in the array during the traversal is summarized to obtain the filtered container resource data; S22. Calculate the energy consumption data of the container resource data based on the filtered container resource data; S23. Summarize the energy consumption data of container resource data to obtain the processed container resource data.

4. The container resource dynamic allocation system based on reinforcement learning according to claim 3, characterized in that, The calculation of energy consumption data for container resource data based on filtered container resource data includes the following steps: The energy consumption data mainly includes: bandwidth, hard disk resources, CPU and memory utilization, wherein the utilization of hard disk resources is obtained directly through container resource configuration status data; The formula for calculating CPU utilization is shown below: ； in, This indicates the CPU's idle time. This indicates the total CPU runtime. Indicates CPU utilization; The formula for calculating bandwidth utilization is as follows: Calculate the maximum data transmission rate under the current bandwidth of the data transmission line using Shannon's theorem; ； in, This indicates the maximum data transmission rate for the current road segment. This indicates the bandwidth of the current data transmission line being collected. Indicates data symbols; Record the data transmission rate of resource nodes under the current data transmission line bandwidth, and determine the bandwidth utilization rate based on the calculated data limit transmission rate; ； in, Indicates bandwidth utilization. This indicates the data transmission rate of the first resource node under the corresponding bandwidth of the current data transmission line; Memory utilization is calculated separately for independent tasks and partially reused tasks using distributed computing, yielding the memory processing time for any independent task or partially reused task.

5. The container resource dynamic allocation system based on reinforcement learning according to claim 1, characterized in that, The data analysis module is used to analyze the processed container resource data, and the analyzed container resource data includes the following steps: S31. Perform stationarity analysis on the processed container resource data using stationarity analysis. S32. Analyze the processed container resource data using load analysis. S33. Summarize the results of load analysis and stability analysis to obtain the analyzed container resource data.

6. The container resource dynamic allocation system based on reinforcement learning according to claim 5, characterized in that, The process of performing stationarity analysis on the processed container resource data includes the following steps: The processed container resource data is sorted in chronological order to obtain a time series of container resource data. Based on the time series of container resource data, curves of bandwidth, disk resources, CPU and memory utilization are plotted using image plotting methods. Based on the plotted curves, calculate the mean, maximum, minimum, and inflection points of bandwidth, hard disk resources, CPU, and memory utilization, respectively. The time series characteristics are obtained by summarizing the mean, maximum, minimum, and inflection points of bandwidth, hard disk resources, CPU, and memory utilization. The obtained time series features are tested using a stationarity test to determine whether the obtained time series features are stable. The time series features are calculated using an autoregressive process to determine whether the eigenvalues ​​of the time series data are in the unit cell. When the eigenvalues ​​are in the unit cell, it indicates that the corresponding time series... Stable, otherwise unstable; If the corresponding time series For non-stationary time series, perform d-order differencing to output a stationary time series. By summarizing the container resource data corresponding to the stationary time series, we obtain the container resource data after stationarity analysis.

7. The container resource dynamic allocation system based on reinforcement learning according to claim 5, characterized in that, The analysis of the processed container resource data using load analysis includes the following steps: The container resource data load constraints are as follows: Set container resource data status ; in, =0 indicates that the first virtual machine in the i-th physical host is in an abnormal state. =1 indicates that the first virtual machine in the physical host is configured normally. =0 indicates that the i-th virtual machine has not been configured with container resources. =1 indicates that the i-th virtual machine has been configured with container resources; Calculate the start and end times of the task set executed by the configured container resource nodes; The end time of the task set is estimated based on a distributed computing approach, and a load constraint threshold is set based on the difference between the estimated and actual end times. When the difference between the estimated and actual end time exceeds the set load constraint threshold, it indicates that the currently configured container resource nodes are overloaded; otherwise, they are not overloaded.

8. The container resource dynamic allocation system based on reinforcement learning according to claim 1, characterized in that, The container resource load prediction module is used to construct a container resource load prediction model through model building, including the following steps: S41. Initialize the analyzed container resource data, and set chromosome encoding for the analyzed container resource data based on domain-related knowledge and experience; Construct a population set, and based on the size of the analyzed container resource data, set the population size, crossover rate, mutation rate, and fitness function. And the maximum number of iterations; S42. Based on the rules set in step S41, decode, generate, and determine the decimal values ​​of the three parameters—time step, unit, and prediction step—in the analyzed container resource data. The time step represents the time interval between two adjacent sets of container resource data, and the prediction step represents the time interval between two adjacent sets of predicted container resource data. S43. Determine the neural network structure, and process the analyzed container resource data again using the three parameters of time step, unit and prediction step in the analyzed container resource data to convert the analyzed container resource data into a supervised time series. Based on the three parameters of time step, unit, and prediction step from the analyzed container resource data, the current neural network structure is set, and the weights and biases in the current neural network are initialized. Supervised time series data is divided into training and validation sets, and the container resource data of the corresponding time series in the training set is input into the neural network for training. S44. Initialize the LSTM network; LSTM networks include input gates, output gates, and forget gates; The container resource data corresponding to the time series in the training set is used as the input gate to input the LSTM network; S45. Train the LSTM network using forward computation and backpropagation of error. S46. Input the container resource data of the corresponding time series in the test set into the trained LSTM network, calculate the predicted value of each group of corresponding time series container resource data in the trained LSTM network, and perform differential and normalization restoration operations on it. Then, calculate the mean error based on the real data and the predicted restored data. S47. Use the mean error calculated in step S46 as the fitness function. And based on the fitness function Calculate the fitness value of each individual in the population set; S48. Determine whether the maximum number of iterations has been reached based on the number of iterations of the algorithm. If the maximum number of iterations has been reached, output the container resource load prediction model. If the maximum number of iterations has not been reached, continue to execute steps S41-S47. The output container resource load prediction model is set to the trained LSTM network.

9. The container resource dynamic allocation system based on reinforcement learning according to claim 8, characterized in that, The mean error calculated in step S46 is used as the fitness function. And based on the fitness function Calculating the fitness value of each individual in the population includes the following steps: The optimal chromosome codes are selected from the chromosome codes by fitness value ratio selection; A new population is generated by sequentially crossing over selected superior chromosomes according to a set crossover rate. ; A new population is generated after crossover according to the set mutation rate. Mutation is performed to generate a mutated population. ; Compare the fitness difference between the initial population and the population after crossover mutation using the genetic algorithm. When the fitness difference is less than 0, it means that the fitness of the mutated population is higher than that of the initial population, and the population is accepted. When the fitness difference is greater than or equal to 0, it means that the fitness of the mutated population is lower than that of the initial population, and the population is rejected.

10. The container resource dynamic allocation system based on reinforcement learning according to claim 1, characterized in that, The container resource dynamic configuration module is used to set the container resource configuration mechanism and dynamically configure the container resource data in each resource node of the container cloud platform by constructing a container resource load prediction model, including the following steps: When r different execution tasks are received, the execution tasks will be sorted according to the container resource configuration status of the current resource node; The ways to sort r distinct execution tasks include: Configure the container resource configuration status information matrix Ava[ ,…, ], Maximum Container Resource Matrix for Task Execution (Max[) ,…, ]、All allocated container resource matrix All[ ,…, ], Execution task requirement container resource matrix Need[ ,…, ]; Among them, the container resource configuration status information matrix Ava[ ,…, ]: Each element represents the number of container resources of a certain type; Maximum Container Resource Requirement Matrix for Executing Tasks (Max[) ,…, ]: Indicates the maximum resource requirement for each type of container during task execution; All allocated container resource matrix All[ ,…, ]: Indicates the number of container resources of each type currently allocated to each executing task; Execution task requirement container resource matrix Need[ ,…, ]: Indicates the number of various container resources still needed for each task to be executed; When Ava[ ,…, ] <Need[ ,…, When the message "[]" appears, it indicates that the remaining container resources are insufficient to complete the task. A container resource load prediction model is constructed to dynamically configure the container resource data in each resource node of the container cloud platform. When Ava[ ,…, ]≥Need[ ,…, When [the task is executed], the execution order is calculated to complete the various required tasks.

Images