Container resource allocation method and device based on improved PSO algorithm

Abstract

The invention belongs to the technical field of computers, and particularly discloses a container resource allocation method and device based on an improved PSO algorithm. According to the container resource allocation scheme, firstly, resource allocation schemes of all tasks are regarded as a particle, and a particle swarm is initialized based on different task sorting schemes; taking the total number of computing devices required by all tasks as the adaptive value of the particles; iteratively updating the particle swarm, the local optimal solution and the global optimal solution with the purpose of minimizing the adaptive value until an iteration termination condition is reached; and finally, outputting a globally optimal solution, and allocating resources for each task according to the globally optimal solution to create a container. According to the scheme, the purpose of the highest computing resource utilization rate and the minimum consumption is achieved, optimization iteration is conducted on the resource requirements of the tasks through the improved particle swarm optimization algorithm, the container resource allocation method improves the computing resource utilization rate in the cluster environment, and meanwhile the power consumption of the computing cluster is reduced.

Description

Technical Field

[0001] The present application belongs to the field of computer technology, and more specifically, to a container resource allocation method and device based on an improved PSO algorithm. Background Art

[0002] In a cluster environment, container technology has become the core of modern cloud computing and microservice architecture with its lightweight, portable and fast deployment characteristics. Container technology achieves high consistency across different environments by packaging applications and their dependencies in lightweight containers. These containers can be dynamically scheduled and managed to adapt to changing business needs.

[0003] Although container technology provides many advantages in cluster environments, there are still challenges in resource allocation. Insufficient resource utilization is a common problem, which is mainly manifested in container scheduling and resource allocation strategies. Due to the dynamic and diverse nature of containers, traditional resource allocation methods may not be able to accurately predict and meet the resource requirements of containers, resulting in insufficient resource utilization. Summary of the invention

[0004] In view of the defects of the prior art, the present application proposes a container resource allocation method and device based on an improved PSO algorithm, aiming to solve the problem of insufficient resource utilization of the current container allocation method.

[0005] To achieve the above objectives, in a first aspect, the present application provides a container resource allocation method based on an improved PSO algorithm, the container resource allocation method comprising the following steps: The resource allocation scheme of all tasks is regarded as a particle. Different particles are obtained based on different task sorting, and a particle swarm is composed of multiple different particles. The total number of computing devices required to complete all tasks is taken as the fitness value of the particle; Iteratively update the particle swarm, local optimal solution and global optimal solution with the purpose of minimizing the fitness value until the iteration termination condition is reached; Output the global optimal solution and allocate resources to each task to create a container based on the global optimal solution.

[0006] Preferably, there is Tasks , then the particles are specifically ,in, Indicates tasks and the resources required.

[0007] Preferably, the total number of computing devices required to complete all tasks is used as the fitness value of the particle. Specifically, all tasks in the particle are placed in the computing devices in the order in which they are arranged in the particle, and it is determined in real time whether the remaining resources of the computing devices can accommodate new tasks. If not, new computing devices are enabled to accommodate new tasks until all tasks in the particle are accommodated by the computing devices. Finally, the total number of enabled computing devices is used as the fitness value of the particle.

[0008] Preferably, the particle swarm is iteratively updated, specifically, randomly selecting from the global optimal solution sequence; record the global optimal solution On the order task; to place all particles in the same The order of tasks is replaced into the global optimal solution The same order of tasks is obtained, thus a new particle swarm is obtained.

[0009] Preferably, the local optimal solution and the global optimal solution are iteratively updated, specifically, by comparing the fitness values ​​of all particles in the particle swarm, taking the particle with the smallest fitness value as the current local optimal solution, comparing the fitness values ​​of the global optimal solution and the current local optimal solution, and taking the particle with the smallest fitness value as the current global optimal solution.

[0010] Preferably, resources are allocated to each task according to the global optimal solution to create a container. Specifically, according to the resource allocation plan of all tasks in the global optimal solution, computing devices are turned on to allocate resources to create corresponding containers, and it is determined in real time whether the remaining resources in the turned-on computing devices can create a new container. If not, a new computing device is turned on to create a new container, until corresponding containers are created for all tasks in the global optimal solution, and the corresponding tasks are deployed in the containers. Preferably, different particles are obtained based on different task sorting, and a particle group is composed of multiple different particles. Specifically, the order of each task in the particle is randomly disrupted to obtain new particles with different task orders, New particles form a particle swarm.

[0011] Preferably, different particles are obtained based on different task sorting, and a particle group is composed of multiple different particles. Specifically, tasks are sorted in descending or ascending order of the resources required for the tasks to obtain new particles, among which is the type of resources required; randomly shuffle the order of each task in the particle and get new particles with different task orders, and finally New particles form a particle swarm.

[0012] Preferably, it is characterized in that the resources include memory, GPU and video memory, and the GPU resources and video memory resources are divided into different containers through GPU virtualization technology.

[0013] In a second aspect, the present application provides an electronic device comprising: at least one memory for storing programs; and at least one processor for executing the programs stored in the memory. When the program stored in the memory is executed, the processor is used to execute the method described in the first aspect or any possible implementation of the first aspect.

[0014] In a third aspect, the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a processor, the processor executes the method described in the first aspect or any possible implementation of the first aspect.

[0015] In a fourth aspect, the present application provides a computer program product. When the computer program product runs on a processor, the processor executes the method described in the first aspect or any possible implementation of the first aspect.

[0016] In general, the above technical solutions conceived by this application have the following beneficial effects compared with the prior art: (1) The container resource allocation method of the present application is task-oriented. A particle swarm is designed based on the requirements of all tasks for various resources. The optimization goal is to maximize the utilization rate of computing resources and minimize the consumption. An improved particle swarm optimization algorithm is used to iteratively optimize the resource requirements of the tasks. Finally, the optimized resource requirements are used to create a container to execute the task. The container resource allocation method of the present application improves the resource utilization rate in the cluster environment and reduces the power consumption of the computing cluster.

[0017] (2) When updating all particles in the population, the present application will refer to the partial task sorting scheme of the global optimal particle to update the particles, thereby improving the convergence speed and optimization accuracy in the optimization iterative algorithm.

[0018] (3) The resource allocation method of this application combines GPU virtualization technology to share GPU resources with multiple containers to complete tasks, thereby further improving resource utilization, flexibility and scalability of the cluster system. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is one of the flow charts of a container resource allocation method based on an improved PSO algorithm provided in an embodiment of the present application.

[0020] Figure 2 This is the second flow chart of a container resource allocation method based on an improved PSO algorithm provided in an embodiment of the present application.

[0021] Figure 3 It is a schematic diagram of a computing device for task allocation provided by an embodiment of the present application.

[0022] Figure 4 This is a particle update schematic diagram provided in an embodiment of the present application.

[0023] Figure 5 It is a schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0025] The terms "first", "second", etc. in the specification and claims herein are used to distinguish different objects rather than to describe a specific order of objects. For example, a first computing device and a second computing device are used to distinguish different computing devices rather than to describe a specific order of computing devices.

[0026] In the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0027] In the description of the embodiments of the present application, unless otherwise specified, "plurality" means two or more than two. For example, a plurality of containers means two or more than two containers, etc.; a plurality of particles means two or more than two particles, etc.

[0028] First, the technical terms involved in the embodiments of the present application are introduced.

[0029] PSO, particle swarm optimization algorithm; is an optimization algorithm based on swarm intelligence, inspired by the foraging behavior of bird flocks and fish schools. In the algorithm, each particle represents a potential solution, and it finds the optimal solution to the problem by imitating swarm behavior.

[0030] The embodiments of the present application are described below in conjunction with the drawings in the embodiments of the present application.

[0031] Embodiment 1: like Figure 1 As shown in the figure, it is a flowchart of a container resource allocation method based on an improved PSO algorithm in Example 1 of the present application, as shown in the figure, which specifically includes the following steps: (1) Design of single particles and particle swarms based on task-based resource allocation: The resource allocation scheme of all tasks is regarded as a particle, and Tasks , then the particles are specifically ,in, Indicates tasks and the resources required.

[0032] Different particles are obtained based on different task sorting, and a particle swarm is composed of multiple different particles: Determine the number of particles in the particle swarm as , randomly disrupt the order of each task in the particle, by Different rankings are obtained A new particle, New particles form a particle group; Or, determine the number of particles in the particle swarm as , sort the tasks in descending or ascending order of the resources required for the tasks, and get new particles, among which is the type of resources required; randomly shuffle the order of each task in the particle, Different rankings are obtained new particles, and finally by New particles form a particle swarm.

[0033] (2) Define the fitness value of the particle: The total number of computing devices required for all tasks is taken as the fitness value of the particle. Specifically, all tasks in the particle are placed in the computing device in the order in which they are arranged in the particle, and it is determined in real time whether the remaining resources of the computing device can accommodate the new task. If not, a new computing device is opened to accommodate the new task until all tasks in the particle are accommodated by the computing device. The total number of computing devices opened is finally taken as the fitness value of the particle.

[0034] (3) Iterative optimization to obtain the global optimal solution: Iteratively update the particle swarm, local optimal solution and global optimal solution with the purpose of minimizing the fitness value until the iteration termination condition is reached and the global optimal solution is output; Compare the fitness values ​​of all particles in the particle swarm, take the particle with the smallest fitness value as the current local optimal solution, compare the fitness values ​​of the global optimal solution and the current local optimal solution, and take the particle with the smallest fitness value as the current global optimal solution.

[0035] Randomly select from the global optimal solution sequence; record the global optimal solution On the order task; to place all particles in the same The order of tasks is replaced into the global optimal solution The same order of tasks is obtained, thus a new particle swarm is obtained.

[0036] (4) Allocate resources based on the global optimal solution: Output the global optimal solution, and allocate resources to each task and create a container based on the global optimal solution. Specifically, according to the resource allocation plan of all tasks in the global optimal solution, enable computing devices to allocate resources and create corresponding containers, and determine in real time whether the remaining resources in the enabled computing devices can create new containers. If not, enable new computing devices to create new containers until all tasks in the global optimal solution have created corresponding containers and deployed the corresponding tasks in the containers. The resources include memory, GPU and video memory, and GPU resources and video memory resources are divided into different containers through GPU virtualization technology.

[0037] Embodiment 2: Embodiment 2 of the present application is a method for allocating container resources based on an improved PSO algorithm, which specifically includes the following steps: S100, virtualizes the computer’s GPU resources; Deploy vGPU management components in the computing cluster, virtualize the GPU resources of each computer, and create a container image that can call virtualized GPU resources; S200, initialize the task list and population; S201, according to the task requirements and the actual situation of the computing device, set resource tags of memory, GPU computing power, and GPU video memory for N tasks and M computing devices, where the task is consumption and the computing device is a resource pool; S202, arrange N tasks (task numbers are 1, 2, ..., N) in descending order according to the three resource tags to obtain three initial particles; randomly shuffle the order of the tasks n-3 times to obtain n-3 initial particles, all of which have a length of N, representing the placement order of the N tasks; S300, calculate the fitness value of the population, update the particle swarm, and the local optimal solution M lbest and the global optimal solution M gbest ; S301, according to the task order of the particles, using a greedy algorithm to calculate the number of computing devices required for each particle, that is, the fitness value corresponding to each particle; S302, the algorithm goal is set to minimize the number of computing devices, that is, the particle with the smallest fitness value is selected as the local optimal solution of this population; S303, comparing the local optimal solution and the global optimal solution, and taking the smaller value as the global optimal solution.

[0038] S400, determine whether the termination condition is met, if so, output the global optimal solution and enter S500, if not, update the population and enter S300 to continue iteration; S401, if the number of iterations reaches the maximum, the global optimal solution is output and the process proceeds to S500; S402, if the number of iterations has not reached the maximum, randomly select p positions in the global optimal particle, record the p tasks on the p positions of the global optimal particle; replace the positions of the same p tasks in all particles with the same positions as the p tasks in the global optimal solution, thereby obtaining a new particle group; enter S300 and continue iteration; S500, scheduling computing devices and containers according to the global optimal solution; S501, based on the global optimal solution, the cluster management node controls M gbest A computing device is turned on; S502, the cluster creates N resource-limited containers based on the container image that supports GPU resource virtualization according to the resource requirements of the N tasks, and starts the corresponding tasks; S503, deploy the N startup task containers to M gbest computing device.

[0039] Embodiment 3: like Figure 2 As shown, it is a flowchart of a container resource allocation method based on an improved PSO algorithm, which is Example 3 of this application. In this implementation case, 30 computing devices of the same type are equipped. The device uses Feiteng D2000 / 8 as the main CPU, the memory is 32GB DDR4, and is equipped with a GPU card of Tianshu MR100. The device has a BMC and can be remotely controlled to start and stop the machine. There are 50 lightweight tasks to be run in the cluster. Set the population size n=200 of the improved PSO algorithm, and the number of iterations E=80. As shown in the figure, the specific steps include: S100, virtualizes the computer’s GPU resources; S101, deploy the k8s cluster management environment in each computing device, deploy the vGPU management component in the cluster, virtualize the computing power and video memory of each GPU card, with a computing power granularity of 1% and a video memory granularity of 128MB.

[0040] S102, based on the centos basic image, install the GPU card software stack driver and the dependencies required for the task, ensure that all tasks can be run in the image, package the image and upload it to the k8s image warehouse; S103, number 30 computing devices in order o j , j is 1~30, put the total memory o j.mem 32GB, total computing power o j .gpu 100%, total video memory o j .gmem 32GB label; S104, number the 50 tasks in order r i =i, i is 1~50; according to the task requirements, it is r i Type the required memory i .mem, required computing power r i .gpu, required video memory i .gmem requirement tags; S200, initialize the task list and population; S201, r i According to r i .mem, r i .gpu、r i Arrange the size of .gmem in descending order to get three lists containing task numbers from 1 to 50. The lists are particles. At this time, there are three particles: q1, q2, and q3. S202, randomly shuffle the 50 tasks 197 times to obtain initial random particles q4, q5, ..., q 200 ; S300, calculating the fitness value of the population and updating the local and global optimal solutions; S301, calculate the fitness values ​​of 200 particles in the population, such as Figure 3 As shown, according to the list order of the particles, the task containers are placed in the enabled computing devices one by one. When a task container cannot be placed, a new computing device is opened. After all tasks are placed, the number of enabled computing devices is the fitness value of the particle. S302, compare the fitness values ​​of all particles in the population, and take the smallest one as the current local optimal M lbest , M lbest and the global optimal M gbest Compare and take the smaller value as the current global optimal value, and add 1 to the number of iterations; S400, determine whether the termination condition is met, if so, output the global optimal solution and enter S500, if not, update the population and enter S300 to continue iteration; S401, determine whether the number of iterations reaches the maximum, if yes, output the particle corresponding to the global optimal solution, and enter step S500; S402: If the number of iterations has not reached the maximum, randomly select 5 numbers p between 1 and 50. m , as the updated position, such as Figure 4 As shown, the 5 numbers include 2, set M gbest The task number r5 corresponding to position 2 is the update task number.i Find the position 26 of the r5 task number in the particle and put q i The task numbers at position 26 and position 2 of the particle are swapped to obtain the updated particle.

[0041] S403, update all particles in the population, enter step S300, and continue iteration; S500, scheduling computing devices and containers according to the global optimal solution.

[0042] S501, according to the output global optimal solution Mgbest, the computing device BMC is controlled by the cluster management node to start the Mgbest computing devices; S502: Create and start a container in a computing device according to the resource allocation method corresponding to the global optimal solution, and deploy and run the corresponding task application.

[0043] Based on the method in the above embodiment, the embodiment of the present application provides an electronic device, such as Figure 5 As shown, it includes: a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus. The processor can call the logic instructions in the memory to execute the method in the above embodiment.

[0044] In addition, the logic instructions in the above-mentioned memory can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application.

[0045] Based on the method in the above embodiment, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program runs on a processor, the processor executes the method in the above embodiment.

[0046] Based on the method in the above embodiment, an embodiment of the present application provides a computer program product. When the computer program product runs on a processor, the processor executes the method in the above embodiment.

[0047] It is understandable that the processor in the embodiment of the present application may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor.

[0048] The method steps in the embodiments of the present application can be implemented by hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, and the software modules can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC.

[0049] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions may be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state drive (SSD)), etc.

[0050] It should be understood that the various numerical numbers involved in the embodiments of the present application are only used for the convenience of description and are not used to limit the scope of the embodiments of the present application.

[0051] It will be easily understood by those skilled in the art that the above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A container resource allocation method based on an improved PSO algorithm, characterized in that: The container resource allocation method comprises the following steps: The resource allocation scheme of all tasks is regarded as a particle. Different particles are obtained based on different task sorting, and a particle swarm is composed of multiple different particles. The total number of computing devices required to complete all tasks is taken as the fitness value of the particle; Iteratively update the particle swarm, local optimal solution and global optimal solution with the purpose of minimizing the fitness value until the iteration termination condition is reached; Output the global optimal solution and allocate resources to each task to create a container based on the global optimal solution.

2. The container resource allocation method according to claim 1, characterized in that: Features Tasks , then the particles are specifically ,in, Indicates tasks and the resources required.

3. The container resource allocation method according to claim 1, characterized in that: The total number of computing devices required to complete all tasks is taken as the fitness value of the particle. Specifically, all tasks in the particle are placed into the computing devices in the order in which they are arranged in the particle, and it is determined in real time whether the remaining resources of the computing devices can accommodate new tasks. If not, new computing devices are opened to accommodate new tasks until all tasks in the particle are accommodated by the computing devices. Finally, the total number of computing devices opened is taken as the fitness value of the particle.

4. The container resource allocation method according to claim 1, characterized in that: Iteratively update the particle swarm, specifically, randomly select from the global optimal solution sequence; record the global optimal solution On the order task; to place all particles in the same The order of tasks is replaced into the global optimal solution The same order of tasks is obtained, thus a new particle swarm is obtained.

5. The container resource allocation method according to claim 1, characterized in that: Iteratively update the local optimal solution and the global optimal solution. Specifically, compare the fitness values ​​of all particles in the particle swarm, take the particle with the smallest fitness value as the current local optimal solution, compare the fitness values ​​of the global optimal solution and the current local optimal solution, and take the particle with the smallest fitness value as the current global optimal solution.

6. The container resource allocation method according to claim 1, characterized in that: Allocate resources to create containers for each task according to the global optimal solution. Specifically, according to the resource allocation plan of all tasks in the global optimal solution, start the computing device to allocate resources to create the corresponding container, and judge in real time whether the remaining resources in the turned-on computing device can create a new container. If not, start a new computing device to create a new container until all tasks in the global optimal solution have created corresponding containers and deploy the corresponding tasks in the container.

7. The container resource allocation method according to claim 1, characterized in that: Different particles are obtained based on different task sorting, and a particle group is composed of multiple different particles. Specifically, the order of each task in the particle is randomly disrupted to obtain new particles with different task orders, New particles form a particle swarm.

8. The container resource allocation method according to claim 1, characterized in that: Different particles are obtained based on different task sorting, and a particle group is composed of multiple different particles. Specifically, tasks are sorted in descending or ascending order of the resources required for the tasks to obtain new particles, among which is the type of resources required; randomly shuffle the order of each task in the particle and get new particles with different task orders, and finally New particles form a particle swarm.

9. The container resource allocation method according to claim 1, 2, 3, 6 or 8, characterized in that: The resources include memory, GPU and video memory, and GPU resources and video memory resources are divided into different containers through GPU virtualization technology.

10. An electronic device, characterized in that: include: at least one memory for storing a computer program; At least one processor is used to execute the program stored in the memory. When the program stored in the memory is executed, the processor is used to execute the method according to any one of claims 1 to 9.

Images