Docker container scheduling method applied to Slurm job scheduling system

Abstract

The invention discloses a Docker container scheduling method applied to a Slurm job scheduling system, and the method comprises the steps: S1, mapping a super user in a Docker container into a common user on a host machine, and enabling the super user to carry out the limited processing of data in a shared storage system through the authority of the common user; s2, a pre-processing Prolog and a post-processing program Epilog of the Slurm job scheduling system are set in a user-defined mode, and a Docker container is started and closed through a scheduling system service with the super user permission. By mapping the super user in the Docker container into the common user on the host machine, the user in the Docker container can be prevented from maliciously operating or unintentionally damaging the host machine as the super user identity, meanwhile, the data of the super user in the shared storage system is limited, and abuse and out-of-control of storage resources are avoided. According to the method, the Docker container is started and closed by scheduling the system service, so that the situation that a super user permission is directly authorized to a common user can be avoided, the normal use of the Docker container is ensured, and meanwhile, the safety of the system is maintained.

Description

Technical Field

[0001] The present invention relates to the technical field of high-performance cluster job scheduling, and in particular to a Docker container scheduling method applied to a Slurm job scheduling system. Background Art

[0002] In today's high-performance computing field, the Slurm job scheduling system is widely used in various computing clusters due to its excellent resource management and task allocation capabilities. With the rise of container technologies (such as Docker), how to efficiently and securely schedule Docker containers in the Slurm environment has become an important issue.

[0003] As a popular containerization technology, Docker has significant advantages in resource isolation and application deployment, but its permission management mechanism has brought certain troubles to high-performance computing scenarios. Docker stipulates that only super users or users in the Docker user group have the permission to run Docker images. In high-performance computing scenarios, ordinary users do not have super user permissions. In order to run Docker images, ordinary users can only join the Docker user group. But even so, when ordinary users successfully start the container, if they switch to the super user identity in the container to operate, it will cause a series of problems. On the one hand, there are great security risks. Users may deliberately tamper with key configuration files of the host machine, implant malware, etc. for malicious purposes, or accidentally delete important system files or cause system service crashes due to unintentional erroneous operations, thereby damaging the host machine. On the other hand, since the data generated after the user switches to the super user in the Docker container exists as the super user, this will cause the shared storage system of the high-performance cluster to be unable to effectively manage the quota, which will easily lead to the abuse and loss of control of storage resources. Summary of the invention

[0004] In order to solve the problems in the related art, the present invention provides a Docker container scheduling method applied to the Slurm job scheduling system, which can effectively reduce the system security risks and avoid the abuse and loss of control of system storage resources.

[0005] The present invention provides a Docker container scheduling method applied to a Slurm job scheduling system, the scheduling method comprising: S1: mapping a super user in a Docker container to an ordinary user on a host machine, so that the super user performs limited processing of data in a shared storage system with the authority of the ordinary user; S2: customizing the pre-processing Prolog and post-processing program Epilog of the Slurm scheduling system, so that the scheduling system service with super user authority can start and close the Docker container.

[0006] The beneficial effect of this technical solution is that by mapping the super user in the Docker container to an ordinary user on the host machine, it can prevent the user in the Docker container from maliciously operating or unintentionally damaging the host machine as a super user, and at the same time, limit the super user's data in the shared storage system to avoid the abuse and loss of control of storage resources. In addition, the scheduling system service with super user privileges is allowed to start and shut down the Docker container to avoid directly granting super user privileges to ordinary users, thereby ensuring the normal use of the Docker container and maintaining the security of the system.

[0007] Furthermore, the S1 step is specifically as follows: S101: modify the configuration file of the Docker container, add a command to start the user namespace function in the configuration file, so as to build a basic framework based on user namespace isolation; S102: modify the mapping rule of the Docker container configuration file, map the super user in the Docker container to the ordinary user on the host machine; S103: set a clear limit on the available storage capacity of the ordinary user of the mapped host machine in the high-performance cluster shared storage system, and limit the storage scale of the shared data by the ordinary user;

[0008] Furthermore, the S2 step is specifically as follows: S201: performing a custom configuration operation on a pre-processing program Prolog of a Slurm scheduling system, wherein the pre-processing program is configured to set an execution environment that conforms to the Docker container job before the Docker container job is actually executed; S202: starting the Docker container job through a service of the scheduling system with administrator privileges; S203: performing a custom configuration operation on a post-processing program Epilog of the Slurm scheduling system, wherein the post-processing script is configured to clean up the execution environment of the Docker container job after the Docker container job is completed; S204: after the Docker container job is completed, executing a command to close the Docker container through a service of the scheduling system with administrator privileges, and then completely clearing the Docker container instance, releasing resources related to the Docker container and stopping the computing task.

[0009] Furthermore, the S201 step is specifically as follows: the pre-processing program Prolog formulates a processing flow for the Docker container job, and the pre-processing program Prolog checks whether the script submitted by the Docker container job contains the identification variable of the Docker container job; if it contains the identification variable of the Docker container job, then the startup flow of the Docker container job is entered; if it does not contain the identification variable of the Docker container job, then the startup flow of the normal job is entered.

[0010] Furthermore, the S202 step is specifically as follows: the scheduling system first obtains the node number, CPU number, GPU number and memory information to which the task is assigned, passes the node number, CPU number, GPU number and memory information to the startup command parameters of the Docker container, and then executes the startup command of the Docker container.

[0011] Furthermore, the S203 step is specifically as follows: checking through the post-processing program Epilog whether the script submitted by the Docker container job contains the identification variable of the Docker container job; if it contains the identification variable of the Docker container job, entering the exit process of the Docker container job; if it does not contain the identification variable of the Docker container job, entering the exit process of the normal job. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0013] Figure 1 It is a flow chart of the scheduling method of the present invention. DETAILED DESCRIPTION

[0014] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0015] As a popular containerization technology, Docker has significant advantages in resource isolation and application deployment, but its permission management mechanism has brought certain troubles to high-performance computing scenarios. Docker stipulates that only super users or users in the Docker user group have the permission to run Docker images. In high-performance computing scenarios, ordinary users do not have super user permissions. In order to run Docker images, ordinary users can only join the Docker user group. But even so, when ordinary users successfully start the container, if they switch to the super user identity in the container to operate, it will cause a series of problems. On the one hand, there are great security risks. Users may deliberately tamper with key configuration files of the host machine, implant malware, etc. for malicious purposes, or accidentally delete important system files or cause system service crashes due to unintentional erroneous operations, thereby damaging the host machine. On the other hand, since the data generated after the user switches to the super user in the Docker container exists as the super user, this will cause the shared storage system of the high-performance cluster to be unable to effectively manage the quota, which will easily lead to the abuse and loss of control of storage resources.

[0016] In order to solve the problems in the above technology, the present invention provides a Docker container scheduling method applied to the Slurm job scheduling system, which can effectively reduce the system security risks and avoid the abuse and loss of control of system storage resources.

[0017] like Figure 1 As shown:

[0018] The present invention provides a Docker container scheduling method applied to a Slurm job scheduling system. The scheduling method comprises: S1: mapping a super user in a Docker container to an ordinary user on a host machine, so that the super user performs limited processing of data in a shared storage system with the authority of the ordinary user; S2: customizing the pre-processing Prolog and post-processing program Epilog of the Slurm scheduling system, so that the scheduling system service with the super user authority can start and close the Docker container. Users are divided into ordinary user authority and super user (Root) authority. The super user authority is a kind of system authority and is the same concept as the system authority. The super user authority is the highest authority of the entire system, and can start or stop a process, delete or add a user, add or disable hardware, etc. By mapping the super user in the Docker container to an ordinary user on the host machine, it is possible to prevent the user in the Docker container from maliciously operating or unintentionally destroying the host machine as a super user, and at the same time, the super user's data in the shared storage system is limited to avoid the abuse and loss of control of storage resources. By starting and closing the Docker container by scheduling the system service, it is possible to avoid directly authorizing the super user authority to the ordinary user, thereby ensuring the normal use of the Docker container and maintaining the security of the system.

[0019] In an optional embodiment, the S1 step is specifically as follows: S101: modify the configuration file of the Docker container, such as the docker.service file that configures the Docker service, add the --storage-optoverlay2.size=10G parameter in the file, that is, limit the space size of the container system disk to 10G (the specific value can be adjusted according to actual conditions), avoid the container system disk from occupying storage resources without limit, add a command to start the user namespace function in the configuration file, so as to build a basic framework based on user namespace isolation; S102: modify the mapping rule of the Docker container configuration file, map the super user in the Docker container to the ordinary user on the host machine, such as modifying the daemon.json file that configures the Docker service, add "userns-remap":"default" in the file, enable Docker r's user namespace function, and uses the default mapping rules. In this way, when the user in the container switches to root to operate on the host, his permissions will be mapped to ordinary user permissions, thereby limiting his impact on the host system. This is an important security mechanism that can prevent users in the container from maliciously operating or unintentionally damaging the host as root. It also helps to solve the problem that the shared storage system cannot limit the root identity data, because on the host, these data are actually created and managed as ordinary users; S103: Set clear limits on the available storage capacity of the ordinary users of the mapped host in the high-performance cluster shared storage system to limit the storage scale of shared data by the ordinary users; setting limits on these ordinary users in the shared storage system can effectively control the storage amount of the data in the container in the global file system, and avoid excessive data in the global file system after switching to root in the container.

[0020] In an optional embodiment, the S2 step is specifically as follows: S201: performing a custom configuration operation on the pre-processor Prolog of the Slurm scheduling system, the pre-processor is configured to set an execution environment that conforms to the Docker container job before the Docker container job is actually executed, so it can be coded and implemented in Prolog, and a separate processing flow is formulated for the Docker job, that is, checking whether the job submission script contains a container identification variable. If the container identification variable is contained, the Docker container startup flow is entered; otherwise, if it is not contained, the ordinary task startup flow is entered without special processing; S202: starting the Docker container job through the service of the scheduling system with administrator privileges. If it is a Docker container job, the Slurm service first obtains the node number, CPU number and GPU number and memory information assigned to the task, and then passes this information to the docker startup command parameters, for example in the docker In the run command, the corresponding parameters are used to specify the node, CPU, GPU and memory configuration, and finally the command is executed to start the Docker container; S203: a custom configuration operation is performed on the post-processing program Epilog of the Slurm scheduling system. The post-processing script is configured to clean up the execution environment of the Docker container job after the Docker container job is completed. The slurm scheduling system supports a custom post-processing script Epilog, which is used to clean up the job execution environment after the job is completed. Through the slurm post-processing script, it is also checked whether the job submission script contains a container identification variable. If the container identification variable is contained, the Docker container exit process is entered, otherwise, the ordinary task exit process is entered; S204: after the Docker container job is completed, the service of the scheduling system with administrator authority executes a command to close the Docker container, and then completely clears the Docker container instance, releases the resources related to the Docker container and stops the computing task. If it is a Docker container job, the slurm service first executes the docker rm command to close the container, completely clears the container instance, releases the relevant resources, and then stops the computing task, completing the end process of the entire task, ensuring that the system resources are completely recovered and ready for the next task scheduling.

[0021] In an optional embodiment, the S201 step is specifically as follows: the pre-processing program Prolog formulates a processing flow for the Docker container job, and the pre-processing program Prolog checks whether the script submitted by the Docker container job contains the identification variable of the Docker container job; if it contains the identification variable of the Docker container job, then the startup flow of the Docker container job is entered; if it does not contain the identification variable of the Docker container job, then the startup flow of the normal job is entered.

[0022] In an optional embodiment, the S202 step is specifically as follows: the scheduling system first obtains the node number, CPU number, GPU number and memory information to which the task is assigned, passes the node number, CPU number, GPU number and memory information to the startup command parameters of the Docker container, and then executes the startup command of the Docker container.

[0023] In an optional embodiment, the S203 step is specifically as follows: checking through the post-processing program Epilog whether the script submitted by the Docker container job contains the identification variable of the Docker container job; if it contains the identification variable of the Docker container job, entering the exit process of the Docker container job; if it does not contain the identification variable of the Docker container job, entering the exit process of the normal job.

[0024] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the application herein applied. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that the present application does not apply for. The specification and embodiments are considered to be exemplary only, and the true scope and spirit of the present application are indicated by the appended claims.

[0025] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A Docker container scheduling method applied to a Slurm job scheduling system, characterized in that: The scheduling method comprises: S1: Map the super user in the Docker container to a common user on the host machine, so that the super user can process a limited amount of data in the shared storage system with the authority of the common user; S2: Customize the pre-processing Prolog and post-processing Epilog of the Slurm scheduling system, so that the scheduling system service with superuser privileges can start and shut down the Docker container.

2. The method according to claim 1, characterized in that The S1 step is specifically as follows: S101: modifying a configuration file of the Docker container, and adding a command to start the user namespace function in the configuration file, so as to build a basic framework based on user namespace isolation; S102: Modify the mapping rule of the Docker container configuration file to map the super user in the Docker container to the common user on the host machine; S103: setting a clear limit on the available storage capacity of the common user of the mapped host machine in the high-performance cluster shared storage system, so as to limit the storage scale of the shared data by the common user.

3. The method according to claim 1, characterized in that The S2 step is specifically as follows: S201: performing a custom configuration operation on a pre-processing program Prolog of a Slurm scheduling system, wherein the pre-processing program is configured to set an execution environment that conforms to the Docker container job before the Docker container job is actually executed; S202: Starting the Docker container job through the service of the scheduling system with administrator privileges; S203: performing a custom configuration operation on a Slurm scheduling system post-processing program Epilog, wherein the post-processing script is configured to clean up the execution environment of the Docker container job after the Docker container job is completed; S204: After the Docker container job is completed, a command to close the Docker container is executed by the service of the scheduling system with administrator privileges, and then the Docker container instance is completely cleared, the resources related to the Docker container are released, and the computing task is stopped.

4. The method according to claim 3, characterized in that The S201 step is specifically as follows: the pre-processing program Prolog formulates a processing flow for the Docker container job, and the pre-processing program Prolog checks whether the script submitted by the Docker container job contains the identification variable of the Docker container job; if it contains the identification variable of the Docker container job, then the startup flow of the Docker container job is entered; if it does not contain the identification variable of the Docker container job, then the startup flow of the normal job is entered.

5. The method according to claim 3, characterized in that: The S202 step is specifically as follows: the scheduling system first obtains the node number, CPU number, GPU number and memory information to which the task is assigned, passes the node number, CPU number, GPU number and memory information to the startup command parameters of the Docker container, and then executes the startup command of the Docker container.

6. The method according to claim 3, characterized in that The S203 step is specifically as follows: checking, through the post-processing program Epilog, whether the script submitted by the Docker container job contains the identification variable of the Docker container job; if it contains the identification variable of the Docker container job, entering the exit process of the Docker container job; if it does not contain the identification variable of the Docker container job, entering the exit process of the ordinary job.

Images