Container starting method and device, equipment, readable storage medium and program product

Abstract

The invention relates to a container starting method, device and equipment, a readable storage medium and a program product, which are applied to a high-performance computing cluster. The method comprises the steps that firstly, a job request of an artificial intelligence task submitted by a target user and sent by a scheduling system is received, then, the job request is analyzed to obtain a container starting command comprising resource limiting parameters, the resource limiting parameters are used for representing resources needed for executing the job request, and finally, the container starting command is sent to the scheduling system. And starting the target container according to the resource limitation parameter so as to process the operation request through the target container. By the adoption of the method, when the high-performance computing cluster runs the artificial intelligence task, the resource limiting parameters in the job request sent by the scheduling system are analyzed, then the target container is started to process the job request according to the resource limiting parameters, and the target container resources can be limited within the scheduling range of the scheduling system; the container is started according to different requirements of different operation requests, and the container starting mode is more flexible.

Description

Technical Field

[0001] The present application relates to the field of cloud computing technology, and in particular to a container startup method, apparatus, device, readable storage medium, and program product. Background Art

[0002] The Supercomputing Internet intelligently schedules various computing resources and deeply integrates computing, software, and application solutions to build an integrated computing service platform. This platform provides efficient and convenient computing services to the public. The computing service platform offers both High Performance Computing (HPC) and Artificial Intelligence (AI) computing services, but these two services are deployed independently. To enable simultaneous execution of both types of workloads on a single cluster, the HPC cluster needs to be expanded to support AI computing services.

[0003] Currently, artificial intelligence operations are generally performed in containers. Integrating artificial intelligence computing services in high-performance computing clusters requires starting containers in the high-performance computing clusters to perform artificial intelligence operations. However, the current container startup method in high-performance computing clusters is not very flexible. Summary of the Invention

[0004] Based on this, it is necessary to provide a container startup method, device, equipment, readable storage medium and program product to address the above technical problems.

[0005] In a first aspect, the present application provides a container startup method, which is applied to a high-performance computing cluster, and the method includes:

[0006] receiving a job request submitted by a target user and sent by a scheduling system, wherein the job request is an artificial intelligence task; the scheduling system runs on the high-performance computing cluster and is used for cluster management and job scheduling of the high-performance computing cluster;

[0007] Parsing the job request to obtain a container startup command, where the container startup command includes a resource restriction parameter, and the resource restriction parameter is used to represent resources required to execute the job request;

[0008] A target container is started according to the resource restriction parameters, so as to process the job request through the target container.

[0009] In the above embodiment, first, a job request for an artificial intelligence task submitted by a target user is received from the scheduling system. Then, the job request is parsed and processed to obtain a container startup command including resource restriction parameters, wherein the resource restriction parameters are used to characterize the resources required to execute the job request. Finally, the target container is started according to the resource restriction parameters to process the job request through the target container. In this way, when it is necessary to run an artificial intelligence task in a high-performance computing cluster, the resource restriction parameters in the job request sent by the scheduling system are parsed, and then the target container is started according to the resource restriction parameters to process the job request. This can limit the target container resources to the scheduling range of the scheduling system, and enable the startup of containers according to the different requirements of different job requests, making the container startup method more flexible.

[0010] In one embodiment, parsing the job request to obtain a container startup command includes:

[0011] The job request is sent to a container encapsulation tool for parsing and processing to determine the resource restriction parameters; and the container startup command is generated according to the resource restriction parameters.

[0012] In the above embodiment, the Docker container is managed through the container encapsulation tool, which improves the security of container startup. The resource limitation parameters are obtained by parsing, and the container startup command is generated according to the resource limitation parameters to start the container, thereby improving the flexibility of container startup.

[0013] In one embodiment, the method further comprises:

[0014] The job status of the job request is monitored; if the job status indicates that the job request is processed, the target container is closed and resources used by the target container are released.

[0015] In the above embodiment, when it is detected that the job request processing is completed, the post-processing script will automatically trigger the closing and cleaning of the Docker container corresponding to the job request, ensuring that the container resources are released in time, which can improve the utilization efficiency of system resources. At the same time, it also simplifies the subsequent management of container jobs in the high-performance computing cluster management process, so that the life cycle of container jobs can be effectively controlled.

[0016] In one embodiment, monitoring the job status of the job request includes:

[0017] The process information of the target container is obtained through a preset command; and the job status of the job request is determined according to the process information.

[0018] In the above embodiment, the job status of the container corresponding to each job request is obtained through the preset command of the Docker container. The execution status of each job request can be determined based on the job status of each container, and the container after execution is closed, so as to realize the management and control of the entire life cycle of the container and improve the utilization rate of system resources.

[0019] In one embodiment, the resource limitation parameter includes at least one of the number of cores, the amount of memory, the disk space, and the number of I / Os.

[0020] In the above embodiment, by extending resource restriction parameters in different dimensions, it is possible to support the management of multiple resources when starting a container, avoid wasting system resources, and thus improve the flexibility and efficiency of container startup.

[0021] In one embodiment, the job request supports interactive jobs and non-interactive jobs.

[0022] In the above embodiment, Socker is extended to support non-interactive commands and non-interactive jobs, thereby ensuring smooth execution of the jobs.

[0023] In a second aspect, the present application further provides a container startup device, which is applied to a high-performance computing cluster, and includes:

[0024] A receiving module is configured to receive a job request submitted by a target user and sent by a scheduling system, wherein the job request is an artificial intelligence task; the scheduling system runs on the high-performance computing cluster and is used for cluster management and job scheduling of the high-performance computing cluster;

[0025] a parsing module, configured to parse the job request to obtain a container startup command, wherein the container startup command includes a resource restriction parameter, and the resource restriction parameter is used to represent resources required to execute the job request;

[0026] A starting module is configured to start a target container according to the resource restriction parameters, so as to process the job request through the target container.

[0027] In a third aspect, the present application further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the container startup method described in any one of the first aspects is implemented.

[0028] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the container startup method described in any one of the first aspects above is implemented.

[0029] In a fifth aspect, the present application further provides a computer program product, comprising a computer program, which, when executed by a processor, implements the container startup method described in any one of the first aspects above.

[0030] The above-mentioned container startup method, device, equipment, readable storage medium and program product are applied to a high-performance computing cluster. First, a job request for an artificial intelligence task submitted by a target user sent by a scheduling system is received. Then, the job request is parsed and processed to obtain a container startup command including resource restriction parameters, wherein the resource restriction parameters are used to characterize the resources required to execute the job request. Finally, the target container is started according to the resource restriction parameters to process the job request through the target container. In this way, when it is necessary to run an artificial intelligence task in a high-performance computing cluster, the resource restriction parameters in the job request sent by the scheduling system are parsed, and then the target container is started according to the resource restriction parameters to process the job request. This can limit the target container resources to the scheduling range of the scheduling system, and realize the startup of containers according to the different requirements of different job requests, making the container startup method more flexible. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in the embodiments of the present application or related technologies, the following briefly introduces the drawings required for use in the embodiments of the present application or related technical descriptions. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying any creative work.

[0032] Figure 1 Schematic diagram of a process for starting a container in one embodiment;

[0033] Figure 2 A flowchart of steps for generating a container startup command in one embodiment is shown;

[0034] Figure 3 Schematic diagram of a process for closing a target container in one embodiment;

[0035] Figure 4 A flowchart of a step of monitoring a job request in another embodiment;

[0036] Figure 5 Schematic diagram of a flow chart of a container startup method according to another embodiment;

[0037] Figure 6 A flowchart of a container management process in one embodiment;

[0038] Figure 7 This is a structural block diagram of a container starting device in one embodiment;

[0039] Figure 8 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0041] The computing power service platform provides high-performance computing (HPC) and artificial intelligence (AI) computing services, but these two services are deployed independently. To enable simultaneous execution of both types of jobs on a single cluster, the HPC cluster needs to be expanded to support AI computing services. Currently, AI jobs are typically performed in containers. Integrating AI computing services into an HPC cluster requires launching containers within the HPC cluster to perform AI jobs. However, the current container startup methods in HPC clusters are not very flexible.

[0042] Docker containers are a lightweight virtualization technology that allows applications and their dependencies to be packaged and run in an isolated environment. Through operating system-level virtualization, they provide a consistent operating environment across different systems, ensuring cross-platform runtime compatibility and portability of applications. Therefore, integrating AI computing services into high-performance computing clusters can enable AI operations using Docker containers. However, traditional Docker container management methods still have some issues. For example, in terms of security, starting a Docker container requires root permissions on the host machine. Therefore, the started container may lead to unauthorized system access or tampering, affecting other tasks. Furthermore, currently, the scheduling system management of started Docker containers is not convenient.

[0043] In view of this, the present application proposes a container startup method, which is applied to a high-performance computing cluster. First, a job request for an artificial intelligence task submitted by a target user sent by a scheduling system is received. Then, the job request is parsed and processed to obtain a container startup command including resource restriction parameters, wherein the resource restriction parameters are used to characterize the resources required to execute the job request. Finally, the target container is started according to the resource restriction parameters to process the job request through the target container. In this way, when it is necessary to run an artificial intelligence task in a high-performance computing cluster, the resource restriction parameters in the job request sent by the scheduling system are parsed, and then the target container is started according to the resource restriction parameters to process the job request. This can limit the target container resources to the scheduling range of the scheduling system, and enable the startup of containers according to the different requirements of different job requests, making the container startup method more flexible.

[0044] In an exemplary embodiment, Figure 1 As shown, a container startup method is provided, which is described by applying the method to a high-performance computing cluster as an example. The high-performance computing cluster can be a system composed of multiple closely interconnected computers that work together to perform large-scale computing tasks in the form of a single system. The container startup method includes the following steps 101 to 103. Among them:

[0045] Step 101: Receive a job request submitted by a target user from a scheduling system.

[0046] The job request is an AI task, and the scheduling system runs on the HPC cluster, responsible for cluster management and job scheduling. The scheduling system schedules computing tasks submitted by users of the HPC cluster, determining the time and resources for task execution. The HPC cluster's scheduling system typically uses the Slurm scheduling system. Slurm (Simple Linux Utility for Resource Management) is an open-source cluster management and job scheduling system with key functions including resource allocation, task scheduling, task management, load balancing, and job dependency management.

[0047] Because HPC services and AI computing services are deployed independently, users needing to execute both tasks simultaneously would need to operate on two different clusters, which is inconvenient. Therefore, the HPC cluster can be expanded to support AI computing services. The HPC cluster receives job requests for AI tasks submitted by target users. Since AI jobs typically run in containers, AI computing services can be implemented by launching containers in the HPC cluster based on the job requests. Furthermore, the scheduling system's job scheduling enables scheduling and management of AI computing services.

[0048] Step 102: parse the job request to obtain a container startup command.

[0049] Among them, the container startup command includes resource limitation parameters, and the resource limitation parameters are used to characterize the resources required to execute the job request. The job request sent by the scheduling system is parsed and processed to obtain the resource limitation parameters carried in the command request, and then the container startup command is generated. Optionally, there are many container management tools, such as Kubernetes, Podman, Rocket, Singularity, etc. This application uses Docker containers as an example for illustration. If artificial intelligence operations are performed through Docker containers, the container startup command can be a command to start a Docker container. The resource limitation parameters in the container startup command can be used to limit the amount of resources occupied by the started Docker container, so that the container resources of the started Docker container can be limited to the management of the scheduling system. As a mainstream container management tool, Docker has a mature ecosystem that includes a large number of images and tool support, and has good consistency and portability.

[0050] Step 103: Start the target container according to the resource restriction parameters to process the job request through the target container.

[0051] Taking the Docker container as an example, the container startup command is sent to the Docker container. The Docker container starts the target container according to the resource limit parameters in the container startup command. The container resources of the target container are determined by the resource limit parameters, thereby starting the target container under the resource limit of the scheduling system to process the job request, realizing the integration of container resources and cluster scheduling system.

[0052] For example, when multiple artificial intelligence job requests are executed on a high-performance computing cluster, the scheduling system can determine the container resources that need to be occupied by the containers that need to be started for multiple artificial intelligence job requests based on the needs of each artificial intelligence job request, and then start different containers according to various resource limit parameters to process multiple artificial intelligence job requests. In this way, the containers started by each artificial intelligence job request do not need to occupy the same resources, avoiding the waste of some resources. Therefore, the container management method is more flexible.

[0053] The above embodiment is applied to a high-performance computing cluster. First, a job request for an artificial intelligence task submitted by a target user and sent by the scheduling system is received. Then, the job request is parsed and processed to obtain a container startup command including resource restriction parameters, wherein the resource restriction parameters are used to characterize the resources required to execute the job request. Finally, the target container is started according to the resource restriction parameters to process the job request through the target container. In this way, when it is necessary to run an artificial intelligence task in a high-performance computing cluster, the resource restriction parameters in the job request sent by the scheduling system are parsed, and then the target container is started according to the resource restriction parameters to process the job request. This can limit the target container resources to the scheduling range of the scheduling system, and enable the startup of containers according to the different requirements of different job requests, making the container startup method more flexible.

[0054] In one embodiment of the present application, the step of parsing the job request in step 102 to obtain the container startup command is as follows: Figure 2 As shown, including:

[0055] Step 201: Send the job request to the container encapsulation tool for parsing and processing to determine resource restriction parameters.

[0056] Currently, when the Slurm scheduling system manages Docker containers, starting the container requires root privileges on the host machine. The started container may lead to unauthorized system access or tampering, affecting other tasks. Therefore, Docker containers can be managed through container encapsulation tools to improve the security of container management.

[0057] Alternatively, the container encapsulation tool can be Socker, which addresses security issues associated with container management, particularly user permission control. By encapsulating Docker, Socker eliminates the need for root permissions on the host machine when launching a Docker container, but allows root permissions within the Docker container. This improves security while ensuring flexibility in container management.

[0058] The job request sent by the scheduling system may include a Socker command. The container encapsulation tool Socker executes the Socker command and parses it to obtain the resource restriction parameters carried in the job request.

[0059] Step 202: Generate a container startup command based on resource restriction parameters.

[0060] Based on the resource limitation parameters obtained from the above analysis, a container startup command for starting the Docker container is generated. When starting Docker, the container resources occupied by the Docker container are determined according to the resource limitation parameters, so as to realize the startup of the target container according to the resource limitation parameters to perform artificial intelligence task operations.

[0061] In the above embodiment, the Docker container is managed through the container encapsulation tool, which improves the security of container startup. The resource limitation parameters are obtained by parsing, and the container startup command is generated according to the resource limitation parameters to start the container, thereby improving the flexibility of container startup.

[0062] In one embodiment, the resource limitation parameter includes at least one of the number of cores, the amount of memory, the disk space, and the number of I / Os.

[0063] Optionally, the resource limit parameter can be the number of cores, that is, the number of cores used by the started target container. The scheduling system allocates the number of available cores to the job request based on the overall scheduling strategy, and carries the number of cores as a resource limit parameter in the job request. After being parsed by the container packaging tool, a container startup command is generated and sent to the Docker container. The Docker container starts the target container according to the number of cores in the container startup command.

[0064] Optionally, resource limit parameters may also include memory usage, i.e., the amount of memory used by the started target container; disk space, i.e., the amount of disk space occupied by the started target container; and I / O quantity, i.e., the number of input and output ports occupied by the started target container. The above resource limit parameters may include one or more, thereby limiting the container resources of the target container from multiple dimensions.

[0065] In the above embodiment, by extending resource restriction parameters in different dimensions, it is possible to support the management of multiple resources when starting a container, avoid wasting system resources, and thus improve the flexibility and efficiency of container startup.

[0066] In the embodiment of the present application, in order to avoid the waste of system resources, such as Figure 3 As shown, the method further includes:

[0067] Step 301: Monitor the job status of the job request.

[0068] Currently, when using the container encapsulation tool Socker to execute container tasks, the container remains running after the task is completed, which leads to a waste of system resources. Therefore, to avoid wasting system resources, by monitoring the job status of job requests, the container resources occupied by the container can be reclaimed after the job request is completed.

[0069] Job status can include, but is not limited to, job progress information, start time, completion time, current progress percentage, execution logs, error messages, and warnings. By accessing this information, users can intuitively view the detailed status of each job, ensuring timely identification and resolution of potential issues. Furthermore, the container system can include a history record function, allowing users to review past job status changes for performance analysis and optimization.

[0070] Step 302: If the job status indicates that the job request processing is completed, close the target container and release the resources used by the target container.

[0071] If the job status indicates that the job request processing is complete, that is, the target container is no longer needed, the target container can be closed and the resources used by the target container can be released. Optionally, a post-processing script can be written in advance. The post-processing script is used to close the container and release the container resources, monitor the job status of the job request, and execute the post-processing script when the job request processing is completed to close the container and release the container resources. Optionally, all related processes are automatically detected and confirmed to have terminated to ensure that there are no residual tasks affecting subsequent operations. The resources used by the target container may include memory resources, storage space resources, or other computing resources. After closing the target container, the memory, storage space and other computing resources are gradually released to optimize the performance and efficiency of the overall container startup. At the same time, the time when the container is closed and the specific information of the resource release, including time and detailed information, can be recorded in the log for easy review during subsequent maintenance.

[0072] If the job status indicates that processing of the job request is not complete, then proceeding to the step of monitoring the job status of the job request.

[0073] In the above embodiment, when it is detected that the job request processing is completed, the post-processing script will automatically trigger the closing and cleaning of the Docker container corresponding to the job request, ensuring that the container resources are released in time, which can improve the utilization efficiency of system resources. At the same time, it also simplifies the subsequent management of container jobs in the high-performance computing cluster management process, so that the life cycle of container jobs can be effectively controlled.

[0074] In one embodiment, the step of monitoring the job status of a job request is as follows: Figure 4 As shown, including:

[0075] Step 401: Obtain process information of the target container through a preset command.

[0076] Optionally, you can monitor the container's process information to determine the container's job execution status. For example, you can use a preset command to obtain the target container's process information. You can use the docker ps -a command to identify all currently started containers, including those that are running and those that have stopped after job execution. Then, use the docker inspect command to obtain detailed information about a specific container, including the process ID. Finally, use the docker top command to view the current process status of the target container, including details such as the process ID, user, start time, CPU usage, memory usage, and command.

[0077] Step 402: Determine the job status of the job request based on the process information.

[0078] Based on the process information of the target container obtained above, the job status of the job request can be determined, that is, whether the target container is running or the job has completed. Optionally, if the CPU usage and memory usage of the target container are zero, that is, the target container is currently stopped, the job has completed. If the CPU usage and memory usage of the target container are not zero, that is, the target container is currently running.

[0079] In the above embodiment, the job status of the container corresponding to each job request is obtained through the preset command of the Docker container. The execution status of each job request can be determined based on the job status of each container, and the container after execution is closed, so as to realize the management and control of the entire life cycle of the container and improve the utilization rate of system resources.

[0080] In one embodiment, the job request supports interactive jobs as well as non-interactive jobs.

[0081] Interactive jobs are tasks that require real-time user participation and feedback during the execution of a job request. For example, during the execution of a job request, the program may prompt the user to select different options or parameters. Interactive jobs are immediate and dynamic, allowing users to make immediate decisions based on the current job situation. Non-interactive jobs are tasks that do not require real-time user participation during the execution of a job request. For example, batch jobs are typically non-interactive; they can run in the background, processing large amounts of data or performing a series of predefined tasks without real-time user intervention. Non-interactive jobs are generally stable and efficient.

[0082] If the container encapsulation tool Socker is used to manage the Docker container, when Socker processes the non-interactive command of a non-interactive job, if the non-interactive command carries extra parameters, Socker will not be able to recognize the command, which will cause the non-interactive command to fail to execute and the container to execute the non-interactive job. Therefore, the Socker function is optimized and expanded according to the execution process of Socker for interactive commands. When Socker starts the non-interactive command, the extra parameters passed by the non-interactive command are identified and parsed. The Docker container command is determined based on the parsing result, and the Docker container command is then sent to the Docker container for execution, so that the container is started to execute the non-interactive job, so that the non-interactive command of the non-interactive job can be executed normally. In this way, no matter whether the job request is an interactive job or a non-interactive job, it can be executed normally, so that the container startup method can support both interactive jobs and non-interactive jobs.

[0083] In the above embodiment, Socker is extended to support non-interactive commands and non-interactive jobs, thereby ensuring smooth execution of the jobs.

[0084] In the embodiments of the present application, Figure 5 As shown, a container startup method is provided, including:

[0085] Step 501: Receive a job request submitted by a target user from a scheduling system.

[0086] Step 502: Send the job request to the container encapsulation tool for parsing and processing to determine resource restriction parameters.

[0087] Step 503: Generate a container startup command based on the resource restriction parameters.

[0088] Step 504: Start the target container according to the resource restriction parameters to process the job request through the target container.

[0089] Step 505: Monitor the job status of the job request.

[0090] Step 506 : If the job status indicates that the job request processing is complete, close the target container and release the resources used by the target container.

[0091] In the embodiments of the present application, Figure 6Figure 2 shows a flow chart of parameter transfer during container startup. A user submits a job to the Slurm scheduling system. Slurm determines the resource limit parameters, such as the number of cores, to allocate when starting the job based on the scheduling scope. It then sends the resource limit parameters to the container encapsulation tool, Socker, via a Socker command. Socker parses the command, obtains the resource limit parameters, generates a container startup command, and sends it to the Docker container, which starts the target container to process the job request. This effectively limits the number of cores used by the target container and brings the resources of the started target container under the management of the Slurm scheduling system.

[0092] At the same time, the job status of the job request is monitored to confirm whether the job has ended. If the job has ended, the post-processing script is executed to close the Docker container corresponding to the job request and clean up the container resources. If the job has not ended, the job status continues to be monitored.

[0093] In the above embodiment, by parsing the Socker command, determining the resource limitation parameters and generating the container startup command, the resources of the started container are strictly limited to the scheduling range of Slurm, ensuring that the container will not exceed the predetermined resource quota. At the same time, the Socker command is extended to correctly identify and process additional parameters when executing non-interactive commands, thereby ensuring that the job can be executed smoothly, and the job request supports interactive jobs and non-interactive jobs. Furthermore, after the container task corresponding to the job request is completed, the relevant Docker container is automatically closed to avoid resource waste. The effective combination of the scheduling system Slurm and the Docker container is achieved, which not only solves the security issues of the Docker container in terms of permission management, but also gives the scheduling system Slurm the comprehensive scheduling and resource management capabilities of the container job, improves the overall efficiency and security of the high-performance computing cluster, and provides a high-performance computing and artificial intelligence integration method for the supercomputing Internet.

[0094] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0095] Based on the same inventive concept, embodiments of the present application also provide a container startup device for implementing the container startup method described above. The solution provided by this device is similar to the solution described in the method described above. Therefore, the specific limitations of one or more container startup device embodiments provided below can be found in the above-described limitations of the container startup method and will not be further elaborated here.

[0096] In an exemplary embodiment, Figure 7 As shown, a container startup device 700 is provided, which is applied to a high-performance computing cluster. The device includes: a receiving module 701, a parsing module 702 and a startup module 703, wherein:

[0097] Receiving module 701, used to receive job requests submitted by target users from the scheduling system. The job requests are artificial intelligence tasks. The scheduling system runs on a high-performance computing cluster and is used for cluster management and job scheduling of the high-performance computing cluster.

[0098] A parsing module 702 is configured to parse the job request to obtain a container startup command, wherein the container startup command includes a resource restriction parameter, which is used to represent the resources required to execute the job request;

[0099] The starting module 703 is configured to start the target container according to the resource restriction parameters, so as to process the job request through the target container.

[0100] In one embodiment, the parsing module 702 is specifically configured to send the job request to a container encapsulation tool for parsing and processing, determine resource restriction parameters, and generate a container startup command based on the resource restriction parameters.

[0101] In one embodiment, the device further includes a monitoring module for monitoring the job status of the job request; if the job status indicates that the job request processing is completed, closing the target container and releasing the resources used by the target container.

[0102] In one embodiment, the monitoring module is specifically configured to obtain process information of the target container through a preset command; and determine a job status of the job request based on the process information.

[0103] In one embodiment, the resource limitation parameter includes at least one of the number of cores, the amount of memory, the disk space, and the number of I / Os.

[0104] In one embodiment, the job request supports interactive jobs as well as non-interactive jobs.

[0105] Each module in the container startup device described above may be implemented in whole or in part through software, hardware, or a combination thereof. Each module may be embedded in or independent of a processor in a computer device in hardware form, or may be stored in a computer device memory in software form, so that the processor can call and execute the corresponding operations of each module.

[0106] In an exemplary embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as shown in FIG. Figure 8 As shown. The computer device includes a processor, a memory, an input / output interface (I / O) and a communication interface. The processor, memory and input / output interface are connected via a system bus, and the communication interface is connected to the system bus via the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store job data. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, a container startup method is implemented.

[0107] Those skilled in the art will understand that Figure 8 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0108] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented: receiving a job request submitted by a target user and sent by a scheduling system, wherein the job request is an artificial intelligence task, and the scheduling system runs on a high-performance computing cluster and is used for cluster management and job scheduling of the high-performance computing cluster; parsing the job request to obtain a container startup command, wherein the container startup command includes resource limitation parameters, and the resource limitation parameters are used to characterize the resources required to execute the job request; and starting the target container according to the resource limitation parameters to process the job request through the target container.

[0109] In one embodiment, when executing the computer program, the processor further implements the following steps: sending the job request to the container packaging tool for parsing and processing to determine resource restriction parameters; and generating a container startup command based on the resource restriction parameters.

[0110] In one embodiment, when executing the computer program, the processor further implements the following steps: monitoring the job status of the job request; if the job status indicates that the job request processing is complete, closing the target container and releasing resources used by the target container.

[0111] In one embodiment, when executing the computer program, the processor further implements the following steps: obtaining process information of the target container through a preset command; and determining the job status of the job request based on the process information.

[0112] In one embodiment, the resource limitation parameter includes at least one of the number of cores, the amount of memory, the disk space, and the number of I / Os.

[0113] In one embodiment, the job request supports interactive jobs as well as non-interactive jobs.

[0114] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the following steps are implemented: receiving a job request submitted by a target user and sent by a scheduling system, where the job request is an artificial intelligence task, and the scheduling system runs on a high-performance computing cluster and is used for cluster management and job scheduling of the high-performance computing cluster; parsing the job request to obtain a container startup command, where the container startup command includes resource limitation parameters, and the resource limitation parameters are used to characterize the resources required to execute the job request; and starting a target container according to the resource limitation parameters to process the job request through the target container.

[0115] In one embodiment, when the computer program is executed by a processor, the following steps are implemented: sending the job request to a container packaging tool for parsing and processing to determine resource restriction parameters; and generating a container startup command based on the resource restriction parameters.

[0116] In one embodiment, when the computer program is executed by a processor, the following steps are implemented: monitoring the job status of the job request; if the job status indicates that the job request processing is complete, closing the target container and releasing resources used by the target container.

[0117] In one embodiment, when the computer program is executed by a processor, the following steps are implemented: obtaining process information of the target container through a preset command; and determining the job status of the job request based on the process information.

[0118] In one embodiment, the resource limitation parameter includes at least one of the number of cores, the amount of memory, the disk space, and the number of I / Os.

[0119] In one embodiment, the job request supports interactive jobs as well as non-interactive jobs.

[0120] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the following steps: receiving a job request submitted by a target user and sent by a scheduling system, where the job request is an artificial intelligence task, and the scheduling system runs on a high-performance computing cluster and is used for cluster management and job scheduling of the high-performance computing cluster; parsing the job request to obtain a container startup command, where the container startup command includes resource limitation parameters, and the resource limitation parameters are used to characterize the resources required to execute the job request; and starting a target container according to the resource limitation parameters to process the job request through the target container.

[0121] In one embodiment, when the computer program is executed by a processor, the following steps are implemented: sending the job request to a container packaging tool for parsing and processing to determine resource restriction parameters; and generating a container startup command based on the resource restriction parameters.

[0122] In one embodiment, when the computer program is executed by a processor, the following steps are implemented: monitoring the job status of the job request; if the job status indicates that the job request processing is complete, closing the target container and releasing resources used by the target container.

[0123] In one embodiment, when the computer program is executed by a processor, the following steps are implemented: obtaining process information of the target container through a preset command; and determining the job status of the job request based on the process information.

[0124] In one embodiment, the resource limitation parameter includes at least one of the number of cores, the amount of memory, the disk space, and the number of I / Os.

[0125] In one embodiment, the job request supports interactive jobs as well as non-interactive jobs.

[0126] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0127] Those skilled in the art will understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. In particular, any reference to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the various embodiments provided herein may be, but are not limited to, general-purpose processors, central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), programmable logic devices (PLDs), quantum computing-based data processing logic devices, artificial intelligence (AI) processors, and the like.

[0128] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0129] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A container startup method, characterized in that: Applied to a high-performance computing cluster, the method includes: receiving a job request submitted by a target user and sent by a scheduling system, wherein the job request is an artificial intelligence task; the scheduling system runs on the high-performance computing cluster and is used for cluster management and job scheduling of the high-performance computing cluster; Parsing the job request to obtain a container startup command, where the container startup command includes a resource restriction parameter, where the resource restriction parameter is used to represent resources required to execute the job request; A target container is started according to the resource restriction parameters, so as to process the job request through the target container.

2. The method according to claim 1, characterized in that The step of parsing the job request to obtain a container startup command includes: Sending the job request to a container encapsulation tool for parsing and processing to determine the resource restriction parameters; Generate the container startup command according to the resource limitation parameters.

3. The method according to claim 1, characterized in that The method further comprises: monitoring the job status of the job request; If the job status indicates that the job request processing is complete, the target container is closed and resources used by the target container are released.

4. The method according to claim 3, characterized in that The monitoring of the job status of the job request includes: Obtaining process information of the target container through a preset command; The job status of the job request is determined according to the process information.

5. The method according to claim 1, characterized in that The resource limitation parameter includes at least one of the number of cores, the amount of memory, the disk space and the number of I / Os.

6. The method according to claim 1, characterized in that The job request supports interactive jobs as well as non-interactive jobs.

7. A container starting device, characterized in that: Applied to a high-performance computing cluster, the device includes: A receiving module, configured to receive a job request submitted by a target user and sent by a scheduling system, wherein the job request is an artificial intelligence task; the scheduling system runs on the high-performance computing cluster and is used for cluster management and job scheduling of the high-performance computing cluster; a parsing module, configured to parse the job request to obtain a container startup command, wherein the container startup command includes a resource restriction parameter, and the resource restriction parameter is used to represent resources required to execute the job request; A starting module is configured to start a target container according to the resource restriction parameters, so as to process the job request through the target container.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images