A cluster task container management method and system

Abstract

A cluster task container management method is applied to a supercomputing cluster, the supercomputing cluster comprising a management node and a plurality of computing nodes, the method comprising: constructing a lightweight state database to record a task full life cycle state, the life cycle comprising a submission, scheduling, container startup, port mapping and recycling stage; designing a bitmap-based port pool mechanism to generate an available port sequence according to a fixed step length, and ensuring the concurrent uniqueness of port application through memory locking; establishing an SSH tunnel on the management node to forward a local port to an SSH port of a target computing node; implementing container life cycle management to periodically scan the tunnel state, and automatically triggering container state checking and tunnel reconstruction when an exception occurs; and writing all state events into the lightweight state database through an API, the lightweight state database supporting task query and connection health check. The method can support task-level state persistent recording, automatic port scheduling and recovery, and improve platform stability and resource utilization efficiency.

Description

Technical Field

[0001] This invention relates to the field of high performance computing (HPC) technology, and in particular to a cluster task container management method and system. Background Technology

[0002] In the context of the rapid development of high-performance computing (HPC) and artificial intelligence platforms, container technology (such as Singularity) is widely used in scientific computing environments to achieve software dependency isolation and convenient task deployment. However, in cluster environments with multi-user, concurrency, and high-frequency job scheduling, traditional container management methods face the following main problems:

[0003] Port conflict is a serious problem: When multiple container tasks concurrently request host ports for remote communication (such as SSH tunnel forwarding), port resources cannot be allocated efficiently, which can easily lead to conflicts or port leaks.

[0004] Lack of task status tracking mechanism: Traditional script-based container scheduling methods cannot persistently record task status and lifecycle events, making subsequent visualization, diagnosis and recovery impossible;

[0005] Container access path configuration is complex: users need to manually configure port forwarding, local mapping, jump host connection and other steps, which has a high threshold for use and a high error rate.

[0006] Lack of failure recovery mechanism: Once a container exits abnormally or port mapping is disconnected, the system often cannot detect and automatically repair it, which seriously affects task continuity and platform availability. Summary of the Invention

[0007] To address the problems existing in the prior art, embodiments of this application provide a method, system, computing device, computer storage medium, and product containing computer programs for cluster task container management, which can support persistent recording of task-level status, automatic port scheduling and recovery, and improve platform stability, user experience, and resource utilization efficiency.

[0008] In a first aspect, embodiments of this application provide a cluster task container management method applied to a supercomputing cluster, the supercomputing cluster including a management node and multiple computing nodes, comprising: constructing a lightweight state database to record the full lifecycle state of a task, the lifecycle including submission, scheduling, container startup, port mapping, and recycling phases; designing a bitmap-based port pool mechanism to generate a discretely distributed sequence of available ports at fixed steps, and ensuring the concurrent uniqueness of port requests through memory locks; establishing an SSH tunnel on the management node to forward local ports to the SSH ports of the target computing node; implementing container lifecycle management, periodically scanning the tunnel status, and automatically triggering container status checks and tunnel reconstruction when an anomaly occurs; and writing all status events into the lightweight state database through an API, the lightweight state database supporting task queries and connection health checks.

[0009] In some possible implementations, the lightweight state database is implemented using PyDbLite. The task record uses the task's UUID as the primary key and clusterName_user_jobId as the business index, and includes user information, node information, job status, container status, and port mapping status fields.

[0010] In some possible implementations, the memory lock in the port pool mechanism is generated using the MD5 hash algorithm, with the hash input including the cluster name, username, job ID, node IP, and port.

[0011] In some possible implementations, the SSH tunnel is established via a two-layer jumper, including: connecting to the cluster login node via the Paramiko library; passing through from the login node to the target compute node, and creating an encrypted channel using Ed25519 key authentication and the open_channel method.

[0012] In some possible implementations, container lifecycle management is based on the Singularity runtime, and container instance naming follows the rule of service_auto_ <baseport>At startup, the overlay file system, mount path, and environment variables are dynamically injected.

[0013] In some possible implementations, container status checks and tunnel reconstruction are automatically triggered when an exception occurs, including: when an SSH tunnel interruption is detected, querying the lightweight state database to obtain the associated container status; if the container is alive, automatically reconstructing the tunnel; if the container fails, marking an exception and releasing port resources.

[0014] In some possible implementations, the method is compatible with Slurm or Kubernetes scheduling systems, with the lightweight state database deployed as a standalone module without modifying the original scheduling logic of Slurm or Kubernetes.

[0015] Secondly, embodiments of this application provide a cluster task container management system deployed on a supercomputing cluster. The supercomputing cluster includes a management node and multiple computing nodes, comprising: a lightweight state database module for recording the entire lifecycle state of a task, including state information for the submission, scheduling, container startup, port mapping, and recycling stages; a remote jump board connection module for establishing a secure communication channel with the computing nodes through a dual-layer SSH jump board mechanism; a container lifecycle control module for implementing lifecycle management of container startup, shutdown, and status monitoring based on the Singularity runtime; a port scheduling and forwarding module for allocating ports and establishing SSH tunnels through a bitmap mechanism to achieve local-to-container access forwarding; and a port lifecycle monitoring and anomaly recovery mechanism module for periodically detecting port and tunnel status and automatically triggering a recovery process when an anomaly occurs.

[0016] Thirdly, embodiments of this application provide a computer-readable storage medium including computer-readable instructions that, when read and executed by a computer, cause the computer to perform the method as described in any of the first aspects.

[0017] Fourthly, embodiments of this application provide a computing device, including a processor and a memory, wherein the memory stores computer program instructions, which, when executed by the processor, perform the method as described in any of the first aspects.

[0018] Fifthly, embodiments of this application provide a product comprising a computer program that, when the computer program product is run on a processor, causes the processor to perform the method as described in any of the first aspects. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a schematic diagram of a cluster task container management architecture provided in an embodiment of this application;

[0021] Figure 2 This is a flowchart illustrating a cluster task container management method provided in an embodiment of this application;

[0022] Figure 3 This is a schematic diagram of the overall process of cluster task execution provided in an embodiment of this application. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] In this article, the term "and / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent three cases: A exists alone, A and B exist simultaneously, and B exists alone. The symbol " / " in this article indicates that the related objects are in an "or" relationship, for example, A / B means A or B.

[0025] The terms "first" and "second," etc., used in the specification and claims herein are used to distinguish different objects, not to describe a specific order of objects. For example, "first response message" and "second response message," etc., are used to distinguish different response messages, not to describe a specific order of response messages.

[0026] In the embodiments of this application, the words "exemplary" or "for example" are used to indicate that they are examples, illustrations, or descriptions. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design options. Specifically, the use of the words "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0027] In the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more, for example, multiple processing units means two or more processing units, multiple elements means two or more elements, etc.

[0028] To facilitate understanding of the embodiments of this application, the following will provide further explanation and description with reference to the accompanying drawings and specific embodiments. These embodiments do not constitute a limitation on the embodiments of the present invention.

[0029] For example, Figure 1 This illustration shows a schematic diagram of a cluster task container management architecture provided in an embodiment of this application. The system is deployed in a supercomputing cluster, which includes a management node and multiple computing nodes. Figure 1 As shown, the cluster task container management architecture includes a lightweight state database module, a remote jump board connection module, a container lifecycle control module, a port scheduling and forwarding module, and a port lifecycle monitoring and anomaly recovery mechanism module.

[0030] The lightweight state database module is the storage and record module in the entire system architecture, deployed on the cluster management node. It can be implemented using pydblite (a lightweight Python library). The lightweight state database does not rely on any external services and operates directly on local files. It records the entire lifecycle state of each task, from start to finish, including every key step and state change, such as submission, scheduling, container startup, port mapping, and task recycling. Each record in the database represents an independent task. Each task has a globally unique UUID as its primary key. For ease of daily querying, a clusterName_user_jobId is designed as a business index, allowing users to quickly find relevant records directly by user account or job ID. Each record contains fields including, but not limited to, basic user information, the cluster and specific compute node information to which the task was assigned, the current state of the job, the running status of the associated container, the assigned port number and tunnel status, and the last update time. These fields are continuously updated as the task progresses. Through encapsulated interface functions, the system can update the state database in a timely manner at each stage, including task submission, node allocation, container startup, port application, forwarding establishment, anomaly detection, and resource reclamation, achieving full lifecycle tracking and management of tasks. For example, when a user submits a task, a new record is created and marked as "submitted"; after the scheduler allocates compute nodes, the node information and status are updated to "scheduled"; when a container starts, the startup time and process information are recorded; when a port is allocated, the port number is recorded; when a tunnel is established, the tunnel status is recorded; when the system detects a runtime anomaly, error details and recovery operations are also recorded. This full lifecycle tracking ensures that the task's progress is accurate at any time. The lightweight state database does not require a separate database service; data is stored directly in local files on the management node and can be directly manipulated through the Python interface. This makes the lightweight state database easy to integrate into existing systems without incurring additional maintenance burdens, and the file-based storage facilitates backup and migration; it can be packaged and copied to other machines when needed.

[0031] The remote jump server connection module is used for communication between the entire system and the compute nodes. On the cluster resource node side, the system establishes a secure communication channel with the node through the remote jump server connection module. The remote jump server connection module is implemented based on Paramiko (a Python SSH library) and supports a two-layer SSH jump server mechanism to establish and maintain secure connections to the cluster compute nodes. The two-layer jump server design first connects to the cluster's login node, and then, through the login node, directly connects to the final target compute node. This is done to accommodate the security architecture of most high-performance computing clusters, as these clusters typically do not allow direct login to compute nodes and require redirection through a designated login node. Authentication is performed using a private key in Ed25519 format during the connection process. The remote jump server connection module creates a special transparent channel using the `open_channel('direct-tcpi p')` method to establish an information channel between the login node and the compute node. All management commands and data are transmitted through this encrypted information channel, ensuring stable and secure issuance and feedback of remote commands. It's worth noting that, for security reasons, this module does not permanently store SSH keys. Instead, it only uses them temporarily during the current session, and the keys are cleared from memory after the session ends. This significantly reduces the risk of key leakage. When the system needs to perform operations on the compute node, such as starting a container or checking its running status, this module sends commands through a pre-established secure channel and obtains the execution results in real time. If the connection is unexpectedly interrupted, the module will automatically attempt to re-establish the connection. Throughout the process, all connection states and operation records are updated in real time to a lightweight state database, ensuring that every operation is traceable.

[0032] The container lifecycle control module manages the entire lifecycle of containers on compute nodes, using Singularity containers to start, stop, prepare, and query container status. When the system receives a user-submitted task request, the container lifecycle control module begins working, dynamically assembling a complete container startup command. This command includes various necessary parameter configurations. For example, it sets up an overlay file system to provide writable space within the container, mounts the user's home directory to ensure access to personal files, and injects specific environment variables to configure the runtime environment. All these parameters are automatically generated based on task requirements, requiring no manual specification from the user. Next, the `singularity instance start` command is executed on the target node via a remote session, automatically injecting the overlay file system, mount path, environment variables, and other parameters to complete container instantiation. For ease of subsequent identification and management, each started container is named according to a unified rule, such as `nginx_auto_`. <baseport>This format, where `basePort` is the port number assigned to the task, allows for quick identification of the corresponding container by name. After the startup command is executed, the module checks the operation result, indicating whether the container started successfully, by parsing the instance list returned remotely. Specifically, it confirms successful container startup by parsing the output of the `singularity instance list` command. If startup fails, it immediately logs the error message and attempts to recover. During container runtime, the module continuously monitors the container's status in conjunction with other components. When a task completes or an exception occurs requiring termination, the module executes the `singularity instancestop` command to shut down the container instance and clean up related resources. Throughout the entire process, from container startup and runtime status checks to final shutdown, the module updates the lightweight state database in real time at each critical node, ensuring the system always has accurate information on the status of each container.

[0033] The port scheduling and forwarding module enables convenient user access to remote containers, resolving port conflicts and remote access issues in multi-user environments. This module initializes a local pool of available ports using a bitmap structure. When a user task requires remote access, the port scheduling and forwarding module allocates an available port from this pre-planned pool. This port pool is not a simple sequential arrangement of numbers, but rather a sequence of ports distributed according to a certain step size (e.g., generated at fixed intervals, such as 8090, 8100, 8110, taking one port every 10). This discrete distribution significantly reduces the probability of port conflicts. When multiple users simultaneously request a port, the system uses a memLock mechanism. The port scheduling and forwarding module generates an MD5 hash value as a temporary lock based on information such as the cluster name, user, job number, node IP, and base port, ensuring that only one request can operate on a specific port resource at a time. After successful port allocation, the module immediately establishes an SSH tunnel locally, mapping the localhost:basePort port on the management node to the containerNode:22 port on the target compute node, achieving a seamless connection from the local machine to the remote container. During this process, the user only needs to connect to the local port, and data is automatically forwarded to the remote compute node through the encrypted tunnel. To ensure security, all tunnel connections use encrypted transmission, and the system records the allocation of each port and the tunnel status. When the task is completed, the module automatically reclaims the port resources, marking them as idle for use by subsequent tasks.

[0034] The port lifecycle monitoring and anomaly recovery mechanism module addresses port failures or container anomalies. The system periodically scans the status of local SSH tunnels to confirm whether any processes are still listening on each port and whether each SSH tunnel remains operational. When an anomaly is detected in port forwarding, such as an unexpected termination of the tunnel process or a port becoming unresponsive, the anomaly handling process is immediately triggered. First, the lightweight status database is queried to find the specific task, container, and compute node information associated with the port, determining whether the corresponding job and container are still alive. If the container is found to be running normally, but the tunnel connection is faulty, the mechanism automatically triggers the tunnel reconstruction process, re-establishing the SSH forwarding channel from the management node to the compute node. If the container itself has also stopped running, detailed information about this anomaly event is recorded, including the occurrence time, anomaly type, and associated tasks, and all information is updated in the status database. The port can also be marked as abnormal to prevent subsequent tasks from incorrectly reusing it. Under certain configurations, a resource reclamation process can also be triggered to clean up residual port usage information. All these operations generate detailed log records, providing complete clues for subsequent problem analysis.

[0035] The above describes the cluster task container management architecture. Key events throughout the system, including job submission, node allocation, container startup, port application, tunnel establishment, anomaly recovery, and resource reclamation, are all written to the state database through encapsulated API interfaces, forming a complete lifecycle record chain. Simultaneously, the platform supports information retrieval based on the state database, such as querying user tasks by email address, querying port status, and checking expired jobs, providing data support for subsequent system visualization, operation and maintenance management, and fault diagnosis.

[0036] Based on this system architecture, embodiments of this application provide a cluster task container management method. For example, Figure 2 This illustration shows a flowchart of a cluster task container management method provided in an embodiment of this application. The method is applied to, for example... Figure 1 The system architecture is shown below. Figure 2 As shown, the method may include the following steps:

[0037] S21: Receive the task submitted by the user, generate a unique ID and record the initial state, and wait for the scheduler to allocate computing nodes.

[0038] In this embodiment, when a user submits a new task to the management node, the system first generates a unique UUID for the task as an index. Simultaneously, to facilitate subsequent queries, a retrieval identifier composed of the cluster name, username, and job ID is created. This information, along with the user's basic information and submission time, is recorded in the lightweight state database, and the task's status is marked as "submitted." Next, the system hands the task over to the underlying scheduler, such as a cluster scheduling system like Slurm or Kubernetes. The scheduler, based on current resource availability, allocates the task to a specific compute node for execution. Once allocation is complete, the system immediately updates the state database, recording the target node's IP address, hostname, and other information, and changes the task status to "scheduled." During this process, if any anomalies occur, such as scheduling failures or database update errors, the system captures these errors, records detailed error information and the occurrence time in the state database, and marks the task status as abnormal.

[0039] S22: Start the container on the target node via SSH, monitor its running status in real time, and update the database.

[0040] In this embodiment, once a task is successfully scheduled to the target compute node, the system starts the container instance. The entire process begins with establishing a secure connection. The system connects to the cluster's login node via a pre-configured SSH jump server, and then connects from the login node to the assigned compute node. After the connection is established, the system automatically assembles a complete container startup command based on the user-submitted task requirements. Here, the Singularity container's instance start mode is used, essentially running the container as a background service. The startup command contains many automatically generated parameters, such as mounting the user's home directory to allow the container to access personal files, setting up the overlay file system to provide writable space, and injecting various environment variables according to task requirements. To facilitate subsequent management, each container is given a consistent name, usually embedding the port number assigned to the task within the name, such as the format nginx_auto_8080. After the command is sent to the compute node for execution, the system runs the `singularity instance list` command, checks the return results, and confirms that the newly started container is indeed in the running list. Regardless of whether the container startup is successful or not, this status information is fed back to the system in real time. If successful, the container status field in the status database will be updated to "Running," and the entire task status will be changed to "Executing." If it fails, specific error information will be recorded, such as whether the image pull failed or the startup command malfunctioned, and the task status will be marked as abnormal. This continuous monitoring method ensures that the system knows the real status of each container at all times, and all status changes are accurately timestamped, providing complete data support for subsequent troubleshooting and performance analysis.

[0041] S23: Dynamically allocate discrete ports and establish an encrypted SSH tunnel for users to access remote containers.

[0042] In this embodiment, when a user needs to access a running container, the system initiates a port allocation and tunnel establishment process. First, an available port number is determined from a pre-configured port pool. This pool contains ports that are not consecutive numbers, but rather a sequence of ports distributed according to a certain step size, such as 8090, 8100, and 8110. This design significantly reduces the possibility of port conflicts. After determining the port, the system generates a lock identifier. This is achieved by calculating the MD5 hash of the cluster name, username, job ID, node address, and port number, ensuring that only one task can use this port resource at a time. Once the port is obtained, the system starts an SSH tunnel on the management node, mapping the newly allocated port to port 22 of the target compute node. The system records detailed information about the allocated port number and the established tunnel status in a status database, including the time of allocation, which compute node is using it, and which container it is associated with. Subsequently, when a user needs to access the container, they only need to connect to this port on the management node, and the request will be automatically forwarded to the node where the target container resides through the encrypted tunnel, providing both convenience and security. If the port is no longer needed, the system will automatically reclaim it, remark it as idle, and wait for it to be allocated to other tasks next time.

[0043] S24: Periodically check port and container status, automatically recover in case of abnormality, and clean up resources after the task is completed.

[0044] In this embodiment, the system periodically checks all currently used ports and their corresponding SSH tunnels, periodically verifying the connectivity of each communication line. The system assesses connection health by checking the local port listening status and the SSH tunnel process's liveness. If a tunnel corresponding to a port is found to be disconnected, the system first checks which task and container are associated with that port. If the container is still running normally, but the tunnel connection is faulty, the system attempts to re-establish the SSH tunnel to restore communication. However, if the container itself has stopped running, the system records this anomaly, including the time of occurrence and the specific cause, and marks the port as abnormal to prevent other tasks from using it incorrectly. When a task finally completes, whether normally or abnormally, the system initiates a resource reclamation process. The previously allocated port number is released back to the resource pool, and the port is remarked as available. Finally, all related SSH connections are closed. All these operations are recorded in detail in the status database, including the resource reclamation time and operation results, ensuring the entire task lifecycle is traceable.

[0045] Figure 3 This is a schematic diagram illustrating the overall process of cluster task execution provided in an embodiment of this application. For example... Figure 3 As shown, user requests to the API call start, stop, or prepare actions trigger different processes via the instanceAction scheduler. During startup, the target instance is first connected via a double-layer SSH jump (connected to the login node and then passed through to the compute node). `startInstance` is called, and the Singularity startup command is constructed using dynamic parameters generated by `dynamicScriptVariables`. After container startup, the state is updated to a lightweight state database (based on PyDbLiter, using UUID as the primary key and `clusterName_user_jobId` as the index to record the entire lifecycle state). In the preprocessing stage, after connecting to the login node, the `prepare.sh` script is executed to copy model data to the preprocessing component. For port management, a Bitmap port pool is used to generate available port segments in steps. An MD5 hash lock is generated using `memLock` to ensure port uniqueness in a concurrent environment. After requesting a port, an SSH tunnel is established from `localhost:basePort` to `containerNode:22`, and forwarding rules are recorded. During shutdown, the instance is connected to execute the Singularity stop command, and the state is updated to `stopped / revoked`. The system records all critical events through a status database module (including interfaces such as createJobSubmit and checkExpiredRecord). Administrators or inspectors periodically scan the port forwarding status, and automatically rebuild tunnels or reclaim resources when anomalies occur. The entire mechanism does not change the original scheduling logic of Slurm / K8s and has the capabilities of secure isolation, multi-user concurrency support, and anomaly self-healing.

[0046] The above describes the cluster task container management method provided in this application embodiment. A lightweight state database is built using PyDb Lite, with UUID as the primary key to record the entire lifecycle state of tasks, including submission, scheduling, running, and recycling stages. Compute nodes are connected via a dual-layer SSH jump server, and containers are dynamically started using the Singularity runtime, automatically injecting parameters such as the overlay file system and environment variables. A port pool management mechanism is designed, using offline serial port allocation and MD5 hash locks to ensure resource uniqueness during multi-user concurrency. An SSH tunnel is also established to enable secure access from the local machine to the container. The system has anomaly recovery capabilities, periodically scanning port and container status, and automatically rebuilding connections or recycling resources in case of anomalies. This method has the following advantages: high degree of automation, requiring no manual configuration of ports and mapping paths by the user; the system automatically allocates, schedules, and recycles resources. Full-process traceability of task status: the lightweight state database supports real-time updates and historical queries of all task statuses. Robust anomaly recovery capabilities: any container or port anomaly can be automatically identified and triggered for repair. The platform is highly non-intrusive: both the state library and forwarding logic can be deployed without modifying the original Slurm / K8s scheduling logic, ensuring good compatibility. It supports multi-user concurrency: the bitmap + hash lock scheduling mechanism inherently provides concurrent access guarantees.

[0047] It is understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. Furthermore, in some possible implementations, each step in the above embodiments may be selectively executed according to actual circumstances; it may be partially or fully executed, without limitation here. All or part of any feature of any embodiment of this application can be freely and arbitrarily combined without contradiction. The combined technical solutions are also within the scope of this application.

[0048] Based on the methods in the above embodiments, this application provides a computer-readable storage medium storing a computer program that, when run on a processor, causes the processor to execute the methods in the above embodiments.

[0049] Based on the methods in the above embodiments, this application provides a computer program product that, when run on a processor, causes the processor to execute the methods in the above embodiments.

[0050] It is understood that the processor in the embodiments of this application may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. A general-purpose processor may be a microprocessor or any conventional processor.

[0051] The method steps in the embodiments of this application can be implemented in hardware or by a processor executing software instructions. The software instructions can consist of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, portable hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can reside in an ASIC.

[0052] In the above embodiments, implementation can be achieved entirely or partially through software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented entirely or partially in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0053] It is understood that the various numerical designations used in the embodiments of this application are merely for descriptive convenience and are not intended to limit the scope of the embodiments of this application.< / baseport> < / baseport>

Claims

1. A cluster task container management method, characterized in that, Applied to a supercomputing cluster, the supercomputing cluster including a management node and multiple computing nodes, the method includes: Build a lightweight state database to record the state of the task throughout its entire lifecycle, which includes the stages of submission, scheduling, container startup, port mapping, and recycling. Design a bitmap-based port pool mechanism to generate a discretely distributed sequence of available ports with a fixed step size, and use a memory lock to ensure the concurrent uniqueness of port requests; An SSH tunnel is established on the management node, forwarding the local port to the SSH port of the target compute node; wherein, the SSH tunnel is established through a two-layer jumper, including: connecting to the cluster login node through the Paramiko library; passing through from the login node to the target compute node, using Ed25519 key authentication and the open_channel method to create an encrypted channel; Implement container lifecycle management, periodically scan tunnel status, and automatically trigger container status checks and tunnel reconstruction when an anomaly occurs; the automatic triggering of container status checks and tunnel reconstruction when an anomaly occurs includes: when an SSH tunnel interruption is detected, querying the lightweight status database to obtain the associated container status; if the container is alive, automatically rebuilding the tunnel; if the container fails, marking it as abnormal and releasing port resources; All status events are written to the lightweight state database via API, which supports task queries and connection health checks.

2. The method according to claim 1, characterized in that, The lightweight status database is implemented using PyDbLite. The task record uses the task's UUID as the primary key and clusterName_user_jobId as the business index. It includes user information, node information, job status, container status, and port mapping status fields.

3. The method according to claim 1, characterized in that, In the port pool mechanism, the memory lock is generated using the MD5 hash algorithm, and the hash input includes the cluster name, username, job ID, node IP, and port.

4. The method according to claim 1, characterized in that, Container lifecycle management is based on the Singularity runtime, and the container instance naming convention is service_auto_ <baseport> At startup, the overlay file system, mount path, and environment variables are dynamically injected.< / baseport> 5. The method according to claim 1, characterized in that, The method is compatible with Slurm or Kubernetes scheduling systems. The lightweight state database is deployed as an independent module without modifying the original scheduling logic of Slurm or Kubernetes.

6. A cluster task container management system, characterized in that, Deployed in a supercomputing cluster, the supercomputing cluster including a management node and multiple computing nodes, the system is used to perform the method as described in any one of claims 1-5.

Images