High-performance cluster user isolation and task scheduling method and system based on shared account

Abstract

The invention provides a high-performance cluster user isolation and task scheduling method and system based on a shared account, and the method comprises the steps: configuring a springboard machine with a management authority, and connecting the springboard machine with a high-performance cluster; creating a corresponding independent account number for each user on the springboard machine, and associating the independent account number with a shared account number of the high-performance cluster through a secret key; setting access permissions of the independent accounts, and performing task scheduling and job submission with each independent account through the shared account; and controlling the job submission amount of each independent account according to the resource occupation amount. Under the condition that a shared account mechanism is adopted in a high-performance computing environment, user isolation and job control can be realized without system permission, directory-level data access isolation and job identity tag injection are realized for a plurality of users sharing accounts, a scheduler is supported to identify user ownership, and the operation efficiency is improved. User-level job quota control and resource use limitation are supported, and the security, controllability and resource utilization efficiency in a multi-user cluster scene are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of high-performance computing system resource management, and in particular to a high-performance cluster user isolation and task scheduling method, system, device, computer readable medium and computer program product based on a shared account. BACKGROUND

[0002] With the continuous growth of demand for high-performance computing (HPC) capabilities in scientific research, industrial simulation, climate modeling, and artificial intelligence, etc., high-performance computing clusters, as the core computing power platform supporting key tasks such as scientific research, engineering simulation, and artificial intelligence training, their resource scheduling strategies and user isolation mechanisms have a decisive impact on system efficiency, data security, and resource fairness.

[0003] In traditional HPC systems, each user is usually assigned an independent operating system account, and the scheduling system (such as Slurm, PBS, etc.) performs job scheduling and resource management through user identity, thereby achieving user-level resource quota control, job audit tracking, and data access isolation. This mode can effectively guarantee the fair allocation of computing resources and the secure storage of user data.

[0004] However, in actual applications, due to management complexity, isolation cost, or limited permissions, etc., some cluster operators prefer to use a shared account mechanism for system management. Typical application scenarios include:

[0005] 1. Educational and scientific research clusters: universities or research institutions provide computing platforms for experimental teaching and course assignments, and usually assign a unified access account (such as commonuser) to students or temporary users to simplify account management and configuration processes;

[0006] 2. Enterprise rental platforms: due to the need for security control and uniform management of contracts, industry power rental platforms often aggregate multiple enterprise customers under a single rental account for scheduling resource configuration;

[0007] 3. Cloud HPC subsystems: in a platform-as-a-service (PaaS) or software-as-a-service (SaaS) power platform, multiple business modules may run through a unified system account to reduce integration complexity;

[0008] 4. Cross-institutional power interconnection scenarios: when building a power network, each power subject opens only a small number of public accounts to a third-party supercomputer interconnection unified scheduling platform for security protection and resource control requirements.

[0009] In the above scenario, multiple actual users share the same Linux system identity login and task submission, the scheduling system cannot identify the specific job owner, and the operating system layer permission has no difference in authorization, which leads to the following core problems in system management and security control:

[0010] 1. Job cannot be traced: Since the identity of the task submitter has no unique identifier, the scheduling system can only identify the shared account (such as commonuser) as the job initiator, resulting in a lack of auditability of job behavior and difficulty in statistics and archiving of resource usage;

[0011] 2. Data access is not isolated: All users share the same permission view and working directory, lack of boundary restrictions, and are prone to file deletion, illegal access, data leakage and other security risks;

[0012] 3. Resource quota is difficult to control: The scheduling system cannot subdivide user behavior based on account, making it difficult to implement quota management and scheduling intervention of user-level job number, running time, core number, memory and other resource indicators, resulting in resource abuse, system congestion and scheduling unfairness.

[0013] Therefore, in the case of using a shared account mechanism in a high-performance computing environment, an innovative technical solution is needed to achieve user isolation and job control without system permissions. SUMMARY

[0014] The present application provides a high-performance cluster user isolation and task scheduling method, system, device, computer readable medium and computer program product based on a shared account, aiming to achieve user isolation and job control in a restricted environment without high-performance cluster management permissions, lack of high-performance cluster administrator support, and inability to add high-performance cluster system accounts, to improve security, controllability and resource utilization efficiency in a multi-user cluster scenario.

[0015] To achieve the above technical effects, one aspect of the present application provides a high-performance cluster user isolation and task scheduling method based on a shared account, comprising: configuring a jump machine with management permissions and connecting it to a high-performance cluster;

[0016] Creating a corresponding independent account for each user on the jump machine, and associating it with the shared account of the high-performance cluster through a secret key;

[0017] Setting the access permissions of the independent account, and performing task scheduling and job submission through the shared account and each independent account;

[0018] Controlling the number of job submissions of each independent account according to resource occupancy.

[0019] According to the preferred embodiment of the present application, the configuration of the jump machine with management authority and the connection with the high-performance cluster further comprises:

[0020] The jump machine with management authority is connected to the login node of the high-performance cluster through an SSH network channel.

[0021] The jump machine is authenticated and connected to the shared account of the high-performance cluster.

[0022] According to the preferred embodiment of the present application, the creation of a corresponding independent account for each user on the jump machine and the association with the shared account of the high-performance cluster through a secret key further comprises:

[0023] In the jump machine, a corresponding local logical independent account is created for each actual user.

[0024] Log in to the high-performance cluster with the shared account and create a subdirectory corresponding to the independent account.

[0025] The independent account is connected to the subdirectory through a secret key pair for password-free access.

[0026] According to the preferred embodiment of the present application, the connection of the independent account to the subdirectory through a secret key pair for password-free access further comprises:

[0027] For each independent account, an SSH key pair is configured for password-free authentication with the shared account.

[0028] The subdirectory of the shared account is mounted to the jump machine using the SSHFS tool to establish access connection with the corresponding independent account.

[0029] According to the preferred embodiment of the present application, the setting of the access rights of the independent account and the task scheduling and submission through the shared account and each independent account further comprises:

[0030] On the jump machine, the access rights of the subdirectory of each independent account are set so that each independent account can only access its corresponding subdirectory.

[0031] In the shared account of the high-performance cluster, a task scheduling relay service based on the FastAPI framework is configured, and a corresponding identification job ownership script is configured in each independent account of the jump machine.

[0032] The shared account performs task scheduling and job submission with each independent account through the HTTP interface of the task scheduling relay service.

[0033] According to the preferred embodiment of the present application, the control of the job submission amount of each independent account according to resource occupation further comprises:

[0034] Analyze resource occupation required by the job, and count total resource occupation currently running by the independent account submitting the job;

[0035] Determine whether the sum of resource occupation required by the job submitted by the independent account and the total resource occupation exceeds a quota threshold of the independent account;

[0036] Control the job submission amount of the independent account according to the determination result.

[0037] Another aspect of the present application provides a high-performance cluster user isolation and task scheduling system based on a shared account, comprising:

[0038] A jump machine configuration module is configured to configure a jump machine with management authority and connect the jump machine with a high-performance cluster;

[0039] An account management module is configured to create a corresponding independent account for each user on the jump machine, and associate the independent account with a shared account of the high-performance cluster through a secret key;

[0040] A task scheduling module is configured to set access rights of the independent account, and perform task scheduling and job submission through the shared account and each independent account;

[0041] A resource control module is configured to control the job submission amount of each independent account according to resource occupation.

[0042] Another aspect of the present application provides a high-performance cluster user isolation and task scheduling device based on a shared account, the device comprising:

[0043] At least one processor; and

[0044] A memory in communication connection with the at least one processor; wherein,

[0045] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method as described above.

[0046] Another aspect of the present application provides a computer readable storage medium having computer program instructions stored thereon, and the computer program instructions are executable by a processor to implement the method described above.

[0047] Another aspect of the present application provides a computer program product comprising a computer program, and the computer program is executable by a processor to implement the method described above.

[0048] The scheme provided by the present application embodiment has the following beneficial effects:

[0049] The application can realize user isolation and job control without system permission in the case of adopting a shared account mechanism in a high-performance computing environment, realize directory-level data access isolation for users of multiple shared accounts, realize job identity tag injection, support a scheduler to identify user attribution, support user-level job quota control and resource use limitation, solve three core problems of "job untraceability, data non-isolation, and resource quota difficulty", and improve security, controllability, and resource utilization efficiency in a multi-user cluster scenario. BRIEF DESCRIPTION OF DRAWINGS

[0050] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative labor under the premise of the drawings.

[0051] Other features, objects and advantages of the present application will become more apparent from the following detailed description of non-limiting embodiments with reference to the following drawings:

[0052] Figure 1 A flowchart of a high-performance cluster user isolation and task scheduling method based on a shared account provided by an embodiment of the present application;

[0053] Figure 2 An architecture diagram of a high-performance cluster user isolation and task scheduling method based on a shared account provided by an embodiment of the present application;

[0054] Figure 3 A job submission flowchart of an independent account provided by an embodiment of the present application;

[0055] Figure 4 A schematic diagram of a high-performance cluster user isolation and task scheduling system based on a shared account provided by an embodiment of the present application;

[0056] Figure 5 A structural schematic diagram of a device suitable for implementing the scheme in the embodiments of the present application.

[0057] The same or similar reference signs in the drawings represent the same or similar components. DETAILED DESCRIPTION

[0058] In order to make the purposes, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0059] In a typical configuration of the present application, the terminal and the device of the service network each include one or more processors (CPU), input / output interfaces, network interfaces, and memories.

[0060] The memory can include a non-permanent memory in a computer readable medium, random access memory (RAM), and / or a non-volatile memory such as a read-only memory (ROM) or a flash memory (flash RAM). The memory is an example of the computer readable medium.

[0061] The computer readable medium includes a permanent and non-permanent, removable and non-removable medium, and can be implemented by any method or technology to store information. The information can be computer program instructions, data structures, program modules, or other data. Examples of the storage medium of the computer include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape disk storage or other magnetic storage device, or any other non-transmission medium that can be used to store information accessible to a computing device.

[0062] In an actual scenario, the execution subject of the method can be a user device, or a device integrated by a user device and a network device through a network, or also can be an application program running on the above device, the user device includes but is not limited to computers, mobile phones, tablet computers, smart watches, wristbands, and various terminal devices, and the network device includes but is not limited to network hosts, single network servers, multiple network server sets, or computer sets based on cloud computing, which can be used to realize part of the processing function when setting an alarm. Here, the cloud is composed of a large number of hosts or network servers based on cloud computing, wherein the cloud computing is a virtual computer composed of a loose coupling computer set.

[0063] Noun explanation: jumpboard machine: a network device used by system administrators or operation and maintenance personnel to perform batch operations on remote devices, which takes over the access request of the terminal computer through the protocol agent and plays a role in security transfer. Its core functions include cutting off direct access, operation audit and illegal command blocking.

[0064] The embodiment of the application provides a high-performance cluster user isolation and task scheduling method based on a shared account, in which a job scheduling system supporting logical user isolation and resource control is constructed and deployed in a high-performance computing cluster environment without administrator permission, and only a unified shared account is provided externally. The system realizes remote safe submission and job identification control through a jumpboard machine relay, Figure 1 A flowchart of a high-performance cluster user isolation and task scheduling method based on a shared account provided by an embodiment of the application is provided, Figure 2 A high-performance cluster user isolation and task scheduling method based on a shared account provided by an embodiment of the application is provided, as shown in the Figure 1 and Figure 2 The method comprises the following processing steps:

[0065] Step S101, configure a jumpboard machine with management permission and connect to the high-performance cluster.

[0066] Specifically, first, a Linux host is deployed as a jumpboard machine, which is configured to have root management permission, and an SSH service is installed and enabled. By modifying the SSH service configuration file (such as sshd_config), the root user is allowed to log in, and the corresponding firewall port is opened to ensure the availability of SSH connection; second, a network channel is established between the jumpboard machine and the login node of the high-performance cluster through the SSH protocol; finally, the jumpboard machine and the shared account (such as user_share) of the high-performance cluster are connected for authentication, which is used for job personnel to log in to the cluster system and cannot obtain root permission or add system-level users.

[0067] Step S102, create a corresponding independent account for each user on the jumpboard machine, and associate it with the subdirectory of the shared account of the high-performance cluster through a secret key.

[0068] Specifically, first, a corresponding local logical independent account (such as user1, user2, etc.) is created for each actual user in the jumpboard machine using Linux, etc., so that each user has an independent system identity and file workspace for task script management, remote directory mounting and permission control.

[0069] Then log in to the high-performance cluster with the shared account and create subdirectories (such as user1_data and user2_data) corresponding to the independent accounts to store user job-related data. The permissions are set to 700, allowing only the shared account owner to access, serving as the first layer of isolation.

[0070] For example: mkdir -p / home / user_share / user1_data (create user1's subdirectory)

[0071] mkdir -p / home / user_share / user2_data (create user2's subdirectory)

[0072] chmod 700 / home / user_share / user1_data (set user1's access permissions to the shared account)

[0073] chmod 700 / home / user_share / user2_data (set user2's access permissions to the shared account)

[0074] Finally, establish a password-free access connection between the independent account and the subdirectory through a secret key pair. Configure SSH key pairs for each independent account on the jump machine and copy the public key to the cluster login node to complete the encryption authentication configuration, enabling password-free authentication connection with the shared account. Then use the SSHFS tool to mount the shared account's subdirectory to the jump machine and establish an access connection with the corresponding independent account, for example:

[0075] sudo -u user1 ssh-keygen -t rsa -N "" -f / home / user1 / .ssh / id_rsa, which means generating an SSH key pair with the independent account user1: sudo -u user1 specifies the operation subject as user1; ssh-keygen -t rsa generates an RSA type key; -N "" sets an empty password (implements password-free access); -f specifies the key storage path as user1's exclusive directory / home / user1 / .ssh / id_rsa, which automatically generates the corresponding public key id_rsa.pub, laying the foundation for user1's encryption authentication with the cluster shared account.

[0076] sudo -u user1 ssh-copy-id -i / home / user1 / .ssh / id_rsa.pub user_share@HPC_IP, which means that the password-free authentication configuration of user1 and the cluster shared account is completed: sudo -u user1 ensures that user1 is executed; -i specifies the public key path to be copied; user_share@HPC_IP is the cluster shared account and the cluster IP, and after execution, the public key is added to the authorized list (~ / .ssh / authorized_keys) of the cluster user_share, so that user1 can access the cluster shared account through the jump machine without password.

[0077] sudo -u user1 mkdir -p / home / user1 / remote_home, which means that user1 creates a local mounting directory: sudo -u user1 operates as user1; mkdir -p ensures that the directory (including the parent directory) exists, and the path / home / user1 / remote_home is mapped to the local node of the jump machine as a cluster subdirectory, which is used for subsequent data access.

[0078] sudo -u user1 sshfs user_share@HPC_IP: / home / user_share / user1_data / home / user1 / remote_home -o IdentityFile= / home / user1 / .ssh / id_rsa -o reconnect, which means that the mounting connection of the cluster subdirectory and the local directory is realized: sudo -u user1 specifies the operation subject; sshfs mounts the exclusive subdirectory ( / home / user_share / user1_data) of user1 in the cluster to the local directory ( / home / user1 / remote_home) of user1 through the SSH protocol.

[0079] In this way, users can transparently access remote cluster subdirectories on the jump machine, realize data channel docking, and each user has a private access path that is synchronized with the cluster data on the jump machine, and the entire process does not depend on the administrator permission of the cluster side or the support of NFS. Users can log in to the jump machine using an independent account and connect to the shared account login interface provided by the high-performance cluster system, download tasks or submit jobs from the corresponding subdirectory of the shared account by inputting their own secret keys.

[0080] Step S103, set the access permission of the subdirectory, and schedule tasks through the shared account and each independent account.

[0081] Specifically, first, through the user mapping mechanism of SSHFS, set the access permission of each independent account subdirectory on the jump machine, so that each independent account can only access its corresponding subdirectory, for example:

[0082] chown user1:user1 / home / user1 / remote_home, which means setting the owner and group of the directory to the independent account user1, ensuring that user1 has complete control over the directory.

[0083] chmod 700 / home / user1 / remote_home, which means limiting the directory permissions to only the owner user1 can read, write and execute, and other users have no access rights.

[0084] Through the Linux file system permission mechanism, combined with the independent system account structure, the data access isolation of the jump machine is realized. Even if users share the cluster's shared account, they cannot access other user-mounted data paths through the jump machine, forming a minimum permission isolation system based on directory level, thereby building a security boundary.

[0085] Then configure the task scheduling relay service based on the FastAPI framework in the shared account of the high-performance cluster. This relay service runs based on a web interface (such as REST API), which can manage resources at the logical user granularity by parsing task submission parameters. Its core functions include:

[0086] / submit: receive job script upload, parse resource requirements, check quota, and control submission;

[0087] / squeue: query job status in the cluster and filter corresponding logical user jobs based on the --comment field;

[0088] / sinfo: forward cluster node status information.

[0089] This service runs in a non-privileged environment and does not depend on administrator configuration, with good deployability and independence.

[0090] Then configure the corresponding Slurm-like command structure in each independent account of the jump machine to identify job ownership scripts, replacing the original command wrapper scripts, including sbatch, squeue, sinfo, etc. For example, the sbatch command:

[0091] #! / bin / bash

[0092] USER_TAG=$(whoami) means to obtain the independent account (such as user1) of the current login jump machine as the user identifier;

[0093] SCRIPT_PATH="$1" means to receive the job script path (a file in the local mounted directory) input by the user to be submitted;

[0094] RESPONSE=$(curl-F"file=@${SCRIPT_PATH}" means to call the cluster submission interface, and upload the job script to the cluster service (HPC_IP:8000 is the cluster interface address) through the HTTP protocol;

[0095] http: / / HPC_IP:8000 / submit?user=$USER_TAG) means to carry the user identifier USER_TAG, so as to ensure that the cluster verifies and executes the job according to the shared account subdirectory permission;

[0096] echo "$RESPONSE" # Output the job submission result returned by the cluster (such as job ID, status information).

[0097] The script automatically adds the logical username as the --comment=<logical username> parameter before calling the native command, so as to embed the logical identity of the job, identify the job ownership by the transfer service, and initiate an interface call to the task scheduling service. At the same time, the use mode of the original Slurm command is kept unchanged, the remote transfer and status query of the job are completed, and the user experience is improved.

[0098] Finally, the shared account performs task scheduling and job submission with each independent account through the HTTP interface of the task scheduling transfer service.

[0099] Step S104, controlling the job submission amount of each independent account according to the resource occupation amount.

[0100] Specifically, Figure 3 The independent account job submission process provided by an embodiment of the present application is shown in FIG. 1. Figure 3 As shown in FIG. 1, after the task scheduling transfer service of the shared account receives a job submission request of a certain independent account, the resource demand amount declared by the #SBATCH instruction in the script is automatically parsed, for example:

[0101] #SBATCH --ntasks=2

[0102] #SBATCH --cpus-per-task=2

[0103] The total number of CPUs required for the current submitted job is 4, and then the system call scontrol show job and the like command is used to count the total resource occupation of the independent account (identified by the comment field) currently running. According to the independent account quota set by the system (such as a maximum of 4 cores), it is determined whether the preset quota threshold is exceeded, and the job submission amount of the independent account is controlled according to the determination result.

[0104] If the sum of the requested resources and the occupied resources of the current job does not exceed the quota of the independent account, the job forwarding submission is performed by the transfer service, and sbatch --comment=user1 script.sh is called; if the sum of the requested resources and the occupied resources of the current job exceeds the quota of the independent account, an error response is returned to the jump machine, and an error message of exceeding the resource quota is issued, such as: {"error": "CPU quota exceeded. Current usage: 3, requested:2, quota: 4"}, the job submission is rejected, and the system resources are prevented from being excessively occupied. The logical user can adjust the job according to the prompt to ensure fair use of resources and stable operation of the system.

[0105] In summary, the method adopts the path of "jump machine logical user -> SSHFS mapping -> cluster shared directory -> scheduling transfer service -> Slurm scheduling system" to complete task isolation and submission.

[0106] In the scheme provided by the method embodiment, in the case of using a shared account mechanism in a high-performance computing environment, user isolation and job control can be realized without system permissions, directory-level data access isolation is realized for users of multiple shared accounts, job identity tag injection is realized, the scheduler is supported to identify user attribution, user-level job quota control and resource usage limitation are supported, and the three core problems of "job untraceable, data not isolated, and resource difficult to quota" are solved, thereby improving the security, controllability, and resource utilization efficiency in a multi-user cluster scenario.

[0107] Figure 4 A high-performance cluster user isolation and task scheduling system based on a shared account provided by an embodiment of the present application is shown in FIG. 1. Figure 4 As shown in FIG. 1, the system includes:

[0108] The jump machine configuration module 11 is configured to configure a jump machine with management permissions and connect the jump machine with the high-performance cluster.

[0109] The account management module 22 is configured to create a corresponding independent account for each user on the jump machine, and associate the independent account with the shared account of the high-performance cluster through a secret key.

[0110] The task scheduling module 33 is configured to set the access right of the independent account, and perform task scheduling and job submission for each independent account through the shared account.

[0111] The resource control module 44 is configured to control the job submission amount of each independent account according to the resource occupation amount.

[0112] The system can perform the high-performance cluster user isolation and task scheduling method based on a shared account in the foregoing embodiments.

[0113] The jump machine configuration module 11 performs step S101, the account management module 22 performs step S102, the task scheduling module 33 performs step S103, and the resource control module 44 performs step S104.

[0114] Based on the same inventive concept, the electronic device provided in the embodiments of the present application can correspond to the high-performance cluster user isolation and task scheduling method based on a shared account in the foregoing embodiments, and the problem solving principle thereof is similar to that of the method. The electronic device provided in the embodiments of the present application comprises at least one processor, and a memory in communication connection with the at least one processor. The memory stores instructions executable by the at least one processor. The instructions are executed by the at least one processor, so that the at least one processor can execute the method and / or technical solution of the plurality of embodiments of the present application.

[0115] The electronic device can be a user device, or a device integrated by a user device and a network device through a network, or can also be an application program running on the device. The user device includes, but is not limited to, computers, mobile phones, tablet computers, smart watches, wristbands, and various terminal devices. The network device includes, but is not limited to, network hosts, single network servers, multiple network server sets, or computer sets based on cloud computing, which can be used to implement part of the processing function when setting an alarm. Here, the cloud is composed of a large number of hosts or network servers based on cloud computing, wherein cloud computing is a kind of distributed computing, which is composed of a virtual computer formed by a group of loosely coupled computer sets.

[0116] Figure 5The structure of a device suitable for implementing the method and / or technical solution in the embodiments of the present application is shown. The device 1200 includes a central processing unit (CPU) 1201, which can perform various appropriate actions and processes according to programs stored in a read-only memory (ROM) 1202 or loaded from a storage portion 1208 into a random access memory (RAM) 1203. In the RAM 1203, various programs and data required for system operation are also stored. The CPU 1201, the ROM 1202, and the RAM 1203 are connected to each other through a bus 1204. An input / output (I / O) interface 1205 is also connected to the bus 1204.

[0117] The following components are connected to the I / O interface 1205: an input portion 1206 including a keyboard, a mouse, a touch screen, a microphone, an infrared sensor, and the like; an output portion 1207 including a cathode ray tube (CRT), a liquid crystal display (LCD), an LED display, an OLED display, and the like, and a speaker, and the like; a storage portion 1208 including one or more computer readable media such as a hard disk, an optical disk, a magnetic disk, a semiconductor memory, and the like; and a communication portion 1209 including a network interface card such as a LAN (Local Area Network) card, a modem, and the like. The communication portion 1209 performs communication processing via a network such as the Internet.

[0118] In particular, the method and / or embodiments in the embodiments of the present application can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product including a computer program carried on a computer readable medium, the computer program containing program codes for executing the method shown in the flowchart. When the computer program is executed by the central processing unit (CPU) 1201, the above-mentioned functions defined in the method of the present application are performed.

[0119] Another embodiment of the present application also provides a computer readable storage medium having stored thereon computer program instructions, which can be executed by a processor to implement the method and / or technical solution of any one or more embodiments of the present application.

[0120] In particular embodiments, one or more computer programs can be employed, stored on or transmitted over as one or more computer program products, and can be executed by one or more programmable processors. A computer program product can be a computer- readable medium having instructions stored on it or a machine-readable medium having instructions stored on it. Computer programs (also known as programs, software, software applications, applications, components, program components, or code) can be written in any form of programming language, including compiled or interpreted languages, and can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program can be deployed to be executed on one computer or on multiple computers at one site or distributed across multiple sites and

[0121] A computer program product can be tangibly embodied in an information carrier. The computer program product can also contain instructions that, when executed, perform one or more methods, such as those described above. The instructions can be stored on a computer-readable medium, which is any media that can be accessed by a computer. By way of example, and not limitation, a computer-readable medium can comprise computer- storage media, communication media, or memonic devices including the following: volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage and / or transmission of information such as computer readable instructions, data structures, program code, or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, solid state drives, flash cards, other types of memory cards, other types of magnetic media such as hard disks, any other media that can be used to store and / or transport computer readable instructions and / or data structures, and any combination thereof. Computer storage media does not include communication media.

[0122] Computer program products can also include computer-readable media having variations of the foregoing, including both physical and virtual computer-readable media. Computer- readable media can also include communication media including any medium that facilitates transfer of a computer program from one place to another. A network connection, for example, can be one means of

[0123] Another embodiment of the application provides a computer program product, including a computer program tangibly embodied in a computer readable medium, the computer program including instructions that, when executed by a processor, implement a method for high performance cluster user isolation and task scheduling based on shared accounts.

[0124] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages, including an object oriented programming language such as Java, Smalltalk, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider).

[0125] The flow diagrams and block diagrams in the drawings are representative of the architectural, functional, and operational aspects of possible implementations of apparatuses, methods, and computer program products according to various embodiments of the present application. In this regard, each block in the flow diagrams or block diagrams can represent a module, a segment, or a portion of code, which comprises one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently or the blocks may

[0126] Those skilled in the art can clearly understand that, for the convenience and brevity, the specific working process of the system, device and unit described above can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0127] In several embodiments provided by the present application, it should be understood that the disclosed system, device and method can be implemented by other ways. For example, the device embodiments described above are merely schematic, for example, the division of the units is merely a logical function division, and actual implementation can have another division manner, for example, a plurality of units or page components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units or components shown or discussed can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0128] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, i.e. may be located in one place, or may be distributed on multiple network units. Part or all of the units may be selected according to actual needs to achieve the purpose of the embodiment.

[0129] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present alone, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of hardware plus software functional unit.

[0130] The integrated unit realized in the form of software functional unit can be stored in a computer readable storage medium. The software functional unit is stored in a storage medium, including a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute part of the steps of the method described in each embodiment of the present application. The foregoing storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), magnetic disk or optical disk, and various program code storage media.

[0131] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of each embodiment of the present application.

[0132] In addition, it is obvious that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. The plurality of units or devices stated in the device claim can also be realized by one unit or device through software or hardware. The words first, second, etc. are used to indicate names, and do not indicate any specific order.

Claims

1. A high-performance cluster user isolation and task scheduling method based on shared accounts, characterized in that, include: Configure a jump server with administrative privileges and connect it to a high-performance cluster; Create a unique account for each user on the jump server and associate it with a shared account in the high-performance cluster using a key; Set access permissions for the independent accounts, and use the shared account to schedule tasks and submit jobs to each independent account; The number of job submissions per account is controlled based on resource usage.

2. The high-performance cluster user isolation and task scheduling method based on shared accounts according to claim 1, characterized in that, The configuration of the jump server with management privileges and its connection to the high-performance cluster further includes: Configure a jump server with administrative privileges to connect to the login node of the high-performance cluster via an SSH network tunnel; Establish an authentication connection between the jump server and the shared account of the high-performance cluster.

3. The high-performance cluster user isolation and task scheduling method based on shared accounts according to claim 1, characterized in that, The step of creating a corresponding independent account for each user on the jump server and associating it with a shared account in the high-performance cluster via a key further includes: In the jump server, a corresponding local logical independent account is created for each actual user; Log in to the high-performance cluster with a shared account and create a subdirectory corresponding to the independent account; A passwordless access connection is established between the independent account and the subdirectory using a key pair.

4. The high-performance cluster user isolation and task scheduling method based on shared accounts according to claim 3, characterized in that, The step of establishing a passwordless access connection between the independent account and the subdirectory via a key pair further includes: Configure an SSH key pair for each individual account to connect to the shared account without password authentication; Use the SSHFS tool to mount the subdirectories of the shared account to the jump server and establish access connections with the corresponding independent accounts.

5. The high-performance cluster user isolation and task scheduling method based on shared accounts according to claim 4, characterized in that, Setting access permissions for the independent accounts and scheduling and submitting tasks with each independent account through the shared account further includes: Configure subdirectory access permissions for each independent account on the jump server so that each independent account can only access its own corresponding subdirectory; Configure a task scheduling relay service based on the FastAPI framework in the shared account of the high-performance cluster, and configure the corresponding job attribution identification script in each independent account of the jump server; The shared account communicates with each individual account via the HTTP interface of the task scheduling relay service to schedule tasks and submit jobs.

6. The high-performance cluster user isolation and task scheduling method based on shared accounts according to claim 1, characterized in that, The method of controlling the number of job submissions for each individual account based on resource consumption further includes: Analyze the resource consumption required by the job and calculate the total resource consumption currently being used by the individual accounts that submitted the job; Determine whether the sum of the resource usage required by the job submitted by the independent account and the total resource usage exceeds the quota threshold of the independent account; The number of assignments submitted by this independent account will be controlled based on the judgment results.

7. A high-performance cluster user isolation and task scheduling system based on shared accounts, characterized in that, include: Jump server configuration module: Used to configure jump servers with management privileges and connect them to high-performance clusters; Account Management Module: Used to create a unique account for each user on the jump server and associate it with the shared account of the high-performance cluster via a key; The task scheduling module is used to set the access permissions of the independent accounts and to schedule tasks and submit jobs with each independent account through the shared account. The resource control module is used to control the number of job submissions for each individual account based on resource usage.

8. A high-performance cluster user isolation and task scheduling device based on shared accounts, characterized in that, The device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 6.

9. A computer-readable medium having computer program instructions stored thereon, characterized in that, The computer program instructions can be executed by a processor to implement the method as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 6.

Images