Cross-container gpu allocation method, system, and storage medium

Abstract

The application discloses a cross-container GPU allocation method and system and a storage medium, relates to the technical field of artificial intelligence, and comprises the following steps: a GPU allocator analyzes an application starting request, extracts a GPU demand quantity and application program starting parameters, obtains GPU card numbers and GPU allocation environment variable names of all GPU cards, traverses all GPU cards, and locates a lock file corresponding to the GPU card number in a lock file directory in a container; a flock system call is used to lock the lock file corresponding to the GPU card number, and after the GPU allocator determines that the number of lock files successfully locked reaches the GPU demand quantity, the GPU card numbers corresponding to the lock files successfully locked are spliced into a string and bound as a GPU allocation environment variable; a child process of the GPU allocator runs an application program; and an operating system unlocks the lock files successfully locked after determining that the child process ends. The application program in multiple containers can be safely run when sharing a physical GPU resource.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to cross-container GPU allocation methods, systems and storage media. Background Technology

[0002] With the rapid development of fields such as artificial intelligence and high-performance computing, graphics processing units (GPUs) have become core hardware resources for various computationally intensive applications due to their powerful parallel computing capabilities. Currently, most GPU applications require exclusive use of the GPU because if multiple programs simultaneously compete for the same GPU's video memory resources, it can easily lead to memory overflow, program crashes, and other runtime failures. In recent years, containerization technologies (such as Docker and Kubernetes) have been used to encapsulate each application in an independent container, achieving resource isolation between applications and resolving dependency conflicts between different software on the runtime environment. However, this isolation also makes it difficult for applications within different containers to directly communicate and synchronize resource states, potentially leading to resource contention conflicts when multiple applications within multiple containers share physical GPU resources in cross-container scenarios.

[0003] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0004] The main purpose of this application is to provide a cross-container GPU allocation method, system, and storage medium, which aims to solve the technical problem of resource conflicts caused by applications in multiple containers sharing physical GPU resources in cross-container scenarios.

[0005] To achieve the above objectives, this application proposes a cross-container GPU allocation method, the method comprising: The received application launch request is parsed by the GPU allocator inside the container, the number of GPUs required and the application launch parameters are extracted, and the GPU allocator queries the management command line of each graphics card manufacturer to obtain the graphics card number and GPU allocation environment variable name of each graphics card. The GPU allocator iterates through each graphics card, locates the lock file corresponding to the graphics card number in the preset lock file directory, and locks the lock file corresponding to the graphics card number by using the flock system call until the GPU allocator determines that the number of successfully locked lock files reaches the required number of GPUs. The GPU allocator then concatenates the graphics card numbers corresponding to the successfully locked lock files into a string and binds the string to the GPU allocation environment variable name to obtain the GPU allocation environment variable. The application is launched and run by a subprocess of the GPU allocator based on the GPU allocation environment variables and the application startup parameters, wherein the application accesses GPU resources in the graphics card specified by the GPU allocation environment variables during its operation; After the subprocess has finished, the operating system unlocks the successfully locked file.

[0006] Furthermore, to achieve the above objectives, this application also proposes a cross-container GPU allocation system, which includes: The GPU allocator is used to parse the received application launch request, extract the GPU requirement and application launch parameters, and query the management command line of each graphics card manufacturer to obtain the graphics card number and GPU allocation environment variable name of each graphics card. The GPU allocator is also used to traverse each of the graphics cards and locate the lock file corresponding to the graphics card number in the preset lock file directory; The operating system is used to implement the flock system call to lock the lock file corresponding to the graphics card number; The GPU allocator is also used to determine that after the number of successfully locked files reaches the number required by the GPU, it concatenates the graphics card number corresponding to the successfully locked file into a string and binds the string to the GPU allocation environment variable name to obtain the GPU allocation environment variable. A child process is used to start and run the application based on the GPU allocation environment variables and the application startup parameters, wherein the application accesses GPU resources in the graphics card specified by the GPU allocation environment variables during the running process; The operating system is also used to unlock the successfully locked file after determining that the subprocess has ended.

[0007] In addition, to achieve the above objectives, this application also proposes a storage medium, wherein the computer program storage medium is a computer-readable storage medium, and a computer program is stored on the computer program, which, when executed by a processor, implements the steps of the cross-container GPU allocation method described above.

[0008] In addition, to achieve the above objectives, this application also provides a lock file storage medium, which is a computer-readable and writable storage medium. The lock file storage medium stores a lock file of the cross-container GPU allocation method as described above. When the file lock is executed by the processor, the steps of the cross-container GPU allocation method as described above are implemented.

[0009] Upon receiving an application launch request, this application initiates a GPU allocator to parse the required number of GPUs and the application launch parameters, obtaining the graphics card ID and GPU allocation environment variable name. This achieves comprehensive awareness of available GPU resources within the system. Different manufacturers manage graphics cards differently; by uniformly querying the manufacturer's command line, the system can obtain standardized graphics card IDs and environment variable names, ensuring that subsequent resource allocation accurately associates with the specific physical GPU. This avoids resource identification confusion caused by manufacturer differences and provides a unified resource identification system for cross-container resource allocation.

[0010] The GPU allocator locates the lock file corresponding to the GPU ID in the preset lock file directory, uses the `flock` system call to lock it until the required number of GPUs are acquired, and binds the successfully locked GPU ID to an environment variable. This file-locking mechanism achieves mutual exclusion control of GPU resources. When multiple applications request GPUs simultaneously, the `flock` system call ensures that only one application can successfully lock a specific GPU at a time until it acquires a sufficient number of GPU resources. This effectively avoids problems such as memory overflow and program crashes caused by multiple applications simultaneously competing for the same GPU. Binding the GPU ID to an environment variable allows applications to accurately access the allocated GPUs through the environment variable, achieving precise alignment between resource allocation and application access.

[0011] By launching applications via child processes based on environment variables and startup parameters, it is ensured that applications access only their allocated GPU resources during runtime. As independent execution units within the container, child processes strictly adhere to the GPU resources specified in the environment variables, preventing applications from accessing other GPUs without authorization. This further strengthens resource isolation while ensuring accurate resource usage, enabling each application to run stably on its dedicated GPU.

[0012] Unlocking the lock file after the child process ends enables dynamic release and recycling of GPU resources. When the application finishes running, the corresponding lock file is unlocked promptly, allowing other container applications to re-request and use these GPU resources. This improves GPU resource utilization, avoids situations where resources are occupied and idle for extended periods, and resolves resource conflicts while ensuring efficient resource utilization.

[0013] In summary, this application ensures the accuracy of resource access through environment variable binding, while dynamic unlocking improves resource utilization, thereby enabling secure operation of applications within multiple containers when sharing physical GPU resources. Attached Figure Description

[0014] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the principles of this application.

[0015] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is a flowchart illustrating an embodiment of the cross-container GPU allocation method of this application. Figure 2 This is a flowchart illustrating Embodiment 2 of the cross-container GPU allocation method of this application; Figure 3 This is a flowchart illustrating Embodiment 3 of the cross-container GPU allocation method of this application; Figure 4 This is a simplified flowchart illustrating the use of file locks to lock a specified number of lock files according to an embodiment of this application; Figure 5 This is a flowchart illustrating Embodiment 4 of the cross-container GPU allocation method of this application; Figure 6 A simplified flowchart illustrating a cross-container GPU allocation method provided in an embodiment of this application; Figure 7 This is a schematic diagram of the architecture of a cross-container GPU allocation method according to an embodiment of this application; Figure 8 This is a schematic diagram illustrating the application process of a cross-container GPU allocation method according to an embodiment of this application; Figure 9 This is a schematic diagram of the module structure of the cross-container GPU allocation system according to an embodiment of this application; Figure 10 This is a schematic diagram of the device structure of the hardware operating environment involved in the cross-container GPU allocation method in this application embodiment.

[0017] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0018] It should be understood that the specific embodiments described in this embodiment are merely used to explain the technical solutions of this application and are not intended to limit this application.

[0019] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0020] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device capable of performing the above functions. The following description uses an electronic device as an example to illustrate this embodiment and the subsequent embodiments. It should be noted that the operating system running on the electronic device in this embodiment supports containerization technology and has at least one graphics card driver installed.

[0021] Based on this, embodiments of this application provide a cross-container GPU allocation method, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the cross-container GPU allocation method of this application.

[0022] In this embodiment, the cross-container GPU allocation method includes steps S10 to S40: Step S10: The received application launch request is parsed through the GPU allocator in the container, the GPU requirement quantity and application launch parameters are extracted, and the GPU allocator is used to query the management command line of each graphics card manufacturer to obtain the graphics card number and GPU allocation environment variable name of each graphics card.

[0023] Containers are lightweight virtualization units created based on containerization technologies such as Docker and Kubernetes. They encapsulate applications and their dependencies into independent runtime environments, achieving resource isolation and environment consistency between applications. A container can contain at least one application. A GPU allocator is a dedicated component deployed inside a container. Its core functions are parsing application resource requirements, querying GPU resource information, and performing GPU resource locking and releasing. It serves as the central hub connecting application requirements and hardware resources. An application startup request is a user-initiated instruction used to launch a target application, containing the resource requirements (such as the number of GPUs) and configuration parameters (such as running mode and input path) needed for the application to run.

[0024] Upon receiving an application launch request, the GPU allocator is automatically triggered. Once started, the GPU allocator first performs semantic parsing and parameter extraction on the application launch request, identifying and extracting the user-specified GPU requirement quantity to clarify the quantitative target of resource allocation. It also extracts application launch parameters to determine the application's runtime configuration requirements. This parsing process provides a clear and accurate basis for subsequent GPU resource queries and allocation operations, effectively preventing the resource allocation process from deviating from the actual needs of the application and ensuring the accuracy and effectiveness of GPU resource allocation.

[0025] It should be noted that during container startup, the preset lock file directory is mounted to a specified path inside the container via the container engine's mount interface. This ensures that the GPU allocator within the container can access and manipulate the lock file, achieving mutual exclusion control of GPU resources. Furthermore, permissions are configured at container startup to allow the container to access all GPU resources on the host machine, preventing the GPU allocator from being unable to retrieve or use certain GPU resources due to container permission restrictions. It is important to emphasize that the configuration of the aforementioned container startup-related parameters pertains to the container creation phase. For containers that have already been created and are running, as long as the above two parameter configuration requirements are met, or if additional configuration is performed during container operation using container engine tools, the GPU allocator of this embodiment can still be deployed and used for dynamic GPU resource allocation.

[0026] Graphics card manufacturer management command lines are command-line tools provided by different graphics card manufacturers for managing and querying graphics card information, such as NVIDIA's nvidia-smi and AMD's rocm-smi. They can obtain graphics card hardware information, status, and configuration parameters. The graphics card number is a unique identifier for each graphics card in the system, assigned by the graphics card manufacturer's management command line, used to distinguish different physical GPUs. For example, NVIDIA graphics cards are typically numbered 0, 1, 2, etc. GPU-assigned environment variable names are used to specify the environment variable identifiers used by applications for a particular GPU. Different manufacturers use different environment variable names; for example, NVIDIA uses "CUDA_VISIBLE_DEVICES", and AMD uses "ROCR_VISIBLE_DEVICES".

[0027] To accurately identify and manage GPU resources within the system, it's necessary to collect basic information about all available GPUs by querying the management command-line tools provided by each graphics card manufacturer. Specifically, the GPU allocator invokes system commands to execute the management command-line tools of each graphics card manufacturer sequentially. For example, first executing "nvidia-smi --list-gpus" to query NVIDIA graphics cards, then executing "rocm-smi --showid" to query AMD graphics cards. The command-line output is parsed to extract the unique identifier of each graphics card and its corresponding GPU allocation environment variable name. The obtained graphics card identifiers and environment variable names are stored in a mapping table, establishing a correspondence between the graphics card's physical identifier and the environment variable. In essence, querying the manufacturer's management command-line tools provides a comprehensive understanding of the currently available GPU resources, including the unique identifier of each graphics card and its corresponding environment variable name. This solves the problem of difficulty in identifying GPU resources due to manufacturer differences. The established mapping between graphics card identifiers and environment variable names provides an accurate basis for subsequent GPU resource locking and targeted application access, ensuring the consistency and accuracy of cross-vendor GPU resource management.

[0028] Step S20: The GPU allocator traverses each graphics card, locates the lock file corresponding to the graphics card number in the preset lock file directory, and locks the lock file corresponding to the graphics card number by using the flock system call until the GPU allocator determines that the number of successfully locked lock files reaches the required number of GPUs. The GPU allocator concatenates the graphics card numbers corresponding to the successfully locked lock files into a string and binds the string to the GPU allocation environment variable name to obtain the GPU allocation environment variable.

[0029] The default lock file directory is a pre-defined directory used to store GPU lock files. All graphics card lock files are centrally stored in this directory for unified management and access. Each lock file corresponds uniquely to a graphics card and is used to implement exclusive control over GPU resources through a file-based locking mechanism. One lock file corresponds to one physical GPU. The flock system call is an operating system call used to apply exclusive or shared locks to files, ensuring that only one process can operate on a locked file at a time, thus achieving mutual exclusion access to resources.

[0030] It should be noted that if the GPU allocator locates the lock file corresponding to a specific GPU ID in the lock file directory within the container and finds that the lock file does not exist (e.g., it may not have been pre-created when deploying a new machine, or the lock file may have been accidentally deleted), the dynamic creation process of the lock file will be automatically triggered. The GPU allocator calls a file creation function (such as the "open" function in Linux) to create the lock file corresponding to the GPU ID in the lock file directory within the container, and sets the permissions of the lock file to be readable and writable by all users. It is understandable that in a multi-container scenario, different containers may start with different user identities. If the lock file only allows specific users to read and write, the GPU allocator for unauthorized users will be unable to perform locking / unlocking operations. By setting readable and writable permissions for all users, it can be ensured that all containers can access the lock file normally, avoiding interruption of the GPU allocation process due to permission issues.

[0031] The GPU allocator locates the corresponding lock file in the lock file directory based on the previously obtained GPU ID. It uses the `flock` system call to lock these lock files. If a lock file is already locked by another container, the current locking operation fails, and the GPU allocator skips that lock file, continuing to iterate through the lock files corresponding to the next GPU ID until the number of successfully locked lock files reaches the application's required GPU count. The GPU allocator then concatenates these successfully locked GPU IDs into a string and binds it to the corresponding GPU allocation environment variable name, forming the GPU allocation environment variable. In this embodiment, the specific process may be as follows: The GPU allocator accesses a preset lock file directory and finds the lock file path corresponding to each graphics card according to the graphics card number mapping table; iterates through each graphics card and uses the flock system call to perform exclusive locking operations on these lock files; if the locking is successful, the graphics card number is recorded; if the locking fails, the lock file is skipped and the next one is tried; the locking attempts continue until the number of successfully locked graphics cards equals the number of GPUs required; the successfully locked graphics card numbers are concatenated into a string in sequence, and the corresponding GPU allocation environment variable name is found according to the manufacturer of these graphics cards. The string is bound to the environment variable name to generate a GPU allocation environment variable. The GPU allocation environment variable is an environment variable composed of the GPU allocation environment variable name and the successfully locked graphics card number string, which is used to inform the application of the GPU resources that can be accessed.

[0032] Understandably, locking is the process of applying an exclusive lock to a lock file using the `flock` system call. After successful locking, other processes cannot lock the same file until the lock is released. In this embodiment, when the GPU allocator uses the `flock` system call to perform a locking operation on the lock file, it adopts a combination of "exclusive lock + non-blocking" (`LOCK_EX | LOCK_NB`) mode to avoid the GPU allocator getting stuck in a long-term blocking state and ensure the efficiency of the resource allocation process. If a blocking mode is used, when the target lock file is occupied, the current GPU allocator will wait for the lock to be released, during which time it cannot process locking requests from other graphics cards, resulting in a significant increase in resource allocation time. The non-blocking mode allows the GPU allocator to quickly traverse all available graphics cards and prioritize the acquisition of unoccupied GPU resources, significantly shortening the overall resource allocation time and improving the system's response speed in scenarios with multiple container concurrent requests.

[0033] In this embodiment, the combination of locking files and the `flock` system call enables mutual exclusion access control of GPU resources, solving problems such as memory overflow and program crashes caused by multiple containers simultaneously preempting the same GPU. This ensures the security and orderliness of GPU resource allocation. By dynamically locking until the required quantity is met, on-demand allocation of GPU resources is achieved, avoiding over-allocation or under-allocation of resources. The generation of GPU allocation environment variables indicates the accessible GPU resources for the application, achieving precise docking between resource allocation and application access.

[0034] Step S30: The application is started and run by a subprocess of the GPU allocator based on the GPU allocation environment variable and the application startup parameters, wherein the application accesses the GPU resources in the graphics card specified by the GPU allocation environment variable during the running process.

[0035] Child processes are created by the GPU allocator process and are dedicated to running applications. They are independent of their parent processes, facilitating monitoring and management of the application's running status. It's important to note that these child processes inherit the GPU allocator's file descriptors and all environment variables. Furthermore, based on the operating system's process management mechanism, when the GPU allocator parent process is abnormally killed (e.g., by a user executing a "kill" command or a container being forcibly stopped), its created application child processes will be forcibly terminated by the operating system. This linkage mechanism prevents the abnormal scenario where the GPU allocator parent process has terminated, but the application child process is still running, thus avoiding the continuous occupation of allocated GPU resources by the application child process and preventing permanent GPU resource leakage.

[0036] The GPU allocator within the container creates a child process, passing the previously generated GPU allocation environment variables and application startup parameters to the child process. When the child process starts the application, it loads these environment variables and parameters into the application's runtime environment. During runtime, the application loads the GPU driver, reads the GPU allocation environment variables, and obtains the GPU numbers it is allowed to use, thus accessing only these specified GPU resources and preventing unauthorized access to other unallocated GPUs. In essence, by limiting the application's GPU access scope through GPU allocation environment variables, it prevents the application from occupying unallocated GPUs, further strengthening resource isolation and ensuring stability and security when multiple containers share GPU resources.

[0037] Step S40: After the subprocess has ended, the successfully locked file is unlocked by the operating system.

[0038] After the application running the child process terminates normally after completing all computing tasks, or after the application running the child process terminates abnormally due to an error, the GPU allocator automatically ends, the operating system releases the exclusive locks on all locked files in the lock list, and the corresponding GPU resources change from an occupied state to an available state, making them available for other containers to apply for use.

[0039] In this embodiment, a complete cross-container GPU resource management mechanism is formed by creating a container and starting a GPU allocator to parse requirements, querying graphics card information to obtain resource identifiers, implementing exclusive allocation of GPU resources based on lock files and the Flock system call, controlling application-directed access to the GPU through child processes and environment variables, and unlocking resources in a timely manner after the child process ends. This mechanism solves the problem of contention when multiple applications share GPU resources, avoids risks such as memory overflow and program crashes, and improves the utilization rate of GPU resources through dynamic allocation and release of resources. The isolation of the container ensures the independence of the application's runtime environment, the targeted control of environment variables ensures the accuracy of resource access, and the lock file mechanism realizes the security and orderliness of resource allocation. Ultimately, it enables secure, efficient, and orderly sharing of physical GPU resources by multiple applications within the container.

[0040] It is understood that this embodiment has the ability to coordinate GPU resources across container boundaries. That is, through the GPU allocator in each container, applications in the container can apply for GPU resources by deploying the GPU allocator, and applications in different containers can apply for GPU resources through their respective container GPU allocators. The GPU allocators in different containers achieve exclusive allocation of GPU resources based on lock files and the Flock system call, thus avoiding applications in different containers from competing for GPU resources.

[0041] In one feasible embodiment, before step S10: parsing the received application launch request through the GPU allocator within the container, extracting the GPU requirement quantity and application launch parameters, and querying the management command lines of each graphics card manufacturer through the GPU allocator to obtain the graphics card number and GPU allocation environment variable name of each graphics card, the method further includes: Step S01: Read the lock file directory path and mount the lock file directory to the specified path of the container through the mounting interface of the container engine.

[0042] In this embodiment, the lock file directory path can be read from a preset configuration file and mounted to a specified path in the container through the container engine's mounting interface. This method supports batch adjustment of paths by modifying the configuration file without modifying the allocator code. Alternatively, the allocator may have a built-in lock file directory path, which eliminates the need to create an additional configuration file and simplifies the deployment process. The above two implementation methods are merely feasible implementations for reading the lock file directory path and do not constitute a limitation on the process of reading the lock file directory path. The specific implementation can be set according to actual needs.

[0043] The mount interface of a container engine is a programming interface or command-line tool provided by the container engine to map directories or files in the host file system to the inside of the container. Examples include Docker's `-v` parameter and Kubernetes' volume mount mechanism, enabling file sharing between the host and containers. The container-specified path is the path inside the container used to associate the host lock file directory, such as " / tmp / gpu_locks". Processes within the container can access the contents of the host lock file directory through this path; it acts as a bridge for accessing host lock files within the container.

[0044] Before container creation and GPU allocator startup, it's crucial to ensure the container can access the lock file directory on the host machine, as the lock file is the core medium for achieving mutually exclusive allocation of GPU resources across containers. This step first extracts the lock file's storage path on the host from a pre-defined configuration file, clarifying the physical location of resource control. Then, through the container engine's mount interface, it binds the host's lock file directory to the specified path within the container. This provides the foundational path support for the GPU allocator's subsequent operations on the lock file within the container, a prerequisite for cross-container resource coordination. Understandably, managing the lock file directory path uniformly through a pre-defined configuration file improves system configurability and maintainability, allowing adjustments to the lock file storage location without code modification. The mount operation breaks down file isolation between the container and the host, ensuring the GPU allocator within the container can normally access and manipulate the lock file on the host, providing the necessary conditions for subsequent mutually exclusive allocation of GPU resources based on the lock file, and avoiding resource control failures caused by container isolation.

[0045] Step S02: The lock file directory path is written to the container's allocator configuration file and passed to the process within the container via the GPU allocator.

[0046] The allocator configuration file is used to store the runtime configuration parameters of the GPU allocator. Its generation and transmission can be as follows: it can be automatically generated by the GPU allocator at startup, populating basic parameters based on the hardware environment and default configuration strategy; or it can be passed in from outside by the user or upper-layer system, injecting custom configuration parameters into the allocator by specifying a file path. Container processes are processes running inside a container; in this embodiment, it specifically refers to the GPU allocator and the application processes it launches. These processes need to know the lock file directory path to perform lock file operations.

[0047] After mounting the lock file directory, the corresponding lock file directory path inside the container is informed to the process inside the container. By writing this path into the allocator configuration file, the process inside the container can obtain the path information by reading the configuration file, ensuring that the process can accurately locate the lock file directory at runtime. This realizes the transfer of configuration information from the host to the process inside the container, avoids the lack of flexibility caused by hard-coded paths, and ensures that the process can dynamically adapt to different mount path configurations.

[0048] In this embodiment, by reading the path from the preset configuration file and mounting it, the lock file access problem caused by container isolation is solved, ensuring the feasibility of sharing lock files across containers. By writing the path into the allocator configuration file and passing it to the process inside the container, it is ensured that the process inside the container can accurately obtain the lock file location, providing a reliable foundation for subsequent lock file operations.

[0049] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in the first embodiment described above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 2 Step S10: The step of parsing the received application launch request through the GPU allocator within the container and extracting the GPU requirement and application launch parameters includes: Step S101: The GPU allocator checks whether the GPU quantity field in the application startup request has a valid value.

[0050] The GPU count field is a specific field in the application launch request used to specify the number of GPUs required by the application. Examples include fields identified by "--gpu-num" and "gpu_count" in the request parameters. A valid value refers to a GPU count that conforms to preset rules, typically a positive integer, and this value does not exceed the total number of available GPUs in the system, ensuring the request's validity.

[0051] When parsing an application launch request, the GPU allocator first focuses on the GPU quantity field, which is crucial information for determining resource requirements. By checking if this field has a valid value, it can determine whether the user has explicitly specified the required number of GPUs. In this embodiment, the specific process is as follows: After receiving the application launch request, the GPU allocator iterates through the fields in the request, locates the GPU quantity field, and checks if this field has a value. If no value exists, it is determined to be invalid; if a value exists, it is further verified whether the value is a non-negative integer and does not exceed the total number of available GPUs in the system. If both conditions are met, it is determined to be a valid value. It is understandable that by checking the validity of the GPU quantity field, unreasonable resource requirement requests can be identified in advance, avoiding subsequent resource allocation based on erroneous information and reducing invalid operations. At the same time, it provides a clear judgment standard for determining the required number of GPUs based on different situations, ensuring the accuracy and rationality of resource allocation.

[0052] Step S102: If the valid value exists, the value of the GPU quantity field is determined as the GPU requirement quantity, and the field values ​​in the application startup request other than the GPU quantity field are determined as the application startup parameters.

[0053] When a valid value is detected in the GPU quantity field, it indicates that the user has explicitly specified the required number of GPUs. This value is then directly used as the GPU requirement to meet the user's personalized resource needs. Other field values ​​besides the GPU quantity field are extracted from the request as application startup parameters to ensure that other application configurations are executed. Understandably, directly using the valid GPU quantity specified by the user as the required quantity satisfies the differentiated GPU resource needs of different applications, improves the flexibility and targeting of resource allocation, accurately extracts application startup parameters, ensures that the application runs according to the user's expected configuration, and guarantees the application's functionality.

[0054] In this embodiment, by detecting the validity of the GPU quantity field, different situations are handled accordingly. When the user does not explicitly or incorrectly specify the number of GPUs, the default value is used, and when the user explicitly specifies it, the value is used directly. At the same time, the application startup parameters are accurately extracted, which not only ensures the rationality and flexibility of resource allocation, but also improves the fault tolerance and ease of use of the system.

[0055] Furthermore, in this embodiment, if no valid value exists, that is, if the value of the GPU quantity field is negative, a decimal, and / or exceeds the total number of available GPUs in the system, an error is reported to avoid invalid requests continuing to occupy resources and to improve the maintainability and robustness of the system.

[0056] In one feasible embodiment, step S10, which involves querying the management command lines of each graphics card manufacturer through the GPU allocator to obtain the graphics card number and GPU allocation environment variable name of each graphics card, includes: Step S103: The GPU allocator iterates through each graphics card manufacturer in the preset graphics card manufacturer list and calls the management command line of the graphics card manufacturer to send a query command to the graphics card of the graphics card manufacturer.

[0057] The default graphics card manufacturer list is a pre-set list of mainstream graphics card manufacturers, such as NVIDIA, AMD, and Intel. This list is updated dynamically according to industry developments to ensure coverage of common graphics card brands currently on the market. The graphics card manufacturer's management command line is a dedicated command-line tool provided by each manufacturer for their graphics cards. It is used to query and manage the graphics card's hardware information, operating status, etc. Query commands are sent to the graphics card through the management command line to retrieve relevant information. For example, "nvidia-smi --query-gpu=index,name --format=csv" is used to query the serial number and name of an NVIDIA graphics card.

[0058] Since the command-line tools for managing graphics cards differ among manufacturers, categorizing and calling the corresponding command lines by manufacturer ensures the compatibility and effectiveness of the query commands. In this embodiment, the specific process may be as follows: The system loads a preset list of graphics card manufacturers and selects each manufacturer in the list sequentially; for each manufacturer, its corresponding management command-line tool is invoked, and a query command is sent to all graphics cards produced by that manufacturer. The command content covers the need to obtain graphics card hardware information and status information. After sending the command, the system waits for the graphics card to return response information.

[0059] Understandably, traversing the preset manufacturer list ensures that the system covers all mainstream graphics card manufacturers, avoiding the omission of any brand of graphics card, thus improving the system's versatility. Calling the corresponding manufacturer's management command line to send query commands ensures the compatibility of commands with graphics cards, ensuring that the required information can be accurately obtained. This lays the foundation for subsequent extraction of key data and reduces the problem of information retrieval failure due to command incompatibility.

[0060] Step S104: Obtain the graphics card hardware information and graphics card status information returned based on the query command, extract the graphics card number based on the graphics card hardware information, and extract the GPU allocation environment variable name based on the graphics card status information.

[0061] Graphics card hardware information refers to the inherent hardware parameters of the graphics card itself, including the graphics card model, serial number, core frequency, and memory capacity. For example, in "GPU0: NVIDIA GeForce RTX 3090 (UUID:GPU-xxx)", the "0" represents the serial number from the hardware information. Graphics card status information is the current operating status data of the graphics card, including temperature, memory usage, power consumption, and whether it supports specific environment variables. For instance, some graphics card status information will indicate the supported resource control environment variables.

[0062] After receiving the information returned by the graphics card, the information needs to be parsed to extract the key data required for subsequent GPU allocation. The graphics card hardware information is the basis for extracting the graphics card number, because the number is an inherent identifier of the graphics card and does not change with the running state. The GPU allocation environment variable name is related to the graphics card driver and operating mechanism, and is usually contained in the status information. In this embodiment, the specific process can be as follows: divide it into two parts: hardware information and status information; for hardware information, extract the graphics card number by parsing the string, for example, extract "0" from "index=0"; for GPU allocation environment variable name, obtain it according to the graphics card manufacturer. For example, when the graphics card number is successfully obtained by executing nvidia-smi (NVIDIA's graphics card management command), the environment variable name is directly set to "CUDA_VISIBLE_DEVICES" corresponding to NVIDIA; similarly, for graphics cards from other manufacturers, set it according to the predefined environment variable names of their respective manufacturers; associate and store the extracted graphics card number and the corresponding GPU allocation environment variable name to form a "graphics card number-environment variable name" mapping table.

[0063] Understandably, accurately extracting the graphics card ID and GPU allocation environment variable name from the returned information ensures that the target graphics card can be accurately identified and the correct environment variables can be set during subsequent GPU resource allocation. By establishing a mapping between the two, a direct basis is provided for applications to access specific GPUs, avoiding resource allocation chaos caused by incorrect identification and improving the accuracy and reliability of cross-container GPU allocation.

[0064] In this embodiment, by traversing the preset manufacturer list and calling the corresponding management command line to send query commands, comprehensive coverage of graphics cards from multiple manufacturers and compatibility of information query are ensured; by extracting key data from the returned information by category, the accuracy of graphics card number and environment variable name is guaranteed.

[0065] Based on the first and / or second embodiments of this application, in the third embodiment of this application, the content that is the same as or similar to the first and / or second embodiments described above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 3Step S20: The GPU allocator traverses each graphics card, locates the lock file corresponding to the graphics card number in the preset lock file directory, and locks the lock file corresponding to the graphics card number using the `flock` system call until the GPU allocator determines that the number of successfully locked lock files reaches the required number of GPUs. The GPU allocator then concatenates the graphics card numbers corresponding to the successfully locked lock files into a string and binds the string to the GPU allocation environment variable name to obtain the GPU allocation environment variable. This step includes: Step S201: The GPU allocator traverses each graphics card, locates the lock file corresponding to the graphics card number in the preset lock file directory, and locks the lock file corresponding to the graphics card number in an exclusive and non-blocking manner by using the flock system call.

[0066] The exclusive and non-blocking mode is the locking mode of the `flock` system call. "Exclusive" means that only one process is allowed to lock the file at a time, ensuring exclusive use of GPU resources. "Non-blocking" means that if the file is already locked by another process, the current process will not wait but will immediately return a failure to lock, avoiding prolonged blocking. The `flock` system call is a system function used to lock files, supporting exclusive locks, shared locks, and other modes. It achieves resource synchronization between processes through file locks, and in this embodiment, it is used to control access permissions for GPU resources.

[0067] The system iterates through all graphics cards sequentially. For each graphics card, it locates the corresponding lock file in the preset lock file directory based on its ID (e.g., "lock_0.txt" for graphics card ID 0). When using the `flock` system call, it specifies the "exclusive and non-blocking" parameter. This ensures that the same GPU is not occupied by multiple applications simultaneously and prevents the current process from stalling while waiting for the lock to be released. In this embodiment, the specific process can be as follows: the system selects graphics cards sequentially from the graphics card list and obtains their IDs; it locates the lock file path in the lock file directory based on the ID; it uses the `flock` system call, passing in the lock file descriptor and the "exclusive + non-blocking" parameter, to attempt to lock the lock file; if the lock file is not occupied, the lock is successfully acquired; if it is occupied, it immediately returns failure without waiting. It is understandable that the exclusive mode ensures mutual exclusion access to GPU resources, completely avoiding conflicts caused by multiple containers competing for the same GPU; the non-blocking mode improves the efficiency of the locking operation, preventing processes from wasting resources while waiting for the lock to be released, and significantly improving system response speed, especially in high-concurrency scenarios.

[0068] In step S202, if the locking is successful, the lock file corresponding to the graphics card number is added to the preset lock list, and the number of successfully locked files is incremented by one.

[0069] The preset lock list records information about successfully locked files, including the file path and corresponding GPU ID, serving as the basis for subsequent confirmation and release of lock files. The file count represents the total number of successfully locked files, used to determine whether the application requires the required number of GPUs.

[0070] Once a file is successfully locked, its information is stored in a pre-defined lock list, and the number of successfully locked files is incremented. This cumulative count tracks resource acquisition progress, providing a clear indication of whether the required GPU resources have been met. The lock list clearly records the occupied GPU resources, facilitating subsequent unlocking operations and status queries; the cumulative count directly reflects the resource acquisition progress, providing a quick basis for determining whether the requirements are met and avoiding issues of duplicate locking or missed statistics.

[0071] Step S203: If the number of files reaches the number required by the GPU, then the lock files in the lock list are determined as successfully locked files.

[0072] When the number of successfully locked files reaches the GPU's required limit, it indicates that sufficient GPU resources have been acquired. At this point, there's no need to continue iterating through other graphics cards; the locked files in the lock list are directly confirmed as the finally successfully locked files. Understandably, promptly terminating unnecessary locking operations reduces system resource consumption while ensuring that the application can quickly acquire the necessary GPU resources, shortening resource allocation time.

[0073] Step S204: If the number of files does not reach the number required by the GPU, then check whether each of the graphics cards has been traversed.

[0074] When the number of successfully acquired locks is insufficient, it is necessary to determine whether all graphics cards have been traversed. If not, continue attempting to acquire locks; if traversal is complete, a retry strategy must be implemented. Understandably, by checking the traversal status, premature termination of locking attempts before all graphics cards have been traversed is avoided, thus improving the success rate of resource acquisition. Simultaneously, it provides a trigger for the subsequent retry mechanism, ensuring that the required GPU can be acquired through retries when resources are temporarily scarce.

[0075] Step S205: If the traversal of each graphics card is completed, the locked file is released, a pause duration is randomly generated, and after the locking operation is paused until the pause duration, the number of files is set to zero, and the step of traversing each graphics card and subsequent steps are executed.

[0076] Releasing a locked file involves using the `close` system call to close the locked file and having the operating system unlock all files in the lock list, making these GPU resources available again. The pause duration is a randomly generated retry interval used to wait for other containers to release GPU resources, avoiding the system overhead caused by frequent retries in a short period. Simultaneously, the randomized pause duration prevents multiple containers' GPU allocators from retrying locking operations simultaneously, preventing lock contention storms caused by excessive concurrent requests, reducing unnecessary resource occupation, and ensuring the overall stability and resource utilization efficiency of the system. In specific implementations, the pause duration can be randomly generated within a preset range to avoid excessively long pauses that could impact business processing efficiency.

[0077] If, after iterating through all graphics cards, insufficient GPUs are acquired, the locked files are released to prevent resource fragmentation caused by partial resource occupation. The process then resumes after a preset timeout. In this embodiment, the specific process might be: if all graphics cards have been iterated and the number of locked files is insufficient, the `close` system call is used to close all locked files in the lock list, and the operating system performs the unlocking operation; the system pauses the locking operation and waits for the preset timeout; after the wait, the number of successfully locked files is reset to 0, the lock list is cleared, and the process resumes iterating through all graphics cards and performing the locking operation. It is understood that releasing some locked resources avoids resource fragmentation and improves overall resource utilization; the waiting time prevents frequent retries from impacting the system; and the retry mechanism ensures that the current container can acquire resources promptly after other containers release GPUs, improving the flexibility and success rate of resource allocation.

[0078] Step S206: If each of the graphics cards has not been traversed, then the step of locating the lock file corresponding to the graphics card number in the preset lock file directory and subsequent steps are performed on the next graphics card.

[0079] When the traversal of graphics cards is incomplete and the number of locks acquired is insufficient, the system continues processing the next graphics card, repeating the process of locating the lock file and acquiring the lock. In this embodiment, the specific process may be: if there are still untraversed graphics cards, the system selects the next graphics card, locates its corresponding lock file in the lock file directory, uses the `flock` system call to perform the locking operation, and processes the locking result according to the steps described above. This ensures the completeness of the traversal, does not overlook any potentially available GPU resources, improves the locking success rate in resource-rich scenarios, and ensures that the application can acquire the required resources as quickly as possible.

[0080] Step S207: If locking fails, then execute the step of detecting whether each of the graphics cards has been traversed and subsequent steps.

[0081] When locking a certain file fails, the system checks whether the graphics card traversal is complete, and then decides whether to continue traversing to the next graphics card or enter a retry process. It is understood that this embodiment simplifies the handling logic when locking fails, avoids unnecessary resource consumption, ensures the system can quickly switch to other available GPUs, and improves the overall efficiency of the process.

[0082] Please refer to Figure 4 The specific locking process can be as follows: Begin by iterating through all graphics card numbers while resetting the lock count to zero, officially starting the process related to graphics card number iteration and locking; attempt to lock the lock file corresponding to the current number; if the lock is successful, add the lock file corresponding to that graphics card number to the preset lock list and increment the lock count by one. Then, determine if the lock count has reached the required number: if it has, end and return to the lock list; if not, continue to determine if all graphics cards have been iterated through. If not all graphics cards have been iterated through, continue to the next graphics card number and repeat the previous locking attempts; if all graphics cards have been iterated through and the lock count still has not reached the required number, proceed according to the appropriate logic. If attempting to lock the lock file corresponding to the current number fails, the process will again proceed to determine whether all graphics cards have been traversed. If not, the process will continue to attempt to lock the next graphics card number. If all graphics cards have been traversed, all existing locks will be released, and the locking operation will stop after a random pause period. The process will then return to the initial step of "starting to traverse all graphics card numbers and clearing the lock count to zero," and a new round of traversal and locking will begin. This process will continue until the lock count meets the requirements or all traversal conditions are met.

[0083] In this embodiment, the "exclusive and non-blocking" locking method ensures resource mutual exclusion while avoiding process blocking; the lock list and quantity count enable precise tracking of resource acquisition progress; and the traversal control and retry mechanism allow for flexible adjustment of strategies when resources are insufficient, maximizing the success rate of resource acquisition.

[0084] Based on the first, second, and / or third embodiments of this application, in the fourth embodiment of this application, the content that is the same as or similar to the first, second, and / or third embodiments described above can be referred to the above description and will not be repeated hereafter. Based on this, please refer to... Figure 5 Step S30: The step of starting and running the application through the subprocess of the GPU allocator based on the GPU allocation environment variables and the application startup parameters includes: Step S301: Create a child process of the GPU allocator through the GPU allocator, wherein the child process inherits the file descriptors and all environment variables of the GPU allocator.

[0085] The GPU allocator uses the operating system's `fork` and `exec` system calls to create child processes. When a child process is created, it automatically copies the GPU allocator's environment variable list and file descriptors. A file descriptor is a unique identifier for a file opened by the GPU allocator. These files can be lock file directories, allocator configuration files, runtime log files, etc. File descriptors include, but are not limited to, standard input streams, standard output streams, and standard error streams. The standard input stream provides input data to the child process, allowing it to obtain the necessary information for operation. The standard output stream outputs normal runtime information, allowing users to view its running status and results. The standard error stream outputs error messages generated during the child process's execution, enabling developers or users to quickly locate and resolve problems.

[0086] Step S302: Set the GPU allocation environment variables in the subprocess through the GPU allocator.

[0087] The GPU allocator writes the generated GPU allocation environment variables into the environment variable list of child processes. It can be understood that there may be multiple applications within a container. By writing the GPU allocation environment variables through the GPU allocator, compared to the direct inheritance by child processes, the GPU allocation environment variables can be made effective only in child processes and do not affect the parent process.

[0088] Step S303: Execute the application specified by the application startup parameters through the subprocess. When the application starts, it reads the GPU allocation environment variables through the GPU driver library and accesses the GPU resources in the graphics card specified by the GPU allocation environment variables through the GPU driver library.

[0089] GPU driver libraries are low-level software libraries provided by graphics card manufacturers to enable interaction between applications and GPU hardware, providing interfaces for memory allocation, computing core invocation, device management, and other functions.

[0090] The child process calls the corresponding application executable file based on the application startup parameters and passes the startup parameters to the application. After the application starts, it reads the GPU allocation environment variables by calling the device management interface of the GPU driver library. The GPU driver library accesses the GPU resources in the graphics card specified by the GPU allocation environment variables, while blocking other GPU resources. This ensures that the application can only use the allocated GPU and cannot access unlocked GPUs, thus preventing unauthorized use of resources.

[0091] In one feasible embodiment, the method further includes: S50, when the subprocess finishes running, the return code is passed to the GPU allocator through the subprocess, and the GPU allocator uses the return code as its own return code and terminates the operation of the GPU allocator; A return code is an integer code returned by a child process to its parent process upon termination, indicating the child process's running status. Typically, 0 indicates normal termination, while non-zero values ​​represent different types of errors or exceptions. Obtaining the return code after a child process finishes execution provides a clear indication of whether the task was completed successfully or encountered an error, offering a basis for subsequent resource reclamation and status logging. The GPU allocator's return code is an integer identifier returned to the upper-level caller upon termination, representing the final state of the entire cross-container GPU allocation process. Its value is consistent with the child process's return code.

[0092] When the child process terminates, it transmits a return code to the GPU allocator via the operating system. Upon receiving the return code, the GPU allocator performs a series of cleanup operations, including closing open configuration files, releasing temporary memory caches, and shutting down communication channels with the upper-layer system. Then, it calls the process termination interface, passing the received child process return code as a parameter to inform the upper-layer caller, thus terminating the GPU allocator process. The operating system then reclaims the CPU resources and memory space occupied by the GPU allocator process.

[0093] Step S40: After determining that the child process has ended, the step of unlocking the successfully locked file through the operating system includes: Step S401: After determining that the subprocess has ended, the operating system closes the file descriptor of the GPU allocator to release the lock file associated with the file descriptor into an unlocked state.

[0094] After the operating system determines that the child process and the GPU allocator have terminated, it queries the list of file descriptors held by the GPU allocator and filters out the descriptors associated with the "successfully locked file". The operating system then calls the file descriptor closing interface to close the filtered descriptors one by one, thereby releasing the exclusive lock on the file. After the lock file is released, its state changes from "locked" to "unlocked". The GPU allocator can detect that the file has been unlocked by using the `flock` system call and then initiate a new locking request. It is understood that this embodiment ensures that GPU resources are released in a timely manner, avoiding long-term resource occupation, improving GPU resource utilization, and recording resource reclamation status in the host machine's resource management log, providing administrators with a complete resource usage trajectory, facilitating resource auditing, problem investigation, and performance optimization, and enhancing system manageability and traceability.

[0095] For example, to help understand the implementation process of the cross-container GPU allocation method obtained by combining this embodiment with the above embodiment one, please refer to... Figure 6 , Figure 6 A simplified flowchart illustrating a cross-container GPU allocation method is provided, specifically: 1. Launch the application using the GPU allocator. The user adds the GPU allocator script name before the original application launch command. The number of GPUs to be allocated can be specified after the script name; if not specified, it defaults to 1.

[0096] 2. Parse the request parameters to obtain the number of GPUs to be allocated and the parameters of the application to be executed. The GPU allocator determines whether the first parameter is an integer. If it is, it uses that parameter as the number of GPUs to be allocated, and the remaining parameters are used as the application startup parameters.

[0097] 3. Determine the graphics card manufacturer and obtain all graphics card serial numbers and GPU allocation environment variable names. Call the management command line provided by each supported graphics card manufacturer in turn, and check if the execution was successful. If successful, retrieve all graphics card serial numbers from the execution results and set the GPU allocation environment variable name to the value corresponding to that graphics card manufacturer.

[0098] 4. Use file locks to lock a specified number of lock files. Iterate through each graphics card number and attempt to lock the corresponding lock file for each graphics card number using the operating system's `flock` system call in an exclusive (`LOCK_EX`) and non-blocking (`LOCK_NB`) manner, until the number of successful lock attempts reaches the number of GPUs to be allocated. If the number of successful lock attempts for each graphics card number does not reach the number of GPUs to be allocated, the allocator will release all locks and retry the step after a random wait of 1-5 seconds, until the required number of files are successfully locked.

[0099] 5. Set the GPU allocation environment variable. Concatenate all successfully locked GPU IDs together with commas, and set the GPU allocation environment variable of the child process to be started to this concatenated result.

[0100] 6. Start a child process to execute the application. Use the operating system's fork and exec system calls to start the application in a new child process.

[0101] 7. Wait for the child process to finish executing and pass its return code. After the child process finishes running, obtain its return code and use it as the allocator's return code to terminate the allocator's operation.

[0102] In this embodiment, the key to achieving cross-container allocation lies in providing a unified lock file directory mount when starting the container. Based on the operating system's file system mounting and file lock implementation principles, programs in different containers synchronize GPU allocation status information by checking whether the lock file is locked, thus avoiding GPU allocation conflicts across containers.

[0103] In this embodiment, we consider a scenario where only one GPU is available for allocation, but two containers each use their own allocator to launch their applications. The specific architecture can be found in [reference needed]. Figure 7 The diagram illustrates the interaction architecture between lock files, allocators, containers, and applications in a cross-container GPU allocation scenario. Allocator 1 can perform "lock" and "start" operations: The "lock" action associates the lock file of container 1, establishing exclusive ownership of the lock resource for that specific container; simultaneously, the "start" action triggers application 1, driving business logic execution. Allocator 2 is in a ready state, performing "attempt to lock" and "prepare to start," attempting to acquire control of the lock file of container 2, preparing for the startup of application 2. After allocator 1 successfully locks the lock file of container 1, it starts application 1, connecting resource allocation with business execution; if allocator 2 completes the "attempt to lock," it will also subsequently start application 2, allowing the application to run using the allocated GPU and other resources. The lock files of containers 1 and 2 are core resource markers. The lock file of container 1 is "mapped" to the "lock file outside the container", and the lock file of container 2 is also "mapped" to the external lock file. This establishes the connection between the lock resources inside the container and the unified management node outside, so that the lock status across containers can be perceived and coordinated by the outside. This realizes the unified management logic of lock resources in a cross-container environment and ensures the orderliness when multiple containers compete for resources such as GPUs.

[0104] For details on the interaction process of cross-container allocation, please refer to Figure 8 Before allocator 1 in container 1 executes application 1, it locks the lock file within the container. Due to the mounting of the lock file directory, this operation effectively locks the lock file outside the container. After successful locking, allocator 1 starts application 1. If allocator 2 in container 2 then starts application 2, it will also attempt to lock the lock file outside the container. Since allocator 1 in container 1 has not released the lock, this locking attempt will fail. Therefore, allocator 2 in container 2 will wait 1-5 seconds and retry. When application 1 in container 1 finishes, allocator 1 in container 1 will also finish, and the file lock will be automatically released due to the operating system's mechanism. Afterward, allocator 2 in container 2 will successfully lock the file when it attempts to do so. The subsequent process is similar to that of container 1: application 2 starts, application 2 finishes, allocator 2 finishes, and the file lock is automatically released.

[0105] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the cross-container GPU allocation method of this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0106] This application also provides a cross-container GPU allocation system; please refer to [link / reference]. Figure 9 The cross-container GPU allocation system includes: GPU allocator 10 is used to parse the received application launch request, extract the GPU requirement quantity and application launch parameters, and query the management command line of each graphics card manufacturer to obtain the graphics card number and GPU allocation environment variable name of each graphics card. The GPU allocator 10 is also used to traverse each of the graphics cards and locate the lock file corresponding to the graphics card number in the preset lock file directory; Operating system 20 is used to implement the flock system call to lock the lock file corresponding to the graphics card number; The GPU allocator 10 is also used to determine that after the number of successfully locked files reaches the number required by the GPU, it concatenates the graphics card number corresponding to the successfully locked file into a string and binds the string to the name of the GPU allocation environment variable to obtain the GPU allocation environment variable. Subprocess 30 is used to start and run the application based on the GPU allocation environment variables and the application startup parameters, wherein the application accesses GPU resources in the graphics card specified by the GPU allocation environment variables during the running process; The operating system 20 is also used to unlock the successfully locked file after determining that the subprocess has ended.

[0107] The cross-container GPU allocation system provided in this application, employing the cross-container GPU allocation method described in the above embodiments, can solve the technical problem of resource conflicts caused by applications within multiple containers sharing physical GPU resources in cross-container scenarios. Compared with the prior art, the beneficial effects of the cross-container GPU allocation system provided in this application are the same as those of the cross-container GPU allocation method provided in the above embodiments, and other technical features of the cross-container GPU allocation system are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0108] This application provides an electronic device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the cross-container GPU allocation method in Embodiment 1 above.

[0109] The following is for reference. Figure 10 It shows a schematic diagram of the structure of an electronic device suitable for implementing the embodiments of this application. Figure 10 The electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0110] like Figure 10 As shown, the electronic device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory 1002 or a program loaded from a storage device 1003 into a random access memory 1004. The random access memory 1004 also stores various programs and data required for the operation of the electronic device. The processing unit 1001, the read-only memory 1002, and the random access memory 1004 are interconnected via a bus 1005. An input / output interface 1006 is also connected to the bus. Typically, the following systems can be connected to the input / output interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. The communication device 1009 allows the electronic device to communicate wirelessly or wiredly with other devices to exchange data. Although the diagrams show electronic devices with various systems, it should be understood that it is not required to implement or have all of the systems shown. More or fewer systems may be implemented alternatively.

[0111] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from read-only memory 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0112] The electronic device provided in this application, employing the cross-container GPU allocation method described in the above embodiments, can solve the technical problem of resource conflicts caused by applications within multiple containers sharing physical GPU resources in cross-container scenarios. Compared with the prior art, the beneficial effects of the electronic device provided in this application are the same as those of the cross-container GPU allocation method provided in the above embodiments, and other technical features of this electronic device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0113] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0114] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0115] This application provides a computer program storage medium, which is a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, which are used to execute the cross-container GPU allocation method in the above embodiments.

[0116] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0117] The aforementioned computer-readable storage medium may be included in an electronic device or may exist independently without being assembled into an electronic device.

[0118] The aforementioned computer-readable storage medium carries one or more programs that, when executed by an electronic device, cause the electronic device to perform the various steps of the cross-container GPU allocation method described above.

[0119] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0120] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0121] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0122] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described cross-container GPU allocation method. This solves the technical problem of resource conflicts caused by applications within multiple containers sharing physical GPU resources in cross-container scenarios. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the cross-container GPU allocation method provided in the above embodiments, and will not be repeated here.

[0123] This application also provides a lock file storage medium, which is a computer-readable and writable storage medium having a lock file stored thereon. When the lock file is executed by a processor, it implements the steps of the cross-container GPU allocation method as described in the above embodiments.

[0124] The computer-readable and writable storage medium provided in this application can be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems or devices, or any combination thereof. More specific examples of computer-readable and writable storage media are not limited herein. In this embodiment, the computer-readable and writable storage medium can be any tangible medium containing or storing a lock file, which can be used by or in conjunction with an instruction execution system or device. The lock file contained on the computer-readable and writable storage medium can be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0125] The aforementioned computer-readable and writable storage medium may be included in an electronic device; or it may exist independently without being assembled into an electronic device.

[0126] The aforementioned computer-readable and writable storage medium carries one or more programs, which, when executed by an electronic device, cause the electronic device to implement the various steps of the cross-container GPU allocation method described above.

[0127] The readable storage medium provided in this application is a computer-readable and writable storage medium. This medium stores a lock file, which, when executed by the processor, can resolve the technical problem of resource conflicts caused by applications within multiple containers sharing physical GPU resources in cross-container scenarios. Compared with the prior art, the beneficial effects of the computer-readable and writable storage medium provided in this application are the same as those of the cross-container GPU allocation method provided in the above embodiments, and will not be elaborated upon here.

[0128] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the cross-container GPU allocation method as described above.

[0129] The computer program product provided in this application can solve the technical problem of resource conflicts caused by applications within multiple containers sharing physical GPU resources in cross-container scenarios. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the cross-container GPU allocation method provided in the above embodiments, and will not be repeated here.

[0130] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A cross-container GPU allocation method, characterized in that, The cross-container GPU allocation method includes: The received application launch request is parsed by the GPU allocator inside the container, the number of GPUs required and the application launch parameters are extracted, and the GPU allocator queries the management command line of each graphics card manufacturer to obtain the graphics card number and GPU allocation environment variable name of each graphics card. The GPU allocator iterates through each graphics card, locates the lock file corresponding to the graphics card number in the preset lock file directory, and locks the lock file corresponding to the graphics card number by using the flock system call until the GPU allocator determines that the number of successfully locked lock files reaches the required number of GPUs. The GPU allocator then concatenates the graphics card numbers corresponding to the successfully locked lock files into a string and binds the string to the GPU allocation environment variable name to obtain the GPU allocation environment variable. The application is launched and run by a subprocess of the GPU allocator based on the GPU allocation environment variables and the application startup parameters, wherein the application accesses GPU resources in the graphics card specified by the GPU allocation environment variables during its operation; After the subprocess has finished, the operating system unlocks the successfully locked file.

2. The cross-container GPU allocation method as described in claim 1, characterized in that, Before the steps of parsing the received application launch request through the GPU allocator within the container, extracting the GPU requirement and application launch parameters, and querying the management command lines of various graphics card manufacturers through the GPU allocator to obtain the graphics card ID and GPU allocation environment variable name of each graphics card, the method further includes: Read the lock file directory path and mount the lock file directory to the specified path of the container through the container engine's mount interface; The GPU allocator writes the lock file directory path into the container's allocator configuration file and passes it to the process within the container.

3. The cross-container GPU allocation method as described in claim 1, characterized in that, The step of parsing the received application launch request through the GPU allocator within the container and extracting the GPU requirement and application launch parameters includes: The GPU allocator detects whether the GPU quantity field in the application launch request has a valid value. If a valid value exists, the value of the GPU quantity field is determined as the GPU requirement, and the values ​​of the fields in the application startup request other than the GPU quantity field are determined as the application startup parameters.

4. The cross-container GPU allocation method as described in claim 1, characterized in that, The step of retrieving the graphics card number and GPU allocation environment variable name of each graphics card by querying the management command line of each graphics card manufacturer through the GPU allocator includes: The GPU allocator iterates through each graphics card manufacturer in the preset list of graphics card manufacturers and calls the management command line of each graphics card manufacturer to send a query command to the graphics card of that manufacturer. Obtain the graphics card hardware information and graphics card status information returned by the query command, extract the graphics card number based on the graphics card hardware information, and extract the GPU allocation environment variable name based on the graphics card status information.

5. The cross-container GPU allocation method as described in claim 1, characterized in that, The steps of traversing each graphics card through the GPU allocator, locating the lock file corresponding to the graphics card number in the preset lock file directory, and locking the lock file corresponding to the graphics card number using the `flock` system call until the GPU allocator determines that the number of successfully locked lock files reaches the required number for the GPU, and then concatenating the graphics card numbers corresponding to the successfully locked lock files into a string and binding the string to the GPU allocation environment variable name to obtain the GPU allocation environment variable, include: The GPU allocator traverses each graphics card, locates the lock file corresponding to the graphics card number in the preset lock file directory, and locks the lock file corresponding to the graphics card number in an exclusive and non-blocking manner by using the flock system call. If the locking is successful, the lock file corresponding to the graphics card number is added to the preset lock list, and the number of successfully locked files is incremented by one. If the number of files reaches the GPU's required number, then the lock files in the lock list are identified as successfully locked files; If the number of files does not reach the number required by the GPU, then check whether each of the graphics cards has been traversed. If all the graphics cards have been traversed, the locked files are released, a pause duration is randomly generated, and after the locking operation is paused until the pause duration, the number of files is set to zero, and the steps of traversing all the graphics cards and subsequent steps are executed. If each of the graphics cards has not been traversed, then for the next graphics card, the step of locating the lock file corresponding to the graphics card number in the preset lock file directory and subsequent steps are performed. If locking fails, then the step of checking whether each of the graphics cards has been traversed and subsequent steps are executed.

6. The cross-container GPU allocation method as described in claim 1, characterized in that, The step of starting and running the application through a subprocess of the GPU allocator based on the GPU allocation environment variables and the application startup parameters includes: A child process of the GPU allocator is created by the GPU allocator, wherein the child process inherits the file descriptor and all environment variables of the GPU allocator; The GPU allocation environment variables are set in the subprocess via the GPU allocator; The application specified by the application startup parameters is executed through the subprocess. When the application starts, it reads the GPU allocation environment variables through the GPU driver library and accesses the GPU resources in the graphics card specified by the GPU allocation environment variables through the GPU driver library.

7. The cross-container GPU allocation method as described in any one of claims 1 to 6, characterized in that, The cross-container GPU allocation method also includes: When the subprocess finishes running, it passes a return code to the GPU allocator. The GPU allocator then uses the return code as its own return code and terminates its operation. The step of unlocking the successfully locked file via the operating system after determining that the child process has ended includes: After the child process has finished, the operating system closes the file descriptor of the GPU allocator to release the lock file associated with the file descriptor into an unlocked state.

8. A cross-container GPU allocation system, characterized in that, The cross-container GPU allocation system includes: The GPU allocator is used to parse the received application launch request, extract the GPU requirement and application launch parameters, and query the management command line of each graphics card manufacturer to obtain the graphics card number and GPU allocation environment variable name of each graphics card. The GPU allocator is also used to traverse each of the graphics cards and locate the lock file corresponding to the graphics card number in the preset lock file directory; The operating system is used to implement the flock system call to lock the lock file corresponding to the graphics card number; The GPU allocator is also used to determine that after the number of successfully locked files reaches the number required by the GPU, it concatenates the graphics card number corresponding to the successfully locked file into a string and binds the string to the GPU allocation environment variable name to obtain the GPU allocation environment variable. A child process is used to start and run the application based on the GPU allocation environment variables and the application startup parameters, wherein the application accesses GPU resources in the graphics card specified by the GPU allocation environment variables during the running process; The operating system is also used to unlock the successfully locked file after determining that the subprocess has ended.

9. A computer program storage medium, characterized in that, The computer program storage medium is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the cross-container GPU allocation method as described in any one of claims 1 to 7.

10. A lock file storage medium, characterized in that, The lock file storage medium is a computer-readable and writable storage medium, and the lock file storage medium stores a lock file as described in any one of claims 1 to 7. When the lock file is executed by the processor, it implements the steps of the cross-container GPU allocation method as described in any one of claims 1 to 7.

Images