Business processing method, processing device, equipment, storage medium and program product

Abstract

The invention provides a container business processing method and device, equipment, a storage medium and a program product, and relates to the technical field of cloud computing. The container service processing method comprises the steps that in response to a calling request for an acceleration platform driving interface, a configured hook assembly is called to execute interception operation on the calling request, and container services interact with an acceleration platform through the acceleration platform driving interface; in response to the interception operation, requesting a token from a service resource management component; configuring a service space resource and a service time resource for operating the container service based on the request of the token; and determining that the service resource management component distributes the token to the container service based on the configuration result, and calling an acceleration platform driving interface to operate the acceleration platform to process the container service. According to the technical scheme provided by the invention, the service resources are shared among multiple functions in two dimensions of time and space, so that the fine-grained distribution of the service resources is realized, and the utilization rate of the resources is improved.

Description

Technical Field

[0001] The present disclosure relates to the technical field of cloud computing, and in particular to a container service processing method, a container service processing device, an electronic device, a computer-readable storage medium, and a computer program product. Background Art

[0002] "Serverless AI inference business" refers to business activities related to artificial intelligence inference based on serverless architecture. In this business model, developers and enterprises do not need to worry about infrastructure issues such as server configuration, management and maintenance. When an inference task (such as prediction, classification, and recognition of input data) occurs, the system will automatically allocate computing resources according to demand to perform the corresponding AI inference operation, and automatically release resources after the task is completed. Although Serverless AI function computing can automatically elastically expand AI computing functions according to business needs to cope with the surge in user traffic, in the AI ​​container business running Serverless AI inference business, resources such as GPU (Graphics Processing Unit) are bound to a single container for use, resulting in low business resource utilization.

[0003] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0004] The purpose of the present disclosure is to provide a container service processing method, a container service processing device, an electronic device, a computer-readable storage medium and a computer program product, which at least to some extent overcome the problem of low service resource utilization when processing container services in the related art.

[0005] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0006] According to one aspect of the present disclosure, a container business processing method is provided, comprising: in response to a call request to an acceleration platform driver interface, calling a configured hook component to perform an interception operation on the call request, wherein the container business interacts with the acceleration platform through the acceleration platform driver interface; in response to the interception operation, requesting a token from a business resource management component; configuring business space resources and business time resources for running the container business based on the request for the token; determining based on the configuration result that the business resource management component distributes the token to the container business, then calling the acceleration platform driver interface to run the acceleration platform to process the container business using the business space resources and the business time resources.

[0007] In one embodiment of the present disclosure, in response to a call request to an acceleration platform driver interface, before calling a configured hook component to perform an interception operation on the call request, it also includes: setting resource allocation parameters for configuring the business space resources and the business time resources, the resource allocation parameters are used to configure the business space resources and the business time resources, the resource allocation parameters include the percentage of SM computing units allocated to the container business, the upper and lower percentage limits of the business resource time window occupied by the container business, wherein the business resources include multiple SM computing units, and the time window refers to the time period when the container business uses the business resources.

[0008] In one embodiment of the present disclosure, resource allocation parameters for configuring the business space resources and the business time resources are set, including: the container business is deployed based on the Kubernetes platform, and the resource allocation parameters are set based on the CRD resource description file of the Kubernetes controller.

[0009] In one embodiment of the present disclosure, the business space resources and business time resources for running the container business are configured based on the request of the token, including: calculating the percentage of the SM computing unit allocated to the container business based on the first environment variable provided by the multi-process service of the business resources, so as to configure the business space resources based on the percentage of the SM computing unit; configuring the business time resources based on the configured business space resources and the upper and lower percentage limits, as well as the supervision and scheduling mechanism for the container business.

[0010] In one embodiment of the present disclosure, the business time resources are configured based on the configured business space resources and the upper and lower percentage limits, as well as a supervision and scheduling mechanism for container business, including: calling a monitor to collect occupancy data of the container business on multiple SM computing units; calling a scheduler to determine whether the acceleration platform is allowed to process the container business within the time window based on the occupancy data and the upper and lower percentage limits of the business resource time window occupied by the container business; and determining, based on the configuration result, that the business resource management component distributes the token to the container business, including: if the acceleration platform is allowed to process the container business, the business resource management component distributes the token to the container business.

[0011] In one embodiment of the present disclosure, the calling scheduler determines whether the acceleration platform is allowed to process the container service within the time window based on the occupancy data and the upper and lower percentage limits of the container service occupying the service resource time window, including: the occupancy data includes the occupancy time ratio of the container service to the time window, based on the scheduler detecting that the occupancy time ratio is less than or equal to the upper percentage limit, determining that the acceleration platform is allowed to process the container service within the time window; and scheduling the container service to a candidate queue, and configuring the processing order of the acceleration platform for the container service based on the lower percentage limit.

[0012] In one embodiment of the present disclosure, the container service is scheduled to a candidate queue, and the processing order of the container service by the acceleration platform is configured based on the percentage lower limit, including: determining the difference between the percentage lower limit and the occupied time ratio as the remaining quota percentage; and sorting the candidate queues based on the remaining quota percentage.

[0013] In one embodiment of the present disclosure, the business resource management component distributes the token to the container business, including: for each of the container services in the candidate queue, if the remaining business space resources meet the business space resources required by the container service, the business resource management component distributes the token to the container service.

[0014] In one embodiment of the present disclosure, it also includes: if the remaining business space resources do not meet the business space resources required by the container business, the remaining business space resources are periodically checked based on a timer, so that when it is detected that the released business space resources meet the business space resources, the business resource management component distributes the token to the container business; or detects the business similarity with the container business having the token, and if the business similarity meets the resource sharing requirements, the business resource management component distributes the token to the container business.

[0015] In one embodiment of the present disclosure, sorting the candidate queues based on the remaining quota percentage includes: sorting the candidate queues based on the remaining quota percentage and the service priority of the container service.

[0016] In one embodiment of the present disclosure, the method further includes: based on the scheduler detecting that the occupied time ratio is greater than the upper percentage limit, scheduling the container service to run in the next time window.

[0017] In one embodiment of the present disclosure, the resource allocation parameter further includes: GPU memory allocated to the container service.

[0018] In one embodiment of the present disclosure, in response to a call request to an acceleration platform driver interface, before calling a configured hook component to perform an interception operation on the call request, it also includes: setting a second environment variable for specifying a shared library, the shared library is loaded before calling the acceleration platform driver interface, and the shared library is used to configure the business space resources and the business time resources; calling the configured hook component to perform an interception operation on the call request, including: calling the configured hook component to load the shared library based on the second environment variable.

[0019] According to another aspect of the present disclosure, a container business processing device is provided, including: an interception module, which is used to call a configured hook component to perform an interception operation on the call request in response to a call request to an acceleration platform driver interface, wherein the container business interacts with the acceleration platform through the acceleration platform driver interface; a request module, which is used to request a token from a business resource management component in response to the interception operation; a configuration module, which is used to configure business space resources and business time resources for running the container business based on the token request; and an operation module, which is used to determine, based on the configuration result, that the business resource management component distributes the token to the container business, and then calls the acceleration platform driver interface to run the acceleration platform to process the container business using the business space resources and the business time resources.

[0020] According to another aspect of the present disclosure, an electronic device is provided, including: a processor; and a memory for storing executable instructions of the processor; the processor is configured to execute the above-mentioned container service processing method by executing the executable instructions.

[0021] According to another aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned container service processing method is implemented.

[0022] According to another aspect of the present disclosure, a computer program product is provided, on which a computer program is stored, and when the computer program is executed by a processor, the above container service processing method is implemented.

[0023] The container business processing solution provided by the embodiment of the present disclosure intercepts the call request to the acceleration platform driver interface through the configured hook component, so that additional resource management operations can be performed before the container business interacts with the acceleration platform. When the request is intercepted, a token is immediately requested from the business resource management component to break the mode of a single container directly using business resources, making the allocation of resources more controllable. The business space resources and business time resources of the container business are configured based on the token request so that the business resources can be dynamically adjusted according to actual needs. According to the configuration results, the business resource management component distributes tokens to the container business. Only containers that obtain tokens can use business resources, which ensures the orderly allocation and use of resources, so that business resources can be dynamically shared between different containers. By sharing business resources between multiple functions in both time and space dimensions, fine-grained allocation of business resources is achieved, thereby improving resource utilization.

[0024] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0026] Figure 1 A schematic diagram showing a container service processing system in an embodiment of the present disclosure is shown;

[0027] Figure 2 A schematic flow chart showing a method for processing container services in an embodiment of the present disclosure is shown;

[0028] Figure 3 A schematic flow chart showing another method for processing container services in an embodiment of the present disclosure;

[0029] Figure 4 A framework diagram showing a container service processing solution in an embodiment of the present disclosure is shown;

[0030] Figure 5 A schematic flow chart showing another method for processing container services in an embodiment of the present disclosure is shown;

[0031] Figure 6A schematic diagram of a container service processing device in an embodiment of the present disclosure is shown;

[0032] Figure 7 A structural block diagram of an electronic device in an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0033] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the disclosure will be more comprehensive and complete and to fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0034] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0035] Serverless AI function computing (artificial intelligence-related functions running in a serverless architecture) allows users to run AI computing tasks without having to worry about underlying infrastructure such as container business processing, management, and operation and maintenance of GPUs (Graphics Processing Units) in servers. Users only need to flexibly use business resources on demand to run AI computing tasks. Users only need to write AI computing functions and upload them to the cloud service provider's platform, and the cloud service provider will be responsible for managing and maintaining computing resources and network architecture. The advantage of serverless AI function computing is that it can automatically elastically expand AI computing functions according to business needs to cope with the surge in user traffic, without users having to worry about server capacity planning or expansion issues. In addition, serverless AI function computing can also provide highly available services to avoid single points of failure and ensure high availability of applications. With the continuous development of AI technology, especially in the application of large models such as ChatGPT, more and more users are beginning to use AI services, which has also brought about a surge in AI user traffic. Accordingly, in this case, serverless AI reasoning business will usher in greater development opportunities due to its flexibility and scalability. In this context, due to the high cost of business resources, cloud computing vendors face challenges in how to efficiently use business resources and achieve fine-grained sharing of business resources among multiple serverless AI functions.

[0036] k8s: refers to Kubernetes, which is an open source container orchestration engine for automating the deployment, scaling, and management of containerized applications.

[0037] CUDA (Compute Unified Device Architecture): a parallel computing platform and programming model.

[0038] CUDA hook usually refers to inserting custom code or logic at a specific point in a CUDA programming environment or a CUDA-based application through some technical means to monitor, modify or enhance the CUDA runtime behavior.

[0039] CRD (Custom Resource Definition) resource description file: The resource description file of the k8s controller is used to extend the Kubernetes API in Kubernetes, allowing users to define their own custom resource types.

[0040] The SM (Streaming Multiprocessor) computing unit is the computing component of the GPU (Graphics Processing Unit) core. An SM computing unit contains a certain number of CUDA cores, registers, shared memory, etc. Multiple SM computing units can work in parallel to achieve simultaneous processing of large amounts of data, greatly improving the computing power of the GPU.

[0041] Figure 1 1 is a schematic diagram of a computer system provided by an exemplary embodiment of the present application. The system includes: a plurality of tenant terminals 120 and a server terminal 140, wherein the tenant terminal 120 is used to request processing of container services, and the server terminal 140 includes a GPU device.

[0042] The tenant end 120 can be a mobile terminal such as a mobile phone, a game console, a tablet computer, an e-book reader, smart glasses, an MP4 (Moving Picture Experts Group Audio Layer IV) player, a smart home device, an AR (Augmented Reality) device, a VR (Virtual Reality) device, or the tenant end 120 can also be a personal computer (PC), such as a laptop computer and a desktop computer.

[0043] Among them, an application for providing container business processing may be installed in the tenant end 120.

[0044] The tenant end 120 is connected to the server end 140 via a communication network. Optionally, the communication network is a wired network or a wireless network.

[0045] The server 140 is a server, or is composed of several servers, or is a virtualization platform, or is a cloud computing service center. The server 140 is used to provide background services for applications that provide container business processing. Optionally, the server 140 undertakes the main computing work and the tenant 120 undertakes the secondary computing work; or, the server 140 undertakes the secondary computing work and the tenant 120 undertakes the main computing work; or, the tenant 120 and the server 140 adopt a distributed computing architecture for collaborative computing.

[0046] In some optional embodiments, the server 140 is used to store container service processing program information.

[0047] Optionally, the logistics user terminals of the application programs installed in different tenant terminals 120 are the same, or the logistics user terminals of the application programs installed on the two tenant terminals 120 are logistics user terminals of the same type of application programs on different control system platforms. Based on the different terminal platforms, the specific forms of the logistics user terminals of the application programs may also be different, for example, the logistics user terminals of the application programs may be mobile logistics user terminals, PC logistics user terminals, or global wide area network (World Wide Web, Web) logistics user terminals, etc.

[0048] Those skilled in the art will appreciate that the number of the tenant terminals 120 may be more or less. For example, the number of the tenant terminals 120 may be only one, or the number of the tenant terminals may be dozens or hundreds, or more. The embodiment of the present application does not limit the number of terminals and the device type.

[0049] Optionally, the system may also include a management device ( Figure 1 (not shown), the management device is connected to the server 140 via a communication network. Optionally, the communication network is a wired network or a wireless network.

[0050] Optionally, the above-mentioned wireless network or wired network uses standard communication technology and / or protocol. The network is usually the Internet, but it can also be any network, including but not limited to a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a mobile, wired or wireless network, a dedicated network or any combination of a virtual private network). In some embodiments, the data exchanged through the network is represented by technologies and / or formats including Hyper Text Mark-up Language (HTML), Extensible Markup Language (XML), etc. In addition, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), Internet Protocol Security (IPsec) can also be used to encrypt all or some links. In other embodiments, customized and / or dedicated data communication technologies can also be used to replace or supplement the above-mentioned data communication technologies.

[0051] like Figure 2 As shown, a container service processing method according to an embodiment of the present disclosure includes:

[0052] Step S202, in response to a call request to the acceleration platform driver interface, calling a configured hook component to perform an interception operation on the call request, wherein the container service interacts with the acceleration platform through the acceleration platform driver interface.

[0053] Among them, the acceleration platform is a system or software framework for processing container business, such as CUDA.

[0054] The acceleration platform driver interface is an interface for container services to interact with the acceleration platform. By calling the acceleration platform driver interface, container services can request the acceleration platform to process their tasks and use GPU resources to accelerate the execution of tasks.

[0055] The hook component is a software mechanism for intercepting and processing specific system events or function calls. The configured hook component, namely, CUDA hook, is used to intercept the call request to the acceleration platform driver interface.

[0056] Container business refers to applications or tasks running in a containerized environment. Containerization technology can package artificial intelligence applications and their dependencies into an independent operating environment for easy deployment, management, and expansion.

[0057] In some embodiments, the container service may specifically be an artificial intelligence container service, represented as AIFuncPod.

[0058] Step S204: In response to the interception operation, request a token from the business resource management component.

[0059] The business resource management component is used to effectively manage and allocate business resources.

[0060] In some embodiments, the traffic resources include graphics processing unit (GPU) resources.

[0061] The token is used to control the container service's access to GPU resources. By requesting the token, the container service can ensure that sufficient GPU resources are available before using them.

[0062] Step S206: configuring the service space resources and service time resources for running the container service based on the token request.

[0063] Step S208, based on the configuration result, it is determined that the service resource management component distributes tokens to the container service, and then the acceleration platform driver interface is called to run the acceleration platform to process the container service using the service space resources and service time resources.

[0064] If the acceleration platform successfully configures GPU space resources and GPU time resources for the container service, the service resource management component will distribute tokens to the container service, and the container service can use the token to call the acceleration platform driver interface to start the acceleration platform to process its tasks. The acceleration platform will use the allocated GPU resources to execute the computing tasks of the container service.

[0065] In this embodiment, the call request to the acceleration platform driver interface is intercepted by the configured hook component, so that additional resource management operations can be performed before the container business interacts with the acceleration platform. When the request is intercepted, a token is immediately requested from the business resource management component to break the mode of a single container directly using business resources, making resource allocation more controllable. The business space resources and business time resources of the container business are configured based on the token request so that business resources can be dynamically adjusted according to actual needs. According to the configuration results, the business resource management component distributes tokens to the container business. Only the container that obtains the token can use the business resources, which ensures the orderly allocation and use of resources, so that business resources can be dynamically shared between different containers. By sharing business resources among multiple functions in both time and space dimensions, fine-grained allocation of business resources is achieved, thereby improving resource utilization.

[0066] In one embodiment of the present disclosure, before in response to a call request to the acceleration platform driver interface, calling a configured hook component to perform an interception operation on the call request, the method further includes:

[0067] Set resource allocation parameters for configuring business space resources and business time resources. The resource allocation parameters are used to configure business space resources and business time resources. The resource allocation parameters include the percentage of SM computing units allocated to the container business, and the upper and lower percentage limits of the business resource time window occupied by the container business. The business resources include multiple SM computing units, and the time window refers to the time period when the container business uses the business resources.

[0068] The percentage of SM computing units refers to the proportion of the total SM computing units allocated to a specific container service. The time window refers to the period of time when the container service uses service resources (including SM computing units, etc.). The upper limit of the percentage refers to the time proportion of the service resources that the container service can use at most in a specific time window. The lower limit of the percentage refers to the time proportion of the service resources that the container service should use at least in a specific time window.

[0069] In this embodiment, by setting the percentage of SM computing units allocated to container services, fine spatial resource allocation can be performed according to the computing requirements of different container services. By setting the upper and lower limits of the percentage of container services occupying the business resource time window, the use of time resources by container services can be effectively controlled, which is conducive to optimizing the utilization of business resources.

[0070] In one embodiment of the present disclosure, the resource allocation parameter further includes: GPU memory allocated to the container service.

[0071] In one embodiment of the present disclosure, resource allocation parameters for configuring business space resources and business time resources are set, including: container business is deployed based on the Kubernetes platform, and resource allocation parameters are set based on the CRD resource description file of the Kubernetes controller.

[0072] In some embodiments, by customizing the AIFuncPod k8s controller, the CRD resource description file format of the controller is shown in Table 1 below, and the fields for fine-grained GPU time and space resource configuration include four fields, including:

[0073] "aifunc / sm_config" is used to configure the GPU SM computing unit, which indicates the percentage of GPU SM computing units allocated to AIFuncPod. GPU resources are allocated based on the percentage of SM computing units. Different areas are allocated to different AI functions for parallel computing to achieve GPU space sharing.

[0074] "aifunc / time_limit" indicates the upper limit of the percentage of GPU time window occupied by AIFuncPod (the time window length is set by the user and the time window length is consistent across the entire cluster);

[0075] "aifunc / time_request" indicates the lower limit of the percentage of the GPU time window occupied by AIFuncPod. Based on the upper and lower limits of the percentage, allocation can be achieved according to time periods. Allocation according to time periods means that GPU computing resources are allocated to different AI container services in different time periods to achieve GPU time sharing.

[0076] "aifunc / gpu_mem" indicates the GPU memory allocated to AIFuncPod.

[0077] Table 1

[0078]

[0079] In one embodiment of the present disclosure, the service resource management component includes a space resource management module and a time resource management module, and configures the service space resources and service time resources for running the container service based on the token request, including:

[0080] The space resource management module calculates the percentage of the SM computing unit allocated to the container service based on the first environment variable provided by the multi-process service of the service resource, so as to configure the service space resource based on the percentage of the SM computing unit.

[0081] In some embodiments, the GPU space resource management module performs SM computing resource configuration based on the SM configuration environment variable provided by the GPU MPS (Multi-Process Service) Server, namely, the first environment variable CUDA_MPS_ACTIVE_THREAD_PERCENTAGE, wherein the GPU MPS Server provides services for multiple processes to run on the GPU simultaneously, and the first environment variable, namely, CUDA_MPS_ACTIVE_THREAD_PERCENTAGE, specifies the percentage of threads that are active (executing computing tasks) on the GPU's streaming multiprocessor (SM). By adjusting this percentage, the GPU's computing load and resource allocation can be controlled.

[0082] The time resource management module configures the business time resources based on the configured business space resources and the upper and lower percentage limits, as well as the supervision and scheduling mechanism for the container business.

[0083] In some embodiments, a suitable data storage method is set to save the business space resource configuration, percentage parameters and status information of the container business. The time resource management module reads the configured business space resource information from the configuration source (which may be a configuration file, a database or a dynamic configuration service), including the percentage of SM computing units allocated for the container business, etc., and also extracts the upper and lower limits of the percentage of the container business occupying the business resource time window from the configuration source. These parameters will be used as constraints for time resource configuration, and a real-time monitoring mechanism for container business will be established. By interacting with the container runtime environment (such as Kubernetes, Docker, etc.), the operating status and resource usage of the container business can be obtained. According to the information collected by the supervision mechanism, the configured business space resources and percentage parameters, a time resource scheduling decision is made. For example, if the resource utilization rate of a container business exceeds the upper percentage limit, it can be considered to reduce its time resource allocation; if it is lower than the lower percentage limit, it can be considered to increase its time resources or allocate excess resources to other businesses, convert the scheduling decision into actual operations, and adjust the time resources of the container business.

[0084] In this embodiment, through the synergy of the space resource management module and the time resource management module, accurate allocation and dynamic adjustment of resources for container services are achieved, which improves resource utilization efficiency, system stability and business execution efficiency, and provides an effective resource management solution for container services in a multi-process environment.

[0085] like Figure 3 As shown, in one embodiment of the present disclosure, based on the configured business space resources and the upper and lower percentage limits, and the supervision and scheduling mechanism for the container business, the business time resources are configured, including:

[0086] Step S302: calling a monitor to collect occupancy data of container services on multiple SM computing units.

[0087] In some embodiments, the monitor is a component dedicated to monitoring the usage of system resources. It can obtain the occupancy of SM computing units by container services in real time by interacting with the container operating environment or underlying hardware. The occupancy data may include the number of SM computing units currently in use, usage time, usage frequency, etc. These data will provide a basis for subsequent decision-making.

[0088] Step S304: The calling scheduler determines whether to allow the acceleration platform to process the container service within the time window based on the occupancy data and the upper and lower limits of the percentage of the container service occupying the service resource time window.

[0089] In some embodiments, the scheduler is the core component responsible for resource allocation and task scheduling. It receives occupancy data from the monitor and analyzes it in combination with the pre-set upper and lower limits of the percentage of container business occupying the business resource time window. If the current container business occupies the SM computing unit within the percentage range within the time window, that is, it does not exceed the upper limit and is not lower than the lower limit, the scheduler determines to allow the acceleration platform to process the container business.

[0090] Determining that the service resource management component distributes tokens to the container service based on the configuration result includes:

[0091] Step S306: If the acceleration platform is allowed to process the container service, the service resource management component distributes a token to the container service.

[0092] In some embodiments, when the scheduler determines to allow the acceleration platform to process the container service, the service resource management component distributes a token to the container service. After the container service obtains the token, it can use the resources provided by the acceleration platform, such as the SM computing unit, to process its own tasks.

[0093] In this embodiment, through the coordinated work of multiple components such as monitoring, scheduling and resource management, reasonable allocation and efficient utilization of container business resources are achieved, thereby improving the stability, performance and flexibility of the system.

[0094] In one embodiment of the present disclosure, the calling scheduler determines whether to allow the acceleration platform to process the container service within the time window based on the occupancy data and the upper and lower limits of the percentage of the container service occupying the service resource time window, including:

[0095] The occupancy data includes the occupancy time ratio of the container business to the time window. Based on the scheduler detecting that the occupancy time ratio is less than or equal to the upper limit of percentage, it is determined that the acceleration platform is allowed to process the container business within the time window; and the container business is scheduled to the candidate queue, and the acceleration platform is configured to process the container business in an order based on the lower limit of percentage.

[0096] In one embodiment of the present disclosure, the method further includes: based on the scheduler detecting that the occupied time ratio is greater than the upper limit of the percentage, scheduling the container service to run in the next time window.

[0097] In some embodiments, the monitor collects the occupancy data of the container service on multiple SM computing units, including the occupancy time ratio of the container service to the time window. The occupancy time ratio can be obtained by calculating the ratio of the actual time resources used by the container service to the entire time window.

[0098] After the scheduler detects the occupancy time ratio, it compares it with the preset upper limit of the percentage of the container business occupying the business resource time window. If the occupancy time ratio is less than or equal to the upper limit, it means that the container business has not exceeded the limit on the use of time resources. The scheduler determines to allow the acceleration platform to process the container business within the time window, that is, the container business can continue to use the resources provided by the acceleration platform to perform tasks. The scheduler schedules the container business to the candidate queue, which is a collection of container businesses waiting to be processed.

[0099] In addition, the scheduler configures the acceleration platform to process container services in an order based on the percentage lower limit. If the container service's occupancy time ratio is close to or lower than the percentage lower limit, it may be processed first to ensure that it can make full use of the allocated time resources.

[0100] In this embodiment, by comparing the occupied time ratio with the percentage upper limit, the use of time resources by the container business can be effectively controlled to prevent a certain business from excessively occupying resources and affecting the operation of other businesses. At the same time, the processing order is configured based on the percentage lower limit to ensure that resources are fully utilized and prevent resources from being idle.

[0101] In one embodiment of the present disclosure, the container service is scheduled to a candidate queue, and the processing order of the container service by the acceleration platform is configured based on the percentage lower limit, including:

[0102] The difference between the percentage lower limit and the occupied time ratio is determined as the remaining quota percentage; and the candidate queues are sorted based on the remaining quota percentage.

[0103] In this embodiment, the candidate queues are sorted based on the remaining quota percentage, and can be sorted in descending order from high to low according to the remaining quota percentage, that is, the container services with higher remaining quota percentages will be placed at the front of the queue, so as to better balance resource allocation and improve the overall efficiency of resource utilization.

[0104] In one embodiment of the present disclosure, the service resource management component distributes a token to a container service, including:

[0105] For each container service in the candidate queue, if the remaining service space resources satisfy the service space resources required by the container service, the service resource management component distributes a token to the container service.

[0106] In some embodiments, AIFuncPod is a container service, service resources are GPU resources, and the GPU resource management component is used to manage GPU resources. In order to realize the management of GPU time and space resources, the AIFuncPod GPU resource management component includes two modules, namely, a GPU space resource management module and a GPU time resource management module.

[0107] The GPU space resource management module performs SM computing resource configuration based on the SM configuration environment variables provided by the GPU MPS (Multi-Process Service) Server, namely, the first environment variable CUDA_MPS_ACTIVE_THREAD_PERCENTAGE.

[0108] The GPU time resource management module performs GPU time multiplexing between various AIFuncPods while considering the spatial resource configuration. The module adopts a monitoring-scheduling mechanism, in which the monitor collects real-time GPU usage data for each Pod, and the scheduler decides whether to allow the execution of the AI ​​function CUDA kernel in the Pod based on the resource usage of each Pod and the corresponding time quota specified when the AIFuncPod is configured. If allowed, a token is returned to allow the AIFuncPod to occupy GPU resources.

[0109] The decision-making method of the scheduler is as follows: The scheduler maintains a data table to store the number of GPU time T that each AIFuncPod has occupied used , GPU time limit T limit(Configured by "aifunc / time_limit"), minimum GPU time T request (configured via "aifunc / time_request") and GPU SM resource allocation percentage SM ,The scheduler completes the scheduling decision by filtering, queuing of candidate AIFuncPods and token distribution.

[0110] First, calculate the remaining time quota T for each AIFuncPod rest =T request -T used and the remaining time limit T rest_max =T limit -T used , for T rest_max AIFuncPods with a value less than 0 are directly filtered out, preventing them from accessing the GPU and requiring them to wait for the next time window to run.

[0111] Secondly, for T rest_max ≥0 AIFuncPod enters the queue for candidate queuing, and the queue is based on T rest The sizes are sorted in descending order.

[0112] Finally, the scheduler distributes tokens to AIFuncPod if the remaining resources of SM meet the configuration requirements of AIFuncPod, and does not distribute tokens if they do not meet the requirements.

[0113] In this embodiment, by recording the number of GPU times occupied by each AIFuncPod, the upper limit and the minimum value of the occupied time, the use of GPU time resources by each container can be accurately managed. Containers with a remaining time upper limit less than 0 are directly filtered to prevent them from accessing the GPU. This can prevent the container from exceeding its allocated time resources. Containers with a remaining time upper limit greater than or equal to 0 enter the candidate queue and are queued in descending order according to the size of the remaining time quota to give priority to containers with a larger remaining time quota. Fine-grained allocation of GPU resources is achieved by sharing GPU resources among multiple functions in both time and space dimensions, thereby improving GPU resource utilization.

[0114] In one embodiment of the present disclosure, it also includes: if the remaining business space resources do not meet the business space resources required by the container business, the remaining business space resources are periodically checked based on a timer, so that when it is detected that the released business space resources meet the business space resources, the business resource management component distributes tokens to the container business.

[0115] In some embodiments, the timer periodically checks the status of the remaining business space resources and obtains the current remaining business space resource information, including the availability of resources such as the SM computing unit. If it is detected that the released business space resources meet the needs of the container business, that is, the remaining business space resources plus the released resources can meet the business space resources required by the container business, the business resource management component distributes a token to the container business.

[0116] In one embodiment of the present disclosure, it further includes: detecting the service similarity with the container service having the token, and if the service similarity meets the resource sharing requirement, the service resource management component distributes the token to the container service.

[0117] In some embodiments, the task types, data characteristics, computing requirements, etc. of different container services are analyzed to detect business similarities. If the business similarities of the container services meet the resource sharing requirements, for example, tasks can share some resources without interfering with each other, then the business resource management component can also distribute tokens to the current container service, allowing it to share business space resources with containers with similar services.

[0118] In this embodiment, when new resources are detected to be available, they are quickly allocated to the waiting container services, thereby improving the overall utilization rate of service space resources. Detecting service similarity and sharing resources can further make full use of existing resources. In addition, containers of similar services share resources, which can also maximize the role of resources.

[0119] In one embodiment of the present disclosure, sorting the candidate queues based on the remaining quota percentage includes sorting the candidate queues based on the remaining quota percentage and the service priority of the container service.

[0120] In this embodiment, a final priority value is calculated by taking a weighted sum of the remaining quota percentage and the service priority of the container service, and sorting is performed based on the priority value so that container services with higher priorities can be processed preferentially when performing operations such as resource allocation.

[0121] In one embodiment of the present disclosure, before in response to a call request to the acceleration platform driver interface, calling a configured hook component to perform an interception operation on the call request, the method further includes:

[0122] A second environment variable is set to specify a shared library, which is loaded before the acceleration platform driver interface is called, and which is used to configure business space resources and business time resources; the configured hook component is called to perform an interception operation on the call request, including: the configured hook component is called to load the shared library based on the second environment variable.

[0123] In some embodiments, the configured hook component is an AIFunc CUDA hook component, which intercepts calls to CUDA driver APIs (such as cuLaunchKernel and cuCtxSynchronize) made by AI functions in the container instance based on the Linux LD_PRELOAD mechanism. After the interception, the AIFunc CUDA hook component requests a token from the GPU resource management module to obtain GPU resources.

[0124] Among them, in the Linux LD_PRELOAD mechanism, LD_PRELOAD is the second environment variable, which is used to specify the shared library that is loaded first before the program runs. By setting the second environment variable, namely the LD_PRELOAD environment variable, the specified shared library can be loaded before the program loads other libraries.

[0125] In this embodiment, when the hook component performs an interception operation, the specified shared library is loaded based on the second environment variable. After loading the shared library, the hook component calls the function in the shared library to configure business space resources and business time resources, thereby realizing fine-grained allocation of processing resources required for the business container and then dispatching processing resources.

[0126] like Figure 4 As shown, the GPU-based container business processing architecture includes an AIFuncPod k8s controller 402, an AIFuncCUDA hook component 404 and an AIFuncPod GPU resource management component 406, wherein each AIFuncPod includes an AI reasoning task and an AIFunc CUDA hook component 404.

[0127] AIFuncPod k8s controller 402 customizes four key fields for fine-grained GPU time and space resource configuration based on the CRD resource description file, including: "aifunc / sm_config", "aifunc / time_limit", "aifunc / time_request", and "aifunc / gpu_mem".

[0128] GPU resources can be allocated according to SM computing units to achieve GPU space sharing, and according to time period allocation to achieve GPU time sharing. Allocation by SM computing units means partitioning multiple SM computing units on the same GPU, and allocating different areas to different AI functions for parallel computing. Allocation by time period means allocating GPU computing resources to different AI functions in different time periods. The two allocation methods can also be combined.

[0129] Among the four customized key fields, the "aifunc / sm_config" field is used to configure the GPU SM computing unit, which indicates the percentage of GPU SM computing units allocated to AIFuncPod; "aifunc / time_limit" indicates the upper limit of the percentage of GPU time window occupied by AIFuncPod (the time window length is set by the user and the time window length is consistent across the entire cluster); "aifunc / time_request" indicates the lower limit of the percentage of GPU time window occupied by AIFuncPod; "aifunc / gpu_mem" indicates the GPU memory allocated to AIFuncPod.

[0130] The AIFunc CUDA hook component 404 intercepts calls to the CUDA driver API (such as cuLaunchKernel and cuCtxSynchronize) made by the AI ​​functions in the container instance based on the Linux LD_PRELOAD mechanism. After interception, it requests a token from the GPU resource management module to obtain GPU resources.

[0131] The AIFuncPod GPU resource management component 406 includes a GPU space resource management module 4062 and a GPU time resource management module 4064 .

[0132] The GPU space resource management module 4062 performs SM computing resource configuration based on the SM configuration environment variable CUDA_MPS_ACTIVE_THREAD_PERCENTAGE provided by the GPU MPS (Multi-Process Service) Server.

[0133] The GPU time resource management module 4064 performs GPU time multiplexing between various AIFuncPods while taking into account the spatial resource configuration. The module adopts a monitoring-scheduling mechanism, in which the monitor collects real-time GPU usage data for each Pod, and the scheduler decides whether to allow the execution of the AI ​​function CUDA kernel in the Pod based on the resource usage of each Pod and the corresponding time quota specified when the AIFuncPod is configured. If allowed, a token is returned to allow the AIFuncPod to occupy GPU resources.

[0134] The decision-making method of the scheduler is as follows: The scheduler maintains a data table to store the number of GPU time T that each AIFuncPod has occupied used , GPU time limit T limit (Configured by "aifunc / time_limit"), minimum GPU time T request(configured via "aifunc / time_request") and GPU SM resource allocation percentage SM ,The scheduler completes the scheduling decision by filtering, queuing of candidate AIFuncPods and token distribution.

[0135] First, calculate the remaining time quota T for each AIFuncPod rest =T request -T used and the remaining time limit T rest_max =T limit -T used , for T rest_max AIFuncPods with a value less than 0 are directly filtered out, preventing them from accessing the GPU and requiring them to wait for the next time window to run.

[0136] Secondly, for T rest_max ≥0 AIFuncPod enters the queue for candidate queuing, and the queue is based on T rest The sizes are sorted in descending order.

[0137] Finally, the scheduler distributes tokens to AIFuncPod if the remaining resources of SM meet the configuration requirements of AIFuncPod. If not, no tokens are distributed. For AIFuncPod to which tokens are distributed, the scheduler accesses the GPU device through the CUDA API and allocates the GPU device to the container through the Device Plugin.

[0138] like Figure 5 As shown, according to another embodiment of the present disclosure, a container service processing method includes:

[0139] Step S502, configure AIFuncPod GPU resource allocation parameters.

[0140] Configure AIFuncPod GPU resource allocation parameters through the custom AIFuncPod k8s controller CRD resource description file, including key fields such as "aifunc / sm_config", "aifunc / time_limit", "aifunc / time_request", and "aifunc / gpu_mem". After the CRD file is deployed, the relevant fields are registered by AIFuncPod to the AIFuncPodGPU resource management component.

[0141] Step S504: intercepting the original CUDA driver API call request of the AI ​​function in the AIFuncPod based on the AIFunc CUDA hook component.

[0142] The AIFunc CUDA hook component intercepts the calls to the CUDA driver API made by the AI ​​function in the container instance based on the Linux LD_PRELOAD mechanism. After the interception, it requests a token from the GPU resource management module to obtain GPU resources to run the AI ​​function CUDA program.

[0143] Step S506: Based on the AIFuncPod GPU resource management component, SM resources are configured through the GPU space resource management module.

[0144] The GPU space resource management module configures SM computing resources based on the SM configuration environment variable CUDA_MPS_ACTIVE_THREAD_PERCENTAGE provided by the GPU MPS (Multi-Process Service) Server.

[0145] Step S508: Distribute tokens through the GPU time resource management module based on the AIFuncPod GPU resource management component.

[0146] Step S510: The AI ​​function directly accesses the CUDA driver API and starts running the CUDA program.

[0147] If the time token is granted, the AI ​​function can start accessing the CUDA driver API.

[0148] It should be noted that the above figures are only schematic illustrations of the processes included in the method according to the exemplary embodiments of the present disclosure, and are not intended to be limiting. It is easy to understand that the processes shown in the above figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0149] Refer to the following Figure 6 The container service processing device 600 according to the embodiment of the present disclosure is described. Figure 6 The container service processing device 600 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0150] The container service processing device 600 is presented in the form of a hardware module or a software module. The components of the container service processing device 600 may include but are not limited to: an interception module 602, which is used to respond to a call request to the acceleration platform driver interface and call the configured hook component to perform an interception operation on the call request, wherein the container service interacts with the acceleration platform through the acceleration platform driver interface; a request module 604, which is used to request a token from the service resource management component in response to the interception operation; a configuration module 606, which is used to configure the service space resources and service time resources for running the container service based on the token request; an operation module 608, which is used to determine that the service resource management component distributes tokens to the container service based on the configuration result, and then call the acceleration platform driver interface to run the acceleration platform to use the service space resources and service time resources to process the container service.

[0151] like Figure 7 As shown, the electronic device 700 is in the form of a general computing device. The components of the electronic device 700 may include but are not limited to: at least one processing unit 710, at least one storage unit 720, and a bus 730 connecting different system components (including the storage unit 720 and the processing unit 710).

[0152] The storage unit stores program codes, which can be executed by the processing unit 710, so that the processing unit 710 performs the steps according to various exemplary embodiments of the present disclosure described in the above “Exemplary Method” section of this specification. For example, the processing unit 710 can perform the following steps: Figure 2 The scheme described in .

[0153] The storage unit 720 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 7201 and / or a cache storage unit 7202 , and may further include a read-only storage unit (ROM) 7203 .

[0154] The storage unit 720 may also include a program / utility 7204 having a set (at least one) of program modules 7205, such program modules 7205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0155] Bus 730 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0156] The electronic device 700 may also communicate with one or more external devices 770 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 700, and / or communicate with any device that enables the electronic device 700 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 750. Furthermore, the electronic device 700 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 760. As shown, the network adapter 760 communicates with other modules of the electronic device 700 via a bus 730. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 700, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0157] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or an electronic device, etc.) to execute the method according to the implementation of the present disclosure.

[0158] In an exemplary embodiment of the present disclosure, a computer-readable storage medium is also provided, on which a program product capable of implementing the above method of the present specification is stored. In some possible implementations, various aspects of the present disclosure may also be implemented in the form of a program product, which includes a program code, and when the program product is run on an electronic device, the program code is used to enable the electronic device to perform the steps according to various exemplary implementations of the present disclosure described in the above "Exemplary Method" section of the present specification.

[0159] According to the program product for implementing the above method according to the embodiment of the present disclosure, it can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on an electronic device, such as a personal computer. However, the program product of the present disclosure is not limited thereto. In this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, an apparatus or a device.

[0160] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0161] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0162] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0163] The program code for performing the operations of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the IoT terminal, as an independent software package, partially on the user computing device and partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0164] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0165] In addition, although the steps of the method in the present disclosure are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.

[0166] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a mobile terminal, or an electronic device, etc.) to execute the method according to the implementation of the present disclosure.

[0167] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the disclosure herein. This application is intended to cover any variations, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the appended claims.

Claims

1. A container service processing method, characterized in that: include: In response to a call request to the acceleration platform driver interface, calling the configured hook component to perform an interception operation on the call request, wherein the container service interacts with the acceleration platform through the acceleration platform driver interface; In response to the interception operation, requesting a token from a business resource management component; Configure the business space resources and business time resources for running the container business based on the request of the token; Based on the configuration result, it is determined that the service resource management component distributes the token to the container service, and then the acceleration platform driver interface is called to run the acceleration platform to process the container service using the service space resources and the service time resources.

2. The container service processing method according to claim 1, characterized in that: Before calling the configured hook component to perform an interception operation on the call request in response to the call request to the acceleration platform driver interface, the method further includes: Setting resource allocation parameters for configuring the business space resources and the business time resources, wherein the resource allocation parameters are used to configure the business space resources and the business time resources, and the resource allocation parameters include the percentage of SM computing units allocated to the container business, and the upper and lower percentage limits of the business resource time window occupied by the container business, The service resources include a plurality of the SM computing units, and the time window refers to a time period during which the container service uses the service resources.

3. The container service processing method according to claim 2, characterized in that: Setting resource allocation parameters for configuring the business space resources and the business time resources includes: The container service is deployed based on the Kubernetes platform, and the resource allocation parameters are set based on the CRD resource description file of the Kubernetes controller.

4. The container service processing method according to claim 2, characterized in that: Configuring the business space resources and business time resources for running the container business based on the request of the token includes: Calculating a percentage of the SM computing unit allocated to the container service based on a first environment variable provided by the multi-process service of the service resource, so as to configure the service space resource based on the percentage of the SM computing unit; The service time resources are configured based on the configured service space resources and the upper percentage limit and the lower percentage limit, as well as a monitoring and scheduling mechanism for container services.

5. The container service processing method according to claim 4, characterized in that: Configuring the service time resources based on the configured service space resources and the upper percentage limit and the lower percentage limit, and the supervision and scheduling mechanism for the container service, includes: Calling a monitor to collect occupancy data of the container service on the plurality of SM computing units; The calling scheduler determines whether to allow the acceleration platform to process the container service within the time window based on the occupancy data and the upper and lower limits of the percentage of the container service occupying the service resource time window; Determining, based on the configuration result, that the service resource management component distributes the token to the container service includes: If the acceleration platform is allowed to process the container service, the service resource management component distributes the token to the container service.

6. The container service processing method according to claim 5, characterized in that: The calling scheduler determines whether to allow the acceleration platform to process the container service within the time window based on the occupancy data and the upper and lower limits of the percentage of the container service occupying the service resource time window, including: The occupancy data includes an occupancy time ratio of the container service to the time window, and based on the scheduler detecting that the occupancy time ratio is less than or equal to the upper percentage limit, determining to allow the acceleration platform to process the container service within the time window; and The container service is scheduled to a candidate queue, and the processing order of the container service by the acceleration platform is configured based on the lower limit of the percentage.

7. The container service processing method according to claim 6, characterized in that: Scheduling the container service to a candidate queue, and configuring the order in which the acceleration platform processes the container service based on the lower limit of the percentage, including: Determine the difference between the percentage lower limit and the occupied time ratio as the remaining quota percentage; The candidate queues are sorted based on the remaining quota percentage.

8. The container service processing method according to claim 7, characterized in that: The service resource management component distributes the token to the container service, further comprising: For each of the container services in the candidate queue, if the remaining service space resources satisfy the service space resources required by the container service, the service resource management component distributes the token to the container service.

9. The container service processing method according to claim 8, characterized in that: Also includes: If the remaining business space resources do not meet the business space resources required by the container business, the remaining business space resources are periodically checked based on a timer, so that when it is detected that the released business space resources meet the business space resources, the business resource management component distributes the token to the container business; or The service similarity with the container service having the token is detected, and if the service similarity meets the resource sharing requirement, the service resource management component distributes the token to the container service.

10. The container service processing method according to claim 7, characterized in that: Sorting the candidate queues based on the remaining quota percentage includes: The candidate queues are sorted based on the remaining quota percentage and the service priority of the container service.

11. The container service processing method according to claim 6, characterized in that: Also includes: Based on the scheduler detecting that the occupied time ratio is greater than the upper percentage limit, the container service is scheduled to run in the next time window.

12. The container service processing method according to claim 2, characterized in that: The resource allocation parameters also include: The GPU memory allocated to the container service.

13. The container service processing method according to any one of claims 1 to 12, characterized in that: Before calling the configured hook component to perform an interception operation on the call request in response to the call request to the acceleration platform driver interface, the method further includes: Setting a second environment variable for specifying a shared library, the shared library is loaded before calling the acceleration platform driver interface, the shared library is used to configure the business space resource and the business time resource; The configured hook component is called to perform an interception operation on the call request, including: The configured hook component is called to load the shared library based on the second environment variable.

14. A container service processing device, characterized in that: include: An interception module, configured to, in response to a call request to an acceleration platform driver interface, call a configured hook component to perform an interception operation on the call request, wherein the container service interacts with the acceleration platform through the acceleration platform driver interface; A request module, configured to request a token from a business resource management component in response to the interception operation; A configuration module, configured to configure business space resources and business time resources for running the container business based on a request of the token; An operation module is used to determine, based on the configuration result, that the business resource management component distributes the token to the container business, and then call the acceleration platform driver interface to run the acceleration platform to process the container business using the business space resources and the business time resources.

15. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; The processor is configured to execute the container service processing method according to any one of claims 1 to 13 by executing the executable instructions.

16. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the container service processing method according to any one of claims 1 to 13 is implemented.

17. A computer program product having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the container service processing method according to any one of claims 1 to 13 is implemented.

Images