Business processing method, processing apparatus, device, storage medium, and program product

Abstract

The present disclosure provides a container service processing method, processing device, equipment, storage medium and program product, and relates to the technical field of cloud computing. The container service processing method comprises: in response to a calling request for an acceleration platform driving interface, calling a configured hook component to perform an interception operation on the calling request, and the container service interacts with the acceleration platform through the acceleration platform driving interface; in response to the interception operation, requesting a token from a service resource management component; based on the token request, configuring service space resources and service time resources for running the container service; based on the configuration result, determining that the token is distributed to the container service by the service resource management component, calling the acceleration platform driving interface to run the acceleration platform to process the container service. Through the technical scheme of the present disclosure, the fine-grained allocation of service resources is realized by sharing the service resources among multiple functions in the time and space dimensions, thereby improving the utilization rate of resources.

Description

Technical Field

[0001] This disclosure relates to the field of cloud computing technology, and in particular to a container service processing method, a container service processing apparatus, an electronic device, a computer-readable storage medium, and a computer program product. Background Technology

[0002] "Serverless AI inference business" refers to business activities related to artificial intelligence inference based on serverless architecture. In this business model, developers and enterprises do not need to worry about infrastructure issues such as server configuration, management and maintenance. When an inference task (such as predicting, classifying and recognizing input data) is generated, the system will automatically allocate computing resources according to demand to execute the corresponding AI inference operation and automatically release resources after the task is completed. Although Serverless AI function computing can automatically and elastically expand AI computing functions according to business needs to cope with the surge in user traffic, in the AI ​​container business running Serverless AI inference business, resources such as GPU (Graphics Processing Unit) are bound to a single container for use, resulting in low utilization of business resources.

[0003] It should be noted that the information disclosed in the background section above is only used to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0004] The purpose of this disclosure is to provide a container service processing method, a container service processing apparatus, an electronic device, a computer-readable storage medium, and a computer program product, which at least to some extent overcome the problem of low utilization of service resources when processing container services in related technologies.

[0005] Other features and advantages of this disclosure will become apparent from the following detailed description, or may be learned in part from practice of this disclosure.

[0006] According to one aspect of this disclosure, a container service processing method is provided, comprising: responding to a call request to an acceleration platform driver interface, invoking a configured hook component to perform an interception operation on the call request, wherein the container service interacts with an acceleration platform through the acceleration platform driver interface; responding to the interception operation, requesting a token from a service resource management component; configuring service space resources and service time resources for running the container service based on the token request; determining, based on the configuration result, that the service resource management component will distribute the token to the container service, then invoking the acceleration platform driver interface to run the acceleration platform to process the container service using the service space resources and the service time resources.

[0007] In one embodiment of this disclosure, before calling the configured hook component to perform an interception operation on the call request in response to a call request to the acceleration platform driver interface, the method further includes: setting resource allocation parameters for configuring the business space resources and the business time resources. The resource allocation parameters are used to configure the business space resources and the business time resources. The resource allocation parameters include the percentage of SM computing units allocated to the container service, and the upper and lower percentage limits of the time window during which the container service occupies business resources. The business resources include multiple SM computing units, and the time window refers to the period during which the container service uses business resources.

[0008] In one embodiment of this disclosure, setting resource allocation parameters for configuring the business space resources and the business time resources includes: the container service is deployed based on the Kubernetes platform, and the resource allocation parameters are set based on the CRD resource description file of the Kubernetes controller.

[0009] In one embodiment of this disclosure, configuring service space resources and service time resources for running the container service based on the request of the token includes: calculating the percentage of the SM computing units allocated to the container service based on a first environment variable provided by the multi-process service of the service resources, and configuring the service space resources based on the percentage of the SM computing units; configuring the service time resources based on the configured service space resources, the upper and lower percentage limits, and the supervision and scheduling mechanism for the container service.

[0010] In one embodiment of this disclosure, configuring the service time resources based on the configured service space resources and the percentage upper and lower limits, as well as the monitoring and scheduling mechanism for container services, includes: calling a monitor to collect the occupancy data of the container service on multiple SM computing units; calling a scheduler to determine, based on the occupancy data and the percentage upper and lower limits of the time window for the container service to occupy service resources, whether the acceleration platform is allowed to process the container service within the time window; and determining, based on the configuration result, that the service resource management component distributes the token to the container service, including: if the acceleration platform is allowed to process the container service, the service resource management component distributes the token to the container service.

[0011] In one embodiment of this disclosure, the call scheduler determines whether the acceleration platform is allowed to process the container service within the time window based on the occupancy data and the upper and lower percentage limits of the time window in which the container service occupies service resources. This includes: the occupancy data including the ratio of the container service's time occupied by the time window; determining whether the acceleration platform is allowed to process the container service within the time window based on the scheduler detecting that the ratio of the occupied time is less than or equal to the upper percentage limit; and scheduling the container service to a candidate queue and configuring the processing order of the container service by the acceleration platform based on the lower percentage limit.

[0012] In one embodiment of this disclosure, scheduling the container service to a candidate queue and configuring the processing order of the container service by the acceleration platform based on the percentage lower limit includes: determining the difference between the percentage lower limit and the occupied time ratio as the remaining quota percentage; and sorting the candidate queue based on the remaining quota percentage.

[0013] In one embodiment of this disclosure, the distribution of the token by the business resource management component to the container service includes: for each container service in the candidate queue, if the remaining business space resources meet the business space resources required by the container service, then the business resource management component distributes the token to the container service.

[0014] In one embodiment of this disclosure, the method further includes: if the remaining service space resources do not meet the service space resources required by the container service, periodically checking the remaining service space resources based on a timer, so that when it is detected that the released service space resources meet the service space resources, the service resource management component distributes the token to the container service; or detecting the service similarity with the container service that has the token, and if the service similarity meets the resource sharing requirements, the service resource management component distributes the token to the container service.

[0015] In one embodiment of this disclosure, sorting the candidate queue based on the remaining quota percentage includes: sorting the candidate queue based on the remaining quota percentage and the service priority of the container service.

[0016] In one embodiment of this disclosure, the method further includes: if the scheduler detects that the occupancy time ratio is greater than the percentage upper limit, then the container service is scheduled to run in the next time window.

[0017] In one embodiment of this disclosure, the resource allocation parameters further include: GPU memory allocated to the container service.

[0018] In one embodiment of this disclosure, before invoking a configured hook component to intercept the call request in response to a call request to the acceleration platform driver interface, the method further includes: setting a second environment variable for specifying a shared library, the shared library being loaded before the call to the acceleration platform driver interface, the shared library being used to configure the service space resources and the service time resources; invoking the configured hook component to intercept the call request includes: invoking the configured hook component to load the shared library based on the second environment variable.

[0019] According to another aspect of this disclosure, a container service processing apparatus is provided, comprising: an interception module, configured to, in response to a call request to an acceleration platform driver interface, invoke a configured hook component to perform an interception operation on the call request, wherein the container service interacts with the acceleration platform through the acceleration platform driver interface; a request module, configured to, in response to the interception operation, request a token from a service resource management component; a configuration module, configured to, based on the token request, configure service space resources and service time resources for running the container service; and a running module, configured to, based on the configuration result, determine that the service resource management component distributes the token to the container service, and then invoke the acceleration platform driver interface to run the acceleration platform to process the container service using the service space resources and the service time resources.

[0020] According to another aspect of this disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; the processor being configured to perform the container business processing method described above by executing the executable instructions.

[0021] According to another aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the container business processing method described above.

[0022] According to another aspect of this disclosure, a computer program product is provided, on which a computer program is stored, which, when executed by a processor, implements the container business processing method described above.

[0023] The container service processing scheme provided in this disclosure intercepts call requests to the acceleration platform driver interface through a configured hook component. This allows for additional resource management operations before the container service interacts with the acceleration platform. Upon interception, a token is immediately requested from the service resource management component, breaking the pattern of a single container directly using service resources. This makes resource allocation more controllable. Based on the token request, the service space and time resources of the container service are configured, enabling dynamic adjustment of service resources according to actual needs. According to the configuration result, the service resource management component distributes tokens to the container service. Only containers that obtain tokens can use service resources, ensuring orderly allocation and use of resources. This allows service resources to be dynamically shared between different containers. By sharing service resources across multiple functions in both time and space dimensions, fine-grained allocation of service resources is achieved, thereby improving resource utilization.

[0024] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0025] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0026] Figure 1 This diagram illustrates a container service processing system according to an embodiment of the present disclosure.

[0027] Figure 2 A schematic flowchart of a container service processing method according to an embodiment of this disclosure is shown;

[0028] Figure 3 A schematic flowchart of another container service processing method in an embodiment of this disclosure is shown;

[0029] Figure 4 This diagram illustrates a framework diagram of a container service processing solution according to an embodiment of the present disclosure.

[0030] Figure 5 A schematic flowchart of another container service processing method in an embodiment of this disclosure is shown;

[0031] Figure 6This diagram illustrates a container service processing apparatus according to an embodiment of the present disclosure.

[0032] Figure 7 A structural block diagram of an electronic device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0033] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that this disclosure will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0034] Furthermore, the accompanying drawings are merely illustrative of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0035] Serverless AI function computing (AI-related functions running in a serverless architecture) allows users to run AI computing tasks flexibly and on demand, without worrying about the underlying infrastructure such as GPUs (Graphics Processing Units) and other container processing, management, and maintenance on servers. Users simply write AI computing functions and upload them to the cloud service provider's platform, which then manages and maintains the computing resources and network architecture. The advantage of serverless AI function computing lies in its ability to automatically and elastically scale AI computing functions according to business needs, handling surges in user traffic without requiring users to worry about server capacity planning or expansion. Furthermore, serverless AI function computing provides highly available services to avoid single points of failure and ensure application high availability. With the continuous development of AI technology, especially in the application of large models like ChatGPT, more and more users are starting to use AI services, leading to a surge in AI user traffic. Correspondingly, serverless AI inference services, due to their flexibility and scalability, will see significant growth opportunities. However, given the high cost of business resources, cloud computing vendors face challenges in efficiently utilizing these resources and achieving fine-grained sharing of business resources among multiple serverless AI functions.

[0036] k8s: refers to Kubernetes, which is an open-source container orchestration engine used to automate the deployment, scaling, and management of containerized applications.

[0037] CUDA (Compute Unified Device Architecture) is a parallel computing platform and programming model.

[0038] A CUDA hook typically refers to the insertion of custom code or logic at specific points in a CUDA programming environment or CUDA-based application using certain technical means, in order to monitor, modify, or enhance the runtime behavior of CUDA.

[0039] CRD (Custom Resource Definition) resource description file: The resource description file of the Kubernetes controller, used to extend the Kubernetes API in Kubernetes, allowing users to define their own custom resource types.

[0040] Streaming Multiprocessor (SM) computing units are the core computing components of GPUs (Graphics Processing Units). An SM computing unit contains a certain number of CUDA cores, registers, shared memory, etc. Multiple SM computing units can work in parallel, thereby enabling the simultaneous processing of large amounts of data and greatly improving the computing power of the GPU.

[0041] Figure 1 This is a schematic diagram of the structure of a computer system provided in an exemplary embodiment of this application. The system includes: a plurality of tenant terminals 120 and a server terminal 140, wherein the tenant terminals 120 are used for request processing container services, and the server terminal 140 includes a GPU device.

[0042] Tenant terminal 120 can be a mobile terminal such as a mobile phone, game console, tablet computer, e-book reader, smart glasses, MP4 (Moving Picture Experts Group Audio Layer IV) player, smart home device, AR (Augmented Reality) device, VR (Virtual Reality) device, etc. Alternatively, tenant terminal 120 can also be a personal computer (PC), such as a laptop computer and a desktop computer, etc.

[0043] Among them, tenant 120 may have an application installed to provide container business processing.

[0044] The tenant 120 and the server 140 are connected via a communication network. Optionally, the communication network can be a wired network or a wireless network.

[0045] Server 140 is a single server, or a combination of several servers, or a virtualization platform, or a cloud computing service center. Server 140 provides backend services for applications that provide containerized business processing. Optionally, server 140 undertakes the primary computing work, and tenant 120 undertakes the secondary computing work; or, server 140 undertakes the secondary computing work, and tenant 120 undertakes the primary computing work; or, tenant 120 and server 140 collaborate on computing using a distributed computing architecture.

[0046] In some alternative embodiments, server 140 is used to store container business process information.

[0047] Optionally, the logistics user terminals of the applications installed on different tenant terminals 120 are the same, or the logistics user terminals of the applications installed on two tenant terminals 120 are the same type of application logistics user terminals on different control system platforms. Depending on the terminal platform, the specific form of the application's logistics user terminal can also differ; for example, the application's logistics user terminal can be a mobile logistics user terminal, a PC logistics user terminal, or a World Wide Web (Web) logistics user terminal, etc.

[0048] Those skilled in the art will understand that the number of the aforementioned tenant terminals 120 can be more or less. For example, there may be only one terminal, or there may be dozens or hundreds of terminals, or even more. This application does not limit the number of terminals or the type of device.

[0049] Optionally, the system may also include a management device ( Figure 1 (Not shown), the management device is connected to the server 140 via a communication network. Optionally, the communication network can be a wired network or a wireless network.

[0050] Optionally, the aforementioned wireless or wired networks use standard communication technologies and / or protocols. The network is typically the Internet, but can also be any network, including but not limited to Local Area Networks (LANs), Metropolitan Area Networks (MANs), Wide Area Networks (WANs), mobile, wired or wireless networks, private networks, or any combination of virtual private networks. In some embodiments, technologies and / or formats including Hyper Text Markup Language (HTML), Extensible Markup Language (XML), etc., are used to represent data exchanged over the network. Furthermore, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Networks (VPNs), and Internet Protocol Security (IPsec) can be used to encrypt all or some links. In other embodiments, custom and / or dedicated data communication technologies can be used to replace or supplement the aforementioned data communication technologies.

[0051] like Figure 2 As shown, a container service processing method according to an embodiment of this disclosure includes:

[0052] In step S202, in response to the call request to the acceleration platform driver interface, the configured hook component is invoked to intercept the call request. The container service interacts with the acceleration platform through the acceleration platform driver interface.

[0053] An acceleration platform is a system or software framework used to handle containerized tasks, such as CUDA.

[0054] The acceleration platform driver interface is the interface through which container applications interact with the acceleration platform. By calling the acceleration platform driver interface, container applications can request the acceleration platform to process their tasks and utilize GPU resources to accelerate task execution.

[0055] A hook component is a software mechanism used to intercept and handle specific system events or function calls. A configured hook component, namely a CUDA hook, is used to intercept call requests to the acceleration platform driver interface.

[0056] Containerized services refer to applications or tasks that run in a containerized environment. Containerization technology can package artificial intelligence applications and their dependencies into an independent runtime environment for easy deployment, management, and scaling.

[0057] In some embodiments, the container service can specifically be an artificial intelligence container service, represented as AIFuncPod.

[0058] Step S204: In response to the interception operation, request a token from the business resource management component.

[0059] Among them, the business resource management component refers to the component used to effectively manage and allocate business resources.

[0060] In some embodiments, business resources include graphics processing unit (GPU) resources.

[0061] The token is used to control container services' access to GPU resources. By requesting the token, container services can ensure that there are enough resources available before using GPU resources.

[0062] Step S206: Configure the service space resources and service time resources for running container services based on token requests.

[0063] In step S208, if it is determined from the configuration result that the business resource management component will distribute tokens to the container business, then the acceleration platform driver interface is called to run the acceleration platform to process the container business using business space resources and business time resources.

[0064] If the acceleration platform successfully configures GPU space and time resources for the container service, the service resource management component will distribute a token to the container service. The container service can then use the token to call the acceleration platform's driver interface and start the acceleration platform to process its tasks. The acceleration platform will then utilize the allocated GPU resources to execute the container service's computing tasks.

[0065] In this embodiment, a configured hook component intercepts calls to the acceleration platform's driver interface, enabling additional resource management operations before the container service interacts with the acceleration platform. Upon interception, a token is immediately requested from the business resource management component, breaking the pattern of a single container directly using business resources. This makes resource allocation more controllable. Based on the token request, the business space and business time resources of the container service are configured, allowing business resources to be dynamically adjusted according to actual needs. According to the configuration result, the business resource management component distributes tokens to the container service. Only containers that obtain tokens can use business resources, ensuring orderly allocation and use of resources. This allows business resources to be dynamically shared between different containers. By sharing business resources across multiple functions in both time and space dimensions, fine-grained allocation of business resources is achieved, thereby improving resource utilization.

[0066] In one embodiment of this disclosure, before invoking the configured hook component to perform an interception operation on the call request in response to a call request to the acceleration platform driver interface, the method further includes:

[0067] Configure resource allocation parameters for business space resources and business time resources. These parameters include the percentage of SM computing units allocated to container services, the upper and lower limits of the percentage of business resources used by container services within a time window, and the period during which container services use business resources. Business resources include multiple SM computing units, and the time window refers to the period during which container services use business resources.

[0068] The percentage of SM computing units refers to the proportion of total SM computing units allocated to a specific container service. The time window refers to the period during which the container service uses service resources (including SM computing units, etc.). The percentage upper limit refers to the maximum proportion of time during which the container service can use service resources within a specific time window. The percentage lower limit indicates the minimum proportion of time during which the container service should use service resources within a specific time window.

[0069] In this embodiment, by setting the percentage of SM computing units allocated to container services, space resources can be allocated in a fine manner according to the computing needs of different container services. Setting the upper and lower limits of the percentage of service resource time window occupied by container services can effectively control the use of time resources by container services, thereby facilitating the optimization of service resource utilization.

[0070] In one embodiment of this disclosure, the resource allocation parameters further include: GPU memory allocated to container services.

[0071] In one embodiment of this disclosure, resource allocation parameters for configuring business space resources and business time resources are set, including: container services are deployed based on the Kubernetes platform, and resource allocation parameters are set based on the CRD resource description file of the Kubernetes controller.

[0072] In some embodiments, by using a custom AIFuncPod k8s controller, the controller's CRD resource description file format is shown in Table 1 below. The fields used for fine-grained GPU time and space resource configuration include four fields:

[0073] "aifunc / sm_config" is used to configure the GPU SM compute units, indicating the percentage of GPU SM compute units allocated to AIFuncPod. GPU resources are allocated based on the percentage of SM compute units, and different regions are assigned to different AI functions for parallel computation to achieve GPU space sharing.

[0074] "aifunc / time_limit" represents the maximum percentage of GPU time window that AIFuncPod can use (the length of the time window is set by the user and is consistent across the entire cluster);

[0075] "aifunc / time_request" represents the lower limit of the percentage of GPU time occupied by AIFuncPod. Based on the upper and lower percentage limits, it is possible to allocate according to time periods. Allocation according to time periods means that the GPU computing resources are allocated to different AI container services in different time periods to achieve GPU time sharing.

[0076] "aifunc / gpu_mem" represents the GPU memory allocated to AIFuncPod.

[0077] Table 1

[0078]

[0079] In one embodiment of this disclosure, the business resource management component includes a space resource management module and a time resource management module, which configures the business space resources and business time resources for running container services based on token requests, including:

[0080] The percentage of SM computing units allocated to the container service is calculated by the space resource management module based on the first environment variable provided by the multi-process service of business resources, and the business space resources are configured based on the percentage of SM computing units.

[0081] In some embodiments, the GPU spatial resource management module configures SM computing resources based on the SM configuration environment variable provided by the GPU MPS (Multi-Process Service) Server, namely the first environment variable CUDA_MPS_ACTIVE_THREAD_PERCENTAGE. The GPU MPS Server provides services for multiple processes to run simultaneously on the GPU. The first environment variable, CUDA_MPS_ACTIVE_THREAD_PERCENTAGE, specifies the percentage of threads that are active (performing computing tasks) on the GPU's streaming multiprocessor (SM). By adjusting this percentage, the GPU's computing load and resource allocation can be controlled.

[0082] The time resource management module configures business time resources based on the configured business space resources, percentage upper and lower limits, and the supervision and scheduling mechanism for container services.

[0083] In some embodiments, a suitable data storage method is set to save the business space resource configuration, percentage parameters, and container service status information. The time resource management module reads the configured business space resource information from the configuration source (which may be a configuration file, database, or dynamic configuration service), including the percentage of SM computing units allocated to the container service. Similarly, it extracts the upper and lower limits of the percentage of the business resource time window occupied by the container service from the configuration source. These parameters serve as constraints for time resource configuration, establishing a real-time monitoring mechanism for container services. This mechanism can interact with the container runtime environment (such as Kubernetes, Docker, etc.) to obtain information such as the container service's running status and resource usage. Based on the information collected by the monitoring mechanism, the configured business space resources, and the percentage parameters, time resource scheduling decisions are made. For example, if the resource utilization rate of a container service exceeds the percentage upper limit, its time resource allocation can be reduced; if it is below the percentage lower limit, its time resources can be increased or the excess resources can be allocated to other services. This translates the scheduling decision into actual operation, adjusting the time resources of the container service.

[0084] In this embodiment, the synergistic effect of the spatial resource management module and the time resource management module enables precise allocation and dynamic adjustment of resources for container services, improving resource utilization efficiency, system stability and business execution efficiency, and providing an effective resource management solution for container services in a multi-process environment.

[0085] like Figure 3 As shown, in one embodiment of this disclosure, configuring service time resources based on configured service space resources and percentage upper and lower limits, as well as a monitoring and scheduling mechanism for container services, includes:

[0086] Step S302: Call the monitor to collect data on the usage of multiple SM computing units by the container service.

[0087] In some embodiments, the monitor is a component specifically designed to monitor system resource usage. It can interact with the container runtime environment or underlying hardware to obtain real-time information on the container's SM compute unit usage. Usage data may include the number of SM compute units currently in use, usage time, and usage frequency. This data will provide a basis for subsequent decision-making.

[0088] Step S304: The scheduler is invoked to determine whether to allow the acceleration platform to process container services within the time window based on the occupied data and the upper and lower percentage limits of the time window in which the container service occupies service resources.

[0089] In some embodiments, the scheduler is the core component responsible for resource allocation and task scheduling. It receives usage data from the monitor and analyzes it in conjunction with the pre-set upper and lower limits of the percentage of service resources occupied by container services within a time window. If the current usage of SM computing units by container services is within the percentage range of the time window, i.e., it does not exceed the upper limit and is not lower than the lower limit, the scheduler determines that the acceleration platform is allowed to process the container service.

[0090] Based on the configuration results, the business resource management component determines how to distribute tokens to container services, including:

[0091] Step S306: If the acceleration platform is allowed to process container services, the service resource management component distributes tokens to the container services.

[0092] In some embodiments, when the scheduler determines that the acceleration platform is allowed to process container services, the service resource management component distributes a token to the container service. After obtaining the token, the container service can use the resources provided by the acceleration platform, such as SM computing units, to process its own tasks.

[0093] In this embodiment, the coordinated operation of multiple components such as monitoring, scheduling, and resource management enables the rational allocation and efficient utilization of container service resources, thereby improving the stability, performance, and flexibility of the system.

[0094] In one embodiment of this disclosure, the scheduler determines whether to allow the acceleration platform to process container services within a time window based on occupancy data and the upper and lower percentage limits of the time window in which container services occupy service resources, including:

[0095] The occupancy data includes the percentage of time the container service occupies within the time window. Based on the scheduler detecting that the occupancy percentage is less than or equal to the percentage upper limit, it determines whether the acceleration platform is allowed to process the container service within the time window; and it schedules the container service to the candidate queue and configures the processing order of the container service by the acceleration platform based on the percentage lower limit.

[0096] In one embodiment of this disclosure, the method further includes: if the scheduler detects that the occupancy time ratio is greater than the percentage limit, then the container service is scheduled to run in the next time window.

[0097] In some embodiments, a monitor collects data on the usage of multiple SM computing units by container services, including the ratio of the time occupied by the container service within a time window. The ratio of the time occupied can be obtained by calculating the ratio of the actual time resources used by the container service to the entire time window.

[0098] After the scheduler detects the time occupancy ratio, it compares it with the pre-set maximum percentage of time window for container services to occupy service resources. If the time occupancy ratio is less than or equal to the maximum percentage, it means that the container service has not exceeded the limit in terms of time resource usage. The scheduler determines that the acceleration platform is allowed to process the container service within the time window. That is, the container service can continue to use the resources provided by the acceleration platform to perform tasks. The scheduler schedules the container service to the candidate queue, which is a collection of container services waiting to be processed.

[0099] In addition, the scheduler configures the processing order of container services on the acceleration platform based on the percentage lower limit. If the occupancy time of a container service is close to or lower than the percentage lower limit, it may be processed first to ensure that it can make full use of the allocated time resources.

[0100] In this embodiment, by comparing the time occupied ratio with the percentage upper limit, the use of time resources by container services can be effectively controlled, preventing a certain service from excessively occupying resources and affecting the operation of other services. At the same time, configuring the processing order based on the percentage lower limit can ensure that resources are fully utilized and prevent resources from being idle.

[0101] In one embodiment of this disclosure, container services are scheduled to a candidate queue, and the processing order of container services by the acceleration platform is configured based on a percentage lower limit, including:

[0102] The difference between the lower limit of the percentage and the occupied time ratio is determined as the remaining quota percentage; the candidate queue is sorted based on the remaining quota percentage.

[0103] In this embodiment, the candidate queue is sorted based on the remaining quota percentage. The queue can be arranged in descending order of remaining quota percentage, meaning that container services with higher remaining quota percentages will be placed at the front of the queue. This better balances resource allocation and improves the overall efficiency of resource utilization.

[0104] In one embodiment of this disclosure, a business resource management component distributes tokens to container services, including:

[0105] For each container service in the candidate queue, if the remaining service space resources meet the service space resources required by the container service, the service resource management component will distribute a token to the container service.

[0106] In some embodiments, AIFuncPod refers to container services, and service resources refer to GPU resources. The GPU resource management component is used for GPU resource management. In order to manage GPU time and space resources, the AIFuncPod GPU resource management component includes two modules: a GPU space resource management module and a GPU time resource management module.

[0107] The GPU spatial resource management module configures SM computing resources based on the SM configuration environment variables provided by the GPU MPS (Multi-Process Service) Server, namely the first environment variable CUDA_MPS_ACTIVE_THREAD_PERCENTAGE.

[0108] The GPU time resource management module performs GPU time reuse among various AIFuncPods while considering spatial resource configuration. This module adopts a monitoring-scheduling mechanism, in which the monitor collects real-time GPU usage data for each Pod, and the scheduler decides whether to allow the execution of AI function CUDA cores within the Pod based on the resource usage of each Pod and the corresponding time quota specified when configuring the AIFuncPod. If allowed, a token is returned to allow the AIFuncPod to occupy GPU resources.

[0109] The scheduler's decision-making method is as follows: The scheduler maintains a data table storing the number of GPU times T that each AIFuncPod has already used. used GPU time limit T limit(Configured via "aifunc / time_limit") Minimum GPU time T request (Configured via "aifunc / time_request") and GPU SM resource configuration percentage r SM The scheduler makes scheduling decisions through filtering, queuing of candidate AIFuncPods, and token distribution.

[0110] First, calculate the remaining time quota T for each AIFuncPod. rest =T request -T used and the remaining time limit T rest_max =T limit -T used For T rest_max AIFuncPods with a value less than 0 are directly filtered out, preventing them from accessing the GPU, and need to wait for the next time window to run.

[0111] Secondly, for T rest_max AIFuncPods with a value ≥ 0 are entered into a queue for candidate queuing, and the queue is ordered according to T. rest Sort by size in descending order.

[0112] Finally, if the remaining resources in the SM meet the configuration requirements of AIFuncPod, the scheduler will distribute a token to AIFuncPod; otherwise, it will not distribute a token.

[0113] In this embodiment, by recording the amount of GPU time occupied by each AIFuncPod, the upper limit of occupied time, and the minimum value, the use of GPU time resources by each container can be precisely managed. Containers with a remaining time limit of less than 0 are directly filtered out to prevent them from accessing the GPU. This can prevent containers from exceeding their allocated time resources. Containers with a remaining time limit greater than or equal to 0 are put into a candidate queue and queued in descending order of remaining time quota to prioritize the processing of containers with larger remaining time quotas. By sharing GPU resources among multiple functions in both time and space dimensions, fine-grained allocation of GPU resources is achieved, thereby improving GPU resource utilization.

[0114] In one embodiment of this disclosure, the method further includes: if the remaining service space resources do not meet the service space resources required by the container service, the remaining service space resources are checked periodically based on a timer, so that when the released service space resources meet the service space resources, the service resource management component distributes a token to the container service.

[0115] In some embodiments, the timer periodically checks the status of the remaining service space resources to obtain information on the current remaining service space resources, including the availability of resources such as SM computing units. If it is detected that there are released service space resources that meet the needs of the container service, that is, the remaining service space resources plus the released resources can meet the service space resources required by the container service, the service resource management component distributes a token to the container service.

[0116] In one embodiment of this disclosure, the method further includes: detecting the service similarity with a container service that has a token; if the service similarity meets the resource sharing requirements, then the service resource management component distributes the token to the container service.

[0117] In some embodiments, by analyzing the task types, data characteristics, and computing requirements of different container services, business similarity can be detected. If the business similarity of the container services meets the resource sharing requirements, such as tasks being able to share some resources without interfering with each other, then the business resource management component can also issue tokens to the current container service, allowing it to share business space resources with containers that have similar services.

[0118] In this embodiment, when new resources are detected to be available, they are quickly allocated to waiting container services, which improves the overall utilization rate of service space resources. Detecting service similarity and sharing resources can further make full use of existing resources. In addition, sharing resources among containers of similar services can also maximize the role of resources.

[0119] In one embodiment of this disclosure, sorting the candidate queue based on the remaining quota percentage includes: sorting the candidate queue based on the remaining quota percentage and the service priority of the container service.

[0120] In this embodiment, the remaining quota percentage and the service priority of the container service are weighted and summed to calculate the final priority value. The final priority value is then sorted based on the priority value so that container services with higher priority can be processed first when performing resource allocation and other operations.

[0121] In one embodiment of this disclosure, before invoking the configured hook component to perform an interception operation on the call request in response to a call request to the acceleration platform driver interface, the method further includes:

[0122] Set a second environment variable to specify the shared library, which is loaded before calling the acceleration platform driver interface. The shared library is used to configure business space resources and business time resources. Call the configured hook component to perform interception operations on the call request, including: calling the configured hook component to load the shared library based on the second environment variable.

[0123] In some embodiments, the configured hook component is the AIFunc CUDA hook component. The AIFunc CUDA hook component intercepts calls to CUDA driver APIs (such as cuLaunchKernel and cuCtxSynchronize) by AI functions in the container instance using the Linux LD_PRELOAD mechanism. After interception, it requests a token from the GPU resource management module to obtain GPU resources.

[0124] In the Linux LD_PRELOAD mechanism, LD_PRELOAD is the second environment variable, used to specify the shared libraries that should be loaded first before the program runs. By setting the second environment variable, namely the LD_PRELOAD environment variable, the specified shared library can be loaded before the program loads other libraries.

[0125] In this embodiment, when the hook component performs an interception operation, it loads the specified shared library based on the second environment variable. After loading the shared library, the hook component calls the functions in the shared library to configure business space resources and business time resources, thereby realizing fine-grained allocation of the processing resources required by the business container and dispatching processing resources.

[0126] like Figure 4 As shown, the GPU-based container business processing architecture includes an AIFuncPod k8s controller 402, an AIFuncCUDA hook component 404, and an AIFuncPod GPU resource management component 406. Each AIFuncPod includes an AI inference task and an AIFuncCUDA hook component 404.

[0127] The AIFuncPod k8s controller 402 is based on the CRD resource description file and has four custom key fields for fine-grained GPU time and space resource configuration, including: "aifunc / sm_config", "aifunc / time_limit", "aifunc / time_request", and "aifunc / gpu_mem".

[0128] GPU resources can be allocated according to SM (Smart Module) computing units to achieve GPU space sharing, and according to time periods to achieve GPU time sharing. Allocation by SM computing units means partitioning multiple SM computing units on the same GPU and assigning different regions to different AI functions for parallel computation. Allocation by time periods means allocating GPU computing resources to different AI functions within different time periods. The two allocation methods can also be combined.

[0129] Of the four custom key fields, "aifunc / sm_config" is used to configure the GPU SM compute units, representing the percentage of GPU SM compute units allocated to AIFuncPod; "aifunc / time_limit" represents the upper limit of the percentage of GPU time window occupied by AIFuncPod (the length of the time window is set by the user, and the length of the time window is consistent throughout the entire cluster); "aifunc / time_request" represents the lower limit of the percentage of GPU time window occupied by AIFuncPod; and "aifunc / gpu_mem" represents the GPU memory allocated to AIFuncPod.

[0130] The AIFunc CUDA hook component 404 intercepts calls to CUDA driver APIs (such as cuLaunchKernel and cuCtxSynchronize) from the container instance using the Linux LD_PRELOAD mechanism. After interception, it requests a token from the GPU resource management module to obtain GPU resources.

[0131] AIFuncPod GPU resource management component 406 includes GPU spatial resource management module 4062 and GPU temporal resource management module 4064.

[0132] The GPU spatial resource management module 4062 configures SM computing resources based on the SM configuration environment variable CUDA_MPS_ACTIVE_THREAD_PERCENTAGE provided by the GPU MPS (Multi-Process Service) Server.

[0133] The GPU time resource management module 4064 performs GPU time reuse among various AIFuncPods while considering space resource configuration. This module adopts a monitoring-scheduling mechanism, in which the monitor collects real-time GPU usage data for each Pod, and the scheduler decides whether to allow the execution of AI function CUDA cores within the Pod based on the resource usage of each Pod and the corresponding time quota specified when configuring the AIFuncPod. If allowed, a token is returned to allow the AIFuncPod to occupy GPU resources.

[0134] The scheduler's decision-making method is as follows: The scheduler maintains a data table storing the number of GPU times T that each AIFuncPod has already used. used GPU time limit T limit (Configured via "aifunc / time_limit") Minimum GPU time T request(Configured via "aifunc / time_request") and GPU SM resource configuration percentage r SM The scheduler makes scheduling decisions through filtering, queuing of candidate AIFuncPods, and token distribution.

[0135] First, calculate the remaining time quota T for each AIFuncPod. rest =T request -T used and the remaining time limit T rest_max =T limit -T used For T rest_max AIFuncPods with a value less than 0 are directly filtered out, preventing them from accessing the GPU, and need to wait for the next time window to run.

[0136] Secondly, for T rest_max AIFuncPods with a value ≥ 0 are entered into a queue for candidate queuing, and the queue is ordered according to T. rest Sort by size in descending order.

[0137] Finally, if the remaining resources in the SM meet the configuration requirements of AIFuncPod, the scheduler will distribute a token to AIFuncPod. If not, it will not distribute a token. For AIFuncPods that have been issued tokens, they will access the GPU device through the CUDA API and allocate the GPU device to the container through the Device Plugin.

[0138] like Figure 5 As shown, a container service processing method according to another embodiment of this disclosure includes:

[0139] Step S502: Configure AIFuncPod GPU resource allocation parameters.

[0140] Configure AIFuncPod GPU resource allocation parameters through a custom AIFuncPod Kubernetes controller CRD resource description file. This file includes key fields such as "aifunc / sm_config", "aifunc / time_limit", "aifunc / time_request", and "aifunc / gpu_mem". These fields are registered by AIFuncPod to the AIFuncPod GPU resource management component after the CRD file is deployed.

[0141] Step S504: Intercept the original CUDA driver API call requests of AI functions in AIFuncPod based on the AIFunc CUDA hook component.

[0142] The AIFunc CUDA hook component intercepts calls to the CUDA driver API from AI functions in container instances using the Linux LD_PRELOAD mechanism. After interception, it requests a token from the GPU resource management module to obtain GPU resources to run the AI ​​function CUDA program.

[0143] Step S506: Configure SM resources through the GPU space resource management module based on the AIFuncPod GPU resource management component.

[0144] The GPU spatial resource management module configures SM computing resources based on the SM configuration environment variable CUDA_MPS_ACTIVE_THREAD_PERCENTAGE provided by the GPU MPS (Multi-Process Service) Server.

[0145] Step S508: Token distribution is performed through the GPU time resource management module based on the AIFuncPod GPU resource management component.

[0146] In step S510, the AI ​​function directly accesses the CUDA driver API and begins running the CUDA program.

[0147] If a time token is granted, the AI ​​function can begin accessing the CUDA driver API.

[0148] It should be noted that the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of this disclosure, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Furthermore, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0149] The following reference Figure 6 To describe a container service processing apparatus 600 according to an embodiment of the present disclosure. Figure 6 The container service processing device 600 shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments disclosed herein.

[0150] The container service processing device 600 is manifested as a hardware module or a software module. Components of the container service processing device 600 may include, but are not limited to: an interception module 602, used to intercept a call request to the acceleration platform driver interface by invoking a configured hook component, wherein the container service interacts with the acceleration platform through the acceleration platform driver interface; a request module 604, used to request a token from the service resource management component in response to the interception operation; a configuration module 606, used to configure the service space resources and service time resources for running the container service based on the token request; and a running module 608, used to, if the configuration result determines that the service resource management component will distribute a token to the container service, invoking the acceleration platform driver interface to run the acceleration platform to process the container service using the service space resources and service time resources.

[0151] like Figure 7 As shown, the electronic device 700 is manifested in the form of a general-purpose computing device. The components of the electronic device 700 may include, but are not limited to: at least one processing unit 710, at least one storage unit 720, and a bus 730 connecting different system components (including storage unit 720 and processing unit 710).

[0152] The storage unit stores program code that can be executed by the processing unit 710, causing the processing unit 710 to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure. For example, the processing unit 710 can perform actions such as... Figure 2 The scheme described in [the document].

[0153] Storage unit 720 may include a readable medium in the form of a volatile storage unit, such as random access memory (RAM) 7201 and / or cache memory 7202, and may further include a read-only memory (ROM) 7203.

[0154] The storage unit 720 may also include a program / utility 7204 having a set (at least one) program module 7205, such program module 7205 including but not limited to: an operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0155] Bus 730 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0156] Electronic device 700 can also communicate with one or more external devices 770 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with electronic device 700, and / or with any device that enables electronic device 700 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 750. Furthermore, electronic device 700 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 760. As shown, network adapter 760 communicates with other modules of electronic device 700 via bus 730. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 700, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0157] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or electronic device, etc.) to execute the methods according to the embodiments of this disclosure.

[0158] In exemplary embodiments of this disclosure, a computer-readable storage medium is also provided, having stored thereon a program product capable of implementing the methods described above. In some possible implementations, various aspects of this disclosure may also be implemented as a program product including program code that, when run on an electronic device, causes the electronic device to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure.

[0159] The program product for implementing the above-described method according to embodiments of the present disclosure may employ a portable compact disc read-only memory (CD-ROM) and include program code, and may run on an electronic device, such as a personal computer. However, the program product of the present disclosure is not limited thereto. In this document, the readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0160] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0161] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting programs for use by or in conjunction with an instruction execution system, apparatus, or device.

[0162] The program code contained on the readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0163] Program code for performing the operations of this disclosure can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on an IoT terminal, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0164] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0165] Furthermore, although the steps of the method in this disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or a step may be broken down into multiple steps.

[0166] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, mobile terminal, or electronic device, etc.) to execute the method according to the embodiments of this disclosure.

[0167] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the appended claims.

Claims

1. A container business processing method, characterized in that, include: In response to a call request to the acceleration platform driver interface, the configured hook component is invoked to intercept the call request, wherein the container service interacts with the acceleration platform through the acceleration platform driver interface; In response to the interception operation, a token is requested from the business resource management component; The request to configure the service space resources and service time resources for running the container service based on the token includes: calculating the percentage of SM computing units allocated to the container service based on a first environment variable provided by the multi-process service of the service resources, and configuring the service space resources based on the percentage of SM computing units; calling the monitor to collect the occupancy data of the container service on multiple SM computing units; the occupancy data includes the occupancy time ratio of the container service to the time window, and determining that the acceleration platform is allowed to process the container service within the time window based on the scheduler detecting that the occupancy time ratio is less than or equal to the percentage upper limit; Determining that the token be distributed from the business resource management component to the container service based on the configuration result includes: if the acceleration platform is allowed to process the container service, the business resource management component distributes the token to the container service, and then the acceleration platform driver interface is invoked to run the acceleration platform to process the container service using the business space resources and the business time resources.

2. The container service processing method according to claim 1, characterized in that, Before invoking the configured hook component to perform the interception operation on the call request in response to a call request to the acceleration platform driver interface, the method further includes: The system sets resource allocation parameters for configuring the service space resources and the service time resources. These parameters include the percentage of SM computing units allocated to the container service, and the upper and lower limits of the percentage of the service resource time window occupied by the container service. The service resources include multiple SM computing units, and the time window refers to the period during which the container service uses the service resources.

3. The container service processing method according to claim 2, characterized in that, Configure resource allocation parameters for the service space resources and the service time resources, including: The container service is deployed on the Kubernetes platform, and the resource allocation parameters are set based on the CRD resource description file of the Kubernetes controller.

4. The container service processing method according to claim 2, characterized in that, Also includes: The container service is scheduled to the candidate queue, and the processing order of the container service by the acceleration platform is configured based on the percentage lower limit.

5. The container service processing method according to claim 4, characterized in that, The process of scheduling the container service to the candidate queue and configuring the processing order of the container service by the acceleration platform based on the percentage lower limit includes: The difference between the lower limit of the percentage and the occupied time ratio is determined as the remaining quota percentage; The candidate queues are sorted based on the remaining quota percentage.

6. The container service processing method according to claim 5, characterized in that, The distribution of the token from the business resource management component to the container service further includes: For each container service in the candidate queue, if the remaining service space resources meet the service space resources required by the container service, the service resource management component distributes the token to the container service.

7. The container service processing method according to claim 6, characterized in that, Also includes: If the remaining service space resources do not meet the service space resources required by the container service, the remaining service space resources are checked periodically based on a timer. When it is detected that the released service space resources meet the service space resources, the service resource management component distributes the token to the container service. or The service similarity with the container service that has the token is detected. If the service similarity meets the resource sharing requirements, the service resource management component distributes the token to the container service.

8. The container service processing method according to claim 5, characterized in that, Sort the candidate queue based on the remaining quota percentage, including: The candidate queue is sorted based on the remaining quota percentage and the service priority of the container service.

9. The container service processing method according to claim 4, characterized in that, Also includes: If the scheduler detects that the occupied time ratio is greater than the percentage upper limit, the container service will be scheduled to run in the next time window.

10. The container service processing method according to claim 2, characterized in that, The resource allocation parameters also include: The GPU memory allocated to the container service.

11. The container service processing method according to any one of claims 1 to 10, characterized in that, Before invoking the configured hook component to perform the interception operation on the call request in response to a call request to the acceleration platform driver interface, the method further includes: Set a second environment variable to specify the shared library, which is loaded before calling the acceleration platform driver interface. The shared library is used to configure the business space resources and the business time resources. The configured hook component is invoked to intercept the invocation request, including: The hook component configured in the above configuration loads the shared library based on the second environment variable.

12. A container processing device, characterized in that, include: The interception module is used to respond to a call request to the acceleration platform driver interface by calling the configured hook component to perform an interception operation on the call request, wherein the container service interacts with the acceleration platform through the acceleration platform driver interface; The request module is used to request a token from the business resource management component in response to the interception operation; A configuration module is used to configure the service space resources and service time resources for running the container service based on the request of the token, including: calculating the percentage of SM computing units allocated to the container service based on a first environment variable provided by the multi-process service of the service resources, and configuring the service space resources based on the percentage of SM computing units; calling a monitor to collect the usage data of the container service on multiple SM computing units; the usage data includes the usage time ratio of the container service to the time window, and determining that the acceleration platform is allowed to process the container service within the time window based on the scheduler detecting that the usage time ratio is less than or equal to the percentage upper limit; The running module is used to determine, based on the configuration result, whether the business resource management component distributes the token to the container service. This includes: if the acceleration platform is allowed to process the container service, and the business resource management component distributes the token to the container service, then calling the acceleration platform driver interface to run the acceleration platform to process the container service using the business space resources and the business time resources.

13. An electronic device, characterized in that, include: processor; as well as Memory for storing the executable instructions of the processor; The processor is configured to execute the container service processing method of any one of claims 1 to 11 by executing the executable instructions.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the container business processing method according to any one of claims 1 to 11.

15. A computer program product having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the container business processing method according to any one of claims 1 to 11.

Images