Performance optimization method and system, and computer program product and storage medium
Patent Information
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2025-12-22
- Publication Date
- 2026-08-13
Smart Images
Figure CN2025144275_13082026_PF_FP_ABST
Abstract
Description
A performance optimization method, system, computer program product, and storage medium Technical Field
[0001] This disclosure relates to the field of computer technology, and in particular to a performance optimization method, system, computer program product, and storage medium. Background Technology
[0002] In various scenarios such as computationally intensive, data-intensive, or graphics processing, relying solely on the host's Central Processing Unit (CPU) to perform calculations is no longer sufficient to meet computational performance requirements. Graphics Processing Units (GPUs), with their parallel computing capabilities, are well-suited for handling computationally intensive, data-intensive, and image processing tasks. Therefore, GPUs are often used to accelerate host computations in these scenarios. In the process of using GPUs to accelerate host computations, spatial multiplexing mechanisms can be employed to achieve parallel computing within the GPU.
[0003] Currently, the implementation scheme of spatial multiplexing mechanism usually divides the hardware resources of a physical GPU into multiple independent parts in space, and allocates them to different tasks or virtual machines through virtualization technologies such as Multi-Instance GPU (MIG), so that they are logically used as if they are using different physical GPUs for computational acceleration. This is a static hardware isolation scheme, thus ensuring fault isolation between different tasks or virtual machines.
[0004] However, this static hardware isolation scheme does not adequately utilize GPU resources. Summary of the Invention
[0005] This disclosure provides a performance optimization method, system, computer program product, and storage medium to improve resource utilization of acceleration devices while ensuring fault isolation.
[0006] This disclosure provides a performance optimization method, including:
[0007] If any process within the control scope on the host is detected to have exited abnormally, the target acceleration device used by the process on the host is determined, and the process within the control scope has requested concurrency control.
[0008] An exit command is initiated to the service component deployed in the target acceleration device for the process. The service component is used to perform resource scheduling for processes that require concurrent management on the target acceleration device.
[0009] Using the service component, in response to the exit command, the resources occupied by the process are released in the target acceleration device.
[0010] This disclosure also provides a performance optimization system, including a monitoring component deployed within a virtualization instance in a host and a service component deployed within an acceleration device mounted on the host;
[0011] The monitoring component within any virtualization instance is configured to determine the target acceleration device used by the process on the host when it detects an abnormal exit of any process within the virtualization instance that is within the scope of control, and the process within the scope of control has requested concurrency control.
[0012] The monitoring component is also configured to initiate an exit command for the process to a service component deployed in the target acceleration device. The service component is used to perform resource scheduling for processes on the target acceleration device that require concurrent management.
[0013] The service component is configured to release the resources occupied by the process in the target acceleration device in response to the exit command.
[0014] This disclosure also provides a computer-readable storage medium for storing a computer program, which, when executed by one or more processors, causes the one or more processors to perform the aforementioned performance optimization method.
[0015] This disclosure also provides a computer program product, including a computer program that, when executed by one or more processors, causes the one or more processors to perform the aforementioned performance optimization method.
[0016] In this embodiment, a service component is deployed within an acceleration device mounted on a host machine. This service component can be used to schedule resources for processes on the acceleration device that require concurrent management. Based on the real-time resource scheduling function of the service component, multiple processes can simultaneously share resources within the same acceleration device, achieving process-level spatial multiplexing and effectively improving resource utilization within the acceleration device. Furthermore, the embodiment proposes fault monitoring for each process on the host machine that has requested concurrent management. Upon detecting an abnormal exit of a process within the management scope, a proactive exit command is sent to the service component deployed within the acceleration device used by that process, triggering the service component to promptly release the resources occupied by the process. By proactively triggering the service component to reclaim resources not released during abnormal process exit, service component failures caused by abnormal process exits can be effectively prevented, thus avoiding fault propagation and ensuring fault isolation between processes. Attached Figure Description
[0017] The accompanying drawings, which are included to provide a further understanding of this disclosure and form part of this disclosure, illustrate exemplary embodiments of the present disclosure and are used to explain the disclosure, but do not constitute an undue limitation of the disclosure. In the drawings:
[0018] Figure 1 is a schematic diagram of the structure of a performance optimization system provided in an exemplary embodiment of the present disclosure;
[0019] Figure 2 is a schematic diagram of the deployment method of a performance optimization system in a computing cluster provided by an exemplary embodiment of the present disclosure;
[0020] Figure 3 is a schematic diagram of an optional structure of a performance optimization system provided in an exemplary embodiment of the present disclosure;
[0021] Figure 4 is a schematic diagram of an application scheme in an application scenario provided by an exemplary embodiment of this disclosure;
[0022] Figure 5 is a flowchart illustrating a performance optimization method provided in another exemplary embodiment of this disclosure. Detailed Implementation
[0023] To make the objectives, technical solutions, and advantages of this disclosure clearer, the technical solutions of this disclosure will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this disclosure, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without creative effort are within the scope of protection of this disclosure.
[0024] It should be noted that, in the cases involving user information in the embodiments of this disclosure, the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of this disclosure are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse. In addition, the various models involved in this disclosure (including but not limited to language models or large models) comply with relevant laws and standards.
[0025] Before proceeding with a detailed description of the technical solutions provided in the various embodiments of this disclosure, the following is a brief explanation of several technical concepts involved in this disclosure.
[0026] Heterogeneous computing is a computing paradigm that combines different types of computing resources, such as a host processor (CPU) and a graphics processing unit (GPU), to work together. Its aim is to fully leverage the advantages of various computing resources, improving computing performance, energy efficiency, and system flexibility to better handle complex and diverse computing tasks.
[0027] Acceleration devices, in heterogeneous computing technology, can be understood as external devices that work in conjunction with the processor on the host machine to accelerate computational tasks on the host. Typical acceleration devices may include graphics processing units (GPUs) or tensor processing units (TPUs).
[0028] A Graphics Processing Unit (GPU) is a hardware device used for processing graphics and images. It has a large number of processing units and high-bandwidth memory, making it ideal for parallel computing tasks. The parallel computing capabilities of a GPU allow it to process multiple tasks simultaneously. Compared to a CPU, a GPU has more cores and higher memory bandwidth, enabling it to complete more computational tasks in the same amount of time.
[0029] As described in the background section, current methods typically employ virtualization technologies such as MIG to divide the GPU's hardware resources into multiple isolated hardware portions, which are then allocated to multiple tasks or virtual machines for use, thus achieving spatial multiplexing of the GPU by multiple tasks or virtual machines. In essence, this static hardware isolation scheme means that one task or virtual machine occupies a fixed portion of the GPU's hardware. From a spatial perspective, this achieves the effect of multiple tasks or virtual machines using the GPU simultaneously.
[0030] During their research, the inventors discovered that this static hardware isolation scheme ensures that the computational acceleration work between different tasks or virtual machines is completely independent, thus effectively guaranteeing fault isolation between different tasks or virtual machines. However, from the perspective of individual hardware components partitioned within the GPU, resource utilization is not high.
[0031] To address the aforementioned issues, this disclosure provides a solution. The basic idea is to abandon static hardware isolation schemes and support dynamic resource scheduling among concurrent processes, thereby improving resource utilization. Furthermore, a fault isolation mechanism is designed to solve the problem of fault propagation that may be caused by static fault isolation schemes.
[0032] The technical solutions provided by the embodiments of this disclosure are described in detail below with reference to the accompanying drawings.
[0033] Figure 1 is a schematic diagram of a performance optimization system provided in an exemplary embodiment of this disclosure. As shown in Figure 1, the system may include a monitoring component deployed within a virtualization instance in a host and a service component deployed within an acceleration device mounted on the host.
[0034] The acceleration device in this embodiment may include, but is not limited to, a graphics processing unit or a tensor processing unit, and of course, other electronic devices capable of supporting parallel computing. In this embodiment, the type of acceleration device is not limited.
[0035] Furthermore, the acceleration device in this embodiment can employ the aforementioned heterogeneous computing technology, be mounted on a host computer, and accelerate computing tasks on the host computer. The implementation details of heterogeneous computing technology will not be elaborated here, and this embodiment does not limit its scope. This embodiment focuses more on how to optimize the acceleration performance of the acceleration device based on heterogeneous computing technology.
[0036] In this embodiment, one or more virtualization instances can run on the host. A virtualization instance can be understood as an independent and virtual computing resource entity created in a virtualization environment. Virtualization instances are abstracted from the host's physical hardware resources using virtualization technology, and they share the host's physical hardware resources. Therefore, the host contains a host operating system and virtualization instances, and each virtualization instance includes a guest operating system and upper-layer applications running on top of the guest operating system. The host operating system and guest operating system are relative terms; the host operating system can be understood as the operating system of the physical machine, while the guest operating system can be understood as the operating system of the virtualization instance.
[0037] The virtualization instances in this embodiment may include, but are not limited to, cloud servers, Elastic Compute Service (ECS), virtual machines (VMs), or containers. This embodiment does not limit the product form of the virtualization instances.
[0038] Based on this, referring to Figure 1, this embodiment proposes to deploy a monitoring component in each virtualization instance on the host, and a service component in each acceleration device installed on the host.
[0039] The monitoring component can be used to monitor process failures within the virtualization instance, detecting abnormal process exits. The service component can respond to exit commands and perform resource scheduling on the acceleration device to support multiple concurrent processes sharing the hardware resources of the acceleration device.
[0040] Figure 2 is a schematic diagram illustrating the deployment of a performance optimization system within a computing cluster according to an exemplary embodiment of this disclosure. Referring to Figure 2, the computing cluster may contain multiple hosts, and the performance optimization system provided in this embodiment may be deployed on each host. For example, if the computing cluster is a container cluster, the monitoring component provided in this embodiment may be deployed within the containers of the container cluster, and the service component provided in this embodiment may be deployed in the acceleration devices (such as GPUs) installed on each host in the container cluster.
[0041] Referring again to Figure 2, computing tasks requiring acceleration within the computing cluster can be scheduled to virtualized instances on the host. For clarity, in this embodiment, computing tasks requiring acceleration are described as tasks to be accelerated. This embodiment does not limit the implementation scheme for scheduling tasks to be accelerated within the computing cluster. The existing scheduler in the computing cluster can be responsible for scheduling tasks to be accelerated. For example, if the computing cluster is a container cluster, the container management tools (e.g., Kubernetes) within the container cluster can be responsible for scheduling tasks to be accelerated to containers provided on the host within the container cluster.
[0042] For ease of understanding, the following explanation will use any host as an example to illustrate the technical logic of the performance optimization system provided in this embodiment on that host.
[0043] In this embodiment, the virtualization instance on the host can launch one or more processes for the assigned task to be accelerated. The processes launched for the task to be accelerated will be further scheduled to the acceleration devices installed on the host where the virtualization instance resides, so that the scheduled acceleration devices can be used to perform computations for the process. This further scheduling work can still be handled by the scheduler in the computing cluster mentioned above. Of course, this embodiment is not limited to this, and other entities can also be responsible for this further scheduling work, which will not be further exemplified here. This embodiment focuses more on the technical scenario after the process is scheduled to the acceleration device.
[0044] Among the processes initiated for the task to be accelerated, some or all of them may require concurrent management by the service component in this embodiment. Concurrency management in this embodiment refers to the service component performing resource scheduling for multiple concurrent processes on the acceleration device, so as to support multiple processes simultaneously sharing the hardware resources of the acceleration device.
[0045] This embodiment does not limit the concurrency control mechanism used within the service component. In one optional implementation, the service component may employ the concurrency control mechanism proposed in Multi-Process Service (MPS) technology:
[0046] Multiple concurrent processes on an accelerator are distributed across computing units within the accelerator for processing, allowing multiple processes to execute computational tasks simultaneously within the same time slice. This concurrency control mechanism ensures that multiple processes can execute computational tasks in parallel within the accelerator.
[0047] Of course, the concurrency control mechanism proposed in the MPS technology above is optional. The service components in this embodiment may also adopt concurrency control mechanisms that may exist now or in the future, as long as they can ensure that computing tasks are executed in parallel on the acceleration device. No further examples of concurrency control mechanisms will be given here.
[0048] Based on this, in this embodiment, spatial reuse at the process level can be achieved in the acceleration device through the dynamic resource scheduling function designed for the service components deployed in the acceleration device.
[0049] Clearly, implementing process-level spatial multiplexing in the acceleration device in this embodiment can effectively improve the resource utilization rate of the acceleration device.
[0050] During their research, the inventors discovered that after implementing process-level spatial multiplexing in the acceleration device in this embodiment, a fault propagation problem may occur. Fault propagation can be understood as the possibility that a failure in one process may trigger failures in other processes.
[0051] Therefore, in this embodiment, a fault isolation mechanism is further designed in the performance optimization system to address the aforementioned fault propagation problem. The fault isolation mechanism proposed in this embodiment is described below.
[0052] In this embodiment, processes on the host that have requested concurrency management can be considered as processes to be managed. These processes on the host constitute the management scope on the host. That is, all processes within the management scope on the host have requested concurrency management, and the concurrency management work is handled by the service components deployed in the acceleration device in this embodiment. In other words, in this embodiment, the management scope on the host includes processes that require concurrency management by any service component on the host.
[0053] Based on this, this embodiment proposes that a monitoring component within a virtualized instance on the host can be used to monitor whether processes within the host's control scope experience abnormal exits. For any monitoring component deployed within a virtualized instance on the host, it can monitor whether processes within that virtualized instance that fall within the control scope experience abnormal exits. Here, abnormal exit can be understood as a process exiting while using an acceleration device, but without following the normal process exit procedure.
[0054] Based on this monitoring mechanism, the monitoring component deployed within the virtualization instance can determine the target acceleration device used by any process on the host if any process within the virtualization instance that is within the scope of the control exits abnormally.
[0055] The monitoring component can also send an exit command to the service component deployed in the target acceleration device for that process. This exit command can be a pre-agreed interactive command between the service component and the virtualization instance or process to trigger the process exit process.
[0056] In this embodiment, the monitoring component can replace the abnormally exiting process and initiate an exit command to the service component that would normally be initiated by the process or the virtualization instance. It is understood that the monitoring component can, of course, agree with the service component on a different format for the exit command, without needing to use the original exit command format agreed upon between the service component and the virtualization instance or process; this embodiment does not limit this. In this embodiment, the exit command initiated by the monitoring component to the service component only needs to trigger the service component to perform the relevant processing operations required in the process exit procedure.
[0057] The process exit procedure described above can be understood as the interaction process followed by the virtualization instance, process, and service components when a process exits normally. No specific limitations are imposed on the process exit procedure here; in practical applications, it can be designed as needed to ensure the process exits normally within the acceleration device.
[0058] In this embodiment, the service component can release the resources occupied by the process in the target acceleration device after receiving the exit instruction initiated by the monitoring component. Thus, in this embodiment, the active triggering by the monitoring component enables the service component in the target acceleration device to promptly reclaim the resources that were not released when the process exited abnormally, avoiding service component failures due to untimely resource release.
[0059] In this embodiment, after receiving the exit instruction, the resources released by the service component for the process may include, but are not limited to, computing resources, storage resources, and resource contexts created by the service component for the process.
[0060] In this embodiment, by actively triggering the monitoring component, the service component used for concurrent management of abnormally exiting processes can promptly enter the aforementioned process exit process, thereby timely reclaiming the relevant resources that the process did not have time to release in the acceleration device. This can effectively prevent the service component from being triggered by abnormal process exit, and thus effectively prevent other process failures from being triggered by service component failures. Therefore, it can effectively prevent the propagation of failures and ensure fault isolation between processes at the service component level.
[0061] Furthermore, this embodiment proposes that after releasing the resources occupied by the abnormally exiting process in the target acceleration device, the system can monitor whether the service components deployed in the target acceleration device are in an abnormal state. If an abnormal state is detected, the system controls the service components deployed in the target acceleration device to restart. This provides a further safeguard mechanism in cases where the exit command issued by the monitoring component fails to restore the service component to a normal state. Based on this safeguard mechanism, it can be ensured that in the event of an abnormal process exit, the service components involved can eventually recover to a normal state.
[0062] It is worth emphasizing that although the context mentions the possibility of restarting the service component, the inventors have found in practice that the probability of needing to restart the service component is very low. In this embodiment, the exit command initiated by the monitoring component can significantly reduce the probability of service component failure. Therefore, the further safeguard mechanism proposed here will not affect the performance of the acceleration device.
[0063] In this embodiment, a service component is deployed within an acceleration device mounted on a host. This service component can be used to schedule resources for processes on the acceleration device that require concurrent management. Based on the real-time resource scheduling function of the service component, multiple processes can simultaneously share resources within the same acceleration device, achieving process-level space-division multiplexing and effectively improving resource utilization within the acceleration device. Furthermore, fault monitoring is proposed for each process on the host that has requested concurrent management. Upon detecting an abnormal exit of a process within the management scope, an exit command is proactively sent to the service component deployed within the acceleration device used by that process, triggering the service component to promptly release the resources occupied by the process. By proactively triggering the service component to reclaim resources not released during abnormal process exit, service component failures caused by abnormal process exits can be effectively prevented, thus avoiding fault propagation and ensuring fault isolation between processes.
[0064] In the above or following embodiments, various implementation methods can be used to achieve the relevant functions required by the monitoring components deployed in the virtualized instance.
[0065] Figure 3 is a schematic diagram of an optional structure of a performance optimization system provided by an exemplary embodiment of this disclosure. Referring to Figure 3, in one optional implementation, a self-designed function library can be deployed as a monitoring component in a virtualized instance on the host.
[0066] In this context, a function library can be understood as a collection of functions. In this embodiment, the function library may contain one or more custom-designed functions. These functions enable the implementation of the relevant functionalities required by the monitoring component. In this embodiment, the function library can be loaded into the process space corresponding to the process within the virtualization instance.
[0067] Referring to Figure 3, this optional implementation proposes that signal handling functions can be set in a function library and pre-registered with processes within the management scope. Based on this, the signal handling functions registered in the process that has abnormally exited can be used to initiate an exit command for the process to the service components deployed in the acceleration device used by that process.
[0068] Understandably, in this optional implementation, the signal processing function has its own custom function logic designed so that it can execute the operation of initiating an exit instruction.
[0069] In addition, this optional implementation can fully utilize the basic function of signal handling functions—signal processing—to promptly detect whether a process has abnormally terminated. Specifically, this optional implementation can use signal handling functions registered with the process to monitor whether the process has received abnormal system signals. If so, the signal handling function can determine that the process has abnormally terminated and can continue executing the aforementioned operation of initiating the exit command.
[0070] System signals are an asynchronous notification mechanism sent by the operating system to processes to inform them of specific events. For example, if a user presses the Ctrl+C key combination or a process encounters an error, the operating system in the virtualization instance will send the corresponding system signal to the process. Upon receiving an abnormal system signal, the process may terminate abnormally. In this optional implementation, the signal handler function registered with the process can monitor whether the process has received an abnormal system signal. Here, abnormal system signals may include, but are not limited to, the SIGTERM signal (for requesting process termination) and the SIGSEGV signal (for indicating that the process accessed an invalid memory address). It is understood that these system signals are exemplified. In this optional implementation, other system signals that may trigger abnormal process termination can be pre-configured in the signal handler function as abnormal system signals to be monitored; further examples are not provided here.
[0071] Here, the timing of registering signal handling functions into a process is not limited. In a preferred embodiment, the aforementioned signal handling functions can be registered into a process after the service component deployed in the acceleration device has completed resource scheduling for any process.
[0072] This preferred embodiment proposes a registration timing for signal processing functions. The service components deployed in the acceleration device complete resource scheduling for the process, indicating that the acceleration device will normally begin accelerated computation for that process. That is, the process will begin using the resources on the acceleration device. Therefore, this registration timing can effectively focus on fault monitoring of relevant processes that have already begun accelerated computation, thereby effectively reducing the computational load in the fault isolation mechanism and ensuring its processing efficiency.
[0073] Furthermore, in this preferred embodiment, the executor of the operation of registering the signal handling function in the process is not limited. In some design ideas, the service component can register the above-mentioned signal handling function in the process after completing resource scheduling for any process.
[0074] In summary, this embodiment proposes an implementation method for the monitoring component in a virtualization instance. It uses a function library as the monitoring component and further proposes designing a custom signal processing function within the function library. This signal processing function can detect whether a process has abnormally terminated and promptly send an exit command to the relevant service components for the abnormally terminated process. Therefore, this embodiment can promptly detect whether processes within the control scope on the host have abnormally terminated and can promptly initiate an exit command to the service components for the abnormally terminated process. This triggers the service components to release the resources occupied by the abnormally terminated process in the acceleration device, thereby effectively preventing service component failures.
[0075] It is worth noting that the above-described implementation methods for the monitoring component in the virtualization instance are only optional. In this embodiment, other implementation methods can also be used to implement the monitoring component in the virtualization instance. For example, a dynamic link library or a proxy plugin can be used to implement the monitoring component. Further examples of the implementation methods for the monitoring component will not be provided here; the key is to ensure that the monitoring component can perform the fault monitoring and proactively initiate exit commands required in this embodiment.
[0076] In the above or following embodiments, various implementation methods can be used to maintain the control scope on the host.
[0077] The following section will first introduce the component solutions for the scope of control.
[0078] In one alternative implementation, it is proposed that: if the performance optimization system detects that a target process that needs to use the acceleration device to perform computation is started in any virtualization instance on the host, and if it is determined that the target process has requested concurrency control, then the target process is registered to the control scope maintained by the host; on the host, it monitors whether the processes within the control scope have abnormally exited.
[0079] In this optional implementation, a process is included to determine whether a process has requested concurrency control. By setting this process, processes requiring concurrency control from the service components in the performance optimization system can be selected from the processes on the host that need acceleration. Only these selected processes are registered within the host's management scope. This ensures that the management scope covers all processes on the host that need to use the service components.
[0080] It is understood that the processes covered by the control scope in this embodiment are dynamically changing. Once the process exit process corresponding to any process within the control scope has been completed (automatically completed in case of normal exit, and completed by the performance optimization system in this embodiment in case of abnormal exit), the process can be removed from the control scope. In other words, the control scope corresponding to the host in this embodiment covers processes on the host that require concurrent management by service components and have not yet exited.
[0081] Referring to Figure 3, this optional implementation proposes that a management and control component can be deployed in the host, and the management and control component can maintain the management and control scope for the host.
[0082] Based on this, in this optional implementation: when a monitoring component in any virtualization instance on the host detects that a target process that needs to use an acceleration device to perform computation is started in the virtualization instance, it can determine whether the target process has requested concurrency control; if so, the monitoring component can request the management component in the host to register the target process to the management scope.
[0083] Following the optional implementation method of the monitoring component proposed in the previous embodiments—a function library—referring to Figure 3, in an exemplary implementation: a judgment function can be designed independently in the function library, and if the judgment function determines that the target process has requested concurrency control, it will submit the process information corresponding to the target process to the management and control component deployed on the host. The process information may include the identification information of the acceleration device requested by the target process.
[0084] In this exemplary implementation, the judgment function within the virtualization instance can intercept the acceleration device call request initiated by the target process within the virtualization instance. If the judgment function detects a preset identifier in the acceleration device call request, it can determine that the target process has requested concurrency control. The preset identifier can be understood as an identifier used to characterize the need for concurrency control using a service component. Typically, based on the service component's communication protocol, when a process in the virtualization instance initiates an acceleration device call request, it carries the preset identifier defined in the communication protocol, ensuring that the acceleration device call request reaches and is responded to by the service component.
[0085] In practical applications, virtualization instances typically have an interface library provided by acceleration devices. The target process can call the relevant functional interfaces within this library to initiate the aforementioned acceleration device call request. The acceleration device call request may carry identification information of the requested acceleration device, as well as the aforementioned preset identifier indicating the need for concurrency control using service components. Based on this, in this exemplary implementation, the aforementioned judgment function within the virtualization instance can intercept the target process's calls to the relevant functional interfaces within the interface library, that is, intercept the aforementioned acceleration device call request.
[0086] Referring to Figure 3, a further improvement is proposed in this optional implementation: the management component in the host can monitor whether the target service component in the acceleration device requested by the target process is in normal status after receiving the process information corresponding to the target process submitted by the judgment function in the virtualization instance. There are two scenarios here:
[0087] In one scenario, the management component detects that the target service component is in a "normal state" and allows calls to the service component. In this case, the management component can register the target process within the host's corresponding management scope. Furthermore, it can trigger the judgment function within the aforementioned virtualization instance to send the acceleration device call request initiated by the target process to the aforementioned target service component, enabling the target service component to perform resource scheduling for the target process.
[0088] Understandably, in this situation, after the target process initiates a request to call the acceleration device, the target service component is in a normal state. Therefore, resource scheduling can be performed normally for the target process, and the target process can be accelerated normally.
[0089] In another scenario, the control component detects that the target service component within the acceleration device requested by the target process is in an "abnormal state." In response, the control component can restart the target service component to restore it to a normal state; if the target service component remains abnormal after restarting, the control component can send a rejection command to the aforementioned virtualization instance targeting the target process.
[0090] In this scenario, after the target process initiates a request to access the acceleration device, if the target service component becomes abnormal, the management component will proactively attempt to restore the target service component to a normal state, enabling it to perform resource scheduling for the target process. However, if the management component fails to restore the target service component to a normal state after its attempt, it will send a rejection command to the virtualization instance containing the target process. This will prevent further acceleration device requests from reaching the target service component, thus avoiding further exacerbating the target service component's failure.
[0091] Moreover, the control component can proactively issue a rejection command to the target process, which can trigger the target process to promptly re-initiate a new acceleration device call request and use other acceleration devices in a timely manner. This avoids the target process being in a waiting state for a long time due to abnormal status of the target service component, thereby ensuring the acceleration performance of the target process.
[0092] Understandably, in this situation, if the target service component cannot be restored to a normal state, the management component will not be within the scope of the host's management of registering the target process.
[0093] As can be seen from the description of the two situations above, in this further improved solution, when the control component starts a process that needs to use the service component for concurrent control in any virtualization instance on the host, it first confirms whether the target service component is in a normal state. If an abnormal state is found, it actively tries to restore the target service component to a normal state. If the target service component cannot be restored to a normal state, the control component can directly issue a rejection command to the target process to block the acceleration device call request initiated by the target process from reaching the target service component.
[0094] In this way, the scope of control on the host will cover the processes on the host that can be handled normally by the service components and have not yet exited. Compared with the previously determined scope of control, further removing a portion of processes that cannot be handled normally by the service components can further simplify the scope of control on the host.
[0095] Furthermore, echoing the step of registering signal handling functions into processes in the aforementioned embodiments, here, the management component in the host can act as the executor, registering the aforementioned signal handling functions into the process after registering the process into the management scope. It can be understood that the management component registering a process into the management scope indicates that the process will be normally scheduled for resources by the service component, which also corresponds to the registration timing mentioned earlier—after the service component has completed resource scheduling for the process.
[0096] Furthermore, this further improvement also proposes an exemplary monitoring scheme for detecting whether service components are in an abnormal state. This exemplary monitoring scheme proposes:
[0097] After receiving the process information corresponding to the target process, the management component can detect whether the target service component is in a working state;
[0098] If the target service component is detected to be in a working state, the control component will use the preset test process to send a test call request to the target service component.
[0099] If the target service component does not respond to the test call request, the target service component is determined to be in an abnormal state.
[0100] Additionally, if the target service component is detected to be not in a working state, the management component can control the target service component to start.
[0101] In this exemplary monitoring scheme, the control component can first detect whether the target service component is in a working state, that is, whether the target service component has been started. The detection method could be, for example, sending a status query request to the target service component. If no response is received, it can be determined that the target service component is not in a working state. In this exemplary monitoring scheme, if the target service component is in a working state, the control component will further detect whether the target service component's functionality is abnormal. The detection method is as described above, which can utilize a test process to initiate a test call request. This test call request is essentially a device acceleration call request. Therefore, if the target service component functions normally, the test call request will be responded to normally by the target service component. Conversely, if the target service component does not respond to the test call request, it indicates that the target service component's functionality is abnormal, and the control component can determine that the target service component's state is abnormal.
[0102] The above monitoring scheme is only an example. In this further improved scheme, other monitoring schemes can also be used to monitor whether the service component is in an abnormal state. For example, heartbeat detection can be used to monitor whether the service component is in a working state, and status codes can be used to monitor whether the function of the service component is abnormal. No further examples of monitoring schemes will be given here.
[0103] In summary, this embodiment provides a method for maintaining the control scope on the host through the cooperation of judgment functions deployed in virtualization and management components deployed on the host. During the maintenance of the control scope, the management components on the host can proactively monitor the status of service components deployed within the acceleration device, maintaining them in a normal state in a timely and proactive manner. This ensures that processes on the host that require concurrent management using service components can be more smoothly scheduled for resources, thus accelerating them more quickly. The management components can also proactively refuse to allocate processes to a service component if it cannot be restored to a normal state temporarily, allowing these processes to switch to different acceleration devices in a timely manner and accelerate as soon as possible, avoiding unnecessary waiting. Furthermore, by simplifying the control scope on the host, on the one hand, the control scope can be ensured to cover all processes on the host that need to use service components, thus ensuring the comprehensiveness of fault isolation; on the other hand, the control scope in this embodiment can be effectively simplified, reducing the number of processes requiring fault isolation, thereby reducing the computational load in the fault isolation mechanism and improving fault isolation efficiency.
[0104] It is worth noting that, in addition to the implementation methods described above, other implementation methods can also be used in this embodiment to maintain the control scope of the host. For example, the processes allocated to each service component on the host can be monitored, and the monitored processes can be added to the control scope. In this way, processes on the host that need to use service components for concurrency control and have not yet exited can be included in the control scope. The implementation method for maintaining the control scope is not limited here, nor will further examples be provided. It is understood that different processes covered by the control scope may lead to different computational loads in the fault isolation mechanism, but this does not affect the solution to the fault propagation problem in this embodiment.
[0105] In the above or following embodiments, the performance optimization system on the host can also perform task-level storage resource limiting.
[0106] To address this, this embodiment proposes a storage resource limitation mechanism: After intercepting the acceleration device call request initiated by the target process, the requested storage resource usage can be parsed from the acceleration device call request; the cumulative storage resource usage under the task to which the target process belongs can be determined based on the parsed storage resource usage; it can be determined whether the cumulative storage resource usage has exceeded the storage resource usage threshold corresponding to the task; if it has exceeded the threshold, a rejection instruction is issued to the virtualization instance for the target process, and the rejection instruction is used to refuse the target process from using the requested acceleration device.
[0107] Taking a graphics processing unit (GPU) as an example, the storage resources here can be video memory resources.
[0108] Following the judgment function proposed in the previous embodiments, the aforementioned storage resource limitation mechanism can be implemented here by the judgment function in the virtualization instance.
[0109] In this embodiment, flexible configuration of computing and storage resources for computing tasks is supported, allowing for the setting of thresholds for computing and storage resource usage at the task granularity. For example, users can configure these resources independently when submitting computing tasks within the computing cluster. Users do not need to incur additional learning costs; they can simply complete the configuration according to the native task submission mechanism of the computing cluster. Based on this, in this embodiment, the service components in the performance optimization system can use the task-granularity computing resource usage threshold as one of the scheduling criteria during resource scheduling, thereby more rationally allocating computing resources to processes requiring concurrent control on the acceleration device.
[0110] Regarding storage resources, the storage resource limiting mechanism proposed in this embodiment can be used to intercept acceleration device call requests within the virtualization instance and add a storage resource usage judgment step. By adding this judgment step, acceleration device call requests can be filtered within the virtualization instance, thereby achieving task-level storage resource limiting. In other words, in this embodiment, even if the storage resource usage requested by the process itself does not exceed the storage resource usage threshold corresponding to the process itself, if adding the storage resource usage requested by the process causes the cumulative storage resource usage under the task to which the process belongs to exceed the storage resource usage threshold corresponding to that task, then the monitoring component in the virtualization instance can reject the process's resource request, and the process will fail to request resources.
[0111] Accordingly, in this embodiment, a storage resource limitation mechanism can be used to implement task-level storage resource limitation within the virtualization instance, thereby achieving task-level storage resource isolation and better avoiding the risk of storage resource contention in the acceleration device.
[0112] Figure 4 is a schematic diagram of an application scheme in an exemplary embodiment of this disclosure. Referring to Figure 4, the virtualization instance in this application scenario uses a container, and the acceleration device uses a GPU. According to the performance optimization scheme provided in this embodiment, a function library can be set up in the container shown in Figure 4, and the container also contains an interface library provided by the acceleration device. The acceleration device shown in Figure 4 is configured with a management service component, while the host is configured with a control component.
[0113] Referring to Figure 4, the application solution in this scenario can be roughly divided into the following processing steps:
[0114] 1. When a user process in the container starts, the judgment function in the function library can determine whether the user process belongs to the space-division multiplexing type task (that is, it needs to use the service component for concurrency control). If so, the process information is sent to the control component, that is, the registration information is submitted.
[0115] 2. After receiving process information, the management component can monitor the status of the service components in the GPU requested by the user process:
[0116] (1) If the service component is not started, start the service component;
[0117] (2) If the service component is already started and in normal condition, no action is required;
[0118] (3) If the service component is started but in an abnormal state, restart the service component.
[0119] 3. After confirming that the service component is in a normal state, the management component can determine that the user process has successfully registered and can trigger the judgment function within the container to call the interface library within the container, thereby triggering the service component to perform resource scheduling for the user process. If registration fails, the management component can directly report an error to the container to cause the process to exit.
[0120] 4. After successful registration, the user process can be accelerated normally on the GPU;
[0121] 5. Register the signal handling functions in the function library to the user process. That is, register the signal handling functions. If the user process exits abnormally, the signal handling functions can be triggered. The signal handling functions can actively send an exit command to the service component for the user process, so as to trigger the service component to perform safe exit processing for the user process.
[0122] 6. After completing the safe exit process, the management component can probe the status of the service component again. If the status is normal, no action is required; otherwise, the service component can be restarted.
[0123] As can be seen from the application solutions described for the above application scenarios, the performance optimization system provided in this embodiment can realize process-level space-division multiplexing on acceleration devices, thereby effectively improving the resource utilization of acceleration devices. As for the fault propagation problem that may be caused by this, the abnormal exit fault of the process can be actively monitored and the service component can be actively triggered to perform safe exit processing for the abnormally exiting process. This can effectively avoid the failure of the service component caused by the abnormal exit of the process, thereby avoiding the fault propagation through the service component and realizing enhanced fault isolation between processes.
[0124] Figure 5 is a flowchart illustrating a performance optimization method according to another exemplary embodiment of this disclosure. This method can be executed by a performance optimization system, which can be implemented as software, hardware, or a combination of software and hardware, and can be deployed in a host. Referring to Figure 5, the method may include:
[0125] Step 500: If any process within the control scope on the host is detected to have exited abnormally, determine the target acceleration device used by the process on the host, and the process within the control scope has requested concurrency control.
[0126] Step 501: Send an exit command to the service component deployed in the target acceleration device for the process. The service component is used to perform resource scheduling for processes that need to be concurrently managed on the target acceleration device.
[0127] Step 502: In response to the exit command, the service component releases the resources occupied by the process in the target acceleration device.
[0128] In one optional embodiment, an exit command for the process is initiated to a service component deployed in the target acceleration device, including:
[0129] Using the signal processing function registered in the process, an exit command is sent to the service component deployed in the target acceleration device for the process.
[0130] In an optional embodiment, the method further includes:
[0131] The service components deployed in the target acceleration device register signal processing functions with the process after completing resource scheduling for the process;
[0132] If the signal processing function detects that the process has received an abnormal system signal, then it is determined that the process has abnormally terminated.
[0133] In an optional embodiment, the method further includes:
[0134] If it is detected that a target process that needs to use an acceleration device to perform computation is started in any virtualization instance on the host, and if it is determined that the target process has requested concurrency control, then the target process is registered to the control scope maintained for the host.
[0135] On the host, monitor whether any processes within the control scope experience abnormal termination.
[0136] In an optional embodiment, registering the target process to the management scope maintained for the host includes:
[0137] Using the pre-built judgment function within the virtualization instance, the process information corresponding to the target process is submitted to the management and control component deployed on the host. The process information includes the identification information of the acceleration device requested by the target process.
[0138] If the control component detects that the target service component deployed in the acceleration device requested by the target process is in normal status, the control component registers the target process to the control scope maintained for the host, and triggers the judgment function to send the acceleration device call request initiated by the target process to the service component deployed in the acceleration device requested by the target process.
[0139] The control scope is maintained by the control component.
[0140] In an optional embodiment, the method further includes:
[0141] If the control component detects an abnormal state of the target service component, the control component is used to restart the target service component to restore it to a normal state.
[0142] If the target service component remains abnormal after being restarted, the management component will send a rejection command for the target process to the virtualization instance.
[0143] In an optional embodiment, the method further includes:
[0144] After receiving the process information corresponding to the target process, the management and control component is used to detect whether the target service component is in a working state.
[0145] If the target service component is detected to be in a working state, the control component is used to invoke a preset test process to send a test call request to the target service component.
[0146] If the target service component does not respond to the test call request, then the target service component is determined to be in an abnormal state.
[0147] If the target service component is detected to be not in a working state, then control the target service component to start.
[0148] In an optional embodiment, the method further includes:
[0149] Using a pre-defined judgment function within the virtualization instance, the acceleration device call request initiated by the target process is intercepted;
[0150] If the preset identifier is detected in the acceleration device call request using the judgment function, it is determined that the target process has requested concurrency control.
[0151] In an optional embodiment, after intercepting the acceleration device call request initiated by the target process, the method further includes:
[0152] Using the aforementioned judgment function, the requested storage resource usage is parsed from the acceleration device call request;
[0153] The judgment function is used to determine the cumulative storage resource usage under the task to which the target process belongs based on the parsed storage resource usage, and it is determined whether the cumulative storage resource usage has exceeded the storage resource usage threshold corresponding to the task.
[0154] If the limit has been exceeded, the judgment function is used to issue a rejection instruction to the virtualization instance for the target process and the operation of detecting the preset identifier in the acceleration device call request is no longer performed.
[0155] The rejection instruction is used to refuse the target process from using the requested acceleration device.
[0156] In an optional embodiment, after releasing the resources occupied by the process in the target acceleration device, the method further includes:
[0157] Using the management and control components deployed within the host, it is possible to monitor whether the service components deployed in the target acceleration device are in an abnormal state.
[0158] If an abnormal status is detected, the management and control component is used to control the restart of the service components deployed in the target acceleration device.
[0159] The detailed implementation methods and beneficial effects of each step in the method of this embodiment have been described in detail in the foregoing system embodiments, and will not be elaborated here.
[0160] It should be noted that some processes described in the above embodiments and accompanying drawings include multiple operations that appear in a specific order. However, it should be clearly understood that these operations may not be executed in the order they appear herein, or they may be executed in parallel. The operation numbers, such as 501, 502, etc., are merely used to distinguish different operations and do not represent any execution order. Furthermore, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel.
[0161] Accordingly, this disclosure also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, enables the processor to implement the steps in the above-described method embodiments. The computer-readable storage medium includes volatile or non-volatile or a combination thereof, and may be removable or non-removable. Examples of computer-readable storage media include, but are not limited to, phase-change random access memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random-access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), flash memory or other memory technologies, CD-ROM, Digital Video Disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium.
[0162] Accordingly, this disclosure also provides a computer program product, which includes a computer program or instructions that, when executed by a processor, cause the processor to implement the steps in the above method embodiments. It should be understood that each step or combination of steps in the above method flow can be implemented by the computer program or instructions. Furthermore, these computer programs or instructions can be applied to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device, enabling the processor of the general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to function as an apparatus for implementing the corresponding functions in the above method embodiments.
[0163] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.
[0164] The above are merely embodiments of this disclosure and are not intended to limit the scope of this disclosure. Various modifications and variations can be made to this disclosure by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the scope of the claims of this disclosure.
Claims
1. A performance optimization method, comprising: in response to monitoring an abnormal exit of any process within a management range on a host, determining a target acceleration device used by the process on the host, the process within the management range having requested concurrent management; initiating an exit instruction for the process to a service component deployed in the target acceleration device, the service component being configured to schedule resources for the process requiring concurrent management on the target acceleration device; releasing resources occupied by the process in the target acceleration device in response to the exit instruction by the service component.
2. The method of claim 1, wherein, initiating the exit instruction for the process to the service component deployed in the target acceleration device, comprises: initiating the exit instruction for the process to the service component deployed in the target acceleration device by a signal processing function registered in the process.
3. The method of claim 1, wherein, further comprising: registering the signal processing function in the process after the service component deployed in the target acceleration device completes resource scheduling for the process; determining that the process has exited abnormally if the signal processing function monitors that the process has received an abnormal system signal.
4. The method according to any one of claims 1 to 3, wherein, further comprising: in response to monitoring that a target process requiring computation performed by an acceleration device is started in any virtualization instance on the host, registering the target process in the management range maintained for the host if it is determined that the target process has requested concurrent management; monitoring whether the process within the management range on the host has exited abnormally.
5. The method of claim 4, wherein, registering the target process in the management range maintained for the host, comprises: submitting process information corresponding to the target process to a management component deployed on the host by a judgment function preset in the virtualization instance, the process information including identification information of an acceleration device requested to be used by the target process; registering the target process in the management range maintained for the host by the management component if it is monitored by the management component that a target service component deployed in the acceleration device requested to be used by the target process is in a normal state.
6. The method of claim 5, wherein, further comprising: after the target process is registered in the management range maintained for the host, triggering the judgment function by the management component to send an acceleration device call request initiated by the target process to a service component deployed in the acceleration device requested to be used by the target process.
7. The method of claim 5, wherein, further comprising: if it is monitored by the management component that the target service component is in an abnormal state, controlling the target service component to restart by the management component so as to restore the target service component to a normal state; if the target service component is still in an abnormal state after restarting, sending a rejection instruction for the target process to the virtualization instance by the management component.
8. The method of claim 7, wherein, further comprising: after receiving the process information corresponding to the target process, detecting whether the target service component is in a working state by the management component; if it is detected that the target service component is in the working state, initiating a test call request to the target service component by the management component by calling a preset test process. If the target service component does not respond to the test call request, it is determined that the target service component is abnormal; If it is detected that the target service component is not in a working state, the target service component is started.
9. The method of claim 4, wherein, Further comprising: intercepting an acceleration device call request initiated by the target process by using a judgment function pre-installed in the virtualization instance; If a preset identifier is detected in the acceleration device call request by using the judgment function, it is determined that the target process has requested concurrent control.
10. The method of claim 9, wherein, After intercepting the acceleration device call request initiated by the target process, further comprising: resolving the requested storage resource occupation from the acceleration device call request by using the judgment function; determining the cumulative storage resource occupation of the task to which the target process belongs based on the resolved storage resource occupation by using the judgment function, and determining whether the cumulative storage resource occupation has exceeded the storage resource occupation threshold corresponding to the task; If it has exceeded, issuing a rejection instruction for the target process to the virtualization instance by using the judgment function, and the judgment function no longer performs the operation of detecting the preset identifier in the acceleration device call request; The rejection instruction is used to reject the use of the requested acceleration device by the target process.
11. The method of any one of claims 1-3 or 5-10, wherein, Further comprising: After releasing the resources occupied by the process in the target acceleration device, monitoring whether the service component deployed in the target acceleration device is abnormal by using the control component deployed in the host; If an abnormality is monitored, restarting the service component deployed in the target acceleration device by using the control component.
12. A performance optimization system, comprising a monitoring component deployed in a virtualization instance in a host and a service component deployed in an acceleration device assembled on the host; The monitoring component in any virtualization instance is configured to determine a target acceleration device used by any process in a control range in the virtualization instance to exit abnormally, the process in the control range having requested concurrent control, when monitoring that the process exits abnormally. The monitoring component is further configured to initiate an exit instruction for the process to a service component deployed in the target acceleration device, the service component being configured to perform resource scheduling for the process requiring concurrent control on the target acceleration device. The service component is configured to release resources occupied by the process in the target acceleration device in response to the exit instruction.
13. The system of claim 12, wherein, The monitoring component comprises a signal processing function, and the monitoring component is specifically configured to: initiate the exit instruction for the process to the service component deployed in the target acceleration device by using the signal processing function registered in the process.
14. The system of claim 12, wherein, The performance optimization system further comprises a control component, and the control component is configured to: In a case where the monitoring component submits process information of a target process, if it is monitored that a target service component deployed in an acceleration device requested to be used by the target process is in a normal state, the target process is registered into the control range maintained for the host. The control range is maintained by the control component, and the process information is submitted by the monitoring component after it is determined that the target process has requested concurrent control.
15. A computer-readable storage medium storing a computer program which, when executed by one or more processors, causes the one or more processors to perform the performance optimization method as follows: in a case where it is monitored that any process in a control range on a host abnormally exits, the target acceleration device used by the process is determined, and the process has requested concurrent control; an exit instruction for the process is initiated to a service component deployed in the target acceleration device, the service component is used for resource scheduling for processes on the target acceleration device that need to be concurrently controlled; and resources occupied by the process in the target acceleration device are released by the service component in response to the exit instruction.
16. The computer-readable storage medium storing a computer program of claim 15, wherein, The one or more processors are further configured to perform the performance optimization method as follows: an exit instruction for the process is initiated to a service component deployed in the target acceleration device by using a signal processing function registered in the process.
17. The computer-readable storage medium storing a computer program of claim 15, wherein, The one or more processors are further configured to perform the performance optimization method as follows: after the service component deployed in the target acceleration device completes resource scheduling for the process, a signal processing function is registered in the process; and if it is monitored by the signal processing function that the process receives an abnormal system signal, it is determined that the process abnormally exits.
18. The computer-readable storage medium storing a computer program according to any one of claims 15 to 17, wherein, The one or more processors are further configured to perform the performance optimization method as follows: in a case where it is monitored that a target process that needs to perform computation using an acceleration device is started in any virtualization instance on the host, if it is determined that the target process has requested concurrent control, the target process is registered into the control range maintained for the host; and on the host, it is monitored whether a process in the control range abnormally exits.
19. The computer-readable storage medium storing a computer program of claim 18, wherein, The one or more processors are further configured to perform the performance optimization method as follow: process information corresponding to the target process is submitted to a control component deployed on the host by using a judgment function preset in the virtualization instance, the process information includes identification information of an acceleration device requested to be used by the target process; and if it is monitored by the control component that a target service component deployed in the acceleration device requested to be used by the target process is in a normal state, the target process are registered into the control range maintained for the host by using the control component.
20. The computer-readable storage medium storing a computer program of claim 19, wherein, The one or more processors are further configured to perform a performance optimization method as follows: after the target process is registered into the management scope maintained for the host, the judgment function triggered by the management component sends a request for an acceleration device call initiated by the target process to a service component deployed in an acceleration device requested to be used by the target process.