Service management method and device, electronic equipment and storage medium

Abstract

The invention discloses a service management method and device, electronic equipment and a storage medium, and relates to the technical field of computers, and the method comprises the steps: receiving service configuration information; wherein the service configuration information comprises the minimum starting instance number of each service and a resource constraint condition; according to the service configuration information, service instances meeting the corresponding minimum starting instance number and the corresponding resource constraint condition are deployed for each service in an accelerator resource pool; and acquiring a service load index, calculating a target instance number of each service based on the service load index by using a proportional integral derivative controller, and performing elastic scaling scheduling on the service instance of each service according to the target instance number of each service. In the process of scheduling the accelerator resources, the resource utilization rate is improved, and the service stability is ensured.

Description

Technical Field

[0001] This application relates to the field of computer technology, and more specifically, to a service management method, apparatus, electronic device, and storage medium. Background Technology

[0002] In related technologies, accelerator computing power service management typically employs static resource configuration or elastic scheduling mechanisms triggered by simple thresholds. Graphics Processing Units (GPUs), as the current mainstream accelerator type, are widely used in high-performance computing scenarios such as artificial intelligence inference. However, these technologies have significant shortcomings when facing scenarios with multiple services coexisting, complex resource constraints, and dynamically changing loads. On the one hand, deploying services based solely on a fixed number of instances makes it difficult to balance the availability of core services with overall resource utilization; on the other hand, using threshold-based elastic scaling strategies can easily lead to frequent instance additions and removals due to load fluctuations, causing scheduling jitter, resource waste, and even service instability.

[0003] Therefore, how to improve resource utilization and ensure service stability during the scheduling of accelerator resources is a technical problem that needs to be solved by those skilled in the art. Summary of the Invention

[0004] The purpose of this application is to provide a service management method, apparatus, electronic device, and storage medium that improves resource utilization and ensures service stability during the scheduling of accelerator resources.

[0005] To achieve the above objectives, this application provides a service management method, comprising: Receive service configuration information; wherein, the service configuration information includes the minimum number of instances to be started and resource constraints for each service; Based on the service configuration information, deploy service instances in the accelerator resource pool that meet the corresponding minimum number of startup instances and the corresponding resource constraints for each service. Collect service load metrics, use a proportional-integral-derivative controller to calculate the target number of instances for each service based on the service load metrics, and perform elastic scaling and scheduling of service instances for each service according to the target number of instances for each service.

[0006] The resource constraints include any one or a combination of deployment mode constraints, affinity constraints, and anti-affinity constraints. The deployment mode constraints describe how service instances occupy resources on the accelerator, and include exclusive mode and shared mode. The affinity constraints indicate that service instances of the target service and service instances of other specified services should be deployed on the same accelerator. The anti-affinity constraints indicate that service instances of the target service and service instances of other specified services should not be deployed on the same accelerator. The accelerators in the accelerator resource pool are abstractly represented by a multi-slot resource model, with each accelerator being abstracted into multiple virtual slots; If the deployment mode constraint of the target service is exclusive mode, then the service instance of the target service occupies all slots of the accelerator. If the deployment mode constraint of the target service is shared mode, then the service instance of the target service occupies one slot of the target accelerator and shares the target accelerator with the service instances occupying other slots of the target accelerator.

[0007] The step of deploying service instances in the accelerator resource pool for each service, based on the service configuration information, that meet the corresponding minimum number of startup instances and the corresponding resource constraints, includes: All service instances to be deployed are prioritized according to the resource constraints of their respective services; wherein, the service instances to be deployed are service instances of the services to be deployed, and the services to be deployed are those that do not meet the corresponding minimum number of instances to be started; Based on the sorting results, all the service instances to be deployed are traversed sequentially, and candidate accelerators are determined in the accelerator resource pool based on the resource constraints of the currently traversed service instances to be deployed. The matching score of the candidate accelerator is determined based on a preset scoring strategy; wherein, the preset scoring strategy includes: if the resource constraint of the service to which the currently traversed service instance to be deployed belongs is a shared mode, then the matching score of the candidate accelerator that is occupied but still has free slots is higher than the matching score of the candidate accelerator that is completely free. Deploy the currently traversed service instances to the candidate accelerator with the highest matching score.

[0008] The step of using a proportional-integral-derivative controller to calculate the target number of instances for each service based on the service load metric includes: Calculate the stress index for each service based on the service load metrics; The theoretical target number of instances for each service is calculated based on the stress index of each service using a proportional-integral-derivative controller. The maximum of the theoretical target number of instances and the minimum number of instances to start for each service is used as the target number of instances for each service.

[0009] The step of calculating the stress index for each service based on the service load metric includes: The stress index for each service is calculated according to the stress index calculation formula; wherein, the stress index calculation formula is: ; in, Let be the pressure index at time t. Let t be the service latency collected at time t. The preset delay target value, Let be the length of the request queue collected at time t. The target value for the preset request queue length. , These are the weighting coefficients. .

[0010] The step of using a proportional-integral-derivative controller to calculate the theoretical target number of instances for each service based on the stress index of each service includes: The theoretical target number of instances for each service is calculated using a proportional-integral-derivative (PID) controller based on a control formula; wherein the control formula is: ; in, Let be the theoretical number of target instances at time t. Let be the current instance number at time t. Let be the error at time t. , Let be the pressure index at time t. This is the proportional gain coefficient. This is the integral gain coefficient. For integration variables, for Time error, This is the differential gain coefficient.

[0011] This also includes: Service requests are received through a unified gateway, and the service requests are routed and distributed according to the current service instance status.

[0012] To achieve the above objectives, this application provides a service management device, comprising: A receiving module is used to receive service configuration information; wherein, the service configuration information includes the minimum number of instances to be started and resource constraints for each service; The deployment module is used to deploy service instances in the accelerator resource pool for each service based on the service configuration information, which meet the corresponding minimum number of startup instances and the corresponding resource constraints. The scheduling module is used to collect service load indicators, calculate the target number of instances for each service based on the service load indicators using a proportional-integral-derivative controller, and perform elastic scaling scheduling of service instances for each service according to the target number of instances for each service.

[0013] To achieve the above objectives, this application provides an electronic device, comprising: Memory, used to store computer programs; A processor for implementing the steps of the service management method described above when executing the computer program.

[0014] To achieve the above objectives, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the service management method described above.

[0015] The service management method provided in this application firstly ensures that each service has basic availability and resource compliance in its initial state by receiving service configuration information containing the minimum number of instances to be launched and resource constraints, and deploying service instances that meet these two requirements in the accelerator resource pool for each service. Secondly, by collecting service load metrics and using a Proportional-Integral-Derivative (PID) controller to dynamically calculate the target number of instances for each service based on these metrics, it can respond to real-time load changes. The PID controller achieves smooth and continuous calculation of the target number of instances by comprehensively controlling the current error (proportional term), historical error accumulation (integral term), and error change trend (derivative term). This approach overcomes the problem of frequent jumps in the number of instances caused by instantaneous load fluctuations in traditional threshold-triggered mechanisms, making the elastic scaling process more stable and adaptive, and reducing performance jitter and scheduling overhead caused by frequent instance starts and stops. Finally, the service instances are elastically scaled and scheduled according to the calculated target number of instances, enabling the system to dynamically adjust resource allocation based on the actual load while ensuring the minimum number of instances. When the load increases, the number of instances can be increased to improve processing capacity; when the load decreases, the number of instances can be reduced appropriately, provided it does not fall below the minimum number, to release resources. This ensures both continuous service availability and on-demand allocation and efficient utilization of resources, thereby improving the overall scheduling efficiency of the resource pool and the stability of system operation. In summary, the service management method provided in this application, by combining a minimum instance guarantee mechanism with dynamic elastic scaling based on PID control, ensures basic service availability while achieving smooth and adaptive scheduling of accelerator resources, effectively improving resource utilization and service stability. This application also discloses a service management device, an electronic device, and a computer-readable storage medium, which can achieve the same technical effects.

[0016] It should be understood that the above general description and the following detailed description are merely exemplary and do not limit this application. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. The drawings are used to provide a further understanding of this disclosure and constitute a part of the specification. They are used together with the following detailed description to explain this disclosure, but do not constitute a limitation of this disclosure. In the drawings: Figure 1 This is a flowchart illustrating a service management method according to an exemplary embodiment; Figure 2 A flowchart illustrating another service management method according to an exemplary embodiment; Figure 3 This is an architecture diagram of a service management system provided in one application embodiment of this example; Figure 4(a) is a schematic diagram of the GPU resource model in exclusive mode in an application embodiment provided in this embodiment; Figure 4(b) is a schematic diagram of a shared GPU resource model in an application embodiment provided in this embodiment; Figure 5 This is a flowchart of a static scheduling algorithm provided in an application embodiment of this example; Figure 6 This is a structural diagram illustrating a service management device according to an exemplary embodiment; Figure 7 This is a structural diagram of an electronic device according to an exemplary embodiment. Detailed Implementation

[0018] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0019] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0020] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0021] This application discloses a service management method that improves resource utilization and ensures service stability during the scheduling of accelerator resources.

[0022] See Figure 1 A flowchart illustrating a service management method according to an exemplary embodiment, such as... Figure 1 As shown, it includes: S101: Receive service configuration information; wherein, the service configuration information includes the minimum number of instances to be started and resource constraints for each service; Here, a service refers to a software unit deployed on the accelerator resource pool that can run independently and provide specific functions, such as an image recognition algorithm or a speech processing model. Service configuration information is a set of structured parameters provided by the user when creating or managing a service, used to guide the system in deploying and scheduling the service. The minimum number of instances required to start refers to the minimum number of instances the system must maintain and run to ensure basic availability of the service at all times. Resource constraints refer to the restrictive requirements on the computing resources a service requires during deployment, and can include deployment mode constraints, affinity constraints, and anti-affinity constraints. Deployment mode constraints describe how service instances occupy resources on the accelerator, including exclusive mode and shared mode. Affinity constraints indicate that service instances of the target service and service instances of other specified services should be deployed on the same accelerator, while anti-affinity constraints indicate that service instances of the target service and service instances of other specified services should not be deployed on the same accelerator.

[0023] In this step, service configuration information from the user is received and persistently stored through a management interface. This management interface performs format and validity checks on the input information to ensure data integrity and accuracy. The minimum number of instances required to start for different services can be the same or different; this embodiment does not impose a specific limitation.

[0024] As can be seen, this step, by clearly defining and receiving the minimum number of service instances to be launched and resource constraints, lays a precise input foundation for subsequent automated and compliant resource scheduling, enabling the system to balance service stability requirements with flexible resource allocation.

[0025] S102: Based on the service configuration information, deploy service instances in the accelerator resource pool that meet the corresponding minimum number of startup instances and the corresponding resource constraints for each service. An accelerator resource pool refers to a collection of computing power resources consisting of multiple physical or virtual GPUs or other dedicated computing accelerator cards. A service instance is a specific running copy of a service, capable of independently receiving and processing requests.

[0026] In this step, an initial scheduling plan is generated based on the configuration information of all services. This scheduling process is a constraint satisfaction problem, aiming to find a placement scheme in the resource pool such that the number of instances for each service reaches at least its minimum number of instances to start, and the deployment location of all instances fully meets their respective resource constraints. This step can be implemented using a scheduler that employs heuristic algorithms, for example, prioritizing scheduling services with strict resource requirements (such as requiring exclusive access) or a large number of minimum instances, and comprehensively considering the degree of matching between the node's remaining resources and constraints when selecting specific accelerator nodes.

[0027] As can be seen, this step ensures that the service cluster is in a stable and compliant state from the start by performing an initial deployment that aims to guarantee the minimum number of instances and uses resource constraints as a hard rule, thus providing basic resource guarantees for the availability of core services.

[0028] In a preferred embodiment, the accelerators in the accelerator resource pool are abstractly represented by a multi-slot resource model, with each accelerator being abstracted into multiple virtual slots. If the deployment mode constraint of the target service is exclusive mode, then the service instance of the target service occupies all the slots of the accelerator. If the deployment mode constraint of the target service is shared mode, then the service instance of the target service occupies one slot of the target accelerator and shares the target accelerator with the service instances occupying other slots of the target accelerator.

[0029] In practical implementation, a multi-slot resource model is introduced as an abstract layer when managing accelerator resources to achieve fine-grained segmentation and unified management of accelerators. Specifically, each physical accelerator in the accelerator resource pool is no longer simply regarded as an indivisible whole resource, but is abstractly represented by the multi-slot resource model as a logical resource container composed of multiple virtual slots. Each virtual slot represents a basic computing power unit or resource partition on that accelerator that can be used by a service instance. Deployment mode constraints are a specific type of resource constraint in service configuration information, used to define how service instances are required to occupy accelerator resources.

[0030] When deploying and scheduling based on service configuration information, if the deployment mode constraint of the target service (i.e., the service currently to be scheduled) is specified as exclusive mode, it means that an instance of this service needs to exclusively use all the computing resources of the entire accelerator card during runtime to avoid resource contention with other services and ensure performance isolation. Therefore, when allocating resources to this service instance, the system binds it to a completely idle accelerator card and marks all virtual slots of the accelerator as occupied by this instance. Conversely, if the deployment mode constraint of the target service is specified as shared mode, it means that an instance of this service is allowed to run concurrently with other compatible service instances on the same physical accelerator to improve resource utilization. In this mode, the system uses the total number of virtual slots preset on the accelerator card as the allocable unit. When a shared mode service instance needs to be deployed, the scheduler only needs to find an unoccupied free virtual slot on the target accelerator and allocate it to the instance. Subsequently, this instance shares the underlying physical accelerator's computing resources with other service instances occupying slots on the same accelerator. These instances typically achieve time-sharing or soft isolation of resources through techniques such as time-slice rotation, streaming multiprocessor partitioning, or containerized isolation.

[0031] As can be seen, this implementation method uses a multi-slot resource model to virtualize and abstract the physical accelerator, and clearly defines two deployment modes: exclusive and shared. This enables the system to handle various service requirements, from strong isolation to high-density sharing, in a unified and flexible manner. Under complex constraints, it achieves fine-grained, hybrid resource scheduling, which not only meets the performance determinism requirements of key businesses, but also improves the overall resource pool utilization efficiency.

[0032] S103: Collect service load metrics, use a proportional-integral-derivative controller to calculate the target number of instances for each service based on the service load metrics, and perform elastic scaling scheduling of service instances for each service according to the target number of instances for each service.

[0033] Service load metrics refer to quantitative data reflecting the real-time workload of service instances, such as request processing latency, request rate per unit time, and request queue backlog length. A proportional-integral-derivative (PID) controller is a feedback control algorithm used to dynamically adjust the output value based on system errors. The target instance count is the theoretically optimal number of service instances calculated by the PID controller, used to guide elastic scaling decisions.

[0034] In this step, the monitoring module periodically collects load metrics for each service instance, which serve as the input error signal for the PID controller. The controller calculates the theoretical target number of instances based on preset proportional gain, integral gain, and derivative gain coefficients. Subsequently, the system compares this value with the minimum number of instances to be started in the service configuration, taking the larger value and rounding it down as the final number of instances to be executed. If the current number of instances is less than this value, expansion is triggered, deploying new instances from the remaining resources; if it is greater than this value, shrinkage is triggered, prioritizing the destruction of dynamically created instances to release resources.

[0035] As can be seen, this step achieves smooth and adaptive control of the number of service instances by introducing a feedback adjustment mechanism based on a PID controller. It overcomes the frequent oscillation problem caused by traditional threshold-triggered scaling, and can respond more delicately to load change trends. While ensuring the minimum number of service instances, it dynamically optimizes resource allocation, thereby improving resource utilization while maintaining service performance stability and service quality.

[0036] As a feasible implementation method, this embodiment also includes: receiving service requests through a unified gateway and routing and distributing the service requests according to the current service instance status.

[0037] In practical implementation, a unified gateway is deployed to receive all invocation requests from external systems or user clients targeting various services on the accelerator resource pool. The unified gateway, as an independent software component or service, serves as the sole standardized access point for all services. It is responsible for receiving network requests and performing necessary preprocessing, such as protocol conversion, authentication and authorization, and traffic metering. The current service instance status refers to the set of real-time status information maintained by the system's core scheduling module regarding all running instances of each service, including but not limited to each instance's network access address, health status (e.g., whether it can respond normally), current load weight, and the physical or logical resource location it belongs to.

[0038] When an external request arrives at the unified gateway, the gateway first determines the target service based on the service identifier contained in the request. Then, the gateway queries the service discovery component or state management module to obtain a real-time status list of all available and healthy instances of the target service. Based on this list and preset routing policies, the gateway executes a routing distribution decision. Common routing policies include, but are not limited to: round-robin, which forwards the request sequentially to each instance in the list to achieve basic load balancing; weighted round-robin, which allocates different proportions of the request volume according to the preset weight of the instance or real-time load indicators; or location affinity, which prioritizes forwarding requests to instances with lower network latency or in the same availability zone as the request source. Finally, the gateway proxies the request to a selected specific service instance and forwards the response returned by that instance back to the requester, thus completing a full service call.

[0039] As can be seen, this implementation method, by introducing a unified gateway as a centralized request access and routing hub, effectively shields the complex service topology, which consists of multiple potentially dynamically changing service instances, from upstream requesters, providing a transparent and stable access endpoint. This not only greatly simplifies the calling logic of service consumers and avoids the complexity of directly managing backend instances, but also lays the architectural foundation for achieving advanced features such as unified traffic control and security policy implementation, thereby enhancing the manageability and security of the entire computing power service management system.

[0040] The service management method provided in this application firstly receives service configuration information containing the minimum number of instances to be started and resource constraints, and deploys service instances that meet these two requirements for each service in the accelerator resource pool, ensuring that each service has basic availability and resource compliance in its initial state. Secondly, by collecting service load indicators and using a proportional-integral-derivative (PID) controller to dynamically calculate the target number of instances for each service based on these indicators, it can respond to real-time load changes. The PID controller achieves smooth and continuous calculation of the target number of instances by comprehensively controlling the current error (proportional term), historical error accumulation (integral term), and error change trend (derivative term). This approach overcomes the problem of frequent jumps in the number of instances caused by instantaneous load fluctuations in traditional threshold-triggered mechanisms, making the elastic scaling process more stable and adaptive, and reducing performance jitter and scheduling overhead caused by frequent instance starts and stops. Finally, the service instances are elastically scaled and scheduled according to the calculated target number of instances, enabling the system to dynamically adjust resource allocation based on the actual load while ensuring the minimum number of instances. When the load increases, the number of instances can be increased to improve processing capacity; when the load decreases, the number of instances can be reduced appropriately, provided it does not fall below the minimum number, to release resources. This ensures both continuous service availability and on-demand resource allocation and efficient utilization, thereby improving the overall scheduling efficiency of the resource pool and the stability of system operation. In summary, the service management method provided in this application, by combining a minimum instance guarantee mechanism with dynamic elastic scaling based on PID control, ensures basic service availability while achieving smooth and adaptive scheduling of accelerator resources, effectively improving resource utilization and service stability.

[0041] This application discloses a service management method. Compared with the previous embodiment, this embodiment further explains and optimizes the technical solution. Specifically: See Figure 2 A flowchart illustrating another service management method according to an exemplary embodiment, such as Figure 2 As shown, it includes: S201: Receive service configuration information; wherein, the service configuration information includes the minimum number of instances to be started and resource constraints for each service; S202: Prioritize all service instances to be deployed according to the resource constraints of their respective services; wherein, the service instances to be deployed are service instances of the services to be deployed, and the services to be deployed are those that do not meet the corresponding minimum number of instances to be started; In this step, services that have not yet met their minimum number of startup instances are first selected from all services and marked as services to be deployed. For each service to be deployed, a corresponding number of service instance objects to be deployed are generated based on the number of instances it still lacks. Subsequently, these services are sorted as a whole according to the resource constraint type of the service to which each service instance belongs.

[0042] A typical sorting logic is as follows: Deployed service instances requiring exclusive mode (i.e., whose services require exclusive deployment mode constraints) are given higher priority and placed at the front of the queue; while deployed service instances requiring shared mode are given relatively lower priority and placed at the back of the queue. The motivation behind this design is that exclusive mode instances consume the entire accelerator card, resulting in large resource granularity. If scheduled later, it may be difficult to find a complete idle accelerator due to the scattered deployment of shared instances in the early stages, leading to scheduling failures or a large amount of resource fragmentation. Conversely, scheduling exclusive instances first can prioritize locking complete accelerator resources, ensuring these hard requirements are met; when scheduling shared instances subsequently, they can be flexibly filled into idle slots outside of accelerators already occupied by exclusive instances, or into shared accelerators already partially occupied in the early stages of scheduling, resulting in more efficient resource utilization.

[0043] For example, suppose there is service A (exclusive mode, requiring 2 instances) and service B (shared mode, requiring 4 instances), both currently have 0 instances, meaning neither meets the minimum number of instances required to start. The system generates 2 (A1, A2) and 4 (B1, B2, B3, B4) service instances to be deployed. According to the priority sorting rules, the two exclusive mode A instances (A1, A2) will be at the front of the queue, followed by the four shared mode B instances (B1-B4). The scheduler will process them in this order.

[0044] As can be seen, this step establishes an ordered processing queue for static scheduling by prioritizing based on constraints, guiding the system to make deployment decisions in an order that is more conducive to overall resource integration and utilization. This provides an effective heuristic starting point for solving complex constraint satisfaction problems and helps improve scheduling success rate and the rationality of long-term resource pool planning.

[0045] S203: According to the sorting result, all the service instances to be deployed are traversed in sequence. Based on the resource constraints of the currently traversed service instances to be deployed, candidate accelerators are determined in the accelerator resource pool. Based on the preset scoring strategy, the matching score of the candidate accelerator is determined. The currently traversed service instances to be deployed are deployed to the candidate accelerator with the highest matching score. In this step, for the currently iterating service instance to be deployed in the queue, the scheduler first filters candidate accelerators from the accelerator resource pool. The filtering process mainly involves constraint predicate checks: traversing all accelerator nodes in the pool, checking whether each node meets all the hard resource constraints of the current instance. These conditions include not only deployment mode (exclusive / shared), but may also include affinity / anti-affinity constraints (e.g., cannot share a card with a specific service), hardware specifications, etc. Accelerator nodes that pass the check are added to the candidate accelerator list. Subsequently, for each candidate accelerator in the list, a matching score is calculated according to a preset scoring strategy. The preset scoring strategy includes: if the resource constraints of the service to which the currently iterating service instance belongs are in shared mode, then the matching score of a candidate accelerator that is already occupied but still has free slots is higher than the matching score of a completely free candidate accelerator. This scoring strategy aims to quantify the suitability of each candidate accelerator for deploying the current instance, and one of its core principles is: for instances to be deployed in shared mode, priority is given to filling rather than expanding. Specifically, if the current instance is in shared mode, the scoring strategy assigns higher matching scores to candidate accelerators that are already occupied (i.e., at least one other service instance is running) but still have free virtual slots; while assigning relatively lower matching scores to candidate accelerators that are completely idle (i.e., all slots are unoccupied). This strategy is called the worst-case match or fill-first strategy. Its purpose is to encourage the consolidation of multiple shareable instances onto the same accelerator card as much as possible, thereby reserving more complete, unoccupied accelerators for future expansion of new services or instances that may require exclusive mode. For instances in exclusive mode, the scoring strategy typically favors completely idle accelerators to avoid resource conflicts and performance interference. Finally, the scheduler selects the candidate accelerator with the highest matching score, performs the deployment operation, and updates the accelerator's resource occupancy status (e.g., marking slots occupied or the entire card as exclusive).

[0046] For example, following the S202 example, we first process the exclusive instance A1. The scheduler checks the resource pool and finds that GPU-1 and GPU-2 are completely idle and meet other constraints, making them candidates. According to the exclusive mode scoring rules (which favor completely idle cards), both have the same score, so we assume GPU-1 is randomly selected for deployment on A1. Next, we process A2, similarly selecting GPU-2 for deployment. Then we begin processing the shared instance B1. At this point, the resource pool status is: GPU-1 and GPU-2 are exclusively occupied with no free slots; GPU-3 is completely idle (4 slots); GPU-4 has one slot occupied (running other services), leaving 3 free slots. The candidate accelerators are GPU-3 and GPU-4 (assuming both meet affinity constraints, etc.). According to the shared mode scoring strategy, the matching score of the occupied GPU-4 is higher than that of the completely idle GPU-3. Therefore, B1 is deployed to one of the free slots of GPU-4. This strategy ensures that the complete GPU-3 card is retained.

[0047] As can be seen, this step, by combining constraint filtering and intelligent scoring strategies, especially the rule of prioritizing the filling of occupied accelerators in the shared mode, achieves refined and forward-looking resource allocation. It not only meets the deployment needs of individual instances, but also proactively reduces resource fragmentation from the perspective of global resource pool optimization, reserving space for potential large-granularity resource requests in the future, thereby improving the long-term utilization and scheduling flexibility of the entire accelerator resource pool.

[0048] S204: Collect service load indicators, calculate the stress index of each service based on the service load indicators, use the proportional-integral-derivative controller to calculate the theoretical target number of instances for each service based on the stress index of each service, and take the maximum value of the theoretical target number of instances and the minimum number of instances to be started as the target number of instances for each service.

[0049] In this step, service load metrics are continuously collected through a monitoring agent, and multiple key load metrics are combined to calculate a comprehensive stress index. The stress index is a scalar value calculated through normalization or weighting, used to quantify the current overall stress level of the service. A positive stress index usually indicates that the service is overloaded, a negative value indicates that the load is too low, and a value near zero indicates that the load is balanced.

[0050] As a feasible implementation method, calculating the stress index of each service based on the service load index includes: calculating the stress index of each service according to the stress index calculation formula; wherein, the stress index calculation formula is: ; in, Let be the pressure index at time t. Let be the service latency collected at time t. This can be the latency of a preset percentage of services. The preset percentage (95%) of all services collected at time t will have a latency less than 1 / 2 ms. , The preset delay target value, Let be the length of the request queue collected at time t. The target value for the preset request queue length. , These are the weighting coefficients. .

[0051] Next, the pressure index calculated in real time for each service is input into a proportional-integral-derivative (PID) controller. The PID controller compares the pressure index with the desired target pressure index (usually set to 0, representing an ideal equilibrium state) to obtain an error signal. Then, based on the error signal, it outputs a theoretical target instance count.

[0052] As a feasible implementation method, the step of calculating the theoretical target number of instances for each service based on the stress index of each service using a proportional-integral-derivative (PID) controller includes: calculating the theoretical target number of instances for each service using a PID controller based on a control formula; wherein the control formula is: ; in, Let be the theoretical number of target instances at time t. Let be the current instance number at time t. Let be the error at time t. , Let be the pressure index at time t. This is the proportional gain coefficient. This allows the controller to respond quickly to the current error. This is the integral gain coefficient. For integration variables, for Time error, This term is used to accumulate historical errors, with the aim of eliminating steady-state errors. The differential gain coefficient, The term is used to measure the rate of change of response error. Its purpose is to predict future error trends, suppress system oscillations, and prevent overshoot caused by excessively fast response.

[0053] The theoretical target number of instances is a continuous value, which may be a non-integer. From a control theory perspective, it provides a smoothed-adjustment recommendation for the number of instances needed to eliminate current and near-term stress errors. However, the theoretical target number of instances may be lower than the minimum number of instances required to ensure basic service availability. Therefore, to ensure that the Service Level Agreement (SLA) bottom line is not breached, the theoretical target number of instances is compared with the minimum number of instances required to start, and the larger value is determined as the final target number of instances for the service. , The final target number of instances to be executed. The minimum number of instances configured for the i-th service. This means that the dynamic scaling recommendation will never reduce the number of instances below the minimum number of instances, thus maintaining the bottom line of service reliability while pursuing resource efficiency.

[0054] As can be seen, this step comprehensively assesses the load status by integrating multiple stress indices, and uses a PID controller to achieve smooth and oscillation-resistant target capacity calculation. Finally, by coupling with a minimum instance count guarantee mechanism, it achieves an organic unity between elastic scaling and stability assurance. This ensures that while dynamic scheduling actively responds to load fluctuations and optimizes resource utilization, its decision boundary is always limited by the basic requirements of service availability, thus constructing a flexible and reliable adaptive resource management system.

[0055] S205: Perform elastic scaling and scheduling of service instances for each service based on the target number of instances for each service.

[0056] In specific implementation, if Then, perform expansion and add... In this instance, the algorithm in step S203 can be invoked to find a deployable slot in the available accelerator resources and deploy it. If Then, shrink the size and destroy the file. For each instance, instances dynamically expanded will be destroyed first, and their resources will be released.

[0057] Therefore, this embodiment demonstrates that in the static scheduling phase, by employing a priority ranking mechanism based on resource constraint types and a worst-case matching scoring strategy designed for the shared mode, it can intelligently plan the initial deployment, effectively solving the problem of resource allocation under large-scale complex constraints, and particularly avoiding resource fragmentation caused by improper scheduling order. By prioritizing exclusive services and encouraging the integrated deployment of shared services, it not only ensures the immediate availability of services with strict resource requirements but also reserves complete accelerator resources for possible subsequent elastic scaling, thereby significantly improving the overall utilization rate of the resource pool and the rationality of long-term planning. In the dynamic scaling phase, by integrating multiple indicators to calculate the comprehensive stress index and introducing a PID controller for smooth adjustment, and finally performing a protective maximum value calculation with the minimum number of instances to start, more accurate and stable elastic scaling control is achieved. This method overcomes the frequent oscillation problem caused by instantaneous load fluctuations in traditional threshold triggering mechanisms, ensuring the continuity and stability of scaling up / down decisions. At the same time, the final target number of instances is always not lower than the service guarantee baseline, thus flexibly responding to load changes and improving resource efficiency while adhering to the core commitment of service availability, achieving an optimal balance between efficiency and stability.

[0058] The following describes an application embodiment provided in this application. The service management system needs to perform the following functions: manage a resource pool consisting of multiple GPU computing machines on a cloud platform, and provide algorithm services for multiple projects. The service management system needs to meet the following requirements: 1. Service Assurance Requirements: Core business operations require the system to always maintain at least N service instances running (i.e., minimum number of instances to start). 2. Resource efficiency requirements: Services with low traffic should be able to share the same GPU card to save costs; 3. Resource isolation requirements: Some services need to exclusively use the entire GPU card, or cannot be deployed on the same card as other services (anti-affinity). 4. Elasticity Requirements: During peak access periods, the system needs to be able to automatically increase the number of service instances to achieve elastic scaling; after the peak, it should automatically reduce the number of instances to achieve elastic scaling down, but it cannot be less than the set minimum number of instances to start. 5. Unified management requirements: All service requests must enter through a unified gateway, where the system will route and distribute them.

[0059] The architecture diagram of the service management system is as follows: Figure 3As shown, the computing power management service includes a unified gateway module, a service definition module, a resource status module, a static scheduler, a dynamic scheduler, and a monitoring module. The computing power server source pool includes an OS (Operating System) / GPU computing power resource pool, containing multiple service instances, such as service instance A1, service instance A2, service instance A3, service instance B1, and service instance B2. The unified gateway module is the sole entry point for all service requests, responsible for routing, authentication, and distribution. External API (Application Programming Interface) requests are routed through the unified gateway module and distributed to the corresponding service instances. The service definition module allows users to define algorithm services, including configuring the minimum number of instances to be launched and detailed resource constraints (exclusivity, sharing, affinity, etc.). Computing power operation and maintenance personnel configure the minimum number of instances to be launched and detailed resource constraints through the computing power management UI (User Interface) and send it to the service definition module. The resource status module maintains the real-time status of the entire GPU resource pool, abstracting and recording the occupancy of each GPU card through a resource model. The static scheduler is responsible for implementing the deployment of the minimum number of instances to be launched. The dynamic scheduler is responsible for enabling elastic scaling deployment. The monitoring module is responsible for collecting real-time load metrics of service instances (such as request volume, latency, and queue backlog).

[0060] To achieve refined sharing and exclusive strategies, GPU resource abstraction no longer treats the GPU as a single unit, but rather abstracts it into a multi-slot model. As shown in Figure 4(a), in exclusive mode, service instances occupy all slots on the GPU card, and their status is marked as Exclusive. As shown in Figure 4(b), in shared mode, the maximum number of instances M per card must be specified during service definition. The system abstracts the GPU card into M virtual slots, with each deployed instance occupying one slot.

[0061] The goal of static scheduling is to satisfy all resource constraints (exclusivity, sharing, affinity / anti-affinity) while meeting the minimum number of instances required to launch all services. This is an NP-Hard (Nondeterministic Polynomial-time Hard) constraint satisfaction problem. A worst-case matching heuristic search algorithm is used to improve the utilization of GPU computing resources fragmentation, prioritizing the filling of shared cards that are already occupied but still have available slots. The algorithm flow is as follows: Figure 5 As shown, it includes the following steps: Step 1, Priority Sorting: Sort all service instances to be scheduled (those that do not meet the minimum number of instances to start) according to their resource requirements (e.g., exclusive mode has priority, shared mode is on demand).

[0062] Step 2, Node Filtering: Traverse all GPU nodes and perform predicate checks: Check whether each node meets the hard constraints of the service (such as anti-affinity, and cannot be deployed on the same card as service X). Filter out nodes that do not meet the constraints.

[0063] Step 3, Node Scoring: Score the nodes that pass the filter.

[0064] Exclusive mode: Prioritizes GPU cards that are completely idle.

[0065] Shared mode: Employs a worst-case matching strategy, prioritizing GPU cards that are already occupied but still have available slots and meet affinity requirements. The goal is to fill a single card with similar, shareable service instances, reserving more free full-card slots for future dedicated services.

[0066] Step 4, Binding: Select the node with the highest score for deployment and update the resource status module.

[0067] Step 5: Repeat steps 1-4 until the minimum number of instances required to start for all services is met.

[0068] Once the minimum number of instances for all services is met, the dynamic scheduler starts to handle real-time load. This includes the following steps: Step 1: Monitoring and Metric Calculation: The monitoring module collects real-time load metrics for the service. A comprehensive service stress index is defined. : ; in, Let be the pressure index at time t. Let be the service latency collected at time t. This can be the latency of a preset percentage of services. The preset percentage (95%) of all services collected at time t will have a latency less than 1 / 2 ms. , The preset delay target value, Let be the length of the request queue collected at time t. The target value for the preset request queue length. , These are the weighting coefficients. .

[0069] Step 2, Target Instance Count Calculation (PID Control): To avoid frequent scaling and jitter caused by traditional threshold-based expansion (e.g., expanding when load > 80%), a PID controller is used to smoothly calculate the theoretical target instance count. .

[0070] First, define the error. , Let be the pressure index at time t, and 1 be the desired target pressure value, i.e. At that time, the system load is considered balanced.

[0071] Secondly, the theoretical target instance number is calculated using the PID control formula. : ; in, Let be the theoretical number of target instances at time t. Let be the current instance number at time t. Let be the error at time t. , Let be the pressure index at time t. This is the proportional gain coefficient. This allows the controller to respond quickly to the current error. This is the integral gain coefficient. For integration variables, for Time error, This term is used to accumulate historical errors, with the aim of eliminating steady-state errors. The differential gain coefficient, The term is used to measure the rate of change of response error. Its purpose is to predict future error trends, suppress system oscillations, and prevent overshoot caused by excessively fast response.

[0072] Step 3, Execution and Constraints: Calculated The number of instances to be executed needs to be compared with the minimum number of instances to be started and rounded to obtain the final target number of instances. : , The minimum number of instances to start for the i-th service.

[0073] if Then, perform expansion and add... In this instance, the algorithm in step S203 can be invoked to find a deployable slot in the available accelerator resources and deploy it. If Then, shrink the size and destroy the file. For each instance, instances dynamically expanded will be destroyed first, and their resources will be released.

[0074] Therefore, firstly, this embodiment combines static SLA constraint-based scheduling with dynamic PID control scheduling for traffic handling within the same system, ensuring the stability of core services while maximizing resource utilization. Secondly, this embodiment proposes a multi-slot GPU resource abstraction model, making it possible to achieve high-density hybrid deployments with complex constraints such as exclusive, shared, affinity / anti-affinity, etc., on GPUs. Furthermore, this embodiment abandons simple threshold triggering and uses a PID controller to calculate the target instance count, which effectively suppresses service load fluctuations (based on high latency or request backlog) and prevents instance count oscillations during scaling up / down, resulting in stronger system stability. Further, this embodiment uses a worst-case matching heuristic algorithm in static scheduling, prioritizing the filling of already occupied shared GPU cards, reserving more complete resources for future exclusive services, and improving the overall utilization of the GPU resource pool. Finally, this embodiment, through a centralized service management system gateway, makes the underlying complex scheduling (static, dynamic, shared, exclusive) completely transparent to upper-layer businesses. All projects only need to access services through a unified entry point, greatly simplifying management and access complexity.

[0075] The following describes a service management device provided in an embodiment of this application. The service management device described below and the service management method described above can be referred to each other.

[0076] See Figure 6 A structural diagram of a service management device according to an exemplary embodiment is shown, such as Figure 6 As shown, it includes: The receiving module 100 is used to receive service configuration information; wherein, the service configuration information includes the minimum number of instances to be started and resource constraints for each service; The deployment module 200 is used to deploy service instances in the accelerator resource pool for each service that meet the corresponding minimum number of startup instances and the corresponding resource constraints, based on the service configuration information. The scheduling module 300 is used to collect service load indicators, calculate the target number of instances for each service based on the service load indicators using a proportional-integral-derivative controller, and perform elastic scaling scheduling of service instances for each service according to the target number of instances for each service.

[0077] The service management device provided in this application firstly receives service configuration information containing the minimum number of instances to be started and resource constraints, and deploys service instances that meet these two requirements for each service in the accelerator resource pool, ensuring that each service has basic availability and resource compliance in its initial state. Secondly, by collecting service load indicators and using a proportional-integral-derivative (PID) controller to dynamically calculate the target number of instances for each service based on these indicators, it can respond to real-time load changes. The PID controller achieves smooth and continuous calculation of the target number of instances by comprehensively controlling the current error (proportional term), historical error accumulation (integral term), and error change trend (derivative term). This approach overcomes the problem of frequent jumps in the number of instances caused by instantaneous load fluctuations in traditional threshold-triggered mechanisms, making the elastic scaling process more stable and adaptive, and reducing performance jitter and scheduling overhead caused by frequent instance starts and stops. Finally, the system performs elastic scaling scheduling of service instances based on the calculated target number of instances, enabling the system to dynamically adjust resource allocation according to the actual load while ensuring the minimum number of instances. When the load increases, the number of instances can be increased to improve processing capacity; when the load decreases, the number of instances can be reduced appropriately to release resources, provided that the minimum number of instances is not lowered. This ensures both continuous service availability and on-demand allocation and efficient utilization of resources, thereby improving the overall scheduling efficiency of the resource pool and the stability of system operation. In summary, the service management device provided in this application, by combining a minimum instance guarantee mechanism with dynamic elastic scaling based on PID control, ensures basic service availability while achieving smooth and adaptive scheduling of accelerator resources, effectively improving resource utilization and service stability.

[0078] Based on the above embodiments, as a preferred implementation, the resource constraints include any one or a combination of deployment mode constraints, affinity constraints, and anti-affinity constraints. The deployment mode constraints describe the resource occupancy method of service instances on the accelerator, and include exclusive mode and shared mode. The affinity constraints indicate that the service instances of the target service and the service instances of other specified services should be deployed on the same accelerator. The anti-affinity constraints indicate that the service instances of the target service and the service instances of other specified services should not be deployed on the same accelerator. The accelerators in the accelerator resource pool are abstractly represented by a multi-slot resource model, and each accelerator is abstracted into multiple virtual slots. If the deployment mode constraint of the target service is exclusive mode, then the service instances of the target service occupy all the slots of the accelerator. If the deployment mode constraint of the target service is shared mode, then the service instances of the target service occupy one slot of the target accelerator and share the target accelerator with the service instances occupying other slots of the target accelerator.

[0079] Based on the above embodiments, as a preferred implementation, the deployment module 200 is specifically used for: prioritizing all service instances to be deployed according to the resource constraints of their respective services; wherein, the service instances to be deployed are service instances of services to be deployed, and the services to be deployed are services that do not meet the corresponding minimum number of startup instances; sequentially traversing all the service instances to be deployed according to the sorting results, and determining candidate accelerators in the accelerator resource pool based on the resource constraints of the currently traversed service instances to be deployed; determining the matching score of the candidate accelerators based on a preset scoring strategy; wherein, the preset scoring strategy includes: if the resource constraint of the service to which the currently traversed service instance to be deployed belongs is in a shared mode, then the matching score of the candidate accelerator that is occupied but still has free slots is higher than the matching score of the completely free candidate accelerator; deploying the currently traversed service instance to be deployed to the candidate accelerator with the highest matching score.

[0080] Based on the above embodiments, as a preferred embodiment, the scheduling module 300 includes: The first calculation unit is used to calculate the stress index of each service based on the service load index. The second calculation unit is used to calculate the theoretical target number of instances for each service based on the stress index of each service using a proportional-integral-differential controller. The comparison unit is used to take the maximum of the theoretical target number of instances and the minimum number of instances to start for each service as the target number of instances for each service.

[0081] Based on the above embodiments, as a preferred implementation, the first calculation unit is specifically used to: calculate the stress index of each service according to the stress index calculation formula; wherein, the stress index calculation formula is: ; in, Let be the pressure index at time t. Let t be the service latency collected at time t. The preset delay target value, Let be the length of the request queue collected at time t. The target value for the preset request queue length. , These are the weighting coefficients. .

[0082] Based on the above embodiments, as a preferred implementation, the second calculation unit is specifically used to: calculate the theoretical target number of instances for each service using a proportional-integral-derivative controller based on a control formula; wherein, the control formula is: ; in, Let be the theoretical number of target instances at time t. Let be the current instance number at time t. Let be the error at time t. , Let be the pressure index at time t. This is the proportional gain coefficient. This is the integral gain coefficient. For integration variables, for Time error, This is the differential gain coefficient.

[0083] Based on the above embodiments, as a preferred embodiment, it further includes: The routing module is used to receive service requests through a unified gateway and to route and distribute the service requests according to the current service instance status.

[0084] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0085] Based on the hardware implementation of the above program modules, and in order to implement the method of the embodiments of this application, the embodiments of this application also provide an electronic device. Figure 7 This is a structural diagram of an electronic device according to an exemplary embodiment, such as... Figure 7 As shown, the electronic device includes: Communication interface 1 enables information exchange with other devices, such as network devices; Processor 2 is connected to communication interface 1 to enable information interaction with other devices and to execute the service management methods provided by one or more of the above-mentioned technical solutions when running computer programs. The computer program is stored on memory 3.

[0086] Of course, in practical applications, the various components in an electronic device are coupled together through bus system 4. It can be understood that bus system 4 is used to achieve communication and connection between these components. In addition to the data bus, bus system 4 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 7 The general will label all buses as Bus System 4.

[0087] The memory 3 in this embodiment is used to store various types of data to support the operation of the electronic device. Examples of such data include any computer program used to operate on the electronic device.

[0088] It is understood that memory 3 can be volatile memory or non-volatile memory, or both. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), ferromagnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM); magnetic surface memory can be disk storage or magnetic tape storage. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), SyncLink Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM).The memory 3 described in the embodiments of this application is intended to include, but is not limited to, these and any other suitable types of memory.

[0089] The methods disclosed in the embodiments of this application can be applied to processor 2, or implemented by processor 2. Processor 2 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in processor 2 or by instructions in the form of software. The processor 2 may be a general-purpose processor, DSP, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Processor 2 can implement or execute the methods, steps and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium, which is located in memory 3. Processor 2 reads the program in memory 3 and completes the steps of the aforementioned method in combination with its hardware.

[0090] When processor 2 executes the program, it implements the corresponding processes in the various methods of the embodiments of this application. For the sake of brevity, these will not be described in detail here.

[0091] In an exemplary embodiment, this application also provides a storage medium, namely a computer storage medium, specifically a computer-readable storage medium, such as a memory 3 that stores a computer program, which can be executed by a processor 2 to complete the steps described in the aforementioned method. The computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface memory, optical disc, or CD-ROM.

[0092] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

[0093] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device (which may be a personal computer, server, network device, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

[0094] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A service management method, characterized in that, include: Receive service configuration information; wherein, the service configuration information includes the minimum number of instances to be started and resource constraints for each service; Based on the service configuration information, deploy service instances in the accelerator resource pool that meet the corresponding minimum number of startup instances and the corresponding resource constraints for each service. Collect service load metrics, use a proportional-integral-derivative controller to calculate the target number of instances for each service based on the service load metrics, and perform elastic scaling and scheduling of service instances for each service according to the target number of instances for each service.

2. The service management method according to claim 1, characterized in that, The resource constraints include any one or a combination of deployment mode constraints, affinity constraints, and anti-affinity constraints. The deployment mode constraints describe how service instances occupy resources on the accelerator. The deployment mode constraints include exclusive mode and shared mode. The affinity constraints indicate that service instances of the target service and service instances of other specified services should be deployed on the same accelerator. The anti-affinity constraints indicate that service instances of the target service and service instances of other specified services should not be deployed on the same accelerator. The accelerators in the accelerator resource pool are abstractly represented by a multi-slot resource model, with each accelerator being abstracted into multiple virtual slots; If the deployment mode constraint of the target service is exclusive mode, then the service instance of the target service occupies all slots of the accelerator. If the deployment mode constraint of the target service is shared mode, then the service instance of the target service occupies one slot of the target accelerator and shares the target accelerator with the service instances occupying other slots of the target accelerator.

3. The service management method according to claim 2, characterized in that, The step of deploying service instances in the accelerator resource pool for each service, based on the service configuration information, that meet the corresponding minimum number of startup instances and the corresponding resource constraints, includes: All service instances to be deployed are prioritized according to the resource constraints of their respective services; wherein, the service instances to be deployed are service instances of the services to be deployed, and the services to be deployed are those that do not meet the corresponding minimum number of instances to be started; Based on the sorting results, all the service instances to be deployed are traversed sequentially, and candidate accelerators are determined in the accelerator resource pool based on the resource constraints of the currently traversed service instances to be deployed. The matching score of the candidate accelerator is determined based on a preset scoring strategy; wherein, the preset scoring strategy includes: if the resource constraint of the service to which the currently traversed service instance to be deployed belongs is a shared mode, then the matching score of the candidate accelerator that is occupied but still has free slots is higher than the matching score of the candidate accelerator that is completely free. Deploy the currently traversed service instances to the candidate accelerator with the highest matching score.

4. The service management method according to claim 1, characterized in that, The step of using a proportional-integral-derivative controller to calculate the target number of instances for each service based on the service load metric includes: Calculate the stress index for each service based on the service load metrics; The theoretical target number of instances for each service is calculated based on the stress index of each service using a proportional-integral-derivative controller. The maximum of the theoretical target number of instances and the minimum number of instances to start for each service is used as the target number of instances for each service.

5. The service management method according to claim 4, characterized in that, The calculation of the stress index for each service based on the service load metric includes: The stress index for each service is calculated according to the stress index calculation formula; wherein, the stress index calculation formula is: ； in, Let be the pressure index at time t. Let t be the service latency collected at time t. The preset delay target value, Let be the length of the request queue collected at time t. The target value for the preset request queue length. , These are the weighting coefficients. .

6. The service management method according to claim 4, characterized in that, The calculation of the theoretical target number of instances for each service using a proportional-integral-derivative controller based on the stress index of each service includes: The theoretical target number of instances for each service is calculated using a proportional-integral-derivative (PID) controller based on a control formula; wherein the control formula is: ； in, Let be the theoretical number of target instances at time t. Let be the current instance number at time t. Let be the error at time t. , Let be the pressure index at time t. This is the proportional gain coefficient. This is the integral gain coefficient. For integration variables, for Time error, This is the differential gain coefficient.

7. The service management method according to claim 1, characterized in that, Also includes: Service requests are received through a unified gateway, and the service requests are routed and distributed according to the current service instance status.

8. A service management device, characterized in that, include: A receiving module is used to receive service configuration information; wherein, the service configuration information includes the minimum number of instances to be started and resource constraints for each service; The deployment module is used to deploy service instances in the accelerator resource pool for each service based on the service configuration information, which meet the corresponding minimum number of startup instances and the corresponding resource constraints. The scheduling module is used to collect service load indicators, calculate the target number of instances for each service based on the service load indicators using a proportional-integral-derivative controller, and perform elastic scaling scheduling of service instances for each service according to the target number of instances for each service.

9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor for executing the computer program to implement the steps of the service management method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed, implements the steps of the service management method as described in any one of claims 1 to 7.

Images