A serverless vector database system resource scheduling method

Abstract

The application discloses a kind of Serverless vector database system resource scheduling methods, including the load characteristics of Serverless vector database is collected and cluster resource state is acquired;State space is constructed based on load characteristics and cluster resource state and is normalized, and the Actor-Critic framework of DPPO algorithm is generated to generate the scaling strategy parameter, the scaling strategy parameter includes Pod maximum concurrent threshold, target utilization and Pod quantity;According to the scaling strategy parameter, Pod instance expansion and shrinkage operation is executed, periodically collect service quality index and calculate reward value, update DPPO algorithm model parameter to optimize scheduling strategy.The application realizes the accurate adaptive control of Pod instance expansion and shrinkage, to improve system throughput, reduce response delay and improve resource utilization.

Description

Technical Field

[0001] This invention mainly relates to the technical fields of the Internet, cloud computing, big data services, etc., and in particular to a resource scheduling method for a serverless vector database system based on the reinforcement learning algorithm DPPO. Background Technology

[0002] Serverless architecture, as an emerging cloud computing model, achieves on-demand execution and efficient resource utilization through FaaS technology. Meanwhile, vector databases have emerged due to the rapid development of information retrieval, machine learning, and other fields, enabling them to efficiently process large-scale vector data and meet needs such as similarity search.

[0003] With the rapid expansion of the cloud database market, the combination of serverless and vector databases has become a research hotspot in academia and industry, aiming to further reduce usage costs and improve system efficiency.

[0004] Currently, resource scheduling in serverless vector databases mainly relies on traditional static threshold strategies, heuristic algorithms, or basic reinforcement learning schemes, which have significant technical limitations. Traditional static threshold strategies (such as Kubernetes' native KPA) trigger scaling up and down based solely on fixed thresholds for single resource metrics such as CPU and memory. This cannot adapt to the differentiated resource consumption characteristics of different loads (such as querying, deleting, and similar nearest neighbor retrieval) in vector databases, easily leading to untimely scaling up causing a surge in response latency, or delayed scaling down causing resource waste.

[0005] Moreover, heuristic algorithms rely on manually preset rules and lack adaptability to dynamic loads, making it difficult to balance Quality of Service (QoS) and resource utilization in complex scenarios. Among basic reinforcement learning solutions (such as Q-Learning and PPO), Q-Learning is limited by discrete state space representation and cannot accurately characterize the multidimensional state features of vector databases; although the PPO algorithm supports continuous space modeling, its training stability is insufficient in distributed scenarios, its sample utilization is low, and it is difficult to cope with the burstiness and non-stationarity of load in serverless environments.

[0006] The unique operational characteristics of vector databases further exacerbate the difficulty of resource scheduling. On the one hand, the high-dimensional vector retrieval tasks they handle consume significant CPU and memory resources, and the data transmission and communication overhead between nodes directly impacts overall performance, requiring scheduling strategies to consider both computational resources and network status. On the other hand, the internal characteristics of vector databases, such as index types, data sharding, and query patterns, can lead to fluctuations in resource demands under the same load. Traditional scheduling strategies do not fully integrate these database-level characteristics, making them prone to distorted resource allocation decisions.

[0007] Furthermore, the impact of cold start latency of Pod instances and locality of node data on task execution efficiency in a serverless environment also requires scheduling strategies to balance real-time response and long-term optimization in scaling decisions. This is a core problem that existing solutions have failed to effectively address.

[0008] In summary, existing resource scheduling strategies for serverless vector databases have shortcomings in dynamic load adaptability, multi-dimensional feature fusion capabilities, training stability, and scheduling accuracy, making it difficult to simultaneously meet the demands of high throughput, low response latency, and high resource utilization in complex scenarios. Therefore, developing an intelligent scheduling strategy that can accurately perceive system status and load characteristics, adapt to dynamic changes, and balance performance and resource efficiency has become a key technological breakthrough for promoting the large-scale application of serverless vector databases. Summary of the Invention

[0009] The main objective of this invention is to address the problem that existing scheduling strategies for serverless vector databases are poorly adaptable and struggle to balance service quality and resource utilization under dynamic load scenarios. This invention provides an intelligent resource scheduling scheme based on the DPPO algorithm, which integrates multi-dimensional system states and load characteristics. This scheme enables precise adaptive control of Pod instance scaling, thereby improving system throughput, reducing response latency, and increasing resource utilization.

[0010] Based on the first main aspect of the present invention, a resource scheduling method for a serverless vector database system is provided, comprising the following steps performed by a computer system:

[0011] The load characteristics of the Serverless vector database are collected, including average CPU utilization, total CPU usage share, average memory utilization, total memory usage, and current concurrency configuration.

[0012] Obtain the cluster resource status, which includes the CPU utilization, memory utilization, network bandwidth utilization, disk I / O, request queue length, index cache hit rate, and data locality factor of each node.

[0013] A state space is constructed based on load characteristics and cluster resource status and normalized. Scaling strategy parameters are generated through the Actor-Critic framework of the DPPO algorithm. The scaling strategy parameters include the maximum concurrency threshold of Pods, target utilization, and number of Pods.

[0014] Execute Pod instance scaling operations based on scaling policy parameters, periodically collect service quality indicators and calculate reward values, and update DPPO algorithm model parameters to optimize scheduling strategies.

[0015] Optionally, in the aforementioned method, the normalization of the state space includes: continuous indices are transformed to the [0,1] interval using minimum-maximum normalization to eliminate the bias between different physical quantities; and discrete and categorical features are expressed using one-hot encoding to maintain the separability of categorical information.

[0016] During training, all features are dynamically normalized to adapt to scale changes caused by load fluctuations in real time.

[0017] Optionally, in the aforementioned method, the values ​​and adjustment methods of the scaling strategy parameters are as follows: the maximum concurrency threshold of Pod is in the range of [10, 200] with a step size of 10, the target utilization rate is in the range of [30, 100] with a step size of 10, and the number of Pods is in the range of [1, 5].

[0018] The adjustment operations for the three parameters are uniformly abstracted as a ternary discrete set {-1, 0, +1}, which corresponds to decreasing, keeping unchanged, and increasing respectively. During execution, the configuration items are adjusted according to the corresponding step size.

[0019] Optionally, in the aforementioned method, the reward value is calculated using the following formula:

[0020]

[0021] in Average throughput, The average response time, To respond to the delay, The response time metric is weighted and set to 0.8; the average CPU utilization of all worker Pods is also included. and average memory utilization If any indicator exceeds the threshold range Then the reward value is multiplied by the discount factor. As punishment; This represents the minimum reasonable threshold for CPU / memory utilization. This represents the maximum reasonable threshold for CPU / memory utilization.

[0022] Optionally, in the aforementioned method, the model parameter update process of the DPPO algorithm is as follows: the following advantage function is calculated through generalized advantage estimation:

[0023]

[0024] in, ;

[0025] Model updates use a pruning objective function:

[0026] ;

[0027] In the above content, For the first The advantage function estimate of the first step is used to measure the advantage function of the second step. The superiority of the actions taken by the step compared to the average strategy; This represents the current time step or iteration step number. This refers to the total number of time steps or iterations. The discount factor is set to 0.9; The smoothing parameter is set to 0.95; For the first The timing difference error of the step; For the first The instant reward value for each step; For the first Step System Status The estimated value; For the first The system state of the step; For the first Step system status The estimated value; For time step The expectation operator; This represents the probability ratio between the old and new strategies. This is the clipping function; The pruning threshold is set to 0.1; 1 represents the probability ratio between the old and new strategies. The baseline value.

[0028] Optionally, in the aforementioned method, the time granularity of load feature acquisition is 30 seconds, and the acquired data is stored in the database after being associated with the corresponding function ID for reference during the training and inference stages.

[0029] When the load type changes or resource consumption characteristics fluctuate, the system automatically triggers a re-collection of load characteristics and updates the load characteristic constraints. This ensures that the load characteristics are consistent with the actual operating conditions.

[0030] Optionally, in the aforementioned method, the triggering conditions and execution steps of the scaling operation are as follows:

[0031] The real-time concurrency is obtained by monitoring the current concurrency configuration in the load characteristics. This real-time concurrency is compared with the maximum concurrency threshold of Pods in the scaling strategy parameters as the strategy threshold. When the real-time concurrency exceeds the strategy threshold, the scaling operation is triggered.

[0032] The target utilization lower limit in the scaling strategy parameters is used as the low watermark. When the average CPU utilization and average memory utilization in the load characteristics both drop below the low watermark, the scaling down operation is triggered.

[0033] When creating a new Pod, the target node is selected by combining the node's CPU utilization, memory utilization, network bandwidth utilization, and data locality factor.

[0034] The sampling batch size for model parameter updates is 8, the Mini-Batch size is 4, and the experience buffer stores the state transition quadruple. Expired samples are cleaned up after every 100 rounds of training to ensure the validity of the training data; among them, For the first The system state of the step; For the first The execution of the step; For the first The instant reward value for each step; For the first Step system status.

[0035] Based on the second main aspect of the present invention, a serverless vector database system is provided, including a Kubernetes cluster, a monitoring module, a scheduling module, and a storage module;

[0036] The Kubernetes cluster includes Master nodes and Worker nodes;

[0037] The monitoring module is used to collect load characteristics and cluster resource status. The load characteristics include average CPU utilization and total CPU usage share. The cluster resource status includes CPU utilization and memory utilization of each node.

[0038] The scheduling module adopts the Actor-Critic framework of the DPPO algorithm, generates scaling strategy parameters based on the data collected by the monitoring module, and performs scaling up and down operations.

[0039] The storage module is used to store load characteristics, cluster resource status and model parameters, and the scheduling module updates the model parameters periodically to optimize the scheduling strategy.

[0040] According to a third key aspect of the present invention, an electronic device is provided, comprising: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0041] The memory stores a computer program, which, when executed by the processor, causes the processor to perform the aforementioned serverless vector database system resource scheduling method.

[0042] Based on a fourth key aspect of the present invention, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed, implements the aforementioned serverless vector database system resource scheduling method.

[0043] Compared with existing technologies, the technical solution of this invention revolves around the core of DPPO-based intelligent scheduling, exhibiting significant advantages in dynamic adaptability, training efficiency, scheduling accuracy, and resource performance balance, as detailed below:

[0044] First, the present invention has strong dynamic load adaptation capability. This is because the present invention integrates cluster resource status (CPU / memory / network bandwidth, etc.), vector database-specific characteristics (index cache hit rate, data locality), and load dynamic characteristics (concurrency configuration, query mode). Through the DPPO algorithm, it continuously interacts and learns with the environment, which can accurately adapt to complex scenarios such as tidal loads and sudden requests, and solve the problem that traditional static threshold strategies are difficult to match load fluctuations.

[0045] Secondly, this invention employs a Distributed Proximal Policy Optimization (DPPO) framework. By using multiple workers for parallel sampling and pruning the objective function to limit the policy update magnitude, compared to algorithms such as PPO and Q-Learning, the training process is more stable, the convergence speed is faster, and the sample utilization rate is higher, enabling rapid adaptation to dynamic load changes in serverless environments. Furthermore, this invention designs a ternary discrete action space (maximum Pod concurrency threshold, target utilization rate, and number of Pods), supporting precise step-size adjustments; it also performs dynamic normalization and one-hot encoding on state features to eliminate decision distortion caused by differences in units, ensuring a high degree of matching between scheduling parameters and actual load requirements and resource status.

[0046] In addition, the reward function of this invention takes into account average throughput, response latency and resource utilization, and introduces a resource threshold penalty mechanism (the reward is halved if the indicator exceeds the limit). This ensures high throughput and low latency quality of service (QoS) of the system, while avoiding resource waste caused by excessive expansion, and achieves the optimal balance between performance and cost.

[0047] Finally, the state space of this invention incorporates vector database-specific metrics (such as vector dimension, index type, and number of data shards) rather than a general resource scheduling scheme. The decision logic is more aligned with the resource requirements of core tasks such as high-dimensional data retrieval and similarity calculation, and the scheduling effect is better than that of general serverless scheduling strategies. Attached Figure Description

[0048] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, obtaining other drawings based on these drawings without creative effort still falls within the scope of the present invention.

[0049] Figure 1 The following is an execution flowchart of a serverless vector database system resource scheduling method according to an embodiment of the present invention. Detailed Implementation

[0050] The preferred embodiments of the present invention will be described in detail below to provide a clearer understanding of the purpose, features, and advantages of the invention. It should be understood that the following embodiments are not intended to limit the scope of the invention, but are merely illustrative of the essential spirit of the technical solution of the invention.

[0051] In the following description, certain specific details are set forth for the purpose of illustrating various disclosed embodiments in order to provide a thorough understanding of the various disclosed embodiments. However, those skilled in the art will recognize that embodiments may be practiced without one or more of these specific details. In other instances, well-known techniques associated with the invention may not have been shown or described in detail to avoid unnecessarily obscuring the description of the embodiments.

[0052] Throughout this specification, references to "an embodiment" or "an embodiment" indicate that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment. Therefore, the appearance of "in an embodiment" or "an embodiment" in various places throughout the specification does not necessarily refer to the same embodiment. Furthermore, a particular feature, structure, or characteristic may be combined in any manner in one or more embodiments.

[0053] The core technical terms and their meanings that may be involved in the following embodiments are as follows:

[0054] Serverless architecture: An architecture that eliminates the need for users to manage underlying servers, allocates resources on demand, and charges based on actual usage. In this invention, it is used to deploy a vector database and adapt to dynamic load fluctuations.

[0055] Kubernetes cluster (K8s cluster): an open-source container orchestration and management platform consisting of one Master node (control plane) and multiple Worker nodes (worker nodes), providing the underlying environment for the deployment, scheduling, and scaling of Pod instances in this invention.

[0056] Pod: The smallest deployment unit in a Kubernetes cluster. In this invention, it serves as the running platform for a vector database instance. A single Pod can handle multiple concurrent requests in parallel.

[0057] Master node: The control core of the Kubernetes cluster, responsible for scheduling Pod allocation, monitoring cluster status, executing scaling commands, and ensuring that scheduling policies are implemented.

[0058] Worker nodes: The worker nodes in a Kubernetes cluster. When containers are deployed, they actually support the running of Pod instances and provide hardware resources such as CPU, memory, and network.

[0059] DingoDB: A distributed vector database that, when combined with a Kubernetes cluster, builds the Serverless vector database system of this invention, supporting high-dimensional data retrieval and similarity calculation.

[0060] DPPO (Distributed Near-End Policy Optimization): The core scheduling algorithm of this invention is a distributed extension of the PPO algorithm. It achieves efficient policy iteration through parallel sampling by multiple workers and stable training of the objective function by pruning the objective function.

[0061] The Actor-Critic framework is the core architecture of the DPPO algorithm, consisting of an Actor network (policy network) and a Critic network (value network), which are responsible for generating scheduled actions and evaluating the value of actions, respectively.

[0062] Actor Network: The policy generation module in DPPO, which outputs scaling policy parameters (maximum Pod concurrency threshold, target utilization, number of Pods) based on system status to guide scaling operations.

[0063] Service Quality: The core indicator system for measuring system service performance, which includes average throughput, average response time, and response latency in this invention.

[0064] Average throughput: The number of requests processed by the system per unit of time, which is a key indicator for measuring the system's processing capacity.

[0065] Average response time: The average time it takes for all requests to go from initiation to receiving a response, reflecting the system's response efficiency.

[0066] Response latency: The time taken to respond to a single request, supplemented by average response time to fully characterize service latency.

[0067] This invention discloses a serverless vector database resource scheduling method, apparatus, computer-readable storage medium, and serverless vector database system based on DPPO. The technical solution of this invention will be described in detail below with reference to specific implementation environment and steps, so that those skilled in the art can accurately implement this invention.

[0068] like Figure 1 As shown, in one embodiment, a resource scheduling method for a serverless vector database system based on DPPO according to the present invention includes the following steps 100-400 executed by a computer system:

[0069] Step 100: Collect the load characteristics of the Serverless vector database, including average CPU utilization, total CPU usage share, average memory utilization, total memory usage, and current concurrency configuration.

[0070] Step 200: Obtain the cluster resource status, which includes the CPU utilization, memory utilization, network bandwidth utilization, disk I / O, request queue length, index cache hit rate, and data locality factor of each node.

[0071] Step 300: Construct a state space based on load characteristics and cluster resource status and perform normalization processing. Generate scaling strategy parameters through the Actor-Critic framework of the DPPO algorithm. The scaling strategy parameters include the maximum concurrency threshold of Pods, target utilization rate, and number of Pods.

[0072] Step 400: Execute Pod instance scaling up and down operations according to scaling policy parameters, periodically collect service quality indicators and calculate reward values, and update DPPO algorithm model parameters to optimize scheduling strategies.

[0073] In one possible implementation, in step 100, the collection of load characteristics and cluster resource status is performed synchronously with a 30-second time granularity: the load characteristics of the Serverless vector database are collected in real time through the system monitoring module, specifically including the average CPU utilization, total CPU usage share, average memory utilization, total memory usage, and current concurrency configuration of all working Pods. The collected data is associated with the corresponding function ID and stored in the database to form load characteristic constraints.

[0074] In one possible implementation, the time granularity for load feature acquisition is 30 seconds. The acquired data is associated with the corresponding function ID and stored in a database for reference during the training and inference phases. When the load type changes or resource consumption characteristics fluctuate, the system automatically triggers a re-acquisition of load features and updates the load feature constraints. This ensures that the load characteristics are consistent with the actual operating conditions.

[0075] In step 200, common metrics such as CPU utilization, memory utilization, network bandwidth utilization, disk I / O rate, and request queuing length of each node in the Kubernetes cluster are collected, along with vector database-specific index cache hit rate and data locality factor, and summarized to form cluster resource status data.

[0076] In step 300, when constructing the state space, the collected load features are integrated with the cluster resource status, and dynamic load characteristics such as query type distribution and average top-k are added to form a complete state vector and normalize it: continuous indicators are transformed to the [0,1] interval using minimum-maximum normalization, and discrete and categorical features are expressed using one-hot encoding, and dynamic normalization is maintained during training to adapt to load fluctuations.

[0077] The Actor-Critic framework based on the DPPO algorithm initializes the model. The Actor network outputs scaling strategy parameters based on the normalized state space. The maximum concurrency threshold of Pods is in the range of [10, 200) with a step size of 10, the target utilization rate is in the range of [30, 100] with a step size of 10, and the number of Pods is in the range of [1, 5].

[0078] In step 300, the scaling up and down operations are executed according to the scaling strategy parameters. Real-time concurrency is obtained by monitoring the current concurrency configuration. Scaling up is triggered when this value exceeds the Pod's maximum concurrency threshold. A target utilization lower limit is used as the low-water mark. Scaling up is triggered when both average CPU utilization and average memory utilization fall below the low-water mark. When creating a new Pod, the target node is selected based on node resource status and data locality factors. Simultaneously, service quality indicators such as average throughput, average response time, and response latency are collected at 30-second granularities. An immediate reward value is calculated based on the reward function. If the average CPU or memory utilization exceeds the threshold range, the reward value is multiplied by a discount factor of 0.5 as a penalty.

[0079] In most embodiments, the scaling strategy parameters are set and adjusted as follows: the maximum concurrency threshold for Pods is set in the range of [10, 200] with a step size of 10; the target utilization rate is set in the range of [30, 100] with a step size of 10; and the number of Pods is set in the range of [1, 5]. The adjustment of the three parameters is uniformly abstracted as a ternary discrete set {-1, 0, +1}, which corresponds to decreasing, keeping unchanged, and increasing, respectively. During execution, the configuration items are adjusted according to the corresponding step size.

[0080] The reward value is calculated using the following formula:

[0081]

[0082] in Average throughput, The average response time, To respond to the delay, The response time metric is weighted and set to 0.8; the average CPU utilization of all worker Pods is also included. and average memory utilization If any indicator exceeds the threshold range Then the reward value is multiplied by the discount factor. As punishment; This represents the minimum reasonable threshold for CPU / memory utilization. This represents the maximum reasonable threshold for CPU / memory utilization.

[0083] In possible implementations, the triggering conditions and execution steps for scaling operations are as follows:

[0084] The real-time concurrency is obtained by monitoring the current concurrency configuration in the load characteristics. This real-time concurrency is compared with the maximum concurrency threshold of Pods in the scaling strategy parameters as the strategy threshold. When the real-time concurrency exceeds the strategy threshold, the scaling operation is triggered.

[0085] The target utilization lower limit in the scaling strategy parameters is used as the low watermark. When the average CPU utilization and average memory utilization in the load characteristics both drop below the low watermark, the scaling down operation is triggered.

[0086] When creating a new Pod, the target node is selected by combining the node's CPU utilization, memory utilization, network bandwidth utilization, and data locality factor.

[0087] The sampling batch size for model parameter updates is 8, the Mini-Batch size is 4, and the experience buffer stores the state transition quadruple. Expired samples are cleaned up after every 100 rounds of training to ensure the validity of the training data; among them, For the first The system state of the step; For the first The execution of the step; For the first The instant reward value for each step; For the first Step system status.

[0088] In step 400, the model parameter update process of the DPPO algorithm is as follows: the following advantage function is calculated through generalized advantage estimation:

[0089]

[0090] in, ;

[0091] Model updates use a pruning objective function:

[0092] ;

[0093] In the above content, For the first The advantage function estimate of the first step is used to measure the advantage function of the second step. The superiority of the actions taken by the step compared to the average strategy; This represents the current time step or iteration step number. This refers to the total number of time steps or iterations. The discount factor is set to 0.9; The smoothing parameter is set to 0.95; For the first The timing difference error of the step; For the first The instant reward value for each step; For the first Step System Status The estimated value; For the first The system state of the step; For the first Step system status The estimated value; For time step The expectation operator; This represents the probability ratio between the old and new strategies. This is the clipping function; The pruning threshold is set to 0.1; 1 represents the probability ratio between the old and new strategies. The baseline value.

[0094] During the model parameter update phase, the state transition quadruple (the system state at step t, the scaling action performed, the immediate reward obtained, and the new system state after the action) is stored in the experience buffer. Data is sampled in batches periodically, the advantage function is calculated through generalized advantage estimation, the Actor network parameters are updated using the pruning objective function, the Critic network is optimized by minimizing the mean square error, expired samples are cleaned up after every 100 rounds of training, and the scheduling strategy is continuously iterated and optimized to achieve a dynamic balance between service quality and resource utilization.

[0095] The following is a possible embodiment.

[0096] This embodiment deploys a serverless vector database based on a Kubernetes (K8s) cluster (using Milvus 2.3 as an example). All operations are performed by the computer system of the cluster control plane (CPU: Intel Xeon E5-2680 v4, memory: 64GB, operating system: Ubuntu 20.04). Core dependent tools include:

[0097] Monitoring and data collection: Kubernetes Metrics Server (real-time collection of resource metrics), Prometheus (collection of database business metrics); Algorithm implementation: PyTorch 2.0 framework (deploying the Actor-Critic network for the DPPO algorithm); Scaling and scaling execution: K8s HPA (Horizontal Pod Autoscaler) custom controller (interfacing with scaling policy parameters generated by DPPO). Cluster topology: 1 Master node (responsible for scheduling decisions) + 3 Worker nodes (QueryNode / IndexNode Pod instances running Milvus). In Serverless mode, Pod instances support on-demand creation / destruction.

[0098] The following are possible implementation steps.

[0099] Step 100: Collect load characteristics of the Serverless vector database

[0100] This step is executed at a fixed granularity of 10 seconds per instance. It retrieves raw data from the Milvus monitoring interface ( / metrics) using Prometheus and calculates the load characteristics specified in the claims. The specific implementation is as follows:

[0101] Average CPU utilization: Calculate the arithmetic mean of the CPU utilization of all current Milvus Pods (number of CPU cores used by the Pod / number of allocated CPU cores). The formula is:

[0102]

[0103] in This represents the number of Milvus Pods currently running, initially in the experiment. ,initial .

[0104] Total CPU usage share: This summarizes the actual CPU usage share of all Milvus Pods (in millicores). The initial value in the experiment was 1800 millicores (i.e., 1.8 cores).

[0105] Average memory utilization: Calculated using the same logic as average CPU utilization, the formula is:

[0106]

[0107] Initially in the experiment .

[0108] Total memory usage: Summarizes the actual memory used by all Milvus Pods (in MB). The initial value in the experiment was 8192 MB (8GB). Current concurrency configuration: Reads the queryNode.maxConcurrency parameter from the Milvus configuration file (milvus.yaml), with an initial value of 100 (i.e., the maximum number of concurrent queries allowed per QueryNode Pod). After collection, all load characteristics are encapsulated into vectors and stored in the cluster's local cache (Redis).

[0109] Step 200: Obtain cluster resource status

[0110] This step is executed at the same granularity as step 100. It uses K8s Metrics Server and Prometheus to jointly collect the cluster resource status specified in the claim, covering all worker nodes. The specific implementation is as follows: Node-level basic resource indicators: collect the CPU utilization (initial value: node 1=68%, node 2=70%, node 3=62%), memory utilization (initial value: node 1=75%, node 2=78%, node 3=68%), network bandwidth utilization (actual transmission rate of node network card / maximum bandwidth, initial value is 45%), and disk I / O (disk read / write IOPS / maximum IOPS, initial value is 35%) of each worker node.

[0111] The relevant business metrics include the following: Request queue length: the number of unprocessed tasks in the request queue of each Milvus Pod, initial values: node1=15, node2=18, node3=12; Index cache hit rate: the percentage of Milvus index data that is hit in memory, initial values ​​are all 85%; Data locality factor: the proportion of vector data stored locally on a node to the data required for processing tasks on that node, initial values: node1=80%, node2=75%, node3=88%.

[0112] After the data collection is completed, all node data is summarized and stored in association with the load feature vector from step 100.

[0113] Step 300: Construct and normalize the state space, and generate scaling strategy parameters.

[0114] 3.1 State Space Construction and Normalization

[0115] The load characteristics (5 dimensions) from step 100 and the cluster resource status (8 dimensions per node, 24 dimensions for 3 nodes) from step 200 are integrated into a 29-dimensional state vector. ,in For load feature vectors, This is the resource state vector for all nodes.

[0116] Using Min-Max normalization, all indicators are mapped to the [0,1] interval. The normalization formula is as follows:

[0117]

[0118] Among them: CPU / memory / network / disk utilization , ; Request queue length , Index cache hit rate / data locality factor , ;

[0119] Total CPU usage share , millicore; Total memory usage , MB; Concurrent configuration , After normalization, a dimensionless state vector is obtained, which serves as the input to the DPPO algorithm.

[0120] 3.2 DPPO Algorithm for Generating Scaling Strategy Parameters

[0121] The Actor-Critic dual-network framework for DPPO is implemented using PyTorch. The network structure and training parameters are as follows:

[0122] Actor Network (Policy Network): Input a 29-dimensional normalized state vector, output a 3-dimensional probability distribution of scaled policy parameters, activation function is Tanh; Critic Network (Value Network): Input a 29-dimensional normalized state vector, output state value (evaluating the quality of the current policy), activation function is ReLU; Training parameters: Actor learning rate = 3e-3, Critic learning rate = 2e-3, discount factor = 0.9, Clip coefficient = 0.2, batch size = 64.

[0123] After the algorithm is executed, scaling strategy parameters that conform to the claims are generated:

[0124] Maximum concurrency threshold for Pods: Adjust queryNode.maxConcurrency to 120 (a 20% increase to adapt to the current load pressure); Target utilization: CPU target utilization = 70%, memory target utilization = 75% (as the core threshold for scaling up and down); Number of Pods: Expand from the initial 2 to 3 (add 1 QueryNode Pod and deploy it to node 3, as the load on this node is relatively low).

[0125] Step 400: Perform scaling up / down and update DPPO model parameters

[0126] 4.1 Perform Pod instance scaling operations

[0127] The specific execution logic involves using a custom HPA controller in Kubernetes to interface with scaling policy parameters generated by DPPO.

[0128] Based on the parameter "Pod count = 3", a Pod creation request is sent to the K8s APIServer to deploy a new QueryNode Pod (configuration: CPU = 1 core, memory = 4GB) on node 3; based on the parameter "Pod maximum concurrency threshold = 120", the configuration files of all Milvus Pods are automatically modified and hot-loaded without restarting the instance; based on the parameter "target utilization = CPU 70% / memory 75%", HPA trigger rules are set: when the average CPU utilization of the cluster's Milvus Pods exceeds 70% for 30 consecutive seconds, the cluster is expanded; when it falls below 50%, the cluster is scaled down; the same applies to memory utilization.

[0129] 4.2 Collect Quality of Service (QoS) metrics and calculate reward values.

[0130] After scaling up or down, QoS metrics are collected every 30 seconds (to evaluate the effectiveness of the policy), including:

[0131] Average response latency: The average processing time for vector query requests (before scaling up / down = 280ms, after scaling up / down = 210ms); System throughput: The number of query requests processed per unit time (before scaling up / down = 180 QPS, after scaling up / down = 240 QPS); Task drop rate: The percentage of requests not processed due to node overload (before scaling up / down = 3.5%, after scaling up / down = 1.2%).

[0132] The reward value is calculated using a multi-dimensional weighted formula (aligning with the reward design logic of reinforcement learning):

[0133]

[0134] in: QPS The calculated reward value is 0.4 × (240 / 300) - 0.3 × (210 / 500) - 0.3 × 0.012 = 0.32 - 0.126 - 0.0036 = 0.1904 (positive reward, indicating the strategy is effective).

[0135] 4.3 Update DPPO algorithm model parameters

[0136] Store the triple of "state vector-scaling strategy-reward value" into the experience replay pool. When the amount of data in the pool reaches the batch size (64), perform a DPPO model update:

[0137] Calculate the advantage function: ( For the value of the action, (The state value output by the Critic network); the Actor network is updated using the PPO-Clip strategy: minimizing the loss function. Update the Critic network using mean squared error (MSE): minimize ( (This is for accumulated discount rewards); after updating, the model parameters are saved to the cluster's shared storage to complete one round of strategy optimization. The next round of scaling strategy will be generated based on the new parameters.

[0138] After this round of scheduling, the average CPU utilization of the Milvus Pod cluster dropped to 68% (close to the target of 70%), the average memory utilization dropped to 73% (close to the target of 75%), the average response latency decreased by 25%, the throughput increased by 33.3%, and the task drop rate decreased by 65.7%, verifying the effectiveness of the scheduling method.

[0139] Various embodiments of the systems and techniques described above in this invention can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0140] The program code used to implement the methods of the present invention can be written in any combination of one or more programming languages. This program code can be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code can be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0141] In the context of this invention, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0142] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including voice input, speech input, or tactile input).

[0143] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0144] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0145] The acquisition, storage, and application of user personal information involved in the technical solution of this invention all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0146] It should be understood that the various forms of processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this invention does not impose any limitations on them.

[0147] The technical terms, principles, or means related to the technical solutions of the present invention mentioned in the above embodiments, which are not described in detail above, are all well-known technologies or common practices that are known to those skilled in the art.

[0148] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.

Claims

1. A resource scheduling method for a serverless vector database system, characterized in that, This includes the following steps performed by the computer system: The load characteristics of the Serverless vector database are collected, including average CPU utilization, total CPU usage share, average memory utilization, total memory usage, and current concurrency configuration. Obtain the cluster resource status, which includes the CPU utilization, memory utilization, network bandwidth utilization, disk I / O, request queue length, index cache hit rate, and data locality factor of each node. A state space is constructed based on load characteristics and cluster resource status and normalized. Scaling strategy parameters are generated through the Actor-Critic framework of the distributed near-end strategy optimization algorithm. The scaling strategy parameters include the maximum concurrency threshold of Pods, target utilization, and number of Pods. Execute Pod instance scaling operations based on scaling policy parameters, periodically collect service quality indicators and calculate reward values, and update distributed near-end policy optimization algorithm model parameters to optimize scheduling strategies. The scaling strategy parameters are set and adjusted as follows: the maximum concurrency threshold for Pods is set in the range of [10, 200] with a step size of 10; the target utilization rate is set in the range of [30, 100] with a step size of 10; and the number of Pods is set in the range of [1, 5]. The adjustment operations for the three parameters are uniformly abstracted as a ternary discrete set {-1, 0, +1}, corresponding to decrease, remain unchanged, and increase, respectively. During execution, the configuration items are adjusted according to the corresponding step size; the reward value is calculated using the following formula: in For average throughput, The average response time, To respond to the delay, The response time metric is weighted and set to 0.8; the average CPU utilization of all worker Pods is also included. and average memory utilization If any indicator exceeds the threshold range Then the reward value is multiplied by the discount factor. As punishment; This represents the minimum reasonable threshold for average CPU utilization or average memory utilization. The maximum reasonable threshold for average CPU utilization or average memory utilization; the model parameter update process of the distributed near-end strategy optimization algorithm is as follows: the following advantage function is calculated through generalized advantage estimation: in, ; Model updates use a pruning objective function: ； In the above content, For the first The advantage function estimate of the first step is used to measure the advantage function of the second step. The superiority of the actions taken by the step compared to the average strategy; This represents the current time step or iteration step number. This refers to the total number of time steps or iterations. The discount factor is set to 0.9; The smoothing parameter is set to 0.95; For the first The timing difference error of the step; For the first The instant reward value for each step; For the first Step System Status The estimated value; For the first The system state of the step; For the first Step system status The estimated value; For time step The expectation operator; This represents the probability ratio between the old and new strategies. This is the clipping function; The pruning threshold is set to 0.1; 1 represents the probability ratio between the old and new strategies. The baseline value.

2. The resource scheduling method for a serverless vector database system according to claim 1, characterized in that, The normalization process of the state space includes: continuous indices are transformed to the [0,1] interval using minimum-maximum normalization to eliminate the bias between different physical quantities; and discrete and categorical features are expressed using one-hot encoding to maintain the separability of categorical information. During training, all features are dynamically normalized to adapt to scale changes caused by load fluctuations in real time.

3. The resource scheduling method for a serverless vector database system according to claim 1, characterized in that, The time granularity of the load feature acquisition is 30 seconds. The acquired data is stored in the database after being associated with the corresponding function ID, and is used for reference in the training and inference stages. When the load type changes or resource consumption characteristics fluctuate, the system automatically triggers a re-collection of load characteristics and updates the load characteristic constraints. This ensures that the load characteristics are consistent with the actual operating conditions.

4. The resource scheduling method for a serverless vector database system according to claim 1, characterized in that, The triggering conditions and execution steps for the scaling up and down operation are as follows: The real-time concurrency is obtained by monitoring the current concurrency configuration in the load characteristics. This real-time concurrency is compared with the maximum concurrency threshold of Pods in the scaling strategy parameters as the strategy threshold. When the real-time concurrency exceeds the strategy threshold, the scaling operation is triggered. The target utilization lower limit in the scaling strategy parameters is used as the low watermark. When the average CPU utilization and average memory utilization in the load characteristics both drop below the low watermark, the scaling down operation is triggered. When creating a new Pod, the target node is selected by combining the node's CPU utilization, memory utilization, network bandwidth utilization, and data locality factor. The sampling batch size for model parameter updates is 8, the Mini-Batch size is 4, and the experience buffer stores the state transition quadruple. Expired samples are cleaned up after every 100 rounds of training to ensure the validity of the training data; among them, For the first The system state of the step; For the first The execution of the step; For the first The instant reward value for each step; For the first Step system status.

5. A serverless vector database system, characterized in that, The resource scheduling method for the Serverless vector database system described in claim 1 includes a Kubernetes cluster, a monitoring module, a scheduling module, and a storage module. The Kubernetes cluster includes Master nodes and Worker nodes; The monitoring module is used to collect load characteristics and cluster resource status. The load characteristics include average CPU utilization and total CPU usage share. The cluster resource status includes CPU utilization and memory utilization of each node. The scheduling module adopts the Actor-Critic framework of the distributed near-end policy optimization algorithm, and generates scaling policy parameters based on the data collected by the monitoring module and performs scaling up and down operations. The storage module is used to store load characteristics, cluster resource status and model parameters, and the scheduling module updates the model parameters periodically to optimize the scheduling strategy.

6. An electronic device, comprising: The system comprises a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other via the communication bus; characterized in that the memory stores a computer program, which, when executed by the processor, causes the processor to execute the Serverless vector database system resource scheduling method according to any one of claims 1-4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed, it implements the Serverless vector database system resource scheduling method as described in any one of claims 1-4.

Images