Method and device for adjusting algorithm container quantity, electronic equipment and storage medium

Abstract

The application relates to the technical field of container orchestration, and discloses an algorithm container quantity adjustment method and device, an electronic device and a storage medium, which comprise the following steps: acquiring an actual measured load of a current monitoring period and a predicted load of the current monitoring period predicted in a previous monitoring period; updating the prediction confidence of the previous monitoring period according to the actual measured load of the current monitoring period, to obtain the prediction confidence of the current monitoring period; performing weighted summation on the actual measured load of the current monitoring period and the predicted load of the current monitoring period based on the prediction confidence of the current monitoring period, to obtain a comprehensive load of the current monitoring period; determining an adjustment coefficient according to the difference between the comprehensive load and a preset standard load; and obtaining the adjusted number of algorithm Pods according to the total number of current algorithm Pods and the adjustment coefficient. The application solves the problem of low resource utilization caused by a static resource configuration mode, realizes accurate matching of load demand, and optimizes resource utilization.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of container orchestration, and in particular to an algorithm container quantity adjustment method and device, electronic equipment and storage medium. BACKGROUND

[0002] Currently, for application programs that need GPU to process complex tasks, the usual practice is to deploy GPU algorithm modules in the largest scale (i.e. start as many Pods as possible) in anticipation of the task pool reaching a peak. This approach is based on a conservative estimate, aiming to ensure that there is enough computing power to handle all requests even in the busiest situation. This mode is a static resource configuration mode, using a fixed number of Deployment configurations, and the number of Pods is set based on historical peak load, which cannot adapt to dynamically changing business needs.

[0003] However, although this static resource configuration mode can guarantee service performance under high load, it also leads to low resource utilization: long-term occupation of excessive GPU resources without full utilization, not only increasing hardware costs, but also limiting the execution of other tasks that may be more in need of these resources. SUMMARY

[0004] The purpose of the present application is to at least provide an algorithm container quantity adjustment method, device, electronic equipment and storage medium, which can at least solve the problem of low resource utilization caused by static resource configuration mode, and at least achieve the effect of matching resource allocation with load and improving resource utilization.

[0005] To solve the above technical problems, at least one embodiment of the present application provides an algorithm container quantity adjustment method, comprising:

[0006] obtaining the measured load of the current monitoring period and the predicted load for the current monitoring period predicted in the previous monitoring period;

[0007] updating the prediction confidence of the previous monitoring period according to the measured load of the current monitoring period to obtain the prediction confidence of the current monitoring period;

[0008] based on the prediction confidence of the current monitoring period, the measured load of the current monitoring period and the predicted load of the current monitoring period are weighted and summed to obtain the comprehensive load of the current monitoring period;

[0009] determining an adjustment coefficient according to the difference between the comprehensive load and a preset standard load;

[0010] obtaining the adjusted number of algorithm Pods according to the total number of current algorithm Pods and the adjustment coefficient.

[0011] At least one embodiment of the present application also provides an algorithm container quantity adjustment device, comprising:

[0012] a load acquisition module configured to acquire a measured load of a current monitoring period and a predicted load of the current monitoring period predicted in a previous monitoring period;

[0013] a confidence degree calculation module configured to update the predicted confidence degree of the previous monitoring period according to the measured load of the current monitoring period, to obtain a predicted confidence degree of the current monitoring period;

[0014] a comprehensive load calculation module configured to perform weighted summation on the measured load of the current monitoring period and the predicted load of the current monitoring period based on the predicted confidence degree of the current monitoring period, to obtain a comprehensive load of the current monitoring period;

[0015] an adjustment coefficient determination module configured to determine an adjustment coefficient according to a difference between the comprehensive load and a preset standard load;

[0016] an algorithm Pod quantity adjustment module configured to obtain an adjusted number of algorithm Pods according to a total number of current algorithm Pods and the adjustment coefficient.

[0017] At least one embodiment of the present application also provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the algorithm container quantity adjustment method described above.

[0018] At least one embodiment of the present application also provides a computer-readable storage medium storing a computer program, and the computer program is executed by a processor to implement the algorithm container quantity adjustment method described above.

[0019] The algorithm container quantity adjustment method, device, electronic device and storage medium provided by the embodiments of the present application acquire a measured load and a predicted load, dynamically update a predicted confidence degree according to an error between a current measured load and a current period predicted load. The higher the confidence degree is, the more reliable the prediction is. The prediction error is directly fed back to the confidence degree calculation to form a self-correction mechanism. The measured load and the current period predicted load are weighted and summed with the previous period predicted confidence degree as a weight to obtain a comprehensive load, balancing real-time performance and predictability. An adjustment coefficient is determined according to a difference between the comprehensive load and a preset standard load, which converts the load difference into a quantifiable adjustment signal and avoids subjective judgment. The target Pod number is calculated in combination with the current Pod total number and the adjustment coefficient, which realizes accurate matching of load demand and optimizes resource utilization.

[0020] In some optional embodiments, updating the prediction confidence of the previous monitoring period according to the measured load of the current monitoring period comprises:

[0021] After the actual load of the current monitoring period arrives, a prediction error of the current monitoring period is calculated, and error records in an error sliding window are updated, wherein the error sliding window includes error data of multiple historical monitoring periods;

[0022] Statistical features of errors in the updated error sliding window are calculated, and the statistical features include at least one of mean error, standard deviation, and range;

[0023] A predetermined proportion of quantiles of historical values of the statistical features of each historical monitoring period in the updated error sliding window is taken to generate feature threshold values of the corresponding statistical features;

[0024] A correction coefficient is calculated according to the statistical features and the feature threshold values, and the prediction confidence of the previous monitoring period is adjusted by using the correction coefficient to obtain the prediction confidence of the current monitoring period.

[0025] In this embodiment, a fixed-size error sliding window is maintained to store prediction error data of multiple historical monitoring periods, the window is updated over time, and the expired data is automatically eliminated to ensure that the confidence evaluation reflects the latest load characteristics. The mean error, standard deviation, and range of errors in the sliding window are calculated to more comprehensively quantify the error distribution and improve the accuracy of the prediction confidence of the current monitoring period.

[0026] In some optional embodiments, when the statistical features are multiple, the correction coefficient is calculated according to the statistical features and the feature threshold values, comprising:

[0027] For each statistical feature, a sub-correction coefficient of the statistical feature is calculated according to a ratio of the statistical feature to the feature threshold value corresponding to the statistical feature;

[0028] The correction coefficient is obtained according to a weighted sum of the multiple sub-correction coefficients.

[0029] In this embodiment, the ratio of each statistical feature (such as mean error, standard deviation, and range) to the corresponding threshold value is calculated to generate a sub-correction coefficient, rather than directly using the original value, which standardizes the error influence and unifies the evaluation scale. The structure of independent calculation + weighted fusion of the sub-correction coefficient not only retains the sensitivity of the single feature, but also realizes the global stability through weighting.

[0030] In some optional embodiments, the prediction confidence of the current monitoring period is calculated according to the following calculation formula:

[0031]

[0032] =

[0033] wherein, is the prediction confidence of the current monitoring period, is the prediction confidence of the previous monitoring period, is the statistical feature, , and are the mean error, standard deviation and range, respectively, is the weight of the sub-correction coefficient of the statistical feature i, is the feature threshold.

[0034] In some optional embodiments, the comprehensive load of the current monitoring period is calculated according to the following calculation formula:

[0035]

[0036] wherein, is the comprehensive load of the current monitoring period, is the predicted load of the previous monitoring period, is the measured load of the current monitoring period, is the prediction confidence of the current monitoring period.

[0037] In this embodiment, when the prediction confidence is high, the number of containers is adjusted in advance according to the prediction trend to avoid resource waste or shortage; when the prediction confidence is low, the adjustment is made quickly according to the measured load to ensure that the system can respond to the actual demand in time and improve the performance and stability of the system.

[0038] In some optional embodiments, the adjustment coefficient is calculated according to the following calculation formula:

[0039]

[0040] wherein, is the adjustment coefficient, is the comprehensive load of the current monitoring period, is the preset standard load, is the preset adjustment range.

[0041] In this embodiment, the adjustment coefficient K has a linear proportional relationship with the difference between the comprehensive load and the standard load. The adjustment range is proportional to the load deviation, that is, the larger the deviation, the larger the adjustment range, so that the resource shortage or excess situation can be quickly corrected. By setting a suitable N value, the step size of each adjustment can be controlled, and the adjustment process is more smooth.

[0042] In some optional embodiments, further comprising:

[0043] The length of the next monitoring period is adjusted according to the prediction confidence of the current monitoring period.

[0044] In the embodiment, the confidence is reversely linked with the monitoring frequency, which helps to accurately allocate limited resources to periods with high uncertainty, and maximizes resource efficiency while ensuring system perception capability.

[0045] In some optional embodiments, the length of the next monitoring period is adjusted according to the prediction confidence of the current monitoring period, including:

[0046] The length of the next monitoring period is determined according to the prediction confidence of the current monitoring period based on a preset mapping relationship between confidence and period length, wherein the confidence and the period length are in a positive correlation relationship.

[0047] In the embodiment, the monitoring period length is lower when the confidence is low, so as to shorten the interval intensive monitoring and improve the prediction accuracy; when the confidence is high, the monitoring period length is higher, so as to prolong the interval and save resources, thereby achieving intelligent balance between monitoring accuracy and resource consumption.

[0048] In some optional embodiments, the length of the next monitoring period is calculated according to the following calculation formula:

[0049] ,

[0050] wherein, is the length of the next monitoring period, is a preset minimum monitoring period, is a preset maximum monitoring period, is the prediction confidence of the current monitoring period, is a preset minimum prediction confidence, is a preset maximum prediction confidence, is an exponential coefficient for adjusting the shape of the mapping curve between the confidence and the period length.

[0051] In the embodiment, through the exponential mapping, the monitoring period length rapidly decreases when the confidence is low, thereby improving the response speed of the system. BRIEF DESCRIPTION OF DRAWINGS

[0052] One or more embodiments are exemplarily illustrated by pictures in the drawings corresponding thereto, which do not constitute a limitation on the embodiments.

[0053] Figure 1 is a flow of the adjustment method of the algorithm container quantity provided by an embodiment of the present application Figure One ;

[0054] Figure 2 is another embodiment of the application provides the flow of the algorithm container quantity adjustment method Figure Two ;

[0055] Figure 3 is another embodiment of the application provides the schematic diagram of the algorithm container quantity adjustment device. DETAILED DESCRIPTION

[0056] To make the purpose, technical solutions and advantages of the embodiments of the present application more clear, the embodiments of the present application will be described in detail below with reference to the drawings. However, those skilled in the art can understand that in the embodiments of the present application, many technical details are proposed in order to make the reader better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical solutions claimed by the present application can be implemented. The following embodiments are classified for the convenience of description, and should not constitute any limitation on the specific implementation of the present application, and the embodiments can be combined with each other under the premise of not contradicting each other.

[0057] The present application proposes an algorithm container quantity adjustment method, and the implementation details of the algorithm container quantity adjustment method of the present embodiment will be described in detail below. The following content is only provided for the implementation details of the understanding, and is not necessary for the implementation of the present solution.

[0058] Embodiment one:

[0059] The algorithm container quantity adjustment method of the present embodiment can be applied to electronic devices with communication, calculation and data storage capabilities, and its specific process can be as shown in Figure 1 , including:

[0060] Step 110, obtaining the measured load of the current monitoring period and the predicted load for the current monitoring period predicted in the previous monitoring period;

[0061] In the present embodiment, the monitoring system will trigger a monitoring and adjustment operation once at a preset fixed time interval, for example, 5 minutes or 10 minutes, etc. The measured load of the current monitoring period is the actual measured load value at the end of the current period, such as CPU usage, request QPS, memory occupation, etc. The predicted load is the predicted value of the current period load in the previous monitoring period by a preset prediction model, such as a time series prediction model. The load of the next period is predicted at the end of each monitoring period.

[0062] Step 120, updating the prediction confidence of the previous monitoring period according to the measured load of the current monitoring period to obtain the prediction confidence of the current monitoring period;

[0063] Specifically, the prediction confidence is used to reflect the accuracy of the prediction model. If the error between the current measured load and the predicted load is small, the confidence increases; otherwise, it decreases. The prediction accuracy of the previous period is verified based on the current measured load, and the confidence index is dynamically updated to reflect the reliability of the prediction model.

[0064] In actual use, the prediction model may be inaccurate for a short period of time due to sudden traffic, data noise, etc. If the measured load of the current period is significantly different from the predicted load, it means that the prediction model may not be accurate in the current environment, and the confidence needs to be reduced, so that the weight of the predicted load is reduced in the subsequent weighted sum. By dynamically updating the confidence, the system can quickly respond to such changes and avoid resource allocation errors caused by prediction errors.

[0065] Step 130, based on the prediction confidence of the current monitoring period, the measured load of the current monitoring period and the predicted load of the current monitoring period are weighted and summed to obtain the comprehensive load of the current monitoring period;

[0066] In this embodiment, the comprehensive load is used to realize the smooth adjustment of the number of algorithm Pods. If only the measured load is used for adjustment, it is easy to overreact to transient peaks, but the number of containers is frequently fluctuated after the peak disappears, increasing the system overhead and risk. If only the predicted load is used for adjustment, it is difficult to deal with sudden situations that are not captured by the model, such as hot events causing traffic surge, but the predicted load is lower than the surge load, and the adjusted algorithm Pod number is difficult to cope with the traffic surge. The comprehensive load combines the stability of the prediction and the real-time of the measurement, which helps to realize smooth scaling and avoid resource fluctuations.

[0067] Step 140, determining an adjustment coefficient according to the difference between the comprehensive load and a preset standard load;

[0068] In this embodiment, the standard load is a reasonable load threshold that the algorithm Pod is expected to bear, which can be expressed in the form of resource usage (such as CPU, memory) or business indicators (such as QPS, delay). For example:

[0069] CPU standard load: 70% (reserve 30% buffer for sudden traffic);

[0070] QPS standard load: 5000 requests / second;

[0071] Or custom indicators: the load level corresponding to the prediction model inference delay ≤200ms.

[0072] The target line is defined by the standard load, and the difference direction directly indicates the adjustment direction. When the comprehensive load > standard load, scaling is needed to reduce the pressure on individual Pods; if the comprehensive load < standard load, scaling can be performed to save resources.

[0073] In step 150, the adjusted number of algorithm Pods is obtained according to the total number of current algorithm Pods and the adjustment coefficient.

[0074] In this embodiment, when the adjustment coefficient is greater than 1, the capacity is expanded, and when the adjustment coefficient is less than 1, the capacity is reduced. When calculating the adjustment coefficient, the difference value can be mapped to the adjustment coefficient to realize proportional control. Or a segmented function is used, for example, when the difference value is greater than or equal to a first threshold but less than a second threshold, the adjustment coefficient is 1.2, and when the difference value is greater than the second threshold, the adjustment coefficient is 1.5.

[0075] In summary, the adjustment method for the number of algorithm containers provided in this embodiment obtains the measured load and the predicted load, dynamically updates the prediction confidence according to the error between the current measured load and the predicted load in the current period, and the higher the confidence, the more reliable the prediction. The prediction error is directly fed back to the confidence calculation to form a self-correction mechanism. The prediction confidence of the previous period is used as a weight to perform weighted summation on the current measured load and the predicted load in the current period to obtain a comprehensive load, balancing the real-time and predictability. According to the difference between the comprehensive load and the preset standard load, the adjustment coefficient is determined, which realizes the conversion of the load difference into a quantifiable adjustment signal and avoids subjective judgment. The current total number of Pods and the adjustment coefficient are combined to calculate the target number of Pods, which realizes accurate matching of load demand and optimizes resource utilization.

[0076] In some embodiments, updating the prediction confidence of the previous monitoring period according to the measured load of the current monitoring period comprises: after the actual load of the current monitoring period arrives, calculating the prediction error of the current monitoring period and updating the error records in the error sliding window, wherein the error sliding window includes error data in multiple historical monitoring periods; calculating the statistical characteristics of the errors in the updated error sliding window, the statistical characteristics including at least one of the average error, the standard deviation and the range; taking a preset proportion of the quantile of the historical values of the statistical characteristics in each historical monitoring period in the updated error sliding window to generate a feature threshold value of the corresponding statistical characteristic; calculating a correction coefficient according to the statistical characteristic and the feature threshold value, adjusting the prediction confidence of the previous monitoring period by using the correction coefficient to obtain the prediction confidence of the current monitoring period.

[0077] In this embodiment, the prediction confidence is an indicator representing the reliability of the prediction result, usually taking a value between 0 and 1, the value closer to 1 indicates the prediction is more reliable, and the value closer to 0 indicates the prediction is less reliable. By adjusting the correction coefficient, the prediction confidence can more accurately reflect the actual situation of the current prediction. For example, if the prediction confidence of the previous monitoring period is 0.8, after adjusting the correction coefficient, the prediction confidence of the current monitoring period is 0.7, which indicates that due to the deviation between the actual load and the predicted value of the current monitoring period, the reliability of the prediction has decreased.

[0078] After the actual load of the current monitoring period arrives, the system will compare the actual load value with the predicted load value of the period obtained based on the prediction model. Through a specific error calculation method, such as absolute error (absolute value of actual load - predicted load) or relative error ((actual load - predicted load) / absolute value of predicted load), etc., the prediction error of the current monitoring period is obtained.

[0079] The error sliding window is a data structure for storing error data of multiple historical monitoring periods, with a fixed capacity. When new error data of the current monitoring period is generated, the oldest error data will be removed from the window according to the first-in first-out principle, while the new error data will be added to the window. In this way, the error sliding window always contains the latest and continuous error records within a certain period of time, providing a data basis for subsequent analysis of the trend of prediction error. For example, after the actual load arrives, the prediction error at the current time point is calculated = |measured load - predicted load|; the error sliding window W records the historical error sequence , is the size of the error sliding window, which can be 30.

[0080] The average error is the arithmetic mean of all error data in the error sliding window, reflecting the average level of prediction error in a period of time, and can reflect the accuracy of the prediction as a whole. The standard deviation is used to measure the dispersion of error data, reflecting the deviation between each error value and the average error. The larger the standard deviation, the greater the fluctuation of error data, and the poorer the stability of the prediction; the smaller the standard deviation, the more concentrated the error data, and the more stable the prediction result. The range is the difference between the maximum error value and the minimum error value in the error sliding window, showing the range of error data. The larger the range, the greater the fluctuation of prediction error in extreme cases; the smaller the range, the narrower the range of error. In practical applications, the feature threshold of the corresponding statistical feature can be set by taking a preset proportion of the quantile of the historical value of the statistical feature, or multiple feature thresholds can be set by combining multiple statistical features, and the correction coefficient can be set by combining the feature thresholds.

[0081] The historical value of each statistical feature in the updated error sliding window is taken at a preset proportion of the quantile. The quantile is a value at a specific position after a set of data is arranged in order of size. For example, the median is the 50% quantile, which represents a critical value that divides the data into two parts. By selecting different preset proportions, such as 25% and 75%, the corresponding quantiles can be calculated to reflect the distribution of the statistical feature at different levels. The feature threshold of the corresponding statistical feature is generated according to the calculated quantile. The feature threshold can be used as a judgment standard to measure whether the current statistical feature is within the normal range. If it is not within the normal range, the correction coefficient is calculated according to the statistical feature and the feature threshold. The calculation method of the correction coefficient can be designed according to the specific application scenario and requirements. The prediction confidence of the previous monitoring period is adjusted using the calculated correction coefficient to obtain the prediction confidence of the current monitoring period.

[0082] In some embodiments, when there are multiple statistical features, the correction coefficient is calculated according to the statistical feature and the feature threshold, including: for each statistical feature, the sub-correction coefficient of the statistical feature is calculated according to the ratio of the statistical feature to the feature threshold corresponding to the statistical feature; and the correction coefficient is obtained by weighted summation of the multiple sub-correction coefficients.

[0083] In some embodiments, the prediction confidence of the current monitoring period is calculated according to the following calculation formula:

[0084]

[0085] =

[0086] wherein, is the prediction confidence of the current monitoring period, is the prediction confidence of the previous monitoring period, is the statistical feature, , and are the average error, the standard deviation and the range, respectively, is the weight of the sub-modification coefficient of the statistical feature i, is the feature threshold.

[0087] Specifically, the embodiment integrates the average error, the standard deviation and the range to construct a multi-dimensional confidence function, wherein, is the sub-modification coefficient, is the adjustable weight coefficient corresponding to the sub-modification coefficient. For example, the weight of the sub-modification coefficient corresponding to the average error is 0.4, the weight of the sub-modification coefficient corresponding to the standard deviation is 0.3, and the weight of the sub-modification coefficient corresponding to the range is 0.3. is the overall modification coefficient.

[0088] The embodiment also adds a confidence boundary constraint to avoid the influence of extreme values on system stability. The value range of the confidence is 0.1-0.9.

[0089] In some embodiments, the comprehensive load of the current monitoring period is calculated according to the following calculation formula:

[0090]

[0091] wherein, is the comprehensive load of the current monitoring period, is the predicted load of the previous monitoring period, is the measured load of the current monitoring period, is the prediction confidence of the current monitoring period.

[0092] When the prediction confidence is high, the number of containers is adjusted in advance according to the prediction trend to avoid resource waste or deficiency; when the prediction confidence is low, the system can respond to actual demand in time according to the measured load, thereby improving the performance and stability of the system.

[0093] In some embodiments, the adjustment coefficient is calculated according to the following calculation formula:

[0094]

[0095] wherein, is the adjustment coefficient, is the comprehensive load of the current monitoring period, is the preset standard load, is the preset adjustment range.

[0096] According to the total number of algorithm Pods at present and the adjustment coefficient, the adjusted number of algorithm Pods is obtained, which can be as follows:

[0097]

[0098] wherein, is the adjusted number of algorithm Pods, is the total number of algorithm Pods at present.

[0099] In some embodiments, further comprising: adjusting the length of the next monitoring period according to the prediction confidence of the current monitoring period.

[0100] In the embodiment, when the prediction confidence is high, prolonging the monitoring period can reduce redundant data acquisition and processing, and reduce the calculation. When the confidence is low, the period is shortened, and the limited monitoring resources are concentrated to the high-risk period of predicted failure, so as to avoid the expansion of failure caused by too long monitoring interval.

[0101] In some embodiments, the adjusting the length of the next monitoring period according to the prediction confidence of the current monitoring period comprises: determining the length of the next monitoring period according to the prediction confidence of the current monitoring period based on a preset mapping relationship between confidence and period length, wherein the confidence and the period length are in a positive correlation relationship.

[0102] In the embodiment, the mapping relationship can be proportional mapping, exponential mapping or segmented function mapping, as long as the requirement that the higher the confidence, the longer the period length, and the lower the confidence, the shorter the period length is met.

[0103] In some embodiments, the length of the next monitoring period is calculated according to the following calculation formula:

[0104] ,

[0105] wherein, is the length of the next monitoring period, is a preset minimum value of the monitoring period, is a preset maximum value of the monitoring period, is the prediction confidence of the current monitoring period, is a preset minimum value of the prediction confidence, is a preset maximum value of the prediction confidence, is an exponential coefficient for adjusting the shape of the mapping curve between the confidence and the period length.

[0106] In the embodiment, Greater than 1, which realizes that the curve is steep in the low confidence area and the curve is gentle in the high confidence area, realizes that the system quickly shortens the length of the next monitoring period at low confidence to improve the prediction accuracy of the next monitoring period.

[0107] Low confidence means that the prediction model has failed in the current environment, such as sudden traffic peaks or changes in business patterns. At this time, quickly shortening the monitoring period can capture real load changes faster. Originally, 10 minutes of monitoring, now possibly 5 minutes, the time to discover traffic surges is shortened from an average of 5 minutes to 2.5 minutes, which can better cope with the scenario of business surges.

[0108] Embodiment two:

[0109] On the basis of the above-mentioned embodiments, the present embodiment provides a specific application example. The algorithm container quantity adjustment method provided by the present embodiment is applied to a system for batch reviewing articles. The system has the following technical components:

[0110] Kubernetes (K8s) API: used to interact with the K8s cluster, manage the creation, deletion and replica number adjustment of algorithm module Pods.

[0111] Unified image repository: stores the container images of algorithm modules to ensure that all nodes can pull the latest image version.

[0112] Deployment YAML file: predefines the deployment configuration of each algorithm module, including resource request, limit and other parameters.

[0113] Database-Redis: as a cache database for task pool, quickly reads and writes the number of unprocessed tasks.

[0114] Database-MySQL: persistent storage of task results, provides reliable data storage and query services.

[0115] Front-end management page: responsible for building the user interface, allowing administrators to maintain algorithm tasks and display information such as current task number, processed task number, K8s cluster resource usage, etc.

[0116] Core scheduling service: implements the main business logic, including communication with K8s API, monitoring task pool state, evaluating GPU algorithm module performance, and executing scaling decisions, etc.

[0117] GPU resource pool management: divides all GPUs into three parts: basic resource pool, shared resource pool, and exclusive resource pool, corresponding to different types of resource allocation strategies.

[0118] For example, Figure 2The full flow diagram of algorithm Pod number adjustment when using the above system for article proofreading is shown. For example, 1000 articles are uploaded for review at the same time, and after task splitting, 15,000 review tasks (text + picture) are generated. The system dynamically expands according to the algorithm container number adjustment method described in the embodiments, and automatically expands from 5 Pods to 35 Pods. Distribute tasks according to Pod processing capacity. 35 GPU Pods process at the same time, and finally, the results are aggregated through Spark streaming. After processing is completed, 5 Pods are withdrawn.

[0119] The specific implementation details and core processes between technical components are as follows:

[0120] I. Task receiving and distribution

[0121] The data processing flow is as follows:

[0122] 1.1, Input data: task details submitted by the user (document content, review rules, QoS level label).

[0123] Task parameter verification: the core scheduling service verifies the legality of the input parameters (such as file format, whether the rule ID exists) through regular expressions or JSON Schema.

[0124] 1.2, Processing steps:

[0125] Task ID generation: use UUID to generate a unique task ID to ensure global uniqueness.

[0126] Task serialization: serialize task metadata (ID, timestamp, status, rules, QoS level) into a JSON object.

[0127] Task storage:

[0128] Redis write: push the task JSON into the List structure of Redis (such as task_queue:algorithm_service) using the RPUSH command.

[0129] MySQL persistence: execute INSERT INTO tasks (task_id, status, created_at, qos_level) VALUES (?, 'PENDING', NOW(),?) through JDBC.

[0130] 1.3, Output results: task ID is returned to the user, and task metadata is stored in Redis and MySQL.

[0131] II. Task monitoring and analysis

[0132] The data processing process is as follows:

[0133] 2.1, input data: Redis task queue length, Kubernetes cluster resource indicators (CPU / GPU utilization), historical load data.

[0134] 2.2, processing steps:

[0135] Task queue monitoring:

[0136] Predicted load calculation: Through the pre-trained prediction model, the load in the next 10 minutes (i.e. the next monitoring period) is predicted, and the predicted load is obtained.

[0137] Actual load calculation: Before the end of the current period, the number of unprocessed tasks, i.e. the current task number, is obtained through the LLEN task_queue:algorithm_service. The total Pod processing capacity is obtained through the Kubernetes API call. The actual load of the current monitoring period is obtained according to the following calculation formula:

[0138] Predicted load = (current task number / total Pod processing capacity) * 100, where the Pod processing capacity is based on the historical average task processing rate.

[0139] Three, dynamic expansion and contraction

[0140] 3.1, update the prediction confidence of the previous monitoring period according to the actual load of the current monitoring period to obtain the prediction confidence of the current monitoring period:

[0141] Initialize the confidence =0.5, after the actual load arrives, calculate the prediction error at the current time point =|actual load-predicted load|, maintain an error sliding window W. The error sliding window W records N error data points. After the latest arrives, update the historical error sequence ; based on the updated error sequence, calculate the following statistical features:

[0142] Mean error =

[0143] Standard deviation

[0144] Range

[0145] Construct a multi-dimensional confidence function:

[0146]

[0147] where, , , is the 95th percentile of the error sliding window.

[0148] 3.2, based on the prediction confidence of the current monitoring period, the measured load of the current monitoring period and the predicted load of the current monitoring period are weighted and summed to obtain the comprehensive load of the current monitoring period:

[0149]

[0150] wherein, is the comprehensive load of the current monitoring period, is the predicted load of the previous monitoring period, is the measured load of the current monitoring period, is the prediction confidence of the current monitoring period.

[0151] 3.3, according to the difference between the comprehensive load and the preset standard load, the adjustment coefficient is determined:

[0152]

[0153] wherein, is the adjustment coefficient, is the comprehensive load of the current monitoring period, is the preset standard load, is the preset adjustment range.

[0154] 3.4, according to the total number of algorithm Pods of the current monitoring period and the adjustment coefficient, the number of adjusted algorithm Pods is obtained:

[0155]

[0156] wherein, is the number of adjusted algorithm Pods, is the total number of algorithm Pods of the current monitoring period.

[0157] 3.5, according to the prediction confidence of the current monitoring period, the length of the next monitoring period is adjusted:

[0158] ,

[0159] wherein, is the length of the next monitoring period, is the preset minimum monitoring period, is the preset maximum monitoring period, is the prediction confidence of the current monitoring period, is the preset minimum prediction confidence, For the preset maximum prediction confidence, For the index coefficient, the shape of the mapping curve between confidence and cycle length is adjusted.

[0160] Four, resource pool allocation

[0161] Basic resource pool: fixed allocation to core services (core algorithm modules).

[0162] Shared resource pool: on-demand allocation, adjust the number of replicas of the corresponding algorithm service Deployment.

[0163] Exclusive resource pool: exclusive part of GPU card, bound to a specific point through nodeSelector.

[0164] Kubernetes API call: after calculating the number of pods that need to be scaled up or scaled down, the Deployment configuration in the Kubernetes cluster needs to be actually modified to trigger changes. At the same time, the Kubernetes scheduler will automatically arrange the scheduling of the pod according to the cluster resource situation.

[0165] Update Deployment: use Kubernetes Java client to perform patch operation and update.spec.replicas field.

[0166] Pod scheduling: Kubernetes scheduler selects the optimal node according to the GPU resources of the node (nvidia.com / gpu reported by Device Plugin).

[0167] Output result: new pod starts and joins the task processing queue.

[0168] Five, GPU resource pool management

[0169] Data processing flow:

[0170] Input data: task QoS level, GPU node resource information.

[0171] Processing steps:

[0172] Resource pool division:

[0173] Basic resource pool: mark nodes through nodeLabel (such as gpu-pool: base), and only allocate to high-priority tasks.

[0174] Shared resource pool: support dynamic GPU allocation through Device Plugin (such as NVIDIA DCGM or mGPU CUDA_VISIBLE_DEVICES subdivision).

[0175] Exclusive resource pool: Bind specific nodes through nodeSelector, corresponding to algorithm Pod exclusive whole GPU card.

[0176] Resource scheduling strategy:

[0177] Shared resource allocation: Use nvidia-docker's --gpus parameter to specify GPU memory and computing power ratio (how to use this data later) (such as --gpus '"device=0, compute_cap=3.5, memory=4096"').

[0178] Output result: Tasks are assigned to corresponding resource pool GPU resources according to QoS level.

[0179] Six, task life cycle management

[0180] Data processing flow:

[0181] Input data: task ID, processing status, timeout time.

[0182] Processing steps:

[0183] Task taking and locking mechanism:

[0184] Redis lock: Worker gets task through BLPOP atomic operation, and sets SET task_lock:ID 1 EX120 NX.

[0185] State update: Change task status from PENDING to PROCESSING, and record start time.

[0186] Task timeout processing:

[0187] Timing scanning: Check expired lock (EXPIRED key) through Redis SCAN command, reset task status to PENDING.

[0188] Result persistence:

[0189] MySQL insertion: Execute INSERT INTO results (task_id, result, duration, algorithm_version) VALUES (?,?,?,?).

[0190] Security and fault tolerance mechanism:

[0191] Task retry: Automatically retry 3 times for timeout task, and mark as ERROR after failure.

[0192] Database transaction: Task status updates and result storage use BEGIN...COMMIT transactions to ensure consistency.

[0193] Output result: Task status is updated to COMPLETED, and the result is stored in MySQL.

[0194] New tasks can be added to the system through the front-end interface, and the results can be obtained after processing. The management platform can view the current task progress and the status of the processed tasks.

[0195] Resource configuration: Administrators can set up different types of GPU resource pools and specify which algorithm services should have exclusive access to specific GPU resources and which can be allocated on demand from shared resource pools.

[0196] Real-time monitoring: The front-end provides an intuitive dashboard that displays key indicators such as the number of current tasks, the number of completed tasks, and the resource usage of the K8s cluster.

[0197] Logging and alerting: The system supports logging and can trigger alerts for abnormal situations to notify administrators for timely handling.

[0198] This embodiment can significantly improve GPU resource utilization, increase the flexibility and efficiency of applications deployed on K8s, reduce operational costs, and improve user experience, greatly improving the efficiency of article review. The specific advantages and characteristics are as follows:

[0199] 1. High resource utilization efficiency

[0200] Dynamic adjustment: The system can automatically adjust the number of algorithm module Pods based on real-time task volume and resource usage, ensuring that expensive computing resources such as GPUs are fully utilized.

[0201] Pool management: By dividing GPU resources into basic resource pools, shared resource pools, and exclusive resource pools, resource allocation can be more finely controlled to meet the needs of different types of tasks.

[0202] 2. Strong flexibility and adaptability

[0203] Quick response: The system can quickly respond to changes in workloads by increasing or decreasing the number of Pods to maintain optimal service performance and adapt to request fluctuations at different times.

[0204] Customized strategies: Users can customize resource allocation strategies according to their business needs to flexibly respond to changing work environments.

[0205] 3. High degree of automation

[0206] Intelligent Scheduling: Built-in intelligent decision-making mechanism based on task pool status and algorithm module performance, reducing the need for manual intervention and improving operational efficiency.

[0207] Continuous Optimization: Continuous cycle of monitoring, analysis, and adjustment to ensure the system is always in optimal configuration.

[0208] 4. Scalability

[0209] Microservices Architecture: Adopting containerized deployment and microservices architecture, easy to scale horizontally, supporting larger-scale workload processing capabilities.

[0210] Modular Design: Independent functional components, easy to maintain and upgrade, without affecting the normal operation of other parts.

[0211] 5. User-friendly Interface

[0212] Intuitive Display: The user interface developed by Vue.js provides clear and intuitive display of task statistics, resource usage, and other information, helping administrators better understand and manage the entire system.

[0213] Easy Operation: Simple operation process and friendly interaction design make it easy for non-technical personnel to get started.

[0214] 6. Strong Monitoring and Feedback

[0215] Real-time Monitoring: The system provides comprehensive real-time monitoring functions, including task progress, resource utilization, and other key indicators, ensuring that problems can be discovered and solved in a timely manner.

[0216] Alarm Mechanism: When abnormal conditions are detected, the system will trigger an alarm to notify the administrator, avoiding potential risks.

[0217] 7. Data Persistence and Security

[0218] Stable Storage: Using MySQL database for data persistence storage ensures the safety and reliability of task results.

[0219] Permission Control: Combined with the security mechanism of K8s cluster, strict access permission management is achieved to protect sensitive data from unauthorized access.

[0220] 8. Technological Advancement

[0221] Modern Technology Stack: Adopting the current popular front-end and back-end separation architecture (Vue.js + Java), as well as the container orchestration platform Kubernetes, reflecting the forward-looking technology.

[0222] Integrated innovation: Integrates various advanced technologies and services such as Redis cache, unified image repository, etc., to build a high-efficiency and stable ecosystem.

[0223] In summary, the present solution not only solves the problems of resource waste, slow response speed, and complex technical operation in the prior art, but also provides a set of intelligent, flexible, and easy-to-use solution for users, greatly improving the efficiency of resource management and task processing.

[0224] Embodiment three:

[0225] Another embodiment of the present application relates to an algorithm container quantity adjustment device. The implementation details of the algorithm container quantity adjustment device of the present embodiment are described below. The following details are provided for easy understanding, and are not essential for implementing the present solution. The schematic diagram of the algorithm container quantity adjustment device of the present embodiment can be as shown in Figure 3 The algorithm container quantity adjustment device of the present embodiment includes a load acquisition module 310, a confidence calculation module 320, a comprehensive load calculation module 330, an adjustment coefficient determination module 340, and an algorithm Pod quantity adjustment module 350.

[0226] The load acquisition module 310 is configured to acquire the measured load of the current monitoring period and the predicted load for the current monitoring period predicted in the previous monitoring period.

[0227] The confidence calculation module 320 is configured to update the prediction confidence of the previous monitoring period according to the measured load of the current monitoring period, to obtain the prediction confidence of the current monitoring period.

[0228] The comprehensive load calculation module 330 is configured to perform weighted summation on the measured load of the current monitoring period and the predicted load of the current monitoring period based on the prediction confidence of the current monitoring period, to obtain the comprehensive load of the current monitoring period.

[0229] The adjustment coefficient determination module 340 is configured to determine an adjustment coefficient according to the difference between the comprehensive load and a preset standard load.

[0230] The algorithm Pod quantity adjustment module 350 is configured to obtain the adjusted number of algorithm Pods according to the total number of current algorithm Pods and the adjustment coefficient.

[0231] It is worth mentioning that each module involved in the present embodiment is a logical module. In actual application, a logical unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. In addition, in order to highlight the innovative part of the present application, units that are not closely related to solving the technical problems proposed in the present application are not introduced in the present embodiment, but this does not mean that there are no other units in the present embodiment.

[0232] In some embodiments, the confidence degree calculation module comprises:

[0233] a prediction error calculation unit configured to calculate a prediction error of a current monitoring period after an actual load of the current monitoring period arrives, and update error records in an error sliding window, wherein the error sliding window comprises error data of a plurality of historical monitoring periods;

[0234] an error statistical feature calculation unit configured to calculate statistical features of errors in the updated error sliding window, the statistical features comprising at least one of an average error, a standard deviation and a range;

[0235] a feature threshold calculation unit configured to take a preset proportion of quantiles of historical values of the statistical features of each historical monitoring period in the updated error sliding window to generate feature thresholds of the corresponding statistical features;

[0236] a confidence degree correction unit configured to calculate a correction coefficient according to the statistical features and the feature thresholds, and adjust a prediction confidence degree of a previous monitoring period by using the correction coefficient to obtain a prediction confidence degree of the current monitoring period.

[0237] In some embodiments, when the statistical features are multiple, the confidence degree correction unit comprises:

[0238] a sub-correction coefficient calculation unit configured to calculate, for each statistical feature, a sub-correction coefficient of the statistical feature according to a ratio of the statistical feature to the feature threshold corresponding to the statistical feature;

[0239] a correction coefficient calculation unit configured to obtain the correction coefficient according to a weighted summation result of the multiple sub-correction coefficients.

[0240] In some embodiments, the method further comprises:

[0241] a monitoring period adjustment module configured to adjust a length of a next monitoring period according to the prediction confidence degree of the current monitoring period.

[0242] In some embodiments, the monitoring period adjustment module is configured to determine the length of the next monitoring period according to the prediction confidence degree of the current monitoring period based on a preset mapping relationship between confidence degrees and period lengths, wherein the confidence degrees and the period lengths are in a positive correlation relationship.

[0243] Embodiment four:

[0244] Another embodiment of the present application relates to an electronic device, comprising: at least one processor; and a memory connected with the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the algorithm container quantity adjustment method in the above-mentioned embodiments.

[0245] The memory and the processor are connected in a bus mode, the bus can include any number of interconnected buses and bridges, and the bus connects various circuits of one or more processors and memories together. The bus can also connect various other circuits such as peripheral devices, voltage stabilizers and power management circuits, which are well known in the art, and therefore, further description thereof will not be given herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be one element or multiple elements, such as multiple receivers and transmitters, which provide a unit for communicating with various other devices on the transmission medium. The data processed by the processor is transmitted on the wireless medium through the antenna, and further, the antenna also receives data and transmits the data to the processor.

[0246] The processor is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interface, voltage regulation, power management and other control functions. And the memory can be used to store the data used by the processor in the execution of the operation.

[0247] Embodiment five:

[0248] Another embodiment of the present application relates to a computer readable storage medium, which stores a computer program. The computer program is executed by the processor to implement the method embodiments described above.

[0249] That is, those skilled in the art can understand that all or part of the steps of the above-mentioned embodiment methods can be completed by programs instructing related hardware, the programs are stored in a storage medium, and include a plurality of instructions for causing a device (which can be a single chip microcomputer, a chip, etc.) or a processor to execute all or part of the steps of the method described in each embodiment of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (Read-Only Memory, abbreviated as: ROM), a random access memory (Random Access Memory, abbreviated as: RAM), a magnetic disk or an optical disk and various program code storage media.

[0250] Those skilled in the art can understand that the above-mentioned embodiments are specific embodiments for implementing the present application, and in actual application, various changes can be made in form and detail without departing from the spirit and scope of the present application.

Claims

1. A method for adjusting the number of containers in an algorithm, characterized in that, include: Obtain the measured load for the current monitoring period and the predicted load for the current monitoring period from the previous monitoring period. The prediction confidence level of the current monitoring period is obtained by updating the prediction confidence level of the previous monitoring period based on the measured load of the current monitoring period. Based on the prediction confidence of the current monitoring period, the measured load and the predicted load of the current monitoring period are weighted and summed to obtain the comprehensive load of the current monitoring period. The adjustment coefficient is determined based on the difference between the comprehensive load and the preset standard load; Based on the current total number of algorithm Pods and the adjustment coefficient, the adjusted number of algorithm Pods is obtained; The step of updating the prediction confidence of the previous monitoring period based on the measured load of the current monitoring period includes: After the actual load of the current monitoring period arrives, the prediction error of the current monitoring period is calculated, and the error record in the error sliding window is updated. The error sliding window includes error data from multiple historical monitoring periods. Calculate the statistical characteristics of the error within the updated error sliding window, wherein the statistical characteristics include at least one of the mean error, standard deviation, and range; Take the quantile of the historical values ​​of the statistical features under each historical monitoring period within the updated error sliding window according to a preset proportion to generate the feature threshold of the corresponding statistical features; The correction coefficient is calculated based on the statistical characteristics and the characteristic threshold. The prediction confidence of the previous monitoring period is adjusted using the correction coefficient to obtain the prediction confidence of the current monitoring period.

2. The method for adjusting the number of algorithm containers according to claim 1, characterized in that, When there are multiple statistical features, the step of calculating the correction coefficient based on the statistical features and the feature threshold includes: For each statistical feature, the sub-correction coefficient of the statistical feature is calculated based on the ratio of the statistical feature to the feature threshold corresponding to the statistical feature. The correction coefficients are obtained by weighted summation of the multiple sub-correction coefficients.

3. The method for adjusting the number of algorithm containers according to claim 2, characterized in that, The prediction confidence level for the current monitoring period is calculated using the following formula: = in, The prediction confidence level for the current monitoring period. The confidence level of the prediction for the previous monitoring period. For statistical characteristics, , and These are the mean error, standard deviation, and range, respectively. The weights of the sub-correction coefficients for statistical feature i. This is the feature threshold.

4. The method for adjusting the number of algorithm containers according to claim 1, characterized in that, The overall load for the current monitoring period is calculated using the following formula: in, The overall load for the current monitoring period, The predicted load for the previous monitoring period, The measured load for the current monitoring period. This represents the prediction confidence level for the current monitoring period.

5. The method for adjusting the number of algorithm containers according to claim 1, characterized in that, Calculate the adjustment factor using the following formula: in, To adjust the coefficient, The overall load for the current monitoring period, For the preset standard load, This is the preset adjustment range.

6. The method for adjusting the number of algorithm containers according to any one of claims 1-5, characterized in that, Also includes: The duration of the next monitoring cycle is adjusted based on the prediction confidence level of the current monitoring cycle.

7. The method for adjusting the number of algorithm containers according to claim 6, characterized in that, The adjustment of the duration of the next monitoring cycle based on the prediction confidence of the current monitoring cycle includes: Based on a preset mapping relationship between confidence level and cycle duration, the duration of the next monitoring cycle is determined according to the predicted confidence level of the current monitoring cycle, where there is a positive correlation between confidence level and cycle duration.

8. The method for adjusting the number of algorithm containers according to claim 7, characterized in that, The duration of the next monitoring cycle is calculated using the following formula: ， in, The duration of the next monitoring cycle, This is the minimum value of the preset monitoring cycle. This is the maximum value of the preset monitoring cycle. The prediction confidence level for the current monitoring period. This is the preset minimum prediction confidence level. This is the preset maximum prediction confidence level. It is an exponential coefficient used to adjust the shape of the mapping curve between confidence level and period duration.

9. A device for adjusting the number of algorithm containers, characterized in that, include: The load acquisition module is used to acquire the measured load of the current monitoring period and the predicted load for the current monitoring period predicted in the previous monitoring period. The confidence calculation module is used to update the predicted confidence of the previous monitoring period based on the measured load of the current monitoring period, so as to obtain the predicted confidence of the current monitoring period. The comprehensive load calculation module is used to perform a weighted summation of the measured load and the predicted load of the current monitoring period based on the prediction confidence level of the current monitoring period, so as to obtain the comprehensive load of the current monitoring period. The adjustment coefficient determination module is used to determine the adjustment coefficient based on the difference between the comprehensive load and the preset standard load; The algorithm Pod number adjustment module is used to obtain the adjusted number of algorithm Pods based on the current total number of algorithm Pods and the adjustment coefficient. The confidence calculation module includes: The prediction error calculation unit is used to calculate the prediction error of the current monitoring period after the actual load arrives in the current monitoring period, and update the error record in the error sliding window, wherein the error sliding window includes error data from multiple historical monitoring periods. An error statistical feature calculation unit is used to calculate the statistical features of the error within the updated error sliding window, wherein the statistical features include at least one of mean error, standard deviation, and range; The feature threshold calculation unit is used to take the quantile of the historical values ​​of the statistical features under each historical monitoring period within the updated error sliding window at a preset ratio to generate the feature threshold of the corresponding statistical features. The confidence correction unit is used to calculate a correction coefficient based on the statistical characteristics and the characteristic threshold, and to adjust the prediction confidence of the previous monitoring period using the correction coefficient to obtain the prediction confidence of the current monitoring period.

10. An electronic device, characterized in that, include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the algorithm container number adjustment method as described in any one of claims 1 to 8.

11. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method for adjusting the number of algorithm containers as described in any one of claims 1 to 8.

Images