Distributed model service scale adjustment method, system, device, medium and product

Abstract

This application discloses a method, system, device, medium, and product for scaling up distributed model services, relating to the field of big data processing. The method includes collecting a basic dataset of service metrics, analyzing inter-service call relationships to construct service groups and quantifying the independence of entry services, calculating request pressure values ​​and total resource utilization by combining historical resources and business data, predicting scaling trends and determining adjustment methods, and executing scaling based on independence and adjustment methods. Through multi-dimensional metric fusion and service topology relationship analysis, service priorities are clarified, avoiding decision-making biases based on single metrics. This solves the problems of blind scaling decisions and the inability to balance service stability and resource utilization in related technologies, achieving the technical effects of improving scaling accuracy, ensuring service stability, and enhancing resource utilization efficiency.

Description

Technical Field

[0001] This application relates to the field of big data processing technology, and in particular to methods, systems, devices, media and products for adjusting the scale of distributed model services. Background Technology

[0002] The key architecture for distributed inference of pre-trained language models is the master-slave working set architecture. This architecture treats multiple container groups as a single replica, with each replica containing a master container group and at least one slave container group, adapting to the model inference requirements of multi-node, multi-hardware collaborative computing. Distributed scaling technology is the core technology ensuring the stable operation of pre-trained language model inference services based on the master-slave working set architecture, dynamically adjusting the deployment scale according to service load. However, when applied to this type of inference service, related scaling technologies rely solely on passive decision-making based on historical operating states, lacking the ability to predict future service load and failing to adapt to the unique resource requirements and inter-service call relationships of pre-trained language model inference services. This leads to a disconnect between scaling decisions and actual service load changes, failing to achieve a dynamic balance between service performance, resource utilization efficiency, and service availability. Summary of the Invention

[0003] This application provides a method, system, device, medium, and product for adjusting the scale of a distributed model service, in order to at least solve the problem in related technologies where the scale adjustment of a distributed model service relies on a single indicator, leading to blind and inefficient scaling decisions.

[0004] This application provides a method for scaling up a distributed model service, including:

[0005] Obtain the service metric base dataset for multiple distributed model services. The service metric base dataset includes basic data, resource data, and business data.

[0006] Analyze the calling relationships between distributed model services based on the service metric dataset;

[0007] Multiple service groups are constructed based on the call relationships, and the independence of the entry service corresponding to each service group is calculated.

[0008] The request pressure and total resource utilization of the distributed model service are calculated based on resource data and business data from a preset historical period.

[0009] Based on the request pressure value and the total resource utilization, the scaling up and down of the distributed model service during the target prediction period is predicted to obtain the predicted value.

[0010] Determine the scaling-up / scaling strategy for distributed model services by combining predicted values ​​and total resource utilization.

[0011] The distributed model service is scaled up or down based on independence and scaling strategies.

[0012] This application also provides a distributed model service scaling system, including:

[0013] The data acquisition module is used to acquire the basic dataset of service metrics for multiple distributed model services. The basic dataset of service metrics includes basic data, resource data, and business data.

[0014] The analysis module is used to analyze the call relationships between distributed model services based on the service metric dataset.

[0015] The building module is used to construct multiple service groups based on the call relationship and calculate the independence of the entry service corresponding to the service group;

[0016] The calculation module is used to calculate the request pressure value and total resource utilization of the distributed model service based on resource data and business data for a preset historical period.

[0017] The prediction module is used to predict the scaling up and down of the distributed model service for the target prediction period based on the request pressure value and the total resource utilization, and obtain the predicted value.

[0018] The processing module is used to determine the scaling strategy of the distributed model service by combining the predicted values ​​and the total resource utilization rate.

[0019] The adjustment module is used to adjust the scale of the distributed model service based on independence and scaling strategies.

[0020] This application also provides an electronic device, including: a memory for storing a computer program; and a processor for implementing the steps of any of the above-described distributed model service scaling methods when executing the computer program.

[0021] This application also provides a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, it implements the steps of any of the above-described distributed model service scaling methods.

[0022] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of any of the above-described distributed model service scaling methods.

[0023] The distributed model service scaling method, system, equipment, medium, and product provided in this application integrate multi-dimensional operational data of the distributed model service, combine inter-service call relationships to complete the structured division of the service cluster, quantitatively characterize service independence, and construct a dual indicator system based on business data and resource data to achieve load prediction and scaling up / down. Quantified service priorities serve as the basis for scaling adjustments, constructing a data-driven, distributed service architecture-compliant end-to-end scaling logic. This differs from traditional single-indicator decision-making models. Therefore, it effectively solves the technical problems in traditional distributed model service scaling, such as one-sided decision-making basis, failure to adapt to service topology, lack of reasonable guidance for adjustment execution, leading to blind scaling up / down behavior, insufficient service stability, and mismatch between resource allocation and actual load. It achieves the technical effects of improving the accuracy and rationality of scaling up / down decisions, ensuring stable operation of distributed model services, optimizing resource allocation efficiency, and dynamically adapting service scale to business load and resource status. Attached Figure Description

[0024] To more clearly illustrate the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 A schematic diagram illustrating an application scenario for the distributed model service scale adjustment method provided in this application embodiment;

[0026] Figure 2 A flowchart illustrating the distributed model service scale adjustment method provided in this application embodiment;

[0027] Figure 3 A scaling diagram illustrating the adjustment of the number of replicas in the master-slave working set provided for embodiments of this application;

[0028] Figure 4 This application provides a schematic diagram illustrating the scaling up and down of adjusting the number of working node containers in an embodiment of the invention.

[0029] Figure 5 A schematic diagram of the structure of the distributed model service scaling system provided in this application embodiment;

[0030] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0031] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of this application.

[0032] It should be noted that, in the description of this application, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. The terms "first," "second," etc., in this application are used to distinguish similar objects and are not used to describe a specific order or sequence.

[0033] This application obtains a basic dataset of service metrics for distributed model services, analyzes the call relationships between services to construct service groups, and determines the independence calculation logic for the entry service. Based on resource and business data within a preset historical time period, it calculates request pressure and total resource utilization. Based on these two calculation results, it predicts the scaling behavior of the model service during the target prediction period and generates predicted values. Combining the predicted values ​​and total resource utilization, it determines an appropriate scaling strategy. Finally, based on the independence of the entry service and in conjunction with the determined scaling strategy, it achieves scale adjustment of the distributed model service. By integrating service call relationships, business metrics, and resource metrics to form a unified scale adjustment logic, it constructs a scale adjustment mechanism adapted to the characteristics of distributed model services.

[0034] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0035] The application environment architecture upon which the distributed model service scaling method depends is described here. (References) Figure 1 , Figure 1 This is a schematic diagram illustrating an application scenario of the distributed model service scale adjustment method provided in the embodiments of this application, such as... Figure 1 As shown, it includes: terminal 101 and server 102.

[0036] Terminal 101 is used to initiate distributed model service call requests, upload business interaction data, and display the running status and scale adjustment results of the distributed model service. Server 102 is used to receive requests and data uploaded by the terminal, collect basic datasets of service metrics, analyze service call relationships and calculate independence, request pressure value and total resource utilization, execute distributed model service scaling prediction and scale adjustment operations, generate scale adjustment execution results and feed them back to the terminal.

[0037] Figure 2 A flowchart illustrating the distributed model service scale adjustment method provided in this application embodiment is shown below. Figure 2 As shown, embodiments of this application provide a method for adjusting the scale of a distributed model service. The method is described in detail below:

[0038] Step S201: Obtain the service metric base dataset for multiple distributed model services. The service metric base dataset includes basic data, resource data, and business data.

[0039] Specifically, various relevant data generated during the operation of the distributed model service are collected and integrated to form a basic dataset of service metrics. Basic data refers to the service's basic configuration information, resource data refers to data related to the hardware and software resources used during service operation, and business data refers to data related to the business requests processed by the service. By comprehensively acquiring multi-dimensional data on service operation, data support is provided for subsequent analysis and calculations, ensuring the reliability of the analysis results in each subsequent step, achieving comprehensive coverage of service operation data, and avoiding biases in adjustment decisions due to data gaps.

[0040] Step S202: Analyze the calling relationships between distributed model services based on the service metric dataset.

[0041] Specifically, data related to inter-service interactions is extracted from the service metric dataset, and topology analysis is used to identify the call paths, call frequencies, and call dependencies among the distributed model services. By clarifying the relationships between the services, the call topology of the distributed model services is obtained.

[0042] Step S203: Construct multiple service groups based on the call relationship, and calculate the independence of the entry service corresponding to each service group.

[0043] Specifically, based on service call relationships, services with direct or indirect call associations or functional connections are grouped into the same service group, and the entry service for each service group is determined. Independence refers to the core nature and independent operational capability of the entry service within the service group. The independence value of the entry service is calculated by quantifying the frequency of calls made to the entry service by other services and combining this with the service dependencies within the service group. By quantitatively analyzing the call characteristics of the entry service to determine its independence, the system achieves structured service classification, quantifies the core nature of the entry service, clarifies the core nodes of each service group, and provides a basis for prioritizing scaling.

[0044] Step S204: Calculate the request pressure value and total resource utilization rate of the distributed model service based on the resource data and business data of the preset historical time period.

[0045] Specifically, resource and business data within a preset historical time period are selected. Business data is normalized to eliminate the influence of data dimensions, and then the data is summed to calculate the request pressure value, which refers to the size of the business request load that the service can withstand. Resource data is filtered and corrected to remove abnormal data, and then the total resource utilization rate is calculated through weighted summation. Total resource utilization rate refers to the efficiency of resource usage during service operation. By normalizing data and using weighted summation, the business load and resource usage status of the service are quantified, providing data support for scaling up and down predictions and avoiding the limitations of a single indicator.

[0046] Step S205: Based on the request pressure value and total resource utilization, predict the scaling up and down of the distributed model service for the target prediction period to obtain the predicted value.

[0047] Specifically, based on the request pressure and total resource utilization calculated within a preset historical time period, the changing trends of these two indicators are analyzed through time-series forecasting. Combined with the business scenario characteristics of the target forecast period, the request pressure and total resource utilization of the service within the target forecast period are predicted. The combined results of these two predictions yield a scaling-up / scaling prediction value, which characterizes the required scale adjustment of the service within the target period. By predicting future load changes based on historical data trends and combining these two indicators to predict service load changes within the target period, a forward-looking prediction is provided for determining the scaling-up / scaling method, avoiding delayed or blind adjustments and improving the timeliness of adjustments.

[0048] Step S206: Determine the scaling strategy for the distributed model service by combining the predicted value and the total resource utilization rate.

[0049] Specifically, by pre-setting reasonable threshold ranges for request pressure and total resource utilization, the predicted scaling values ​​are combined with the actual total resource utilization values ​​and compared with the pre-set threshold ranges to determine the current and future load status of the service. Based on the load status, the corresponding scaling method for the service is determined, including scaling up, scaling down, or maintaining the current scale, clarifying the specific direction of adjustment. By combining the predicted values ​​with the real-time resource utilization status, an appropriate scaling strategy is matched, clarifying the specific method of service scale adjustment, ensuring that the adjustment direction is consistent with the service load status, and avoiding ineffective adjustments.

[0050] Step S207: Adjust the scale of the distributed model service according to the independence and scaling strategy.

[0051] Specifically, based on the independence value of the entry service of each service group, the scaling priority of each service group is determined. The higher the independence value, the higher the scaling priority. According to the determined scaling priority and the corresponding scaling method, the scale adjustment operation is performed on each service group and the services within the group. Through priority sorting and combined with the preset scaling method, service scale adjustments are executed in an orderly manner, ensuring the stable operation of core services and achieving orderly and targeted scaling operations. This meets service load requirements while ensuring service stability.

[0052] The distributed model service scale adjustment method provided in this invention combines service topology relationships with multi-dimensional load indicators to achieve precise and orderly adjustment of the distributed model service scale. This solves the problems of one-sided decision-making and lack of reasonable guidance in traditional adjustment methods, thereby improving the accuracy of scaling decisions, ensuring stable service operation, and optimizing resource allocation efficiency.

[0053] This embodiment provides a detailed description of the process for obtaining the basic dataset of service metrics for multiple distributed model services in the above embodiments. The specific implementation of this process includes the following steps:

[0054] Step a1: Retrieve the basic data corresponding to the distributed model service from the preset database.

[0055] Specifically, basic data related to the distributed model service is extracted from a pre-configured database through database queries. This pre-deployed and configured database stores the basic configuration information of the distributed model service. The basic data refers to the fundamental configuration information required for the service to run, representing the service's basic operational attributes. By quickly obtaining accurate basic data, the cumbersome process of basic data collection is avoided, improving data acquisition efficiency and providing fundamental support for subsequent data integration and analysis.

[0056] Step a2: Obtain time-series data of the distributed model service from multiple acquisition nodes in the underlying environment according to the preset sampling frequency. The time-series data includes resource data and business data.

[0057] Specifically, time-series data of the distributed model service is collected from multiple collection nodes in the underlying operating environment according to a pre-set sampling frequency. The pre-set sampling frequency is a fixed time interval for collecting time-series data, which can be set according to actual needs. Collection nodes refer to terminal nodes deployed in the underlying operating environment for real-time collection of service runtime time-series data. Time-series data is data collected continuously in chronological order, reflecting the changes in the service's operating status over time. Resource data is data related to the hardware and software resources occupied during service operation, and business data is data related to various business requests processed by the service.

[0058] By acquiring dynamic data during service operation, we can supplement the deficiencies of basic data, improve data dimensions, and achieve continuous and real-time collection of resource and business data. This ensures the timeliness and completeness of the data and provides dynamic data support for subsequent load quantification calculations.

[0059] Step a3: Associate the basic data and time-series data according to the service identifier to obtain the service indicator basic dataset containing basic data, resource data and business data.

[0060] Specifically, service identifiers are extracted from the basic data and time-series data. Through data association processing, the basic data and time-series data belonging to the same distributed model service are matched and integrated to form a complete service indicator basic dataset. The service identifier is the identification information used to distinguish different distributed model services. It can be in the form of a combination of characters and numbers to ensure that the data of different services are not confused.

[0061] Based on the uniqueness of service identifiers, different types of data are accurately matched and integrated through data association to form a dataset with a unified structure and comprehensive dimensions. This avoids analytical biases caused by data dispersion and ensures the accuracy and efficiency of subsequent analysis and calculation.

[0062] This invention achieves comprehensive, accurate, and efficient acquisition of basic service indicator data by pre-setting a database, collection nodes, and data association, combined with a pre-set sampling frequency. This ensures that the dataset covers the basic attributes and dynamic status of service operation, solving the problems of single-dimensionality, low efficiency, and scattered data in traditional data acquisition methods. It effectively improves data acquisition efficiency, ensures the integrity and accuracy of the dataset, and provides reliable data support for subsequent steps in adjusting the scale of distributed model services.

[0063] In one specific embodiment, step a2 above specifically includes:

[0064] Step a21: Verify the acquisition permissions of each acquisition node in the underlying environment, and establish a data transmission link after the verification is successful.

[0065] Specifically, the legitimacy of the collection permissions of each collection node in the underlying environment is verified through permission checks. This verifies whether the collection nodes have the authority to collect data related to the distributed model service. After verification, a data transmission link is established between the collection nodes and the data receiving end. Permission verification verifies the legality of the collection nodes in collecting data, preventing unauthorized nodes from collecting data. The data transmission link is used for data transfer between the collection nodes and the receiving end. Through permission authentication and link establishment, the legitimacy of collection nodes and the construction of data transmission channels are achieved, ensuring the legality and security of data collection. This provides a secure and stable transmission channel for subsequent data collection, preventing data leakage or anomalies caused by unauthorized collection and ensuring the stability of data transmission.

[0066] Step a22: Based on the preset sampling frequency, control the acquisition node to collect the corresponding resource data and business data to obtain the raw data.

[0067] Specifically, collection commands are sent to each verified collection node according to a pre-set sampling frequency. This controls the collection nodes to collect resource and business data in real time during the operation of the distributed model service. After collection, the raw data is aggregated. Raw data refers to data directly collected by the collection nodes without any processing. By collecting data periodically and quantitatively to obtain dynamic raw data during service operation, the regularity and continuity of data collection are ensured, guaranteeing that the raw data accurately reflects the service's operational status and providing raw input for subsequent data analysis.

[0068] Step a23: Perform outlier removal on the original data to obtain the target data.

[0069] Specifically, outlier detection algorithms can be used to detect outliers in the collected raw data, identify and remove them, and obtain purified target data. Outliers refer to data that deviates from the normal distribution range of data and does not conform to the normal operation rules of the service. Outlier removal processing can remove abnormal and invalid data from the data.

[0070] By cleaning and detecting outliers, abnormal components in the data are identified and removed, the raw data is purified, and the interference of abnormal data on subsequent analysis results is eliminated. This results in accurate and effective target data, avoids deviations in subsequent load calculations and decisions caused by abnormal data, and improves data quality.

[0071] Step a24: Sort the target data by timestamp to obtain time series data.

[0072] Specifically, timestamps are extracted from the target data. These timestamps mark the time of data collection. The target data is then sorted and organized according to the order of the timestamps to obtain time-series data. Time-series data is data arranged continuously in chronological order, reflecting the changing patterns of service operation status over time. By combining the uniqueness and continuity of timestamps, the data is organized in a time-series manner, effectively standardizing the data arrangement format, highlighting the temporal correlation of the data, and adapting to subsequent time-series analysis needs.

[0073] This invention, through authorization authentication, data collection, data cleaning and sorting, combined with preset sampling frequency and outlier detection, achieves secure, standardized and efficient collection and processing of time-series data. It solves the problems of lack of authorization control, data anomalies and time-series disorder in traditional data collection processes, and improves the security, accuracy and standardization of time-series data collection.

[0074] In one specific embodiment, step a23 above specifically includes:

[0075] Step a231: Perform outlier detection on the original data to obtain abnormal data points that exceed the preset value range.

[0076] Specifically, by combining outlier detection (OD) with a preset numerical range, the collected raw data is comprehensively inspected to identify and filter out anomalous data points that exceed the preset reasonable range and deviate from the normal data distribution. The outlier detection algorithm can be, for example, the Isolation Forest Algorithm (IFA), the Z-Score Algorithm (ZA), or other outlier detection algorithms, depending on the requirements. No specific restrictions are imposed here. Through analysis, the system achieves accurate identification of anomalous data points, captures data distribution characteristics, distinguishes between normal and anomalous data, and provides a clear target for subsequent data purification.

[0077] Step a232 involves removing abnormal data points from the original data and filling in missing timestamps.

[0078] Specifically, data filtering completely removes identified outlier data points from the original data, eliminating invalid data interference. For missing timestamps that appear after removing outlier data points, data completion is used to fill in the missing positions. Data completion is a method of reasonably filling in missing values ​​in the dataset to ensure data continuity, and can employ methods such as Linear Interpolation Method (LIM) or Mean Interpolation Method (MIM).

[0079] By filtering and removing invalid data and filling in missing data, we can ensure the integrity and continuity of data, eliminate invalid and abnormal data, fill in data gaps, restore the continuity and integrity of data, eliminate interference from abnormal data, solve the problem of missing data, ensure that the data can fully reflect the service operation status, and avoid data gaps that may lead to biases in subsequent analysis.

[0080] Step a233: The completed original data is classified and stored according to the preset indicator types to obtain the target data.

[0081] Specifically, the indicator identifiers are extracted from the original data after completion processing. According to the pre-set preset indicator types, the data is classified and archived into two categories, resource data and business data, through data classification storage, and finally the purified and standardized target data is obtained. The preset indicator type refers to the data category pre-divided according to the construction requirements of the service indicator basic dataset. The indicator identifier is a special mark used to distinguish different types of data. Classification storage refers to the method of dividing the storage path according to the data category to achieve orderly data management.

[0082] By standardizing data storage formats and clarifying data classification logic, we can obtain target data that is clearly classified, formatted correctly, and of acceptable quality, thereby improving the efficiency of subsequent data retrieval and analysis.

[0083] This invention achieves in-depth purification, completion, and standardized management of raw data through outlier detection, data filtering, data completion, and classified storage. It solves the problems of unremoved abnormal data, missing data, and disordered storage in traditional data processing, improves data quality, ensures data integrity and standardization, and provides reliable support for the subsequent processing of time series data and the construction of basic datasets for service indicators.

[0084] This embodiment details the process described in the above embodiment of constructing multiple service groups based on the call relationships between distributed model services and calculating the independence of the entry service corresponding to each service group. The specific implementation of this process includes the following steps:

[0085] Step b1: Extract the call information of the distributed model service from the service metric basic dataset.

[0086] Specifically, call information between distributed model services is filtered and extracted from the basic service metric dataset. Call information refers to relevant data on initiating and receiving calls between distributed model services, including the initiator, receiver, and call frequency. By accurately filtering target information from multi-dimensional data, basic input is provided for subsequent identification of call relationships and construction of service groups, avoiding interference from irrelevant data and ensuring the accuracy of subsequent correlation analysis.

[0087] Step b2: Identify multiple distributed model services that are directly related to each other based on the call information.

[0088] Specifically, the extracted call information is analyzed, and multiple distributed model services with direct call relationships are identified through correlation analysis, clarifying the direct call relationship logic between each service. Correlation analysis technology is used to identify the relationship between different data items in a dataset; a direct call relationship refers to the relationship between two services that directly initiate and receive calls without the need for intermediate service forwarding. By mining the correlation features in the call information through correlation analysis, direct and indirect calls are distinguished, and the direct relationship between services is clarified, providing a basis for the integration and division of service groups and ensuring the rationality of the service group division.

[0089] Step b3: Integrate the distributed model services with call associations into a service group, and designate the called distributed model service as the entry service of the corresponding service group.

[0090] Specifically, the identified distributed model services that directly call the associated services are integrated into a service group, and the distributed model services that are directly called by other services are designated as the entry service of the service group. The entry service refers to the service in the service group that is directly called by other services and serves as the core of the service group's external interaction, undertaking the interaction function between the service group and external services or terminals.

[0091] By clustering and integrating closely related services into a group, and determining the core nodes of the service group based on the calling relationship, the service group is constructed, the core entry service of each service group is clarified, and the structured classification management of services is realized.

[0092] Step b4: Analyze the number of services directly called by the entry service to obtain the independence value.

[0093] Specifically, the analysis focuses on the number of services directly invoked by other services as the entry service. An independence value is calculated based on this number of invocations using quantitative analysis. The independence value is a quantifiable indicator of the entry service's ability to operate independently and its core status within the service group. The fewer services directly invoked, the higher the entry service's independence value and the stronger its core status. By quantifying the number of direct invocations of the entry service, a quantitative indicator of its core status is generated. This quantification of the entry service's core status and independent operation capability ensures the rationality and scientific nature of subsequent scaling and adjustment priorities.

[0094] This invention, through data extraction, correlation analysis, clustering integration, and quantitative analysis, combined with the mining and processing of call information, achieves the rational construction of service groups and the accurate quantification of the independence of entry services. It solves the problems of chaotic service grouping, inability to quantify the coreness of entry services, and lack of reasonable priority guidance for scaling up and down. It effectively standardizes service grouping, accurately represents the coreness of entry services, provides a basis for adjusting the scale of distributed model services, and improves the pertinence and rationality of scaling up and down.

[0095] In one specific embodiment, step b4 above specifically includes:

[0096] Step b41: Determine if the entry service has a call record.

[0097] Specifically, the system retrieves the call information corresponding to the entry service and verifies whether there are any records of the entry service being directly called by other services through data querying and judgment. Call records refer to detailed information about when the entry service is directly called by other services, including the initiator and the time of the call. By verifying the two call states of the entry service, a classification basis is provided for the subsequent differentiated assignment of independence.

[0098] Step b42: Assign the independence of the entry service with no call record to a preset baseline value.

[0099] Specifically, for entry services without any call records, their independence is directly assigned a preset benchmark value according to preset rules. The preset benchmark value is a fixed quantitative value that is pre-set to characterize the coreness of entry services without call records, and serves as a reference benchmark for independence measurement. By uniformly quantifying entry services without call records, the coreness of these entry services is clearly defined, ensuring the uniformity of independence measurement for such entry services, simplifying the calculation process, and establishing a benchmark standard for independence measurement.

[0100] Step b43: Summarize the number of services directly called by the entry service with a call record to obtain call data.

[0101] Specifically, for entry services with call records, their corresponding call information is extracted. By summarizing the number of services directly called by other services, the summarization result is the call data. Call data refers to quantitative data characterizing the frequency of direct calls to entry services, reflecting the degree of correlation between entry services and other services. By accurately summarizing the number of direct calls to entry services, we obtain quantitative basic data on the correlation degree of entry services, providing input for subsequent accurate calculation of independence, ensuring the reliability of the data source for subsequent independence calculations, and providing support for the differentiated quantification of independence.

[0102] Step b44: Input the called data into the preset calculation model to obtain the corresponding independence value. The independence value is less than the preset benchmark value.

[0103] Specifically, the called data is input into a pre-trained and deployed preset computing model. The model then outputs the corresponding independence value, ensuring that the independence value is less than a preset benchmark value. The preset computing model refers to a quantitative model used to convert the called data into independence values. For example, a linear regression model (LRM) achieves quantitative conversion by establishing a linear relationship between the called data and the independence value. The smaller the independence value, the more frequently the entry service is called, the higher the degree of correlation, and the weaker the ability to operate independently.

[0104] For example, suppose there are N services, represented as: It is divided into N service groups, and the entry service of the i-th service group is The independence of the i-th service is calculated as follows:

[0105]

[0106] When the i-th service is not invoked, independent i A value of 1 indicates the highest degree of independence, requiring only consideration of scaling up or down its own service. The i-th service is directly called by m services. The larger m is, the lower the degree of independence. The lower the independence of a service, the more other services need to be considered for scaling up or down.

[0107] The quantization transformation based on the preset calculation model maps the called data into an independence value that represents the core status of the entry service. This precise quantification of the core status and independent operational capability of entry services with call records enables differentiated representation of independence, yielding accurate and reasonable independence values ​​that conform to the logic that the more calls a service receives, the lower its independence. This provides a quantitative basis for subsequent scaling priority ranking.

[0108] This invention, through data querying, logical judgment, data analysis, and a preset calculation model, combined with preset benchmark values ​​and quantification rules, achieves differentiated and precise quantification of the independence of entry services. It solves the problems of traditional independence calculation methods being singular, inaccurate in quantification, and unable to distinguish the degree of association of entry services, improves the accuracy and rationality of independence quantification, clarifies the differences in the core degree of different entry services, and provides reliable quantitative support for the priority division of distributed model service scale adjustment.

[0109] This embodiment provides a detailed explanation of the process in the above embodiments for calculating the request pressure value and total resource utilization of the distributed model service based on resource data and business data from a preset historical time period. The specific implementation of this process includes the following steps:

[0110] Step c1 involves performing non-linear mapping and normalization on the business data, followed by superposition to obtain the request pressure value.

[0111] Specifically, business data within a preset historical period is extracted, and the business data is transformed through nonlinear mapping (NM) to fit the nonlinear relationship between business data and load pressure. Normalization processing (NP) is used to eliminate the dimensional differences between different types of business data. Normalization methods such as min-max normalization (MMN) can be used. The processed business data is then superimposed to obtain the request pressure value.

[0112] Request pressure value refers to a numerical value that quantifies the business request load that the distributed model service can withstand; nonlinear mapping refers to transforming business data from the original space to the target space to achieve nonlinear relationship fitting; normalization processing refers to transforming data with different dimensions to the same numerical range to eliminate dimension interference.

[0113] For example: Obtaining the prediction result of the i-th service based on historical data. Specifically, it is implemented by using historical data to predict data for a future period and then obtaining the prediction results based on the predicted future data.

[0114] For the i-th service, the request pressure in the j-th hour is:

[0115]

[0116] in, Let P95 be the average length of the request queue in the j-th hour; The P95 value represents the effective concurrency in the j-th hour; Let $\frac{ ... Let P99 be the request delay value for the j-th hour.

[0117] The significant differences in the number of tokens in the distributed model service lead to substantial variations in processing time. Therefore, load cannot be measured solely by QPS (Queries Per Second); it must be considered in conjunction with queue latency and effective concurrency. A logarithmic non-linear mapping is introduced to the request queue items to prevent a single burst of queue backlog from excessively dominating the metric, thus enhancing its stability and robustness. Higher request QPS values ​​and higher concurrency indicate more intense resource contention; longer queues suggest insufficient system processing capacity; and higher request latency is the most sensitive signal of overload.

[0118] By transforming and standardizing data to fit the nonlinear characteristics of business data and eliminating the influence of dimensions, and by superimposing calculations to quantify load pressure, the system converts business data of various types and dimensions into a unified quantitative request pressure value, accurately characterizing the business load status of services, eliminating the interference of dimensions and nonlinear deviations in business data, and obtaining accurate and unified request pressure values, providing reliable business load indicators to support subsequent scaling predictions.

[0119] Step c2 involves correcting the resource data using a preset correction function and then performing a weighted calculation to obtain the total resource utilization rate for each time period.

[0120] Specifically, resource data within a preset historical time period is extracted and input into a pre-defined correction function to correct systematic errors and random biases in the resource data, resulting in corrected resource data. Then, corresponding weights are assigned based on the importance of each type of resource, and a weighted sum (WS) is used to calculate the total resource utilization rate for each time period. Total resource utilization rate is a numerical value that quantitatively represents the efficiency of resource usage during the operation of the distributed model service. The preset correction function refers to a pre-defined mathematical function used to correct resource data biases, such as the Exponential Correction Function (ECF). Weights are coefficients assigned based on resource importance to reflect the degree of influence of different resources on the total utilization rate.

[0121] For example, for the i-th service, because the utilization of different processors and memory exceeds a certain threshold, increasing the utilization actually lowers the overall score. This is because factors such as heat dissipation and queuing can cause efficiency drops. Therefore, a threshold is defined for resource r. ,pass The function handles the portion exceeding the threshold:

[0122]

[0123] in, The value range is [0,1], and a predefined value can be set, for example, 0.79; the threshold values ​​corresponding to processors A and B and memory C. These are preset values, which can be set according to actual needs, such as 0.85, 0.9, and 0.96 respectively.

[0124] The values ​​0.85, 0.9, and 0.96 are example values. In a specific example, processor A is a compute-intensive processor and processor B is a general-purpose processor.

[0125] Processor A tasks have long execution cycles and are not preemptible. For example, when utilization exceeds 85%, significant task queuing and backlog will occur, increasing the average processing latency per task. Simultaneously, a 15% resource redundancy is reserved to handle sudden traffic spikes, system background process overhead, and fault migration buffering, ensuring service availability. Setting the threshold too high can easily lead to service response timeouts under sudden traffic surges; setting it too low can result in a serious waste of computing resources.

[0126] Processors B have a high proportion of short tasks and relatively low context switching overhead. For example, when utilization is below 90%, there is no significant decrease in system scheduling latency and task execution efficiency; however, when utilization exceeds 90%, system kernel processes are squeezed out by user processes, leading to longer scheduling cycles, slower I / O response, and increased processor heat dissipation pressure, which may trigger automatic frequency reduction. Therefore, by setting a threshold, the resource utilization of general-purpose processors can be maximized while ensuring system stability.

[0127] The performance degradation of C memory resources is abrupt. For example, when memory utilization is below 96%, the system does not need to use swap, and memory access latency remains stable. However, when utilization exceeds 96%, the system forcibly swaps some memory data to disk, causing access latency to spike from nanoseconds to milliseconds, resulting in a significant drop in service performance. Reserved 4% of memory for kernel cache, page tables, and burst memory allocation can effectively prevent swap partitioning. The threshold can be adjusted according to actual needs for different memory requirements.

[0128] The normalized total resource utilization score for the j-th hour:

[0129]

[0130] in, Let A be the average utilization of processor A in the j-th hour; Let B be the average utilization rate of the B processor in the j-th hour; Let be the average memory utilization of C in the j-th hour.

[0131] The operational limitations of distributed model services are primarily concentrated on computing resources, followed by memory resources. For example, production environment statistics show that computing resources (processor A + processor B) account for approximately 80% of the service performance degradation, while memory resources account for approximately 20%. Therefore, setting the weight of processor A and processor B to 2 each, and the weight of memory C to 1, for a total weight of 5, accurately reflects the impact of different types of resources on the overall service performance, making the total resource utilization score more closely reflect the actual operating status of the service. The weight values ​​can be set according to actual circumstances; this is just an example, and no specific restrictions are placed on these weight values ​​in practical applications.

[0132] By correcting resource data deviations and highlighting the impact of important resources through error correction and weighted calculation, resource data deviations are eliminated, the overall resource utilization efficiency is quantified, and a unified indicator of resource utilization status for each time period is obtained. This ensures the accuracy and rationality of the total resource utilization rate calculation. At the same time, the weighted calculation is aligned with actual resource usage scenarios, providing reliable resource status indicator support for subsequent expansion and contraction predictions.

[0133] This invention, through nonlinear mapping, normalization, error correction, and weighted calculation, combined with various algorithms and functions, achieves accurate and unified quantification of request pressure and total resource utilization. This improves the quantification accuracy of the two core indicators, providing accurate and reliable dual-dimensional indicator support for distributed model service scaling prediction, and ensuring the scientific nature of subsequent scaling decisions.

[0134] In one specific embodiment, step c1 above specifically includes:

[0135] Step c11: Extract request queue length data, effective concurrency, request query rate, and inference latency data for a preset historical period from the business data.

[0136] Specifically, from business data within a preset historical time period, request queue length, effective concurrency, request query rate, and inference latency data are filtered and extracted. Request queue length data represents the number of business requests waiting to be processed within the preset historical time period; effective concurrency (EC) refers to the number of business requests actually processed by the service within the same time period; request query rate (RQR) refers to the frequency of business query requests received by the service per unit of time; and inference latency data (ILD) refers to the time from receiving a single inference request to returning a result. By filtering and extracting data, core load-related data are accurately obtained from multi-dimensional business data, providing multi-dimensional data input for subsequent request pressure value calculations, ensuring the comprehensiveness of data in subsequent calculations, and avoiding load representation bias caused by single data points.

[0137] Step c12: Perform a non-linear mapping on the request queue length data to obtain the mapped request queue length data.

[0138] Specifically, the extracted request queue length data is input into a preset nonlinear mapping function. The data is transformed through function operations to obtain the mapped request queue length data. Nonlinear mapping refers to the technique of transforming data from the original numerical space to the target numerical space and fitting the nonlinear relationship between the data and the load pressure. Nonlinear mapping functions that can be used include the Sigmoid Mapping Function (SMF), which can be selected according to the actual situation without specific restrictions. It can correct the nonlinear deviation between the request queue length data and the actual load pressure.

[0139] By fitting the non-linear relationship between request queue length and load pressure through function operations, the non-linear distortion of the data is eliminated, making the data more closely reflect the actual business load pressure characteristics. The result is a mapped request queue length that accurately reflects the actual load pressure, improving the accuracy of subsequent request pressure value calculations.

[0140] Step c13: Compare the effective concurrency, request query rate, and inference latency data with their respective preset historical average values ​​to obtain intermediate data.

[0141] Specifically, the average values ​​of effective concurrency, request query rate, and inference latency are calculated within a preset historical period. Then, through data comparison calculations, the differences between the three types of data and their respective average values ​​are calculated to obtain three types of intermediate data. The average value is the arithmetic mean of the corresponding data within the preset historical period, which is used to represent the normal level of the corresponding data. The intermediate data refers to the quantitative data that represents the degree to which each type of data deviates from the normal level.

[0142] By comparing with historical average levels, the fluctuation of various data types is quantified, load characterization dimensions are supplemented, and the calculation basis of request pressure values ​​is improved. Intermediate data reflecting the fluctuation of various business load parameters is obtained, so that subsequent request pressure values ​​can take into account both normal data levels and fluctuations, thus improving the comprehensiveness of quantification.

[0143] Step c14: Overlay the mapped request queue length data with the intermediate data to obtain the request pressure value.

[0144] Specifically, the mapped request queue length data is summed and superimposed with the three types of intermediate data. The result is the request pressure value. The request pressure value is a comprehensive numerical value that quantifies the business load the service is under, integrating load characteristics from four dimensions: queue length, concurrency, query rate, and latency. By integrating multi-dimensional load-related data into a single quantitative indicator, a comprehensive characterization of load pressure is achieved, resulting in a comprehensive and accurate request pressure value that comprehensively reflects the service's business load status and provides reliable quantitative indicators to support subsequent scaling predictions.

[0145] This invention achieves multi-dimensional and accurate quantification of request pressure values ​​through data extraction, nonlinear mapping, data analysis, comparison, and superposition. This effectively improves the comprehensiveness and accuracy of request pressure value quantification, fully characterizes the service load status, and provides high-quality business load indicators to support distributed model service scaling prediction.

[0146] In one specific embodiment, step c2 above specifically includes:

[0147] Step c21: Classify the resource data according to resource type to obtain multiple resource sub-data.

[0148] Specifically, resource data within a preset historical time period is categorized and organized according to resource type, resulting in multiple corresponding resource sub-data. Resource type refers to the different types of hardware and software resources used during the operation of the distributed model service. Resource sub-data refers to the operational data corresponding to a single resource type, used to characterize the occupancy status of that type of resource. By classifying resource data according to resource type differences, different types of resource data are distinguished, eliminating data contamination and interference, resulting in clearly categorized resource sub-data. This standardizes resource data management, ensuring that subsequent processing steps accurately correspond to a single resource type, thereby improving processing efficiency.

[0149] Step c22: Obtain the preset utilization threshold corresponding to the resource sub-data.

[0150] Specifically, the preset utilization threshold refers to the upper limit of a reasonable numerical range set in advance based on the operational characteristics and service requirements of different types of resources, used to determine whether resource utilization is abnormal. Through preset configuration logic, the threshold is accurately matched and retrieved with the corresponding resource sub-data, providing a unified comparison benchmark for subsequent anomaly judgment of resource sub-data and clarifying the reasonable boundaries of resource utilization.

[0151] Step c23: Compare the resource sub-data with the corresponding preset utilization threshold to obtain the resource sub-data that exceeds the preset utilization threshold and needs to be corrected.

[0152] Specifically, each type of resource sub-data is compared with its corresponding preset utilization rate threshold, and resource sub-data whose values ​​exceed the preset utilization rate threshold are selected. Such data is the resource sub-data to be corrected. Resource sub-data to be corrected refers to a single type of resource data whose resource utilization status exceeds the reasonable range and has deviations.

[0153] By identifying data anomalies through threshold comparison, abnormal resource data that exceeds the reasonable range is obtained, providing a clear target for subsequent correction and processing, avoiding the direct participation of abnormal data in subsequent calculations, and reducing the calculation deviation of total resource utilization.

[0154] Step c24: Attenuate the resource sub-data to be corrected using a preset correction function to obtain the corrected resource sub-data.

[0155] Specifically, the selected resource sub-data to be corrected is input into a preset correction function. The function performs attenuation correction to eliminate abnormal deviations in the data, resulting in corrected resource sub-data. The preset correction function is a predefined mathematical function used to correct resource data deviations exceeding a threshold, such as the Exponential Correction Function (ECF). Attenuation correction refers to the process of adjusting abnormal data exceeding the threshold to a reasonable range according to preset rules. By correcting data deviations through the operation logic of the preset function, the data returns to a reasonable range, eliminating abnormal deviations in the resource sub-data to be corrected. This ensures that the resource data accurately reflects the actual resource occupancy status, resulting in accurate and reasonable corrected resource sub-data. This solves the calculation deviation problem caused by data anomalies and provides a reliable data source for subsequent weighted calculations.

[0156] Step c25: Assign preset weights to the corrected resource sub-data according to the resource type.

[0157] Specifically, based on the importance of different resource types to the operation of the distributed model service, corresponding preset weights are assigned to various corrected resource sub-data. The preset weights are pre-set coefficients used to reflect the degree of influence of different types of resources on the overall resource utilization rate. The more important the resource type, the greater the weight assigned.

[0158] Based on the weight allocation logic, the weights are reasonably allocated by combining the differences in the importance of resource types, highlighting the differences in the impact of different types of resources on the total resource utilization rate, so that the subsequent weighted calculation results are more in line with the actual resource use scenarios.

[0159] Step c26: Perform a weighted summation of the corrected resource sub-data according to the preset weights to obtain the total resource utilization rate.

[0160] Specifically, by using weighted summation, each type of corrected resource sub-data is multiplied by its corresponding preset weight, and then all products are summed to obtain the total resource utilization rate. The total resource utilization rate is a comprehensive numerical value that quantitatively represents the overall resource utilization efficiency during the operation of the distributed model service. By integrating multi-type resource data and their corresponding weights, the overall resource utilization efficiency is quantified, transforming multi-dimensional resource data into a unified quantitative comprehensive indicator, resulting in a comprehensive and accurate total resource utilization rate. This comprehensively reflects the overall resource utilization efficiency of the service, providing reliable resource status indicators to support the scaling prediction of the distributed model service.

[0161] This invention achieves accurate and comprehensive quantification of total resource utilization by using data classification, comparison, correction, and weighted calculation, combined with preset thresholds, correction functions, and weight allocation rules. This effectively improves the quantification accuracy of total resource utilization and truly and comprehensively reflects the resource utilization efficiency of distributed model services.

[0162] This embodiment details the process of predicting the scaling up and down of the distributed model service for the target prediction period based on the request pressure value and total resource utilization in the above embodiments. The specific implementation of this process includes the following steps:

[0163] Step d1: Determine the preceding time period and the two preceding time periods adjacent to the target prediction time period based on the time of the target prediction time period.

[0164] Specifically, by defining the time range of the target prediction period, the most recent historical period (previous period) and the two most recent historical periods (previous two periods) directly adjacent to this period are determined using time interval positioning. The target prediction period refers to the specific time interval in which the scaling up and down of the distributed model service needs to be predicted. The previous period refers to the previous historical time interval immediately adjacent to the target prediction period. The two previous periods refer to the two consecutive historical time intervals immediately adjacent to the target prediction period.

[0165] Based on the continuity and adjacency correlation of time series data, the most recent historical period is selected as the benchmark for trend analysis, providing a clear data source for the subsequent trend calculation of pressure and resource utilization, ensuring that the data used in subsequent trend analysis is timely and relevant, and improving the rationality of the forecast.

[0166] Step d2: Calculate the sum of the request pressure values ​​from the first two time periods to obtain the first pressure data.

[0167] Specifically, by extracting the request pressure values ​​corresponding to the first two time periods and summing them, the resulting sum is the first pressure data. The first pressure data refers to the sum of the business load pressure in the first two historical time periods, used to characterize the cumulative state of recent business load. By integrating the load pressure data from the two historical time periods, a reliable input is provided for calculating the pressure trend ratio, supporting the quantitative analysis of pressure change trends.

[0168] Step d3: Compare the request pressure value of the previous period with the first pressure data to obtain the pressure trend ratio.

[0169] Specifically, by using the request pressure value of the previous period as the numerator and the first pressure data as the denominator, a ratio calculation is performed, and the result is the pressure trend ratio. The pressure trend ratio refers to the ratio of the total load pressure of the most recent historical period to the total load pressure of the two previous historical periods, used to quantitatively characterize the changing trend of business load pressure. The ratio calculation reflects the relative degree of change in recent load pressure, quantifies the changing trend of business load pressure, reflects recent pressure fluctuations, clearly shows the recent pattern of business load changes, and provides a basis for determining forecast values.

[0170] Step d4: Calculate the sum of the total resource utilization rates of the first two periods to obtain the first utilization rate data.

[0171] Specifically, the total resource utilization rate for the first two periods is extracted, and the sum of these two total resource utilization rates is the first utilization rate data. The first utilization rate data refers to the sum of resource utilization efficiency in the first two historical periods, used to characterize the cumulative state of recent resource utilization. By integrating resource utilization data from two historical periods, the first utilization rate data can be quickly calculated, providing reliable input for calculating resource trend ratios and supporting the quantitative analysis of resource utilization change trends.

[0172] Step d5: Compare the total resource utilization rate of the previous period with the first utilization rate data to obtain the resource trend ratio.

[0173] Specifically, the resource trend ratio is the ratio of the total resource utilization rate of the most recent historical period to that of the two preceding historical periods. It is used to quantitatively characterize the changing trend of resource utilization efficiency. By calculating this ratio, the relative degree of change in recent resource utilization efficiency is reflected, quantifying the trend of resource utilization efficiency changes and indicating recent fluctuations in resource utilization, thus providing trend support for subsequent forecasts at the resource level.

[0174] Step d6: Compare the pressure trend ratio with the resource trend ratio, and determine the maximum value as the predicted value for the target prediction period of the distributed model service.

[0175] Specifically, by comparing the pressure trend ratio and the resource trend ratio, the larger of the two ratios is selected as the predicted scaling-up / scaling value for the target forecast period. The predicted value is a quantitative indicator of the scaling-up / scaling adjustment range of the distributed model service within the target forecast period; a larger value indicates a greater adjustment is needed. By screening for more significant trends in business load or resource utilization, and integrating trend indicators from both pressure and resource dimensions, a comprehensive and accurate predicted value is obtained. This approach considers both business load and resource utilization as core dimensions, avoiding the one-sidedness of single-dimensional predictions and improving the accuracy and relevance of the forecasts.

[0176] For example, after the service is created, monitor the relevant information about the service:

[0177] Record the service within the first hour after its creation. and ;

[0178] Record the service within 2 hours after its creation. and .

[0179] Based on the above service data, the predicted trend of service request pressure and resource utilization in the third hour is as follows:

[0180]

[0181] Based on the prediction results, determine whether to perform scaling up or down operations. If no operation is performed, continue monitoring for the third hour after creation and record the service data. and .

[0182] Predict the service request pressure and resource utilization trend for the j-th hour using the method described above:

[0183]

[0184] Based on the prediction results, determine whether to scale up or down. If no action is taken, continue monitoring for the j-th hour after creation and record the service data. and .

[0185] This invention, through time interval positioning, data summation, ratio calculation, and extreme value selection, combined with the continuity and correlation of time series data, achieves accurate determination of the scaling prediction values ​​for distributed model services. It solves the problems of traditional predictions that only consider a single dimension, lack historical trend analysis, and have insufficient predictive value fit. It integrates the dual-dimensional trends of business load and resource utilization, improves the accuracy, pertinence, and rationality of scaling predictions, and provides a reliable predictive basis for subsequent scaling methods.

[0186] This embodiment provides a detailed explanation of the process in the above embodiments for determining the scaling-up / scaling strategy of the distributed model service by combining predicted values ​​and total resource utilization. The specific implementation of this process includes the following steps:

[0187] Step e1: Compare the predicted value with the preset expansion interval threshold and the preset shrinkage interval threshold respectively to obtain the expansion / shrinkage prediction results.

[0188] Specifically, the predicted values ​​for the target prediction period are extracted and compared with pre-set preset expansion and contraction thresholds. The expansion / contraction prediction results are obtained based on the comparison results. The preset expansion threshold is a range of predicted values ​​pre-set to determine whether expansion or contraction is necessary. The preset contraction threshold is a range of predicted values ​​pre-set to determine whether contraction is necessary. The expansion / contraction prediction results represent a preliminary conclusion that the service needs to be expanded, contracted, or maintained at its current size. By matching the predicted values ​​with the preset threshold ranges, a preliminary prediction of the expansion / contraction direction is achieved, providing basic guidance for determining the final strategy based on resource status.

[0189] Step e2: Compare the total resource utilization rate corresponding to the previous period with the preset threshold values ​​of multiple resource intervals to obtain the interval to which the total resource utilization rate belongs.

[0190] Specifically, the total resource utilization rate value corresponding to the previous period is extracted. This value is then compared with multiple pre-set resource interval thresholds one by one to determine the specific interval in which the total resource utilization rate falls, thus obtaining the interval to which the total resource utilization rate belongs. The multiple pre-set resource interval thresholds refer to the numerical boundary points that are pre-divided to characterize different resource utilization states. The interval to which the total resource utilization rate belongs refers to the interval range corresponding to the current resource utilization state, which is used to quantitatively characterize the degree of resource utilization tension.

[0191] By comparing numerical values ​​with boundary points, the range of resource utilization status is determined, the actual state of current resource utilization is clarified, the range of resource utilization status is accurately located, the one-sidedness of a single threshold judgment is avoided, and the comprehensiveness of resource status judgment is ensured.

[0192] Step e3: Match the combination of the expansion / shrinkage prediction results and the interval to which the total resource utilization rate belongs with the preset expansion / shrinkage operation type to obtain the expansion / shrinkage strategy.

[0193] Specifically, the scaling prediction results and the range of total resource utilization are integrated to form a combined condition. This combined condition is then matched with predefined preset scaling operation types using rule matching. The preset scaling operation type that matches successfully becomes the final scaling strategy. The preset scaling operation type refers to the predefined specific scaling operation scheme corresponding to different combinations of prediction results and resource ranges; the scaling strategy refers to the specific operation instructions used to guide the adjustment of the scale of the distributed model service, clarifying the direction and magnitude of the adjustment.

[0194] By precisely matching combined conditions with operation types, the system achieves automatic strategy matching, integrates predicted trends with current resource status, determines the final scaling up / down operation plan that fits the actual scenario, and obtains accurate and reasonable scaling up / down strategies. This approach takes into account both business load prediction trends and current resource utilization status, avoids the disconnect between strategies and actual scenarios, and improves the scientificity and feasibility of scaling up / down operations.

[0195] This invention achieves precise matching of scaling-up and scaling-down strategies through data comparison, interval comparison, rule matching, preset thresholds, and matching rules. It integrates business load prediction trends with current resource utilization status, improving the pertinence, scientificity, and feasibility of scaling-up and scaling-down strategies, and providing clear and reliable operational guidance for adjusting the scale of distributed model services.

[0196] In one specific embodiment, step e3 above specifically includes:

[0197] Step e31: If the expansion / shrinkage prediction result is within the preset expansion range threshold and the total resource utilization is within the first resource range threshold, the corresponding expansion / shrinkage operation type is to adjust the number of master-slave working set replicas.

[0198] Specifically, it determines whether the scaling prediction result falls within the preset scaling interval threshold, and simultaneously determines whether the total resource utilization falls within the first resource interval threshold. If both conditions are met, the corresponding scaling operation type is matched as adjusting the number of master-slave working set replicas. For example... Figure 3 As shown, Figure 3 This diagram illustrates the scaling up and down of the master-slave working set by adjusting the number of replicas, as provided in this embodiment. The left side shows a single master-slave working set before adjustment, containing one master service and several worker node containers. The right side shows multiple master-slave working set replicas after adjustment, each replica containing one master service and several worker node containers. Increasing the number of master-slave working set replicas expands the service and improves overall processing capacity; decreasing the number of master-slave working set replicas shrinks the service and releases redundant resources.

[0199] The master-slave working set is a set of services that undertake master-slave collaborative tasks in a distributed service; the number of replicas refers to the number of service replicas in the set; adjusting the number of replicas in the master-slave working set means increasing or decreasing the number of service replicas in the set to improve or decrease the service processing capacity; the first resource interval threshold refers to a preset interval with low resource utilization, indicating that there is still sufficient resource redundancy.

[0200] For expansion scenarios with sufficient resource redundancy, a lightweight adjustment approach is adopted to achieve expansion, avoiding resource waste while rapidly improving service processing capabilities to adapt to the growing business load.

[0201] Step e32: If the expansion / shrinkage prediction result is within the preset expansion range threshold and the total resource utilization rate is within the second resource range threshold, the corresponding expansion / shrinkage operation type is to adjust the number of worker node containers.

[0202] Specifically, when both conditions are met—that the predicted scaling-up / scaling result falls within a preset scaling-up range threshold and the total resource utilization falls within a second resource range threshold—the corresponding scaling-up / scaling operation type is adjusted to increase or decrease the number of container instances on worker nodes. Worker nodes are nodes in a distributed service that undertake specific business processing tasks; containers are lightweight virtualization technologies used to encapsulate the service runtime environment; worker node containers are container instances deployed on worker nodes, and adjusting their number refers to increasing or decreasing the number of container instances on worker nodes; the second resource range threshold refers to a preset range of moderate resource utilization, indicating that resources have some redundancy but require moderate scaling. Figure 4 As shown, Figure 4 This diagram illustrates the scaling up and down process by adjusting the number of worker node containers, as provided in this embodiment. The left side shows the master-slave working set before adjustment, containing one master service and one worker node container; the right side shows the master-slave working set after adjustment, containing one master service and multiple worker node containers. Service scaling up is achieved by increasing the number of worker node containers within the same master-slave working set, improving the processing capacity of a single group; service scaling down is achieved by reducing the number of worker node containers within the same master-slave working set, optimizing resource allocation.

[0203] For scaling scenarios with moderate resource redundancy, moderate scaling can be achieved by adjusting the number of containers, balancing service processing capacity and resource consumption, ensuring the scaling effect while avoiding over-scaling.

[0204] Step e33: If the scaling prediction result is within the preset scaling range threshold and the total resource utilization is within the third resource range threshold, the corresponding scaling operation type is to adjust the resource request value of the worker node container.

[0205] Specifically, if the scaling prediction result falls within a preset scaling range threshold and the total resource utilization falls within a third resource range threshold, then the corresponding scaling operation type is adjusted to adjust the container resource request value of the worker node. The container resource request value refers to the amount of resources requested by the container from the underlying environment during runtime to ensure the normal operation of the container. Adjusting this value means increasing the container's resource request amount to improve the processing capacity of a single container. The third resource range threshold refers to a preset range with relatively high resource utilization, indicating less resource redundancy.

[0206] For scaling scenarios with insufficient resource redundancy, the existing resources can be fully utilized to improve service performance by finely adjusting the container resource request value, without increasing the number of nodes or containers, thus saving resource costs.

[0207] Step e34: If the expansion / shrinkage prediction result is within the preset shrinkage interval threshold and the total resource utilization is within the fourth resource interval threshold, the corresponding expansion / shrinkage operation type is to adjust the number of master-slave working set replicas.

[0208] Specifically, if the scaling-up / scaling prediction result falls within a preset scaling-down threshold and the total resource utilization falls within a fourth resource interval threshold, then the corresponding scaling-up / scaling operation type is adjusted to reduce the number of replicas in the master-slave working set. The fourth resource interval threshold refers to a preset interval with extremely low resource utilization, indicating severe resource redundancy. Adjusting the number of replicas in the master-slave working set specifically refers to reducing the number of replicas to lower resource consumption.

[0209] For scenarios involving scaling down and severe resource redundancy, reducing the number of replicas can quickly release redundant resources, thereby reducing resource waste without affecting the stability of core service functions.

[0210] Step e35: If the expansion / shrinkage prediction result is within the preset shrinkage interval threshold and the total resource utilization is within the fifth resource interval threshold, the corresponding expansion / shrinkage operation type is to adjust the number of worker node containers.

[0211] Specifically, if the scaling-up / scaling prediction result falls within a preset scaling-down interval threshold and the total resource utilization falls within the fifth resource interval threshold, then the corresponding scaling-up / scaling operation type is adjusted to increase the number of worker node containers. The fifth resource interval threshold refers to a preset interval with low resource utilization, indicating that there is some resource redundancy. Adjusting the number of worker node containers specifically refers to reducing the number of container instances to release the resources occupied by redundant containers.

[0212] For scenarios involving scaling down and where resources are somewhat redundant, reducing the number of containers releases redundant resources, optimizes resource allocation, avoids resource idleness, and ensures that service processing capacity is adapted to the current load.

[0213] Step e36: If the scaling up / down prediction result is within the preset scaling down interval threshold and the total resource utilization is within the sixth resource interval threshold, the corresponding scaling up / down operation type is to adjust the resource request value of the worker node container.

[0214] Specifically, if the scaling-up / scaling prediction result falls within a preset scaling-down threshold and the total resource utilization falls within the sixth resource interval threshold, and both conditions are met simultaneously, the corresponding scaling-up / scaling operation type is adjusted to adjust the resource request value of the worker node container. The sixth resource interval threshold refers to a preset interval with moderate resource utilization, indicating relatively low resource redundancy. Adjusting the container resource request value here specifically refers to reducing the resource request amount of the container and releasing some redundant resources.

[0215] For scenarios involving scaling down and limited resource redundancy, the container resource request values ​​can be finely adjusted to maximize the release of redundant resources, optimize resource utilization efficiency, and avoid affecting service stability due to excessive scaling down.

[0216] Step e37: If the expansion / shrinkage prediction result is not within the preset expansion range threshold and the preset shrinkage range threshold, it is determined that no expansion / shrinkage operation will be performed, forming a no-operation expansion / shrinkage strategy.

[0217] Specifically, the system determines whether the predicted scaling up or down falls within a preset scaling up or down threshold range. If neither falls within the threshold, it determines that no scaling up or down operation is needed, resulting in a no-operation scaling up / down strategy. This strategy maintains the current service scale and resource configuration, suitable for scenarios with relatively stable business load and resource utilization. For scenarios without significant scaling up or down requirements, maintaining the current service operation avoids unnecessary adjustments that could cause service fluctuations, while conserving system processing resources and ensuring stable service operation.

[0218] Let's illustrate this with a specific example: based on the prediction results and We jointly determine whether the i-th service should perform a scaling up or down operation, and if so, which type of scaling up or down operation.

[0219] and Expanding capacity involves increasing the number of replicas by one.

[0220] and Expanding capacity involves changing the number of worker node containers, incrementing the number of worker node containers by one.

[0221] and Scaling up is achieved by changing the requested resource value of a worker node container, resulting in the processor and memory values ​​becoming 1.3 times the current value;

[0222] and Scaling down is achieved by varying the number of replicas, reducing the number of replicas by one.

[0223] and Shrink the size by changing the number of worker node containers; the number of worker node containers is reduced by one.

[0224] and Scaling up is achieved by changing the requested resource value of a worker node container, with the processor and memory values ​​becoming 0.8 times the current value;

[0225] In other cases, no expansion or contraction operations are performed.

[0226] Based on the prediction of the j-th hour and the (j-1)th hour The joint judgment mechanism determines whether the service needs to be scaled up or down, and if so, how. It can make the most of existing resource information, adopt different scaling methods for different results, and meet service requirements with the least amount of resources and the shortest scaling time.

[0227] This invention, through its embodiments, determines the combination scenarios of scaling prediction results with thresholds for different resource ranges, matches corresponding refined scaling operation types, and identifies scenarios with no operation. This achieves scenario-based and refined matching of scaling strategies, solving the problems of traditional scaling strategies being simplistic, poorly adapted to different resource states and load scenarios, and prone to over- or under-adjustment. Furthermore, by matching appropriate scaling operations based on different load predictions and resource utilization scenarios, it effectively improves the accuracy and feasibility of scaling strategies, optimizes resource allocation efficiency, and ensures the stable operation of distributed model services.

[0228] This embodiment provides a detailed description of the process for scaling up or down the distributed model service based on independence and scaling strategies in the above embodiments. The specific implementation of this process includes the following steps:

[0229] Step f1: Sort the multiple service groups that need to be expanded and adjusted in descending order of their independence to obtain the service group sequence.

[0230] Specifically, all service groups requiring capacity expansion are identified, the independence value of the entry service for each service group is extracted, and the service groups are sorted and arranged in descending order of independence value to obtain an ordered sequence of service groups. The difference in the house price independence value determines the ordered arrangement of service groups, establishes the expansion priority of service groups, clarifies the expansion execution order, and ensures that core service groups receive expansion resources first.

[0231] Step f2: Remove duplicate service groups from the service group sequence to obtain the expanded execution queue.

[0232] Specifically, the service group sequence is traversed and checked. For example, hash deduplication (HD) can be used to identify and remove duplicate service groups in the sequence. The deduplicated service groups are then rearranged in their original order to obtain the expansion execution queue. Duplicate service groups refer to the same service group that repeatedly enters the sequence due to data association errors or other reasons. Identifying and removing duplicates using unique identifiers avoids performing duplicate expansion operations on the same service group, saving system resources and expansion costs.

[0233] Step f3 sets a preset initial parallelism for capacity expansion adjustment and obtains the parallelism value.

[0234] Specifically, based on the current resource redundancy status and expansion needs of the distributed cluster, a preset initial parallelism for expansion adjustments is configured and set in advance to obtain a specific parallelism value. Parallelism refers to the number of service groups that can execute expansion operations in parallel within the same timeframe. The preset initial parallelism is the initial number of parallel executions pre-set based on the cluster resource status, used to balance expansion efficiency and resource consumption. By setting initial parallel parameters in conjunction with the cluster resource carrying capacity, the initial number of parallel executions for expansion is determined, improving expansion efficiency while avoiding resource overload and ensuring the overall stability of the cluster.

[0235] Step f4: Adjust the capacity according to the expansion execution queue and expansion / shrinkage strategy, and collect cluster resource utilization data in real time.

[0236] Specifically, following the priority order of the expansion execution queue and combining it with the determined scaling-down strategy, expansion operations are executed sequentially on the service groups in the queue. Simultaneously, data acquisition technology is used to collect real-time cluster resource utilization data. Cluster resource utilization refers to the overall resource utilization efficiency of the distributed cluster. The real-time collection process continuously acquires resource utilization data at a preset frequency to ensure data timeliness. By simultaneously advancing expansion operations and resource monitoring, the system aims to execute planned expansion adjustments while simultaneously monitoring the cluster resource status in real time, promptly identifying abnormal resource usage, and providing reliable data for dynamically optimizing the expansion process.

[0237] Step f5: Dynamically adjust the parallelism value based on the cluster resource utilization data, and perform subsequent expansion adjustments according to the adjusted parallelism value until the expansion adjustment of all service groups in the expansion execution queue is completed.

[0238] Specifically, by dynamically optimizing the parallelism parameters through real-time resource data, the scaling process is adaptively adjusted, dynamically balancing scaling efficiency and cluster resource consumption. This ensures that cluster resources are within a reasonable range during the scaling process, avoiding resource overload or idleness, ensuring efficient and stable scaling operations, and guaranteeing the overall stable operation of the distributed cluster.

[0239] For example, scaling up the i-th service actually involves scaling up or down the i-th service group as a whole, S i The service being called is the entry point service.

[0240] This indicates that the entry service for the first group is... The second group's entry service is The third group's entrance service is After calculating the independence of the entry service, duplicate services are removed. The final expansion order becomes: .

[0241] The parallelism setting is used to control the concurrency level during scaling. It can be customized; the default value is 1.

[0242] The first round of expansion operations, initial settings That is, expansion For each service, cluster resource utilization includes cluster processor utilization and cluster memory utilization; in the next round of expansion, if both the cluster processor utilization and cluster memory utilization in the previous round are less than 0.65, then the concurrency level is equal to... If the cluster processor utilization and cluster memory utilization in the previous round were both greater than 0.64 and less than 0.85, then the concurrency level is equal to... If the cluster processor utilization or cluster memory utilization in the previous round was greater than or equal to 0.85, then the concurrency level is equal to... Until all services have been expanded.

[0243] This invention achieves orderly, efficient, and stable execution of distributed model service expansion and adjustment through sorting, deduplication, parallel computing, real-time data acquisition, and feedback control. It solves the problems of low expansion efficiency, resource imbalance, and service instability caused by chaotic priorities, duplicate operations, and fixed parallelism in traditional expansion processes. It clarifies expansion priorities, avoids duplicate operations, and dynamically optimizes parallelism, thereby improving expansion efficiency and resource utilization, while ensuring the stable operation of the distributed model service and the overall cluster.

[0244] In one specific embodiment, step f5 above specifically includes:

[0245] Step f51: Compare the cluster resource utilization data with the preset resource threshold to obtain the cluster resource load status.

[0246] Specifically, real-time cluster resource utilization data is extracted and compared one by one with pre-set preset resource thresholds. The current cluster resource load status is determined based on the comparison results. The preset resource thresholds are pre-defined numerical boundary points used to judge the degree of cluster resource load. The cluster resource load status is a description of the current resource occupancy tension of the cluster, which is divided into three categories: resource availability, moderate resource availability, and resource tension.

[0247] By matching numerical values ​​with boundary points, the resource load level is determined, the current resource carrying capacity of the cluster is clarified, and the parallelism adjustment is made in line with the actual resource situation to avoid blind adjustment.

[0248] Step f52: Based on the cluster resource load status, the current parallelism value is amplified, maintained, or reduced according to preset adjustment rules.

[0249] Specifically, based on the cluster resource load status, preset adjustment rules are retrieved, and the current parallelism value is processed accordingly. If resources are plentiful, the parallelism is increased; if resources are moderate, the parallelism is maintained; and if resources are scarce, the parallelism is decreased. Parallelism refers to the number of service groups that can perform expansion operations in parallel at the same time. The preset adjustment rules are predefined parallelism adjustment schemes corresponding to different resource load statuses, including the increase / decrease ratio, the maintenance condition, and so on. Based on feedback control combined with preset rules, the parallelism is adaptively adjusted, dynamically optimizing the parallelism value, balancing expansion efficiency and cluster resource consumption, ensuring that the parallelism always adapts to the cluster resource status, avoiding resource idleness or resource overload, and ensuring that the expansion process is efficient and stable.

[0250] Step f53: Use the adjusted parallelism value as the execution parallelism for the next round of expansion adjustment, and execute the expansion adjustment and cluster resource utilization data collection steps in the order of the expansion execution queue until all service groups in the expansion execution queue have completed the expansion adjustment.

[0251] Specifically, the adjusted parallelism value is determined as the execution parallelism for the next round of expansion. According to the priority order of the expansion execution queue, the expansion adjustment operation is executed, while continuously collecting cluster resource utilization data. The process from step f51 to this step is repeated, continuously and dynamically adjusting the parallelism, executing expansion, and collecting resource data until all service groups to be expanded in the expansion execution queue have completed the expansion adjustment.

[0252] The scaling process is continuously optimized through iterative iterations and real-time feedback. Its purpose is to ensure that scaling operations continuously adapt to changes in cluster resources, driving the complete implementation of scaling tasks. The effect is to achieve closed-loop management of the scaling process, ensuring efficient completion of scaling tasks while maintaining cluster resources within a reasonable range, avoiding service fluctuations or resource anomalies during scaling.

[0253] This invention achieves real-time adaptive adjustment of the parallelism of expansion and closed-loop advancement of expansion tasks through data comparison, feedback control, and iterative iteration. It dynamically optimizes the parallelism of expansion, improves expansion efficiency, maintains the stability of cluster resources, ensures that expansion tasks are completed in an orderly, complete, and efficient manner, and guarantees the stable operation of distributed model services.

[0254] This embodiment provides a detailed description of the process for scaling up or down the distributed model service based on independence and scaling strategies in the above embodiments. The specific implementation of this process includes the following steps:

[0255] Step g1: Integrate the multiple service groups that need to be scaled down to obtain a service group set.

[0256] Specifically, all service groups requiring scaling down are identified, aggregated, and organized into a unified service group set. This service group set stores all service groups awaiting scaling down, enabling unified management of scaling-down objects. The selection criteria are the service group identifiers explicitly defined in the scaling-down strategy that require scaling down operations. By integrating the scattered service groups awaiting scaling down into a unified whole, and clarifying the specific objects of the scaling-down operation, centralized management of these service groups is achieved. This avoids omissions or confusion, ensuring that all service groups requiring scaling down are included in the processing scope, and improving the standardization and orderliness of the scaling-down operation.

[0257] Step g2: Sort the service groups in the service group set from largest to smallest according to their independence to obtain the shrinking execution sequence.

[0258] Specifically, the independence value of the entry service of each service group in the service group set is extracted, and the service groups are sorted in descending order of independence value to obtain an ordered scaling-down execution sequence. By determining the scaling-down priority order, it is ensured that core service groups are protected first, avoiding service malfunctions caused by prioritizing the scaling down of core service groups.

[0259] Step g3: Perform scaling down operations synchronously on the service groups in the scaling down execution sequence according to the preset parallel execution rules and scaling up / down strategies, and monitor the execution progress of the scaling down operations in real time.

[0260] Specifically, scaling down is performed synchronously on service groups within the sequence according to their priority. Simultaneously, real-time progress data is collected to monitor the completion status of each service group. Pre-defined parallel execution rules regulate the number of service groups and execution conditions for synchronous scaling down, balancing efficiency and service stability. The scaling up / down strategy specifies the exact scaling down operation method for each service group. By synchronously executing and controlling the progress of scaling down operations, efficiency is improved. Real-time monitoring of progress allows for timely detection of anomalies, accelerating the process while ensuring service stability and preventing uncontrolled progress or operational errors, thus ensuring the orderly progress of scaling down.

[0261] Step g4: After the scaling down operation is completed, obtain the running status data of the service group, and verify whether the corresponding distributed model service is running normally based on the running status data.

[0262] Specifically, after the scaling down operation of each service group is completed, the operational status data of that service group is acquired in real time. By comparing the operational status data with the preset normal operation threshold, the normal operation of the corresponding distributed model service is verified. The operational status data refers to the core indicator data that characterizes the operational status of the service group, including service response speed, service availability, etc. The preset normal operation threshold is a pre-set range of indicators used to determine whether the service is operating normally. The verification logic is that if the operational status data falls within the preset range, the service is determined to be operating normally; otherwise, it is determined to be abnormal.

[0263] By comparing operational metrics with preset thresholds, the service operation status after the scaling-down operation is verified, the effectiveness of the scaling-down operation is checked, potential service operation anomalies caused by the scaling-down operation are investigated, and it is ensured that the service can provide normal functionality after scaling-down.

[0264] This invention achieves orderly, efficient, and secure execution of distributed model service scaling-down adjustments through data integration, sorting, parallel computing, real-time monitoring, and status verification. It standardizes the scaling-down operation process, protects core service groups, and improves scaling-down efficiency, while ensuring the stable operation of the distributed model service after scaling-down and optimizing resource allocation efficiency.

[0265] This embodiment provides a detailed description of the process for monitoring the expansion and contraction status in the above embodiments. The specific implementation of this process includes the following steps:

[0266] Step h1: When the distributed model service is scaling up or down, update the running status of the corresponding distributed model service in the preset database to "scaling up or down" and monitor the scaling up or down process in real time.

[0267] Specifically, when the distributed model service initiates a scaling operation, the corresponding running status field in the preset database is updated to "Scaling / Scaling in Progress." Simultaneously, execution parameters and progress data of the scaling operation are continuously collected in real-time, enabling full monitoring of the entire process. The running status refers to an identifier representing the current stage of service operation, including running, scaling in progress, abnormal, and failed scaling operation. Real-time monitoring refers to continuously acquiring operation process data at a preset frequency to ensure controllability throughout the entire operation. By synchronously updating the service status and controlling the entire operation process, the current scaling phase of the service is clearly marked, and the real-time progress and execution status of the scaling operation are monitored, achieving full monitoring of the scaling operation and clear traceability of the service status, preventing the operation from spiraling out of control or status identification from becoming chaotic.

[0268] Step h2: After the scaling up and down operation is detected to be completed, collect the running status data of the distributed model service and verify the running status data.

[0269] Specifically, after real-time monitoring confirms the completion of scaling operations, the operational status data of the distributed model service is extracted and compared with preset normal operation thresholds to complete data verification. Operational status data refers to core indicators characterizing service stability and availability, including service response time, service availability, and resource utilization. The preset normal operation thresholds are pre-defined ranges of indicators used to determine whether a service is operating normally. The verification logic is that if the operational status data falls within this range, it is considered normal; otherwise, it is considered abnormal. By verifying the service's operational status through core indicators, the actual operational status of the service after the scaling operation is confirmed.

[0270] Step h3: If the verification result is normal operation, update the status of the distributed model service in the preset database to "running".

[0271] Specifically, when the verification result indicates that the service is running normally, the running status field corresponding to the distributed model service in the preset database is updated to "running" from the scaling up / down view. By synchronizing the service status with the database record, it is ensured that the database identifier is consistent with the actual running status of the service, facilitating subsequent unified management and status query of the service.

[0272] Step h4: If the verification result is abnormal operation, then update the running status of the distributed model service in the preset database to abnormal.

[0273] Specifically, when the verification result indicates that the service is not operating normally, the running status field corresponding to the distributed model service in the preset database is updated to "abnormal" from the scaling up / down process. An abnormal status refers to a status indicator that the service's operating indicators have not reached the preset normal threshold, cannot provide services normally, or is unstable. By synchronizing the actual abnormal status of the service to the preset database, anomalies after service scaling up / down are marked in a timely manner, reminding administrators to conduct troubleshooting and handling, avoiding delays in fault handling due to incorrect status indicators, and providing clear status guidance for fault troubleshooting.

[0274] Step h5: When a scaling-up / scaling-down operation failure is detected, the running status of the distributed model service is updated to "scaling-up / scaling-down operation failed" and a service rollback operation is triggered to restore the distributed model service to its configuration state before the scaling-up / scaling-down operation.

[0275] Specifically, when a scaling operation fails to execute (such as an operation interruption or parameter error), the service running status is updated to indicate that the scaling operation failed through database update technology. Subsequently, a service rollback operation is triggered. By retrieving the service configuration information stored in the preset database before the scaling operation, the various configurations of the distributed model service are restored to the state before the operation.

[0276] Service rollback refers to a fault recovery operation that restores a service's configuration, operating parameters, and other settings to their pre-operation normal state. Configuration state refers to the collection of configuration information, such as parameters and resource allocation, required for service operation. By using configuration rollback to restore service state, failed scaling operations can be quickly terminated, restoring the service to its normal configuration. This reduces the impact of faults on service operation, minimizes losses from failed scaling operations, avoids prolonged unavailability or instability, and ensures service continuity.

[0277] Step h6 verifies the running status of the distributed model service after the rollback operation.

[0278] Specifically, after the service rollback operation is completed, the runtime status data of the distributed model service is re-collected. The collected data is compared with the preset normal operation threshold to verify the runtime status. This confirms whether the service rollback operation is effective, determines whether the service has recovered to a normal operating state, verifies the rollback effect, and avoids any undetected anomalies after the rollback.

[0279] Step h7: If the distributed model service fails to resume normal operation, a fault feedback log is generated.

[0280] Specifically, when a service fails to return to normal operation, a fault feedback log is generated by collecting detailed information related to the service failure (including failure time, failure symptoms, rollback operation details, etc.). The fault feedback log is a text or data file used to record service failure-related information to facilitate troubleshooting by administrators; it contains key characteristics of the failure and the processing trajectory. By recording and retaining fault-related information through logs, data support is provided for fault investigation.

[0281] Step h8: If the distributed model service resumes normal operation, update the running status of the distributed model service in the preset database to "running".

[0282] Specifically, when a service has returned to normal operation, the running status field corresponding to the service in the preset database will be updated from "failed to scale up / down" to "running"; this clearly marks that the service has returned to normal through a rollback operation, facilitating subsequent service management and status queries.

[0283] Step h9: Continuously collect resource data and business data of all running distributed model services and update them to the preset database.

[0284] Specifically, resource and business data of all running distributed model services are continuously collected at a preset sampling frequency. Data synchronization updates the latest collected data to a preset database in real time, overwriting existing historical data. Resource data refers to data related to the hardware and software resources used during service operation; business data refers to data related to business requests processed by the service; data synchronization refers to the technique of synchronizing and updating the collected real-time data with the data stored in the database to ensure data consistency. By continuously updating service operation data, the service operation data stored in the preset database is ensured to be real-time and accurate, providing reliable data assurance for the full lifecycle management of distributed model services.

[0285] This invention achieves full-process status control, fault handling, and data retention for distributed model service scaling through database operations, real-time monitoring, status verification, service rollback, and data synchronization. It solves the problems of incomplete scaling monitoring, untimely fault handling, chaotic status identification, and asynchronous data that lead to inefficient service management and amplified fault losses in traditional scaling methods. It enables the entire scaling process to be monitored, traceable, and recoverable, timely handling of faults and reduction of losses, ensuring service stability, and providing real-time and accurate data support for subsequent service management and decision-making.

[0286] This embodiment describes the process of scaling up or down the distributed model service according to the independence and scaling-up / scaling strategies in the above embodiments. The specific implementation of this process includes the following steps:

[0287] Step j1: Obtain resource data and business data within a preset time range.

[0288] Specifically, resource and business data are collected within a preset time range after the scaling-up / scaling adjustment, at a preset sampling frequency, and then preliminarily filtered to remove invalid data. The preset time range refers to a fixed time interval used to verify the adjustment effect after the scaling-up / scaling adjustment, ensuring that the collected data reflects the stable operation status of the service after the adjustment. By obtaining valid data that can characterize the service status after the adjustment, data input is provided for subsequent verification of the scaling-up / scaling adjustment effect, ensuring the timeliness and reliability of the verification basis.

[0289] Step j2: Based on the collected resource data and business data, calculate the real-time request pressure value and real-time total resource utilization rate within the preset time period.

[0290] Specifically, the collected business data undergoes nonlinear mapping and normalization to eliminate the influence of dimensions before being superimposed to obtain the real-time request pressure value; the resource data is filtered and corrected, and then the real-time total resource utilization rate is obtained through weighted summation.

[0291] Real-time request load refers to the quantified value of the real-time business request load that the service can withstand after scaling up or down; real-time total resource utilization refers to the quantified value of the real-time resource utilization efficiency of the service after the adjustment. By transforming raw data into verifiable quantitative indicators, the adjusted business load and resource utilization status are accurately quantified, providing a clear basis for subsequent effect verification.

[0292] Step j3: Determine whether the real-time request pressure value is within the preset request pressure threshold range and whether the real-time total resource utilization rate is within the preset total resource utilization rate threshold range.

[0293] Specifically, the calculated real-time request pressure value is compared with the preset request pressure threshold, and the real-time total resource utilization rate is compared with the preset total resource utilization rate threshold to determine whether both indicators fall within their respective preset threshold ranges. The preset request pressure threshold refers to the reasonable fluctuation range of business request load when the service is running normally, and the preset total resource utilization rate threshold refers to the reasonable range of resource utilization efficiency when the service is running stably. By verifying the adjustment effect through dual indicators, the system verifies whether the scaling-up and scaling-down adjustments have achieved the expected results and determines whether the service is in a stable operating state.

[0294] Step j4: If both the real-time request pressure value and the real-time total resource utilization rate are within the corresponding preset threshold range, then the scaling up / down adjustment is confirmed to be complete.

[0295] Specifically, when both core indicators fall within their corresponding preset threshold ranges, the time of completion of the scaling up / down adjustment, the adjustment parameters, and the verification results are recorded, and the service running status is updated to stable operation. Completion of the scaling up / down adjustment means that after adjustment, the service's workload and resource configuration are adapted, enabling stable operation and meeting business needs. Based on the dual indicator compliance verification confirming that the adjustment effect meets expectations, the scaling up / down verification and iteration process is terminated, confirming that the adjustment has achieved its goal, completing the entire scaling up / down process loop, ensuring the service enters a stable operation phase, avoiding ineffective subsequent adjustment operations, and saving system resources.

[0296] Step j5: If the real-time request pressure value is not within the preset request pressure threshold range or the real-time total resource utilization is not within the corresponding preset threshold range, then adjust the parameters of the scaling-up / scaling strategy according to the deviation value between the indicator and the corresponding preset threshold, and return to the step of scaling-up / scaling adjustment of the distributed model service according to the independence and scaling-up / scaling strategy, until the real-time request pressure value is within the preset request pressure threshold range and the real-time total resource utilization is within the preset total resource utilization threshold range.

[0297] Specifically, the deviation value between the unmet indicator and the corresponding preset threshold is calculated. The deviation value is the difference between the actual value of the indicator and the preset threshold, which is used to characterize the degree of deviation from the reasonable range. According to the magnitude and direction of the deviation value, the relevant parameters of the scaling-up and scaling-down strategy (such as the scaling-up range, scaling-down ratio, parallelism, etc.) are adjusted. After the adjustment is completed, the process of scaling-up and scaling-down adjustment, data collection, indicator calculation and threshold judgment is repeated until both core indicators meet the standards.

[0298] By correcting deviations in scaling up and down adjustments through feedback iteration and parameter optimization, we ensure that the adjustment effect ultimately meets the target, avoid service anomalies caused by inaccurate adjustments in a single instance, guarantee long-term stable service operation, and improve resource allocation efficiency.

[0299] This invention achieves comprehensive verification and precise optimization of scaling-up and scaling-down adjustment effects through data collection, quantitative calculation, threshold comparison, and feedback iteration. It effectively avoids problems such as lack of effect verification after scaling-up and scaling-down adjustments, inaccurate single adjustments, and inability to adapt to dynamic load changes. This ensures that the scaling-up and scaling-down adjustment effects meet the standards, achieves optimal adaptation of business load and resource configuration, and guarantees the long-term stable operation of distributed model services.

[0300] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method.

[0301] Figure 5 This is a schematic diagram of the structure of the distributed model service scaling system provided in an embodiment of this application. Figure 5 As shown, embodiments of this application also provide a distributed model service scaling system 50.

[0302] The data acquisition module 501 is used to acquire the basic dataset of service metrics for multiple distributed model services. The basic dataset of service metrics includes basic data, resource data, and business data.

[0303] Analysis module 502 is used to analyze the calling relationships between distributed model services based on the service metric dataset;

[0304] Module 503 is used to build multiple service groups based on the call relationship and calculate the independence of the entry service corresponding to the service group.

[0305] The calculation module 504 is used to calculate the request pressure value and total resource utilization rate of the distributed model service based on resource data and business data of a preset historical period.

[0306] Prediction module 505 is used to predict the scaling up and down of the distributed model service for the target prediction period based on the request pressure value and the total resource utilization rate, and obtain the predicted value.

[0307] Processing module 506 is used to determine the scaling up and down strategy of the distributed model service by combining the predicted value and the total resource utilization rate.

[0308] Adjustment module 507 is used to adjust the scale of the distributed model service according to the degree of independence and scaling strategy.

[0309] For a description of the features in the embodiment corresponding to the distributed model service scaling system, please refer to the relevant description of the embodiment corresponding to the distributed model service scaling method, which will not be repeated here.

[0310] Figure 6 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 6 As shown, the electronic device 60 provided in this embodiment includes at least one processor 601 and a memory 602. Optionally, the electronic device 60 further includes a communication component 603. The processor 601, memory 602, and communication component 603 are connected via a bus.

[0311] In the specific implementation process, at least one processor 601 executes computer execution instructions stored in memory 602, causing at least one processor 601 to execute the above-described distributed model service scale adjustment method embodiment.

[0312] The specific implementation process of processor 601 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0313] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in the application can be directly manifested as being executed by a hardware processor, or executed by a combination of hardware and software modules within the processor.

[0314] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0315] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0316] Embodiments of this application also provide a computer-readable storage medium storing a computer program, wherein the computer program is configured to execute the steps in any of the above embodiments of the distributed model service scaling method when running.

[0317] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard disk, magnetic disk, or optical disk.

[0318] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above embodiments of the distributed model service scaling method.

[0319] Embodiments of this application also provide another computer program product, including a non-volatile computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps in any of the above embodiments of the distributed model service scaling method.

[0320] Any of the components, modules, units, parts, methods, and operations described herein can be implemented using software, firmware, hardware (e.g., fixed logic circuitry), manual processing, or any combination thereof. Alternatively or additionally, any functionality described herein can be executed at least in part by one or more hardware logic components, such as, but not limited to, a central processing unit (CPU), a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), an application-specific standard product (ASSP), a system-on-a-chip (SoC), a complex programmable logic device (CPLD), a microprocessor (MCU), etc. The terms "system," "computing device," or "apparatus" as used herein encompass various means, devices, and machines for processing data, including, for example, one or more programmable processors, computers, SoCs, or combinations thereof. The apparatus may also include code that creates an execution environment for the computer program in question, such as code constituting processor firmware, a protocol stack, a database management system, an operating system, a cross-platform runtime environment, a virtual machine, or one or more combinations thereof. The aforementioned computer program (also known as a program, software, software application, app, script, or code) can be written in any form of programming language, including compiled or interpreted languages, declarative or procedural languages, and can be deployed in any form, including as a standalone program or as a module, component, subroutine, object, or other unit suitable for a computing environment.

[0321] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0322] The above provides a detailed description of the distributed model service scale adjustment method, system, device, medium, and product provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are only intended to help understand the method and core ideas of this application. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of this application.

Claims

1. A method for adjusting the scale of a distributed model service, characterized in that, include: Obtain the service metric base dataset for multiple distributed model services, wherein the service metric base dataset includes basic data, resource data, and business data; Analyze the calling relationships between the distributed model services based on the aforementioned service metric dataset; Multiple service groups are constructed based on the call relationships, and the independence of the entry service corresponding to each service group is calculated. The request pressure value and total resource utilization of the distributed model service are calculated based on resource data and business data from a preset historical period. Based on the requested pressure value and the total resource utilization, the scaling up and down of the distributed model service during the target prediction period is predicted to obtain the predicted value. The scaling strategy for the distributed model service is determined by combining the predicted value and the total resource utilization rate. The distributed model service is scaled up or down according to the independence and the scaling strategy.

2. The method for adjusting the scale of a distributed model service according to claim 1, characterized in that, Multiple service groups are constructed based on the call relationships between the distributed model services, and the independence of the entry service corresponding to each service group is calculated, including: Extract the call information of the distributed model service from the service metric base dataset; Based on the call information, multiple distributed model services that are directly associated with each other can be identified; The distributed model services that have call associations are integrated into a service group, and the called distributed model service is designated as the entry service of the corresponding service group; The independence value is obtained by analyzing the number of services directly invoked by the entry service.

3. The method for adjusting the scale of a distributed model service according to claim 2, characterized in that, The independence value is obtained by analyzing the number of services directly invoked by the entry service, including: Determine if the entry service has a call record; The independence of the entry service with no recorded calls is assigned a preset baseline value; The number of services directly invoked by the entry service with invocation records is summarized to obtain invocation data; The called data is input into a preset calculation model to obtain the corresponding independence value, and the independence value is less than the preset benchmark value.

4. The method for adjusting the scale of a distributed model service according to claim 1, characterized in that, The request pressure and total resource utilization of the distributed model service are calculated based on resource data and business data from a preset historical period, including: The business data is then subjected to nonlinear mapping and normalization, and then superimposed to obtain the request pressure value. The resource data is corrected by a preset correction function and then weighted to obtain the total resource utilization rate for each time period.

5. The method for adjusting the scale of a distributed model service according to claim 4, characterized in that, The business data is superimposed after undergoing nonlinear mapping and normalization processing, including: Extract the request queue length, effective concurrency, request query rate, and inference latency data for a preset historical period from the business data; The request queue length data is non-linearly mapped to obtain the mapped request queue length data; The effective concurrency, request query rate, and inference latency data are compared with their respective preset historical average values ​​to obtain intermediate data; The mapped request queue length data is superimposed with the intermediate data to obtain the request pressure value.

6. The method for adjusting the scale of a distributed model service according to claim 4, characterized in that, The resource data is corrected using a preset correction function and then weighted for calculation, including: The resource data is classified according to resource type to obtain multiple resource sub-data; Obtain the preset utilization threshold corresponding to the resource sub-data; The resource sub-data is compared with the corresponding preset utilization threshold to obtain the resource sub-data that exceeds the preset utilization threshold and needs to be corrected. The resource sub-data to be corrected is attenuated and corrected by a preset correction function to obtain the corrected resource sub-data. Assign preset weights to the corrected resource sub-data according to the resource type; The modified resource sub-data are weighted and summed according to the preset weights to obtain the total resource utilization rate.

7. The method for adjusting the scale of a distributed model service according to claim 1, characterized in that, Based on the requested pressure value and total resource utilization, the scaling up and down of the distributed model service during the target prediction period is predicted, including: Based on the time of the target prediction period, determine the preceding period and the two preceding periods adjacent to the target prediction period; The sum of the requested pressure values ​​in the first two time periods is calculated to obtain the first pressure data; The pressure value of the previous period is compared with the first pressure data to obtain the pressure trend ratio. The first utilization rate data is obtained by summing the total resource utilization rates of the first two time periods. The total resource utilization rate of the previous period is compared with the first utilization rate data to obtain the resource trend ratio. The pressure trend ratio is compared with the resource trend ratio, and the maximum value is determined as the predicted value for the target prediction period of the distributed model service.

8. The method for adjusting the scale of a distributed model service according to claim 7, characterized in that, Determining the scaling strategy for the distributed model service by combining the predicted value and the total resource utilization includes: The predicted values ​​are compared with the preset expansion interval threshold and the preset shrinkage interval threshold to obtain the expansion / shrinkage prediction results. The total resource utilization rate corresponding to the previous time period is compared with multiple preset resource interval thresholds to obtain the interval to which the total resource utilization rate belongs; The expansion / shrinkage strategy is obtained by matching the combination of the expansion / shrinkage prediction results and the interval to which the total resource utilization rate belongs with the preset expansion / shrinkage operation type.

9. The method for adjusting the scale of a distributed model service according to claim 8, characterized in that, Matching the combination of the expansion / shrinkage prediction results and the range to which the total resource utilization rate belongs with preset expansion / shrinkage operation types, including: If the expansion / shrinkage prediction result is within the preset expansion range threshold and the total resource utilization rate is within the first resource range threshold, the corresponding expansion / shrinkage operation type is to adjust the number of master-slave working set replicas. If the expansion / shrinkage prediction result is within the preset expansion range threshold and the total resource utilization rate is within the second resource range threshold, the corresponding expansion / shrinkage operation type is to adjust the number of working node containers. If the scaling up / down prediction result is within the preset scaling up range threshold and the total resource utilization is within the third resource range threshold, the corresponding scaling up / down operation type is to adjust the resource request value of the worker node container. If the expansion / shrinkage prediction result is within the preset shrinkage interval threshold and the total resource utilization rate is within the fourth resource interval threshold, the corresponding expansion / shrinkage operation type is to adjust the number of master-slave working set replicas. If the expansion / shrinkage prediction result is within the preset shrinkage interval threshold and the total resource utilization rate is within the fifth resource interval threshold, the corresponding expansion / shrinkage operation type is to adjust the number of working node containers. If the scaling up / down prediction result is within the preset scaling down interval threshold and the total resource utilization is within the sixth resource interval threshold, the corresponding scaling up / down operation type is to adjust the resource request value of the worker node container. If the expansion / shrinkage prediction result is not within the preset expansion range threshold or the preset shrinkage range threshold, it is determined that no expansion / shrinkage operation will be performed, thus forming a no-operation expansion / shrinkage strategy.

10. The method for adjusting the scale of a distributed model service according to claim 1, characterized in that, The distributed model service is scaled up or down according to the independence and the scaling strategy, including: The service groups that need to be expanded and adjusted are sorted from largest to smallest according to their degree of independence to obtain a service group sequence; The expanded execution queue is obtained by removing duplicate service groups from the service group sequence; A preset initial parallelism is set for the capacity expansion adjustment, and the parallelism value is obtained; The expansion and scaling adjustments are performed according to the expansion execution queue and the expansion and scaling strategy, and cluster resource utilization data is collected in real time. The parallelism value is dynamically adjusted based on the cluster resource utilization data, and subsequent expansion adjustments are performed according to the adjusted parallelism value until the expansion adjustments of all service groups in the expansion execution queue are completed.

11. The method for adjusting the scale of a distributed model service according to claim 10, characterized in that, Dynamically adjusting the parallelism value based on the cluster resource utilization data includes: The cluster resource utilization data is compared with a preset resource threshold to obtain the cluster resource load status. Based on the cluster resource load status, the current parallelism value is amplified, maintained, or reduced according to preset adjustment rules. The adjusted parallelism value will be used as the execution parallelism for the next round of expansion adjustment, and the steps of expansion adjustment and cluster resource utilization data collection will be executed in the order of the expansion execution queue until all service groups in the expansion execution queue have completed the expansion adjustment.

12. The method for adjusting the scale of a distributed model service according to claim 1, characterized in that, The distributed model service is scaled up or down according to the independence and the scaling strategy, including: The multiple service groups that need to be scaled down are integrated to obtain a service group set; The service groups in the service group set are sorted from largest to smallest according to their independence to obtain the shrinking execution sequence; According to the preset parallel execution rules and the scaling-up and scaling-down strategy, the service group in the scaling-down execution sequence is synchronously executed with scaling-down operation, and the execution progress of the scaling-down operation is monitored in real time. After the scaling-down operation is completed, obtain the running status data of the service group, and verify whether the corresponding distributed model service is running normally based on the running status data.

13. The method for adjusting the scale of a distributed model service according to claim 1, characterized in that, The method further includes: When the distributed model service is scaling up or down, the running status of the corresponding distributed model service in the preset database is updated to "scaling up or down", and the process of scaling up or down is monitored in real time. Once the scaling up / down operation is detected as complete, the running status data of the distributed model service is collected and verified. If the verification result is normal operation, the status of the distributed model service in the preset database will be updated to running. If the verification result is abnormal, the running status of the distributed model service in the preset database will be updated to abnormal. When the scaling up / down operation fails, the running status of the distributed model service is updated to "scaling up / down operation failed" and a service rollback operation is triggered to restore the distributed model service to its configuration state before the scaling up / down operation. Verify the running status of the distributed model service after the rollback operation is performed; If the distributed model service fails to resume normal operation, a fault feedback log will be generated. If the distributed model service resumes normal operation, the running status of the distributed model service in the preset database will be updated to "running". Continuously collect resource data and business data from all running distributed model services and update them to the preset database.

14. The method for adjusting the scale of a distributed model service according to claim 1, characterized in that, Obtain the basic dataset of service metrics for multiple distributed model services, including: Retrieve the basic data corresponding to the distributed model service from the preset database; The distributed model service obtains time-series data from multiple acquisition nodes in the underlying environment according to a preset sampling frequency. The time-series data includes resource data and business data. The basic data and the time-series data are associated by service identifiers to obtain a service indicator basic dataset containing basic data, resource data, and business data.

15. The method for adjusting the scale of a distributed model service according to claim 14, characterized in that, The distributed model service obtains time-series data from multiple acquisition nodes in the underlying environment according to a preset sampling frequency, including: The acquisition permissions of each acquisition node in the underlying environment are verified, and a data transmission link is established after the verification is successful. Based on a preset sampling frequency, the acquisition nodes are controlled to collect corresponding resource data and service data to obtain raw data; The original data is processed to remove outliers, resulting in the target data. The target data is sorted by timestamp to obtain time-series data.

16. The method for adjusting the scale of a distributed model service according to claim 15, characterized in that, The outlier removal process for the original data includes: Outlier detection is performed on the original data to obtain abnormal data points that exceed a preset value range; The abnormal data points are removed from the original data, and the missing timestamp positions are filled in. The completed raw data is categorized and stored according to preset indicator types to obtain the target data.

17. A distributed model service scaling system, characterized in that, include: The data acquisition module is used to acquire the basic dataset of service metrics for multiple distributed model services. The basic dataset of service metrics includes basic data, resource data, and business data. The analysis module is used to analyze the calling relationships between the distributed model services based on the service metric dataset. A construction module is used to construct multiple service groups based on the call relationship and calculate the independence of the entry service corresponding to the service group; The calculation module is used to calculate the request pressure value and total resource utilization rate of the distributed model service based on resource data and business data of a preset historical period. The prediction module is used to predict the scaling up and down of the distributed model service during the target prediction period based on the request pressure value and the total resource utilization rate, and obtain the predicted value. The processing module is used to determine the scaling strategy of the distributed model service by combining the predicted value and the total resource utilization rate. The adjustment module is used to adjust the scale of the distributed model service according to the independence and the scaling strategy.

18. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the distributed model service scaling method as described in any one of claims 1 to 16 when executing the computer program.

19. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, it implements the steps of the distributed model service scaling method as described in any one of claims 1 to 16.

20. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the distributed model service scaling method as described in any one of claims 1 to 16.

Images