A large model-based computing power resource planning method and system

Abstract

This invention discloses a computing resource planning method and system based on a large model. The computing resource planning method includes: constructing a structured performance database based on benchmark test documents; generating a set of configuration data points corresponding to each configuration node based on the performance database; generating optimal data points based on the configuration data point sets; obtaining the business target concurrency; traversing the optimal data points corresponding to the configuration nodes; generating at least one candidate deployment scheme that meets the business target concurrency; calculating the cost information of the candidate deployment schemes; and generating the target deployment scheme based on the cost information. Specifically, the optimal data point has a first token latency less than a preset latency time threshold, a single-user throughput greater than or equal to a preset throughput threshold, and the concurrency number is the maximum value in the configuration data point set. This invention achieves accurate prediction of concurrency carrying capacity under strict latency constraints through structured modeling and interpolation calculation, exhibits strong cross-scenario generalization ability, and reduces computing resource planning errors.

Description

Technical Field

[0001] This invention relates to the field of resource planning technology, and specifically to a computing resource planning method and system based on a large model. Background Technology

[0002] With the rapid development of artificial intelligence technology, online inference services for large language models (LLM) and multimodal models are widely used in various Internet platforms, intelligent interactive systems, and other scenarios. Among them, resource planning is the core link in the deployment and operation of AI large model inference services. Its role is to scientifically determine the type, scale, and configuration of computing resources based on the expected concurrent request volume of the business, in order to achieve optimal cost while ensuring service quality.

[0003] Currently, mainstream resource planning solutions in the industry can be summarized into two categories. The first is the experience-based approach and offline benchmarking. This method heavily relies on the personal experience of operations personnel or pre-executed offline stress test data under specific hardware configurations. Performance metrics are obtained through limited test samples (such as fixed concurrency and fixed context length) and used as a reference benchmark for capacity planning. The second is single performance metric-oriented planning. This method typically uses system-level macro-indicators (such as total cluster throughput) as a single optimization objective, estimating the required resource scale through linear extrapolation or simple threshold judgment.

[0004] However, the aforementioned methods, relying on fixed experience or limited stress test data, lack systematic modeling of multi-dimensional configuration parameters such as different GPU models, number of cards, model types, and dynamic context lengths. When business scenarios or hardware environments change, the planning results are difficult to reuse directly, have weak generalization ability, and cannot perform accurate interpolation calculations, resulting in large prediction errors. Furthermore, these methods unilaterally pursue macro-level indicators such as system-level throughput, failing to incorporate strict constraints on first-token latency and single-user throughput—which directly impact user experience—into a unified optimization framework. This leads to excessively high planned system latency, failing to meet service quality requirements. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention provides a computing resource planning method and system based on a large model, aiming to improve the accuracy of predicting the concurrent carrying capacity of configuration nodes, enhance cross-scenario generalization capabilities, and reduce computing resource planning errors.

[0006] This invention discloses a computing resource planning method based on a large model, comprising: A structured performance database is built based on benchmark documents. The benchmark documents are used to record the performance metrics of multiple configuration nodes under multiple concurrency levels. The performance metrics include first token latency, total throughput, and latency increment. The performance database stores the relationship between configuration nodes, concurrency levels, and performance metrics based on a hierarchical index format. Based on the performance database, a configuration data point set is generated for each configuration node. The configuration data point set is used to characterize the first token latency of the configuration node under different concurrency levels. The optimal data point is generated based on the configuration data point set. The optimal data point is the data point that meets the first constraint and the second constraint, and whose concurrency is the maximum value in the configuration data point set. The first constraint indicates that the delay of the first token is less than the preset delay time threshold, and the second constraint indicates that the throughput of a single user is greater than or equal to the preset throughput threshold. The throughput of a single user is the ratio of the total throughput to the concurrency. The optimal data point is associated with the configuration node. Obtain the target concurrency level for the business, and iterate through the optimal data points corresponding to the configuration nodes to generate at least one candidate deployment scheme that meets the target concurrency level for the business. Calculate the cost information of candidate deployment schemes, and generate at least one target deployment scheme based on the cost information. The target deployment scheme is used to characterize the deployment scheme of computing resources. A visualization report is generated based on the target deployment plan. The visualization report includes a comparison of candidate deployment plans, cost comparison data, concurrency-latency curves, and concurrency-throughput curves.

[0007] Preferably, the configuration node includes configuration information, which includes GPU model, model name, GPU card number, input length, and output length; The hierarchical index format includes: GPU model as the first-level index, model name as the second-level index, GPU card number as the third-level index, and input length and output length as the fourth-level index.

[0008] Preferably, generating optimal data points based on the configuration data point set includes: Sort the data points in the configuration data point set in ascending order of concurrency; Piecewise linear interpolation is performed on the sorted data points to solve for the maximum number of concurrent connections and the maximum throughput corresponding to the maximum number of concurrent connections that satisfy the first constraint. The single-user throughput is generated based on the maximum concurrency and the maximum total throughput. If the single-user throughput satisfies the second constraint, the data point corresponding to the maximum concurrency and the maximum throughput is taken as the optimal data point. If the single-user throughput does not meet the second constraint, then select the preferred data point set that meets the first constraint from the set of configured data points, and select the optimal data point that meets the second constraint from the set of preferred data points.

[0009] Preferably, piecewise linear interpolation is performed on the sorted data points to solve for the maximum concurrency and the maximum throughput corresponding to the maximum concurrency, satisfying the first constraint condition, including: Traverse the sorted set of configuration data points and determine whether there are any data points that satisfy the third constraint condition. The third constraint condition is that the delay of the first token is greater than or equal to the preset delay time threshold. If it exists, locate the first data point in the configuration data point set that satisfies the third constraint condition, locate the second data point based on the first data point, and calculate the maximum concurrency and maximum throughput based on the concurrency and total throughput corresponding to the first and second data points; the second data point is the data point preceding the first data point. If it does not exist, the data point with the highest concurrency in the configuration data point set will be used as the target data point, and the maximum concurrency and maximum throughput will be obtained based on the target data point.

[0010] Preferably, the maximum concurrency and maximum throughput are calculated using the following formulas: ; ; ; in, Indicates the maximum number of concurrent connections. Indicates the maximum throughput. This indicates the number of concurrent connections corresponding to the first data point. This indicates the number of concurrent connections corresponding to the second data point. This indicates the target preset delay time threshold. This indicates the preset delay time threshold. For a very small positive value, This indicates the delay of the first token corresponding to the first data point. This indicates the delay of the first token corresponding to the second data point. This represents the total throughput corresponding to the first data point. This represents the total throughput corresponding to the second data point.

[0011] Preferably, selecting the optimal data point that satisfies the second constraint from the preferred data point set includes: Calculate the candidate single-user throughput for each data point in the priority data point set; Candidate data points with a single-user throughput rate greater than or equal to a preset throughput threshold are selected, and the data point with the highest concurrency is selected as the optimal data point from the candidate data points.

[0012] Preferably, the optimal data points corresponding to the configuration nodes are traversed to generate at least one candidate deployment scheme that meets the business target concurrency, including: Calculate the ratio of the target concurrency to the concurrency corresponding to the optimal data point to obtain the target number of nodes; Based on the number of target nodes and the configuration nodes corresponding to the optimal data points, candidate deployment schemes are generated.

[0013] Preferably, the process includes calculating cost information for candidate deployment schemes and generating at least one target deployment scheme based on the cost information, including: Calculate the product of the target number of nodes and the number of GPUs corresponding to the optimal data point to obtain the target total number of GPUs; Calculate cost information based on the target total number of cards and the purchase price of the configuration node corresponding to the optimal data point; Candidate deployment schemes are sorted in ascending order of cost information, and the scheme with the lowest total cost (pre-defined number) is selected as the target deployment scheme.

[0014] Preferably, the computing resource planning method based on a large model includes: testing the configuration nodes corresponding to the target deployment scheme using automated testing tools to obtain measured performance indicators; Update the data in the performance database based on measured performance metrics.

[0015] This invention discloses a large-model-based computing resource planning system for executing a large-model-based computing resource planning method. The large-model-based computing resource planning system includes: The data construction module is configured to: build a structured performance database based on benchmark documents, which record performance metrics of multiple configuration nodes under multiple concurrency levels, including first token latency, total throughput, and latency increment; the performance database stores the relationships between configuration nodes, concurrency levels, and performance metrics based on a hierarchical index format. The optimal point generation module is configured to: generate a set of configuration data points for each configuration node based on a performance database, the set of configuration data points being used to characterize the first token latency of the configuration node under different concurrency levels; and generate an optimal data point based on the configuration data point set, the optimal data point being the data point that meets the first constraint and the second constraint, and whose concurrency level is the maximum value in the configuration data point set, the first constraint characterizing that the first token latency is less than a preset latency time threshold, the second constraint characterizing that the single-user throughput is greater than or equal to a preset throughput threshold; the single-user throughput is the ratio of total throughput to concurrency; the optimal data point is associated with the configuration node; The candidate solution generation module is configured to: obtain the business target concurrency, traverse the optimal data points corresponding to the configuration nodes, and generate at least one candidate deployment solution that meets the business target concurrency. The target deployment scheme generation module is configured to: calculate the cost information of candidate deployment schemes and generate at least one target deployment scheme based on the cost information. The target deployment scheme is used to characterize the deployment scheme of computing resources. The report generation module is configured to generate a visual report based on the target deployment scheme. The visual report includes a comparison of candidate deployment schemes, cost comparison data, concurrency-latency curves, and concurrency-throughput curves.

[0016] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention addresses the weaknesses of traditional empirical methods, such as weak generalization and large prediction errors, by constructing a structured performance database to systematically model multi-dimensional configuration parameters including GPU model, model type, and number of GPUs. Simultaneously, it incorporates first-to-last-to-last-to-last-to-last-user throughput into a unified optimization framework, using piecewise linear interpolation to accurately solve for the optimal data points that satisfy both constraints, ensuring both user experience and resource utilization efficiency. Furthermore, it generates candidate solutions based on business target concurrency and filters target solutions based on cost, ultimately outputting a visual report to aid decision-making. This achieves a balance between the scientific, accurate, and economical aspects of computing resource planning, effectively improving the resource allocation efficiency and service quality of large-scale model inference service deployment. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating the computing resource planning method based on a large model provided by the present invention. Figure 2 This is a schematic diagram of the structure of the computing resource planning system based on a large model provided by the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] The present invention will now be described in further detail with reference to the accompanying drawings.

[0020] like Figure 1 As shown in the figure, this embodiment of the invention provides a computing resource planning method based on a large model, including the following steps.

[0021] S1. Build a structured performance database based on benchmark test documents.

[0022] In this embodiment of the invention, benchmark documents are used to record performance metrics of multiple configuration nodes under multiple concurrency levels. These performance metrics include first token latency, total throughput, and latency increment. The performance database stores the relationships between configuration nodes, concurrency levels, and performance metrics based on a hierarchical index format.

[0023] In this embodiment of the invention, the configuration node includes configuration information, which includes GPU model, model name, GPU card number, input length, and output length. The hierarchical index format includes: GPU model as the first-level index, model name as the second-level index, GPU card number as the third-level index, and input length and output length as the fourth-level index.

[0024] For example, the benchmark documentation covers the performance of different configuration nodes in multi-concurrency scenarios. For instance, it tests the first token latency (in milliseconds), total throughput (in tokens / second), and latency increment (i.e., the change in first token latency for each additional 1 concurrency) as the number of concurrent connections changes from 1 to 64. This includes combinations of GPU models (such as A100, H100, L40S), model types (such as GPT-3.5, LLaMA2-70B, ChatGLM3-6B), number of GPUs (1, 2, 4, 8), input context length (2k, 4k, 8k, 16k), and output length (512, 1024, 2048).

[0025] The performance database uses a hierarchical index storage: the first level index is the GPU model, the second level is the model name, the third level is the number of GPUs, and the fourth level is a composite index of input and output lengths (e.g., "8k_1024"). For example, when querying the configuration "A100+LLaMA2-70B+4 cards+input 8k+output 1024", the index can quickly locate the performance metric data corresponding to all concurrency levels under that configuration, avoiding a full database scan and improving data retrieval efficiency.

[0026] S2. Generate a set of configuration data points for each configuration node based on the performance database.

[0027] In this embodiment of the invention, the configuration data point set is used to characterize the first token delay of the configuration node under different concurrency levels.

[0028] For example, the configuration data point set of configuration node A includes multiple data points, each of which can be represented by (c, d), where c represents the number of concurrent connections and d represents the first token delay of configuration node A when the number of concurrent connections is c.

[0029] S3. Generate the optimal data point based on the configuration data point set.

[0030] In this embodiment of the invention, the optimal data point is the data point that meets the first constraint and the second constraint, and whose concurrency is the maximum value in the set of configured data points. The first constraint indicates that the delay of the first token is less than a preset delay time threshold, and the second constraint indicates that the throughput of a single user is greater than or equal to a preset throughput threshold. The throughput of a single user is the ratio of the total throughput to the concurrency. The optimal data point and the configured node are associated.

[0031] Specifically, in the process of generating optimal data points, the data points in the configuration data point set are first sorted in ascending order of concurrency. For example, the configuration data point set of a certain configuration node, after sorting, becomes [(1, 120), (2,135), (4, 150), (8, 180), (16, 220), (32, 300)], where the first element is the concurrency c, and the second element is the first token delay d (in milliseconds). Next, piecewise linear interpolation is performed on the sorted data points to calculate the maximum concurrency and the corresponding maximum throughput that satisfy the first constraint. The specific steps are as follows: traverse the sorted set of configuration data points and determine if there are any data points that satisfy the third constraint (i.e., data points where the first token delay is greater than or equal to a preset delay time threshold). If a data point satisfies the third constraint, locate the first data point in the configuration data point set that satisfies the third constraint. Based on the first data point, locate the second data point, which is the data point preceding the first data point. Then, calculate the maximum concurrency and maximum throughput based on the concurrency and total throughput corresponding to the first and second data points.

[0032] In this embodiment of the invention, the maximum concurrency and maximum throughput are calculated using the following formulas: ; ; ; in, Indicates the maximum number of concurrent connections. Indicates the maximum throughput. This indicates the number of concurrent connections corresponding to the first data point. This indicates the number of concurrent connections corresponding to the second data point. This indicates the target preset delay time threshold. This indicates the preset delay time threshold. For a very small positive value, This indicates the delay of the first token corresponding to the first data point. This indicates the delay of the first token corresponding to the second data point. This represents the total throughput corresponding to the first data point. This represents the total throughput corresponding to the second data point.

[0033] Furthermore, if no data point satisfies the third constraint, the data point with the highest concurrency in the configuration data point set is selected as the target data point, and the maximum concurrency and maximum throughput are obtained based on the target data point.

[0034] After calculating the maximum concurrency and maximum total throughput, a single-user throughput rate is generated based on these parameters. If the single-user throughput rate satisfies the second constraint, the data point corresponding to the maximum concurrency and maximum throughput is selected as the optimal data point. If the single-user throughput rate does not satisfy the second constraint, a preferred set of data points satisfying the first constraint is selected from the configured data point set, and then the optimal data point satisfying the second constraint is selected from the preferred data point set. Specifically, the candidate single-user throughput rate for each data point in the preferred data point set is calculated; candidate data points with a single-user throughput rate greater than or equal to a preset throughput threshold are selected, and the data point with the highest concurrency rate is selected as the optimal data point from among the candidate data points.

[0035] In this embodiment of the invention, a maximum first token delay threshold is given. (i.e., a preset delay time threshold) for data point sets sorted by concurrency. Perform piecewise linear interpolation to find the solution that satisfies Maximum concurrency and corresponding throughput. The interpolation calculation logic is as follows: if it exists... Then find the first point that satisfies the condition. and the point before it .make (in If the value is a very small positive value (to ensure it is strictly less than the constraint), then: If all Then take the point with the highest concurrency in the dataset. As a candidate solution.

[0036] If the length of the user's output token is not in the hierarchical index data structure but the length of the input token is in the data structure (the output length requirements vary depending on the business scenario), assuming a fixed output length l, the throughput calculation formula is as follows: ; If single-user throughput ( The default throughput threshold is 10 tokens / second. That is the optimal solution. Otherwise, if it satisfies... Among all data points, the search satisfies Maximum concurrency and its swallowing If necessary, interpolation can also be used to solve the problem.

[0037] In this way, through the dual screening of the first and second constraints, the real-time nature of user interaction is guaranteed (the delay of the first token does not exceed the threshold), while avoiding the excessive distribution of resources under high concurrency, which would lead to a decline in the user experience for a single user (the throughput of a single user is not lower than the threshold), thus achieving a balance between concurrency capability and service quality.

[0038] S4. Obtain the business target concurrency and traverse the optimal data points corresponding to the configuration nodes to generate at least one candidate deployment scheme that meets the business target concurrency.

[0039] In this embodiment of the invention, the ratio of the target concurrency to the concurrency corresponding to the optimal data point is calculated to obtain the target number of nodes; based on the target number of nodes and the configuration nodes corresponding to the optimal data point, a candidate deployment scheme is generated.

[0040] S5. Calculate the cost information of candidate deployment schemes, and generate at least one target deployment scheme based on the cost information.

[0041] In this embodiment of the invention, the target deployment scheme is used to characterize the deployment scheme of computing power resources. In the process of generating the target deployment scheme, firstly, the product of the number of target nodes and the number of GPUs corresponding to the optimal data point is calculated to obtain the total number of target GPUs; then, the cost information is calculated based on the total number of target GPUs and the purchase price of the configuration node corresponding to the optimal data point; finally, the candidate deployment schemes are sorted in ascending order of the cost information values, and the scheme with the lowest total cost is selected as the target deployment scheme.

[0042] For example, given a target business concurrency M, iterate through all candidate GPU models and card combinations. For each combination, calculate the maximum concurrency per node, and then calculate: Number of nodes required: ; Total number of cards: Card count; Total cost: .

[0043] Finally, the top-N optimal deployment schemes are output, sorted in ascending order of total cost.

[0044] In this way, by matching the business target concurrency with the optimal concurrency of a single node, and combining the quantitative calculation of GPU hardware costs, a closed loop from "performance constraints" to "cost optimization" is achieved. This avoids the waste of resources caused by blindly adding hardware and ensures the economic feasibility of the solution.

[0045] In this embodiment of the invention, automated testing tools are used to test the configuration nodes corresponding to the target deployment scheme to obtain measured performance metrics. Then, the data in the performance database is updated based on these measured performance metrics. This creates a dynamic iterative mechanism of "test-optimization-feedback," continuously improving the accuracy and coverage of the performance database.

[0046] For example, when the target deployment scheme adopts the configuration of "A100 + ChatGLM3-6B + 2 cards + 4k input + 512 output", automated testing tools simulate concurrent requests under real business scenarios, and collect real-time measured data such as first token latency and total throughput under different concurrency levels. If the measured data deviates from the existing historical data in the performance database, the corrected measured data is updated to the corresponding index position. If the configuration is a new combination not covered by the database (such as the newly added "L40S + GPT-3.5 + 1 card + 16k input + 2048 output"), a new index entry is created for it according to the hierarchical indexing rules, and the measured data is entered to form a new configuration data point set. This dynamic update method enables the performance database to adapt to scenarios such as large model version iterations and hardware environment changes, providing more realistic basic data support for subsequent computing resource planning.

[0047] S6. Generate a visual report based on the target deployment plan.

[0048] In this embodiment of the invention, the visualization report includes a comparison of candidate deployment schemes, cost comparison data, concurrency-latency curves, and concurrency-throughput curves. Through intuitive charts and data comparisons, the visualization report helps technical personnel and decision-makers quickly understand the performance and cost differences between different deployment schemes.

[0049] like Figure 2 As shown, the present invention also provides a computing resource planning system based on a large model for executing the above-mentioned computing resource planning method based on a large model. The computing resource planning system based on a large model includes: a data construction module 201, an optimal point generation module 202, a candidate solution generation module 203, a target solution generation module 204, and a report generation module 205.

[0050] Specifically, the data construction module 201 is configured to: build a structured performance database based on benchmark documents, which are used to record the performance metrics of multiple configuration nodes under multiple concurrency levels, including first token latency, total throughput, and latency increment; the performance database stores the relationship between configuration nodes, concurrency levels, and performance metrics based on a hierarchical index format.

[0051] The optimal point generation module 202 is configured to: generate a set of configuration data points for each configuration node based on the performance database, the set of configuration data points being used to characterize the first token latency of the configuration node under different concurrency levels; and generate an optimal data point based on the set of configuration data points, the optimal data point being the data point that meets the first constraint and the second constraint, and whose concurrency level is the maximum value in the set of configuration data points, the first constraint characterizing that the first token latency is less than a preset latency time threshold, the second constraint characterizing that the single-user throughput is greater than or equal to a preset throughput threshold; the single-user throughput is the ratio of total throughput to concurrency; the optimal data point is associated with the configuration node.

[0052] The candidate solution generation module 203 is configured to: obtain the business target concurrency, traverse the optimal data points corresponding to the configuration nodes, and generate at least one candidate deployment solution that meets the business target concurrency. The target solution generation module 204 is configured to: calculate the cost information of the candidate deployment solutions, and generate at least one target deployment solution based on the cost information. The target deployment solution is used to characterize the deployment scheme of computing resources. The report generation module 205 is configured to: generate a visualization report based on the target deployment solution. The visualization report includes a comparison of candidate deployment solutions, cost comparison data, a concurrency-latency curve, and a concurrency-throughput curve.

[0053] As can be seen from the above technical solution, this application provides a computing resource planning method and system based on a large model. The computing resource planning method includes: constructing a structured performance database based on benchmark test documents; generating a set of configuration data points corresponding to each configuration node based on the performance database; generating optimal data points based on the set of configuration data points; obtaining the business target concurrency; traversing the optimal data points corresponding to the configuration nodes; generating at least one candidate deployment scheme that meets the business target concurrency; calculating the cost information of the candidate deployment schemes; and generating the target deployment scheme based on the cost information. Specifically, the optimal data point has a first token latency less than a preset latency time threshold, a single-user throughput greater than or equal to a preset throughput threshold, and the concurrency number is the maximum value in the set of configuration data points. This invention achieves accurate prediction of concurrency carrying capacity under strict latency constraints through structured modeling and interpolation calculation, exhibits strong cross-scenario generalization ability, and reduces computing resource planning errors.

[0054] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A computational resource planning method based on a large model, characterized in that, include: A structured performance database is constructed based on benchmark test documents, which are used to record performance metrics of multiple configuration nodes under multiple concurrency levels. The performance metrics include first token latency, total throughput, and latency increment. The performance database stores the relationship between the configuration nodes, the concurrency levels, and the performance metrics based on a hierarchical index format. Based on the performance database, a configuration data point set is generated for each configuration node. The configuration data point set is used to characterize the first token delay of the configuration node under different concurrency levels. Optimal data points are generated based on the configuration data point set. The optimal data point is the data point that meets the first constraint and the second constraint, and whose concurrency is the maximum value in the configuration data point set. The first constraint indicates that the delay of the first token is less than a preset delay time threshold, and the second constraint indicates that the throughput of a single user is greater than or equal to a preset throughput threshold. The throughput of a single user is the ratio of the total throughput to the concurrency. The optimal data point is associated with the configuration node. Obtain the target concurrency level for the business, and traverse the optimal data points corresponding to the configuration nodes to generate at least one candidate deployment scheme that satisfies the target concurrency level for the business. Calculate the cost information of the candidate deployment schemes, and generate at least one target deployment scheme based on the cost information. The target deployment scheme is used to characterize the deployment scheme of computing resources. A visualization report is generated based on the target deployment scheme. The visualization report includes a comparison of candidate deployment schemes, cost comparison data, concurrency-latency curves, and concurrency-throughput curves.

2. The computing resource planning method according to claim 1, characterized in that, The configuration node includes configuration information, which includes GPU model, model name, GPU card number, input length, and output length. The hierarchical index format includes: the GPU model as the first-level index, the model name as the second-level index, the GPU card number as the third-level index, and the input length and the output length as the fourth-level index.

3. The computing resource planning method according to claim 1, characterized in that, The step of generating optimal data points based on the configured data point set includes: Sort the data points in the configuration data point set in ascending order of the concurrency level; Piecewise linear interpolation is performed on the sorted data points to solve for the maximum number of concurrent connections that satisfy the first constraint and the maximum throughput corresponding to the maximum number of concurrent connections. The single-user throughput is generated based on the maximum concurrency and the maximum total throughput. If the single-user throughput satisfies the second constraint, the data point corresponding to the maximum concurrency and the maximum throughput is taken as the optimal data point. If the single-user throughput does not meet the second constraint, then a preferred set of data points that meets the first constraint is selected from the set of configured data points, and the optimal data point that meets the second constraint is selected from the set of preferred data points.

4. The computing resource planning method according to claim 3, characterized in that, The step of performing piecewise linear interpolation on the sorted data points to solve for the maximum concurrency and the maximum throughput corresponding to the maximum concurrency that satisfy the first constraint includes: Traverse the sorted set of configuration data points and determine whether there are any data points that satisfy the third constraint condition, wherein the third constraint condition is that the delay of the first token is greater than or equal to the preset delay time threshold. If it exists, locate the first data point in the configuration data point set that satisfies the third constraint condition, locate the second data point based on the first data point, and calculate the maximum concurrency and the maximum throughput based on the concurrency and total throughput corresponding to the first data point and the second data point; the second data point is the data point preceding the first data point; If it does not exist, the data point with the largest concurrency in the configuration data point set is taken as the target data point, and the maximum concurrency and the maximum throughput are obtained based on the target data point.

5. The computing resource planning method according to claim 4, characterized in that, The maximum concurrency and the maximum throughput are calculated using the following formulas: ； ； ； in, This represents the maximum number of concurrent connections. This indicates the maximum throughput. This indicates the number of concurrent connections corresponding to the first data point. This indicates the number of concurrent connections corresponding to the second data point. This indicates the target preset delay time threshold. This represents the preset delay time threshold. For a very small positive value, This indicates the delay of the first token corresponding to the first data point. This indicates the delay of the first token corresponding to the second data point. This represents the total throughput corresponding to the first data point. This represents the total throughput corresponding to the second data point.

6. The computing resource planning method according to claim 3, characterized in that, The step of selecting the optimal data point that satisfies the second constraint from the preferred data point set includes: Calculate the candidate single-user throughput for each data point in the priority data point set; Candidate data points with a single-user throughput rate greater than or equal to the preset throughput threshold are selected, and the data point with the highest concurrency is selected from the candidate data points as the optimal data point.

7. The computing resource planning method according to claim 1, characterized in that, The step of traversing the optimal data points corresponding to the configuration nodes to generate at least one candidate deployment scheme that satisfies the business target concurrency includes: Calculate the ratio of the target concurrency to the concurrency corresponding to the optimal data point to obtain the target number of nodes; The candidate deployment scheme is generated based on the number of target nodes and the configuration node corresponding to the optimal data point.

8. The computing resource planning method according to claim 7, characterized in that, The step of calculating the cost information of the candidate deployment schemes and generating at least one target deployment scheme based on the cost information includes: Calculate the product of the number of target nodes and the number of GPUs corresponding to the optimal data point to obtain the total number of target GPUs; Calculate cost information based on the target total number of cards and the purchase price of the configuration node corresponding to the optimal data point; The candidate deployment schemes are sorted in ascending order of the cost information values, and the schemes with the lowest total cost are selected as the target deployment schemes.

9. The computing resource planning method according to claim 1, characterized in that, Also includes: The configuration nodes corresponding to the target deployment scheme are tested using automated testing tools to obtain actual performance indicators; The data in the performance database is updated based on the measured performance indicators.

10. A large-scale model-based computing resource planning system, used to execute the large-scale model-based computing resource planning method as described in any one of claims 1 to 9, characterized in that, include: The data construction module is configured to: build a structured performance database based on a benchmark test document, wherein the benchmark test document is used to record the performance metrics of multiple configuration nodes under multiple concurrency levels, and the performance metrics include first token latency, total throughput, and latency increment; the performance database stores the relationship between the configuration nodes, the concurrency levels, and the performance metrics based on a hierarchical index format; The optimal point generation module is configured to: generate a set of configuration data points for each configuration node based on the performance database, wherein the set of configuration data points is used to characterize the first token delay of the configuration node under different concurrency levels; Furthermore, an optimal data point is generated based on the configuration data point set. The optimal data point is a data point that meets the first constraint and the second constraint, and whose concurrency is the maximum value in the configuration data point set. The first constraint indicates that the delay of the first token is less than a preset delay time threshold, and the second constraint indicates that the throughput of a single user is greater than or equal to a preset throughput threshold. The throughput of a single user is the ratio of the total throughput to the concurrency. The optimal data point is associated with the configuration node. The candidate solution generation module is configured to: obtain the business target concurrency, traverse the optimal data points corresponding to the configuration node, and generate at least one candidate deployment solution that satisfies the business target concurrency; The target deployment scheme generation module is configured to: calculate the cost information of the candidate deployment schemes, and generate at least one target deployment scheme based on the cost information, wherein the target deployment scheme is used to characterize the deployment scheme of computing power resources; The report generation module is configured to generate a visualization report based on the target deployment scheme. The visualization report includes a comparison of candidate deployment schemes, cost comparison data, a concurrency-latency curve, and a concurrency-throughput curve.

Images