Performance modeling and extrapolation method for parallel programs on domestic supercomputers

Abstract

The application discloses a parallel program performance modeling and extrapolation prediction method for a domestic supercomputer platform, and relates to the technical field of high-performance computing and parallel program analysis, wherein the method comprises the following steps: firstly, obtaining a program and a prediction scale parameter, constructing a performance structure tree containing communication leaf nodes and program unit internal nodes through double-layer program analysis; collecting high-precision benchmark data and global statistical characteristics as a training set based on the performance structure tree by using a hybrid replay; building sub-models at each granularity node of the tree and aggregating the sub-models from bottom to top to obtain a global prediction model; splitting the overall performance into multiple independent components to complete fitting calibration and extrapolation to a target scale; and finally, outputting large-scale performance prediction results by using high-precision benchmark data to calibrate key path prediction values. The application constructs a performance structure tree for hierarchical modeling and aggregation, clearly represents the program hierarchical structure, and solves the problems of missing traditional modeling structure and poor interpretability.

Description

Technical Field

[0001] This invention relates to the field of high-performance computing and parallel program analysis technology, and in particular to a method for parallel program performance modeling and extrapolation prediction for domestic supercomputing platforms. Background Technology

[0002] With the continuous evolution of next-generation high-performance computer architectures, their application scale and concurrency are increasing, and system interconnection layers are becoming more complex. The actual performance of parallel programs is affected by multiple factors, including computation, memory access, communication, synchronization, I / O, and system noise. In the development and optimization of high-performance computing applications, accurate performance prediction of ultra-large-scale parallel execution behavior is an important foundation for guiding application porting, performance tuning, and large-scale resource allocation.

[0003] Currently, existing performance prediction methods face two main challenges. Firstly, achieving high prediction accuracy often requires large-scale operational testing on massive resources, or fine-grained observations and replays at a microarchitecture level, significantly increasing testing costs, including substantial machine time and human resources consumption. Limited by the availability and scheduling constraints of large-scale systems, such methods are difficult to implement widely in the early stages of application development or under resource-constrained conditions. Secondly, if simplified analytical models or statistical extrapolation methods are used to reduce testing overhead, it becomes difficult to capture detailed behavioral characteristics at the microarchitecture level, leading to decreased prediction accuracy, especially with significant error accumulation effects during large-scale scaling. The inherent contradiction between accuracy and efficiency restricts the applicability of existing methods in practical engineering.

[0004] Furthermore, parallel programs themselves possess multi-layered structural characteristics, including function calls, nested loops, branching paths, and communication operations. Existing methods, in the performance modeling process, lack accurate identification and unified representation of these structural relationships, making it difficult to achieve effective acquisition of structured performance data, independent modeling of substructures, and cross-level aggregation inference, thus affecting the interpretability of the model and the reliability of the extrapolation results.

[0005] As the bottleneck of modern high-performance computing systems gradually shifts from computational peaks to data movement overhead, the performance behavior of communication, memory access, and I / O processes has an increasingly prominent impact on the overall program execution time. In particular, asynchronous strategies such as compute-communication overlap and pipelined scheduling are widely used in practical applications. Their performance is highly sensitive to factors such as interconnect topology, communication protocols, and system noise. However, current evaluations of such asynchronous overlap strategies mainly rely on experience and extensive experimental methods, lacking systematic and low-cost quantitative analysis and performance simulation mechanisms.

[0006] Therefore, there is an urgent need for a method that can predict the performance of large-scale or even ultra-large-scale parallel programs at a low cost under single-node or small-scale resource conditions. This method should be able to balance microarchitectural accuracy and statistical coverage, while also having the ability to explicitly express program structure and aggregate modeling capabilities, and be able to perform interpretable and computable performance simulation and extrapolation of asynchronous overlapping strategies. Summary of the Invention

[0007] The main objective of this invention is to provide a method for parallel program performance modeling and extrapolation prediction for domestic supercomputing platforms.

[0008] Another object of the present invention is to provide an electronic device.

[0009] A third objective of this invention is to provide a non-transitory computer-readable storage medium.

[0010] To achieve the above objectives, a first aspect of this invention proposes a method for parallel program performance modeling and extrapolation prediction for domestically produced supercomputing platforms, comprising:

[0011] Obtain the parallel program and its corresponding target prediction scale parameters, perform intra-process analysis and inter-process analysis on the parallel program, and construct a performance structure tree to characterize the hierarchical structure of the program. The nodes of the performance structure tree include leaf nodes representing communication function calls and internal nodes representing program structure units. Based on the performance structure tree, deterministic replay is used on the target path and bottleneck area on a small scale to obtain high-precision benchmark data at the microarchitecture level, and representative replay is used on the overall execution behavior to identify representative processes and behavior patterns from the execution space to obtain statistical features. The high-precision benchmark data and statistical features are used as training data. Based on the training data, sub-models are established at different granularity nodes of the performance structure tree, and hierarchical aggregation is performed from bottom to top according to the performance structure tree to obtain the overall program performance prediction model. Based on the overall performance prediction model of the program, the overall performance is decomposed into multiple performance components that can be independently modeled. After fitting and calibrating the performance components based on the training data, the results are extrapolated to the target prediction scale parameter. The high-precision benchmark data obtained by the deterministic replay is used to calibrate the extrapolation results of the target path, and the performance prediction results under the target scale are output to complete the performance extrapolation prediction.

[0012] Optionally, the intra-process analysis and the inter-process analysis include: The process analysis establishes a function structure tree for each function. The node types of the function structure tree include function nodes representing the current function, loop nodes representing loop structures, branch nodes representing branch structures, branch path nodes representing a certain path under a branch structure, communication nodes representing communication function calls, and call nodes representing calls to non-communication functions. The inter-process analysis assembles the function structure trees into a global structure tree based on function call relationships. The global structure tree takes the function node corresponding to the main function as the root node, and during the assembly process, the calling node is replaced with the function structure tree of the called function, or the calling node is deleted when the set rules are met.

[0013] Optionally, in the performance structure tree, the leaf nodes are used to represent communication function calls, wherein the communication function calls include point-to-point communication primitive calls and set communication primitive calls, the internal nodes are used to represent program structure units, wherein the program structure units include at least branch structures, loop structures and functions, and the edges of the performance structure tree are used to represent hierarchical relationships between nodes, wherein the hierarchical relationships include inclusion relationships, calling relationships and control structure nesting relationships.

[0014] Optionally, based on the performance structure tree, deterministic replay is used on the target path and bottleneck region on a small scale of resources to obtain high-precision benchmark data at the microarchitecture level, including: Run a parallel program on the target platform and perform deterministic replay of the identified target path and bottleneck region. The target path is the path in the performance structure tree whose execution frequency is higher than a set threshold, and the bottleneck region is the node and its subtree region in the performance structure tree whose time consumption ratio is higher than a set threshold. Parallel execution is represented as an ordered sequence of events, with each event recorded as a quadruple, which includes a timestamp, process identifier, action type, and related data. The action type includes computation segment, communication call, and synchronization point, and the related data includes message size, destination process, source process, and code segment identifier. During deterministic replay, by recording the order in which events occur and the dependencies between events, it is ensured that the replay execution is consistent with the original execution at the event sequence level, and microarchitecture-level performance counter data corresponding to each event is collected; Clock offset is controlled by a logical clock synchronization model or a clock calibration mechanism provided by the target platform to maintain timing consistency. The logical clock synchronization model establishes timing constraints based on the partial order relationship between events, while the clock calibration mechanism achieves clock reference alignment between multiple cores and multiple nodes based on the period counter synchronization interface provided by the target platform hardware.

[0015] Optionally, representative replay of the overall execution behavior can be used to identify representative processes and behavioral patterns in the execution space to obtain statistical characteristics, including: A multidimensional feature space is constructed, and the execution trajectory of each process is mapped to a feature vector in the feature space through a feature extraction function. The feature extraction function takes the instruction mixture, memory access address sequence and communication event sequence in the process execution trajectory as input, and outputs a fixed-dimensional numerical feature vector. The feature vector includes computation density features, memory access features, communication mode features and load balancing features. After extracting feature vectors from the trajectories of each process, process clusters are obtained by similarity measurement and hierarchical clustering. Based on probability coverage and importance sampling theory, representative processes and sample sizes are selected from each process cluster to minimize the difference between the distribution of the selected representative processes in the feature space and the distribution of all processes. Variance analysis is used to control sampling error.

[0016] Optionally, sub-models can be built at different granularity nodes of the performance structure tree, including: A communication delay model is established for the communication nodes that are leaf nodes. The communication delay model is related to the message size, communication topology and communication type. The communication types include neighbor communication, cross-rack remote communication and aggregate communication. Corresponding delay functions are established for different communication types. The computation time of the computing node is estimated based on the computation feature vector and calibration parameters. The computation feature vector includes the number of instructions per clock cycle, cache hit rate, branch prediction rate and memory bandwidth utilization. The calibration parameters include cache miss penalty cycle, branch prediction miss penalty cycle and pipeline stall penalty cycle. The results are obtained by experimental measurement on the target platform by running standard benchmark test programs. The time consumption of child nodes is weighted and summarized based on the number of iterations and the branch probability for loop nodes and branch nodes respectively. The total time consumption of loop nodes is the product of the time consumption of a single iteration and the number of iterations, and the total time consumption of branch nodes is the sum of the products of the time consumption of each branch path node and the execution probability of that branch path. The time consumed by substructures of a function node is aggregated according to the calling relationship and the inclusion relationship. The total time consumed by a function node is the sum of the time consumed by each sub-node in the function body plus the calling overhead introduced by the function call itself.

[0017] Optionally, the overall performance is decomposed into multiple independently modelable performance components. These components are then fitted and calibrated based on the training data and extrapolated to the target prediction scale parameter, including: The overall performance is decomposed into computation time components, nearest neighbor communication time components, cross-rack remote communication time components, aggregate communication time components, synchronization overhead components, I / O overhead components, and system noise components, and the components satisfy an additive decomposition relationship. Based on the training data, each of the performance components is independently fitted and calibrated. The fitting process involves determining the model parameters of each component based on small-scale sampling data, and the calibration process involves comparing the fitted model output with the high-precision benchmark data obtained by deterministic replay and adjusting the parameters to reduce the deviation. After each performance component completes independent fitting and calibration, the value of each component under the target scale is calculated according to the scale variable on which each component depends in the target prediction scale parameter. Then, the components are combined and summed according to the decomposition relationship to obtain the overall performance prediction value extrapolated to the target prediction scale parameter.

[0018] To achieve the above objectives, a second aspect of this application proposes an electronic device, including a processor and a memory; wherein the processor reads executable program code stored in the memory to run a program corresponding to the executable program code, so as to implement a parallel program performance modeling and extrapolation prediction method for a domestic supercomputing platform as described in the first aspect embodiment.

[0019] To achieve the above objectives, a third aspect of this application proposes a non-transitory computer-readable storage medium storing a computer program that, when executed by a processor, implements a parallel program performance modeling and extrapolation prediction method for a domestically produced supercomputing platform as described in the first aspect embodiment.

[0020] The embodiments of the present invention have the following beneficial effects: (1) The present invention can achieve performance prediction on ultra-large scale systems by relying on data collection of a single node or a small-scale cluster, without the need for actual operation on the actual target scale, which greatly reduces the machine time cost and manpower expenditure required for performance evaluation.

[0021] (2) The present invention adopts a hybrid replay strategy. Deterministic replay is applied in the critical path and bottleneck area to obtain high-precision benchmark data at the micro-architecture level to ensure prediction accuracy. Representative replay is applied in the overall execution behavior to identify representative processes and behavior patterns from the massive execution space to improve collection efficiency and sample coverage, thereby achieving a balance between accuracy and efficiency.

[0022] (3) This invention achieves explicit expression of parallel program substructures by constructing a performance structure tree, so that each granular node can be modeled independently and aggregated hierarchically, overcoming the modeling difficulties caused by the lack of structural relationships in traditional methods, and enhancing the interpretability and generalizability of the model.

[0023] (4) This invention deconstructs the total running time into components and uses high-precision data obtained by deterministic replay to calibrate the extrapolation results of the critical path, thus avoiding the error accumulation problem caused by fitting a single total time, making the extrapolation prediction results more robust and reliable. Attached Figure Description

[0024] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein: Figure 1 A flowchart illustrating a method for parallel program performance modeling and extrapolation prediction for domestic supercomputing platforms, provided as an embodiment of the present invention; Figure 2 This is a schematic diagram of the performance structure tree (PST) hierarchy provided in an embodiment of the present invention. Figure 3 This is a flowchart of a representative replay multidimensional feature clustering process provided in an embodiment of the present invention; Figure 4 A bar chart comparing the performance prediction errors of different level modeling schemes provided in the embodiments of the present invention; Figure 5 This is a schematic diagram of the overall architecture of a parallel program performance modeling and extrapolation prediction system for a domestic supercomputing platform, provided as an embodiment of the present invention. Detailed Implementation

[0025] It should be noted that, unless otherwise specified, the embodiments and features described in the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0026] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0027] The following describes, with reference to the accompanying drawings, a method for parallel program performance modeling and extrapolation prediction for domestic supercomputing platforms, based on an embodiment of the present invention.

[0028] Example 1 This invention provides a method for parallel program performance modeling and extrapolation prediction for domestically developed supercomputing platforms. Figure 1 This is a flowchart illustrating a method for parallel program performance modeling and extrapolation prediction for domestically produced supercomputing platforms, provided in an embodiment of the present invention. Figure 1 As shown, the method includes the following steps: Step S1: Obtain the parallel program and its corresponding target prediction scale parameters, perform intra-process analysis and inter-process analysis on the parallel program, and construct a performance structure tree to characterize the hierarchical structure of the program. The nodes of the performance structure tree include leaf nodes representing communication function calls and internal nodes representing program structure units.

[0029] In this embodiment, the startup phase of the entire prediction method requires the collection of four core basic input contents, specifically including the source code or executable file of the parallel program to be detected, the hardware configuration indicators corresponding to the target processor platform, the user-defined target size parameter, and the cluster resource range that can be used for small-scale data sampling. Among them, the target size parameter is the core control variable for performance extrapolation, mainly including two types: problem size N and the number of parallel processes / processors P. All subsequent model fitting, parameter calibration, and large-scale extrapolation calculations revolve around these two types of variables. Through complete input data preparation, this step ultimately constructs a performance structure tree PST as the core output carrier. This hierarchical tree structure can fully carry all the operational structural features of the parallel program. All subsequent sub-model construction, feature aggregation, and performance prediction modeling work are carried out in an orderly manner based on this performance structure tree.

[0030] After completing the input data acquisition, the two-layer program structure analysis for parallel programs can be initiated. This analysis is divided into two progressively executing stages: intra-process analysis and inter-process analysis. Through this two-layer analysis, a complete analysis from the local structure of a single function to the global structure of the entire program can be achieved. The intra-process analysis is the local structure analysis stage. Its core task is to traverse all independent functions within the parallel program, disassemble the internal execution logic of each function individually, generate a dedicated function structure tree, and accurately depict the code execution architecture within a single function. In this embodiment, the function structure tree has six types of standardized functional nodes, which can comprehensively cover various code execution behaviors within a function. These include: function nodes representing the current overall function body, loop nodes representing the code's cyclic execution logic, branch nodes carrying the program's branch judgment logic, branch path nodes representing a single independent execution path under the corresponding branch structure, communication nodes marking various communication interface call behaviors, and call nodes indicating ordinary non-communication sub-function call behaviors. Through the combination and arrangement of these six types of nodes, the internal code structure and execution logic of any function can be completely and precisely reconstructed.

[0031] After completing the intra-process analysis of all single functions and generating the complete function structure tree, a higher-dimensional inter-process global analysis is then conducted. This analysis stage uses the original function call logic of the program as its core basis, sorting out the call and association relationships between functions, and unifying and splicing all the scattered and independent function structure trees to obtain a global structure tree that can represent the entire parallel program's execution architecture. To ensure the standardization and simplicity of the global structure tree, the function node corresponding to the program's main function is fixed as the top-level root node of the entire tree, achieving unified tracing of the entire program structure. Simultaneously, during the assembly and splicing of the global structure tree, two standardized processing rules are set for the call nodes generated by the intra-process analysis, which can be adaptively adjusted according to the actual program structure: the first processing rule is replacement processing, which directly replaces the call node with the complete function structure tree of the called sub-function it points to, fully preserving all structural features within the sub-function; the second processing rule is deletion processing, where call nodes that meet the system's preset filtering rules and have no need for retention are directly deleted. Based on the optimization of these two rules, the final global structure tree no longer explicitly retains call nodes, effectively simplifying the global structure and reducing the complexity of subsequent modeling.

[0032] After two-layer analysis (intra-process and inter-process), structural assembly, and optimization, a complete and clearly hierarchical performance structure tree (PST) can be generated. To intuitively illustrate the hierarchical architecture, node relationships, and program structure mapping of the PST, this application embodiment includes a standardized PST hierarchical architecture diagram, as shown below. Figure 2 As shown in the diagram, this is a schematic diagram of the Performance Structure Tree (PST). It clearly and intuitively displays the nesting relationships, code inclusion relationships, and function call relationships among the hierarchical nodes in the entire tree structure, providing a visual basis for subsequent structural modeling and performance analysis. Combined with... Figure 2 The architecture shown clearly shows that the performance tree structure is strictly divided into two categories in terms of node type: leaf nodes and internal nodes. At the same time, the hierarchical relationship between different nodes is uniformly defined through the edge structure of the tree, specifically covering three core relationship forms: code inclusion relationship, function call relationship, and program control flow nesting relationship.

[0033] From the perspective of node function and definition, the internal nodes of the performance structure tree are mainly used to carry various basic program structural units, covering at least the three core program units: branching structures, looping structures, and functions. They are the core components constituting the overall program architecture. Combined with... Figure 2The specific structure is explained below. The top-level root node in the diagram is the function node corresponding to the main function of the program. As the core tracing node of the entire PST, the main function node extends downwards into three types of core child nodes: loop nodes, branch nodes, and function nodes corresponding to child functions, completely replicating the overall execution framework of the main function. Among them, the loop node is configured with a dedicated iteration count k attribute to quantify the loop execution frequency. Its lower-level nodes simultaneously attach computation nodes and communication leaf nodes, each performing its own function of storing core parameters: computation nodes mainly correspond to program computation-type code segments, used to store the feature vectors and micro-architecture penalty parameters of the corresponding code segments; communication leaf nodes specifically correspond to communication execution behavior, used to record key data such as communication message size and communication call count. The branch node is configured with execution probabilities p1 and p2 corresponding to two branch paths to quantify the execution weight of different branch paths. The lower-level nodes correspond to two independent branch path nodes, realizing the decomposition and parsing of branch logic; the end of path 1 attaches the leaf node corresponding to point-to-point communication or set-type communication, and the end of path 2 attaches a computation node carrying a unique code segment ID. Overall, loop nodes, branch nodes, and various function nodes all belong to the category of internal nodes, which are responsible for building the program structure hierarchy and carrying logic.

[0034] Unlike internal nodes, leaf nodes in the performance structure tree are terminal functional nodes with no lower-level child nodes. Their sole function is to accurately represent the various communication function call behaviors in parallel programs, covering the two most core communication modes in parallel computing: point-to-point communication primitive calls and global set communication primitive calls. Figure 2 The structural design shows that all communication nodes at the end of the diagram without any lower-level extended child nodes are PST leaf nodes that conform to the definition. They are specifically used to focus on the communication behavior of the parsing program and provide accurate structural support for subsequent communication delay modeling and communication overhead decomposition.

[0035] While constructing the hierarchical and node architecture of the performance structure tree, this application also binds unique multi-dimensional attribute information to each node in the PST, achieving deep binding between structure and parameters. Specifically, the bound node attributes include the node's own structure type, the unique identifier of the corresponding code segment, the specific category of the communication primitive, the message size of a single communication action, the total number of executions of the communication operation, the estimated iteration count of the loop structure, and the estimated execution probability of the branch path. These refined attribute data bound to each node can comprehensively characterize the operational features of each segment of the program, providing a solid data and structural foundation for subsequent sub-model construction at different granularities, accurate calculation of the time consumption of each module, and overall performance model aggregation and optimization.

[0036] Step S2: Based on the performance structure tree, deterministic replay is used on the target path and bottleneck area on a small scale of resources to obtain high-precision benchmark data at the microarchitecture level. Representative replay is used on the overall execution behavior to identify representative processes and behavior patterns in the execution space to obtain statistical features. The high-precision benchmark data and statistical features are used as training data.

[0037] After completing the construction and attribute labeling of the Performance Structure Tree (PST), this embodiment employs a hybrid replay data acquisition strategy. It collects data from parallel program execution using small-scale cluster resources, balancing high-precision data acquisition of key bottleneck areas with low-overhead statistical feature extraction of the entire program. This results in a complete dataset for subsequent model training. This step is primarily divided into two acquisition modules: deterministic replay and representative replay. These two methods complement each other, effectively addressing the technical shortcomings of traditional single-data acquisition methods, such as insufficient accuracy, high overhead, and poor sample representativeness.

[0038] To identify critical paths and bottleneck regions in a program, this application employs a deterministic replay approach to collect high-precision benchmark data at the microarchitecture level. In this embodiment, both critical paths and bottleneck regions are precisely determined based on a pre-constructed performance tree structure (PST). A critical path is defined as a program execution path in the PST whose cumulative execution frequency exceeds a preset system threshold. A bottleneck region is defined as a node in the PST whose single-node execution time accounts for a proportion of the total program runtime exceeding a preset threshold, along with all its subordinate subtrees. These regions are core modules determining the overall program performance and are the main source of performance prediction bias, requiring precise characterization based on high-precision data. In a small-scale cluster environment on the target platform, a complete deterministic replay operation is performed on the selected critical paths and bottleneck regions to fully reproduce the original execution logic and runtime sequence of the program.

[0039] In deterministic replay execution, the overall parallel execution behavior of the parallel program is abstracted into an ordered, standardized sequence of events. Each independent program event is recorded using a unified four-tuple format, and the corresponding event expression is: Each parameter in the quadruple has a clearly defined purpose: t represents the system timestamp of the current event, used to identify the timing of the event; pid is the unique process identifier, used to distinguish execution events generated by different parallel processes; type is the event action type, mainly covering three core behaviors: program computation segment execution, communication interface call, and program synchronization point waiting; payload is the event's associated payload data, containing fine-grained parameters such as communication message size, communication destination process ID, communication source process ID, and the unique identifier of the corresponding code segment, comprehensively recording the associated characteristics of the event.

[0040] To ensure the authenticity and validity of the replay data, the deterministic replay process strictly records the chronological order of all events and the data dependencies between them. This ensures that the replay execution flow is completely aligned with the original program execution flow in terms of event sequence, eliminating issues such as timing discrepancies, missing events, and dependency deviations. Simultaneously, it collects the original processor microarchitecture performance counter data corresponding to each event, forming high-precision, high-reliability benchmark data. Addressing the industry pain point of inconsistent hardware clocks and clock skew interference with timing consistency in multi-core, multi-processor, and multi-node cluster scenarios, this application's embodiments set up two timing calibration schemes that can adaptively adapt to different hardware platforms. The first is a logical clock synchronization model, drawing on the core idea of ​​Lamport's logical clock, building a global timing system based on the partial order constraints between events to eliminate timing errors caused by hardware clock deviations. The second is the target platform's native clock calibration mechanism, which calls the platform's built-in period counter synchronization interface to achieve unified alignment of clock benchmarks between multiple cores and nodes, ensuring the timing consistency of all collected events throughout the process.

[0041] To analyze the overall execution behavior of the program, this application employs a representative replay approach to filter representative processes and behavioral patterns from a massive program execution space. This method acquires global statistical features with extremely low data collection overhead. The complete clustering and filtering process is as follows: Figure 3 As shown, Figure 3 This is a representative replay clustering diagram, which intuitively illustrates the complete process of massive processes from feature extraction, clustering and grouping to representative sample selection. In this embodiment, the execution trajectories of massive parallel processes have a large number of redundant and similar features. It is not necessary to collect all processes in full. By replaying representative data, the sampling overhead can be significantly reduced while ensuring data accuracy.

[0042] Specifically, a multi-dimensional feature space is first constructed. A dedicated feature extraction function is then used to map the original execution trajectory of each process, transforming the complex trajectory data into a fixed-dimensional numerical feature vector. The input to the feature extraction function includes mixed instruction features, memory access address sequence features, and communication event sequence features during process execution. The output feature vector comprehensively covers four core dimensions: computational density features, memory access features, communication pattern features, and load balancing features, enabling a comprehensive characterization of the execution behavior of a single process.

[0043] After extracting feature vectors from all process trajectories, a similarity measurement algorithm combined with hierarchical clustering is used to group all process samples in the high-dimensional feature space into clusters. Processes with similar behavioral characteristics are grouped into the same cluster, achieving refined classification of massive processes. After clustering, probability coverage theory and importance sampling theory are combined to select an appropriate number of representative process samples and corresponding sample sizes from each process cluster. The core principle of selection is to minimize the difference between the distribution pattern of the selected representative processes in the feature space and the overall distribution pattern of all original processes, thus preserving global execution features to the greatest extent. Simultaneously, this application introduces analysis of variance to strictly control sampling errors, effectively suppressing data fluctuations caused by random sampling and ensuring the stability and reliability of the collected statistical features.

[0044] Finally, the high-precision benchmark data at the microarchitecture level obtained from deterministic replay is integrated and summarized with the global behavioral statistical features obtained from representative replay to form complete training data covering both local bottlenecks and the overall program, providing comprehensive and accurate data support for subsequent multi-granularity sub-model construction and model aggregation fitting.

[0045] Step S3: Based on the training data, sub-models are established at different granularity nodes of the performance structure tree, and hierarchical aggregation is performed from bottom to top according to the performance structure tree to obtain the overall program performance prediction model.

[0046] After acquiring complete hybrid replay training data, this embodiment leverages the hierarchical and multi-granularity node characteristics of the performance structure tree to construct dedicated sub-models for different types and levels of PST nodes, accurately adapting to the operational patterns of each program structure. Subsequently, based on the tree-like hierarchical relationship of the performance structure tree, a bottom-up aggregation approach is used to fuse the multiple sub-models, ultimately constructing an overall performance prediction model capable of fully representing the global operational characteristics of parallel programs. Compared to traditional single global modeling methods, this multi-granularity hierarchical modeling approach significantly improves prediction accuracy, with corresponding model error performance improvements such as... Figure 4 As shown, Figure 4 This is a schematic diagram illustrating the performance prediction error effect based on the performance structure tree, which can intuitively demonstrate the technical advantages of hierarchical aggregation modeling.

[0047] In the specific modeling process, differentiated modeling schemes are adapted for different functional nodes of PST to achieve accurate and targeted performance fitting. First, for the communication nodes, which are the leaf nodes of PST, a dedicated communication latency model is built. The core input variables of this model include three categories: communication message size, cluster communication topology, and communication type. Among them, the communication type is further subdivided into three mainstream parallel communication modes: neighbor communication, cross-rack remote communication, and aggregated communication. This application constructs an independent latency calculation function for each type of communication to adapt to the latency variation patterns of different communication scenarios, solving the problems of poor adaptability and large errors of traditional unified communication models.

[0048] Secondly, for the program computation node, code segment time estimation is completed based on platform calibration parameters and computation feature vectors. The core inputs to the computation node are the standardized computation feature vector and the target platform calibration penalty parameters, where the computation feature vector is defined as... The four parameters within the vector represent, in order, the number of instructions that can be executed per clock cycle, cache hit rate, branch prediction success rate, and memory bandwidth utilization. This application includes a dedicated time-consuming prediction formula:

[0049] in, Represents the i-th independent code segment; This represents the total number of instructions contained in the i-th code segment; This represents the number of instructions per cycle corresponding to the i-th code segment; Represents the original clock cycles consumed by the execution of the i-th code segment; This represents the cache miss penalty period corresponding to the i-th code segment; This represents the branch prediction missing penalty period corresponding to the i-th code segment; This represents the pipeline pause penalty cycle corresponding to the i-th code segment; These are adjustable weight parameters. All values ​​of the above calibration parameters and weight parameters were obtained through multiple sets of comparative experiments using a standard benchmark test program run on the target platform, ensuring that the parameter accuracy is fully compatible with the target operating platform.

[0050] For two typical control structure nodes in PST—loop nodes and branch nodes—a weighted aggregation method is used to calculate the time consumption of child nodes. For loop nodes, the overall time consumption is determined by the total time consumption of child nodes in a single iteration and the number of loop iterations. Specifically, it is the product of the cumulative time consumption of all lower-level child nodes in a single iteration and the number of iterations, accurately reflecting the cumulative time consumption characteristics of loop iterations. For branch nodes, the probabilistic randomness of branch execution is fully considered. The execution probability of each branch path is used as a weighting coefficient. The total time consumption of each branch path node is multiplied by the corresponding execution probability, and finally, the calculation results of all branch paths are summed to obtain the overall running time of the branch node, which closely matches the dynamic characteristics of actual branch execution in the program.

[0051] For function nodes, which serve as upper-level aggregation units, the time consumption data of all subordinate substructures are aggregated layer by layer, strictly following the call and containment relationships in the performance structure tree. The overall time consumption of a function node consists of two parts: the sum of the time consumption of all lower-level child nodes within the function body, and the additional call overhead generated by the function call process itself. This fully covers all sources of time consumption during function execution, avoiding model bias caused by omissions of local time consumption.

[0052] After all node sub-models of all granularities and types have been constructed and their parameters fitted, the global model aggregation process is initiated. This application adopts a bottom-up core aggregation logic, starting from the leaf nodes at the bottom of the PST, and aggregating the time consumption calculation results of each sub-node level by level upwards, synchronously updating the performance parameters of the upper-level internal nodes, and completing the sub-model fusion layer by layer. In some implementation scenarios, top-down global constraints can be used for auxiliary correction to further improve the rationality and accuracy of model aggregation, ultimately converging to obtain a global overall performance prediction model covering the entire program structure and process. Figure 4 It is clear that the PST multi-node hierarchical modeling and layer-by-layer aggregation scheme adopted in this application can significantly reduce performance prediction errors and effectively improve the overall prediction accuracy and stability of the model compared with the traditional single-level modeling method.

[0053] Step S4: Based on the overall performance prediction model of the program, the overall performance is decomposed into multiple independently modelable performance components. After fitting and calibrating the performance components based on the training data, they are extrapolated to the target prediction scale parameter.

[0054] After constructing the overall program performance prediction model, to achieve accurate performance prediction from small-scale sampling scenarios to large-scale target scenarios, this application adopts a component-based decomposition modeling approach. The overall program complexity and performance overhead are decomposed into multiple independent, individually modelable standardized performance components. Based on small-scale training data, parameter fitting and accuracy calibration of each component are completed. Finally, the performance of all components is extrapolated using the target prediction scale parameters to obtain the initial value of the overall performance prediction for large-scale scenarios. This component-based decomposition method effectively avoids the cumulative error problem of overall model extrapolation and significantly improves the adaptability of predictions for large-scale scenarios.

[0055] This application decomposes the overall performance of parallel programs into seven independently modelable performance components. Each component satisfies an additive decomposition relationship, has low coupling, and can be independently fitted and optimized. The overall performance decomposition formula is:

[0056] in, The parameter representing the size of the program problem; Represents the total number of parallel processors / processes; Components representing the total computation time of the program; Components representing the total communication time of the program; This represents the total overhead component for parallel synchronization operations; This component represents the total time spent on system I / O reads and writes. This represents the additional overhead components introduced by system-level noise.

[0057] Based on this, the communication time consumption component can be further subdivided and broken down, with the following breakdown formula:

[0058] In the formula, This represents the time spent on inter-process nearest neighbor communication. Represents the time consumed for cross-rack remote communication. This represents the time taken for global collection communication.

[0059] In addition, the calculation time component has a dedicated extended calculation formula:

[0060] In the formula This is the performance degradation factor obtained from small-scale test sampling. This represents the time required for the program to execute completely serially under a problem size of N. This represents the number of parallel processes.

[0061] All of the above components together constitute the overall program execution time. The performance changes of any component can be reflected independently, which facilitates targeted modeling and error optimization.

[0062] After decomposing the performance components, two core operations are performed for each type of independent performance component: fitting and calibration. The core purpose of the fitting operation is to use the small-scale sampled training data collected in step S2 to solve for the unknown parameters within each component model, determine the functional relationships and initial parameter values ​​of the basic models of each component, and enable the model to adapt to the program's operating rules in small-scale scenarios. The calibration operation, based on the fitting, compares the model's output calculation results with the high-precision microarchitecture-level benchmark data obtained through deterministic replay. Based on the comparison differences, the model parameters of each component are iteratively fine-tuned to minimize the deviation between the calculated model values ​​and the actual operating values, ensuring the prediction accuracy of the single-component model.

[0063] After all performance components have completed independent fitting and accurate calibration, large-scale performance extrapolation calculations are formally carried out. Combining the target prediction scale parameters obtained in step S1, namely the problem size N and the number of processes P, the scale variables upon which each performance component depends are precisely located, and substituted into each component model to solve for its time consumption value under the target large-scale parameters. Finally, based on the additive decomposition relationship of each component, the large-scale prediction results of all components are summed and aggregated to complete the multi-component fusion calculation, ultimately obtaining the initial value of the overall program performance prediction extrapolated to the target prediction scale parameters, providing reliable basic data for subsequent accurate calibration and correction.

[0064] Step S5: Use the high-precision reference data obtained by the deterministic replay to calibrate the extrapolation result of the target path, output the performance prediction result under the target scale, and complete the performance extrapolation prediction.

[0065] After the component-based extrapolation calculation in step S4, initial values ​​for program performance prediction under the target large-scale scenario can be obtained. However, since the parallel interaction logic, communication topology characteristics, and system noise interference of large-scale clusters differ from those of small-scale sampling scenarios, the results of extrapolation based solely on small-scale data will have certain systematic biases. To further eliminate prediction errors and improve the accuracy of results, this embodiment relies on high-precision benchmark data collected through deterministic replay to perform specific calibration and correction on the extrapolation results of critical paths and bottleneck regions, thereby achieving secondary optimization of prediction accuracy.

[0066] In the specific calibration process, the core critical path and performance bottleneck areas of the program are first accurately located again based on the performance structure tree. These areas are the core modules that affect the overall performance of the program and are also the main concentrated areas of large-scale extrapolation errors. Then, the micro-architecture-level high-precision time-series data and performance statistics data retained by deterministic replay for these areas in step S2 are retrieved. Using the high-precision benchmark data as the true value, the large-scale extrapolation results obtained in step S4 are compared and iteratively calibrated module by module. The prediction deviations of the critical path and bottleneck modules are corrected in a targeted manner, effectively solving the performance distortion problem when migrating small-scale models to large-scale scenarios.

[0067] After completing the global calibration and correction, complete and accurate performance prediction results for the target scale can be output. The output comprehensively covers all core performance indicators of the program, specifically including the overall total runtime of the program under the target large-scale parameters, the detailed time consumption of each module corresponding to each level of the performance structure tree, the proportion of various communication overheads, synchronization overheads, and I / O overheads in the total runtime, and the complete time-series breakdown data of the program's critical path. This completes the entire process of parallel program performance modeling and extrapolation prediction, from program structure analysis, mixed data acquisition, multi-granularity modeling and aggregation, component-based extrapolation to accurate calibration output.

[0068] Example 2 This invention also relates to a parallel program performance modeling and extrapolation prediction system for domestically produced supercomputing platforms. Figure 5 This is a schematic diagram of the overall architecture of a parallel program performance modeling and extrapolation prediction system for domestic supercomputing platforms, provided by an embodiment of the present invention. Based on the parallel program performance modeling and extrapolation prediction method described above, the system builds a modular execution architecture that integrates hardware and software. It fully covers the entire business process, including program input parsing, hierarchical structure modeling, hybrid replay data acquisition, feature clustering and filtering, multi-granularity sub-model aggregation, large-scale performance extrapolation, and result calibration output. There are standardized data interaction and feedback calibration mechanisms between the various functional modules. It is adaptable to various supercomputing cluster environments equipped with domestic processors, enabling accurate performance prediction from small-scale sampling scenarios to large-scale parallel scenarios with millions of cores.

[0069] Combination Figure 5 The system's various components and data flow logic are fully described. The entire system comprises six core functional modules and four types of external input sources. The data transmission, constraint, and return flow relationships between the inputs and modules are as follows: The system's external input sources are divided into four categories. The first category is parallel programs, specifically including parallel program source code files or compiled executable programs, serving as the basic program carrier for structure analysis and replay acquisition. The second category is platform parameters, covering a full set of hardware indicators such as domestic processor hardware models, processor operating frequencies, number of cores per node, cluster interconnection network topology, and communication bandwidth, providing platform benchmark parameters for microarchitecture feature calculation, communication latency modeling, and performance component calibration. The third category is target scale parameters, including user-defined prediction variables such as program problem size N and the number of parallel processes and processors P, used to drive the extrapolation prediction module to complete large-scale scene performance solutions. The fourth category is small-scale sampling constraint parameters, used to limit the range of cluster nodes and processes that can be used in the replay acquisition stage, controlling the resource overhead of data acquisition.

[0070] The structural analysis module receives parallel program input as the processing object. This module integrates intra-process analysis sub-units and inter-process analysis sub-units, and can sequentially complete the generation of single-function function structure trees, the assembly and splicing of global structure trees, and finally output a hierarchical performance structure tree (PST). The PST data carrying full node attribute information is synchronously pushed to the downstream modeling and aggregation module, providing a structured framework support for the construction of multi-granularity node sub-models.

[0071] The replay acquisition module simultaneously receives two types of basic data: parallel program input and platform parameters. Internally, it integrates two sets of parallel acquisition branches: a deterministic replay acquisition unit and a representative replay acquisition unit. Relying on small-scale cluster resources, it completes hybrid strategy data acquisition. The deterministic replay unit performs time-series-fidelity replay of critical paths and performance bottleneck areas identified by the performance structure tree, outputting standardized event sequences and high-precision benchmark data of the micro-architecture. The representative replay unit captures raw data from the execution trajectories of all processes, outputting a massive number of unfiltered process trajectory samples. The data produced by the two acquisition branches are uniformly distributed to the feature and clustering module for secondary processing.

[0072] The feature and clustering module receives the full process execution trajectory data output by the replay acquisition module. Internally, the module fully implements the complete logic for multi-dimensional feature mapping, similarity measurement, hierarchical clustering, and representative sample selection. First, it uses a feature extraction function to map a single process trajectory to a d-dimensional feature space. The system generates process feature vectors that include four dimensions: computational density, memory access, communication mode, and load balancing. Then, it uses weighted distance to measure sample similarity and performs hierarchical clustering to obtain multiple process clusters. Finally, it combines probability coverage constraints, importance sampling rules, and variance error thresholds to select representative process samples. The system outputs a simplified global statistical feature dataset, which is then transmitted to the modeling and aggregation module to participate in sub-model training and fitting.

[0073] The modeling and aggregation module synchronously receives the performance tree structure PST output from the structure analysis module and the training statistical features output from the feature and clustering module. Internally, the module configures dedicated sub-model fitting units for different types and granularities of nodes within the PST: for communication leaf nodes, it builds multi-type communication latency models; for computing nodes, it utilizes penalty parameters calibrated with domestically produced processors and calculates feature vectors. Complete the time consumption calculation for a single code segment, and use a corresponding prediction model. To address the performance losses caused by quantization cache missing, branch prediction failure, and pipeline pauses, weighted aggregation and hierarchical aggregation calculation logic are configured for nodes within loops, branches, and functions. After all node sub-models have completed parameter fitting, the module aggregates the sub-model results according to bottom-up tree traversal rules, generates a complete global performance prediction model for the program, and pushes this model to the downstream extrapolation prediction module.

[0074] The extrapolation prediction module simultaneously receives the global performance prediction model output by the modeling and aggregation modules and the target scale parameters from external input. Internally, the module employs an additive component decomposition approach to complete large-scale performance extrapolation. The overall performance decomposition formula is as follows: The computation time component relies on the extended model. After scaling up, the communication timing component was further subdivided into three sub-components: nearest neighbor communication, cross-rack remote communication, and global aggregated communication, corresponding to a decomposed approach. The synchronization overhead, system I / O overhead, and system noise overhead components are independently fitted and calibrated based on small-scale sampled data; the module substitutes the target scale parameters N and P to complete the large-scale time-consuming solution of each component and sum them, and outputs the uncalibrated initial extrapolation performance prediction results.

[0075] The output module receives the initial prediction data from the extrapolation prediction module. This module has a built-in critical path calibration and correction unit, which can retrieve and replay the high-precision benchmark data stored in the acquisition module. It corrects the prediction bias for bottlenecks and critical path modules marked in the performance structure tree, significantly reducing the system prediction error in large-scale scenarios. After calibration, the module outputs standardized prediction results, including the total program runtime at the target scale, the time consumption of each module at each level of the PST, the proportion of various communication / synchronization / I / O overhead, and a complete time sequence breakdown of the critical path.

[0076] besides, Figure 5The system also includes a training data / calibration feedback loop. This loop transmits the high-precision benchmark data output from the replay acquisition module back to the extrapolation prediction module. During the extrapolation solution phase, it continuously iterates and fine-tunes the model parameters, forming a closed-loop optimization mechanism of acquisition-modeling-extrapolation-calibration. This continuously improves the prediction accuracy and generalization adaptability of the entire system under different domestic supercomputing hardware and different parallel programs. The modules of the entire system have clear division of labor and a closed-loop data interaction logic. It can fully implement the parallel program performance modeling and extrapolation prediction method described above, solving the industry's technical pain points such as the difficulty in accurately predicting the running time of large-scale parallel programs on domestic supercomputing platforms, the inability to identify bottleneck modules in advance, and the huge resource consumption of traditional performance simulation.

[0077] Example 3 To implement the methods of the above embodiments, the present invention also provides an electronic device, which includes a memory and a processor; wherein the processor reads executable program code stored in the memory to run a program corresponding to the executable program code, so as to implement the various steps of the methods described above.

[0078] Example 4 To implement the above embodiments, this application also proposes a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the method described in the foregoing embodiments.

[0079] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0080] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0081] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified.

Claims

1. A method for parallel program performance modeling and extrapolation prediction for domestically developed supercomputing platforms, characterized in that, include: Obtain the parallel program and its corresponding target prediction scale parameters, perform intra-process analysis and inter-process analysis on the parallel program, and construct a performance structure tree to characterize the hierarchical structure of the program. The nodes of the performance structure tree include leaf nodes representing communication function calls and internal nodes representing program structure units. Based on the performance structure tree, deterministic replay is used on the target path and bottleneck area on a small scale to obtain high-precision benchmark data at the microarchitecture level, and representative replay is used on the overall execution behavior to identify representative processes and behavior patterns from the execution space to obtain statistical features. The high-precision benchmark data and statistical features are used as training data. Based on the training data, sub-models are established at different granularity nodes of the performance structure tree, and hierarchical aggregation is performed from bottom to top according to the performance structure tree to obtain the overall program performance prediction model. Based on the overall performance prediction model of the program, the overall performance is decomposed into multiple performance components that can be independently modeled. After fitting and calibrating the performance components based on the training data, the results are extrapolated to the target prediction scale parameter. The high-precision benchmark data obtained by the deterministic replay is used to calibrate the extrapolation results of the target path, and the performance prediction results under the target scale are output to complete the performance extrapolation prediction.

2. The method according to claim 1, characterized in that, The intra-process analysis and the inter-process analysis include: The process analysis establishes a function structure tree for each function. The node types of the function structure tree include function nodes representing the current function, loop nodes representing loop structures, branch nodes representing branch structures, branch path nodes representing a certain path under a branch structure, communication nodes representing communication function calls, and call nodes representing calls to non-communication functions. The inter-process analysis assembles the function structure trees into a global structure tree based on function call relationships. The global structure tree takes the function node corresponding to the main function as the root node, and during the assembly process, the calling node is replaced with the function structure tree of the called function, or the calling node is deleted when the set rules are met.

3. The method according to claim 2, characterized in that, In the performance structure tree, the leaf nodes are used to represent communication function calls, which include point-to-point communication primitive calls and set communication primitive calls. The internal nodes are used to represent program structure units, which include at least branching structures, looping structures, and functions. The edges of the performance structure tree are used to represent hierarchical relationships between nodes, which include inclusion relationships, calling relationships, and nested control structure relationships.

4. The method according to claim 3, characterized in that, Based on the aforementioned performance structure tree, deterministic replay is used on the target path and bottleneck regions with small-scale resources to obtain high-precision benchmark data at the microarchitecture level, including: Run a parallel program on the target platform and perform deterministic replay of the identified target path and bottleneck region. The target path is the path in the performance structure tree whose execution frequency is higher than a set threshold, and the bottleneck region is the node and its subtree region in the performance structure tree whose time consumption ratio is higher than a set threshold. Parallel execution is represented as an ordered sequence of events, with each event recorded as a quadruple, which includes a timestamp, process identifier, action type, and related data. The action type includes computation segment, communication call, and synchronization point, and the related data includes message size, destination process, source process, and code segment identifier. During deterministic replay, by recording the order in which events occur and the dependencies between events, it is ensured that the replay execution is consistent with the original execution at the event sequence level, and microarchitecture-level performance counter data corresponding to each event is collected; Clock offset is controlled by a logical clock synchronization model or a clock calibration mechanism provided by the target platform to maintain timing consistency. The logical clock synchronization model establishes timing constraints based on the partial order relationship between events, while the clock calibration mechanism achieves clock reference alignment between multiple cores and multiple nodes based on the cycle counter synchronization interface provided by the target platform hardware.

5. The method according to claim 4, characterized in that, Representative replay of the overall execution behavior is used to identify representative processes and behavioral patterns in the execution space to obtain statistical characteristics, including: A multidimensional feature space is constructed, and the execution trajectory of each process is mapped to a feature vector in the feature space through a feature extraction function. The feature extraction function takes the instruction mixture, memory access address sequence and communication event sequence in the process execution trajectory as input, and outputs a fixed-dimensional numerical feature vector. The feature vector includes computation density features, memory access features, communication mode features and load balancing features. After extracting feature vectors from the trajectories of each process, process clusters are obtained by similarity measurement and hierarchical clustering. Based on probability coverage and importance sampling theory, representative processes and sample sizes are selected from each process cluster to minimize the difference between the distribution of the selected representative processes in the feature space and the distribution of all processes. Variance analysis is used to control sampling error.

6. The method according to claim 5, characterized in that, Sub-models are built at different granularity nodes of the performance structure tree, including: A communication delay model is established for the communication nodes that are leaf nodes. The communication delay model is related to the message size, communication topology and communication type. The communication types include neighbor communication, cross-rack remote communication and aggregate communication. Corresponding delay functions are established for different communication types. The computation time of the computing node is estimated based on the computation feature vector and calibration parameters. The computation feature vector includes the number of instructions per clock cycle, cache hit rate, branch prediction rate and memory bandwidth utilization. The calibration parameters include cache miss penalty cycle, branch prediction miss penalty cycle and pipeline stall penalty cycle. The results are obtained by experimental measurement on the target platform by running standard benchmark test programs. The time consumption of child nodes is weighted and summarized based on the number of iterations and the branch probability for loop nodes and branch nodes respectively. The total time consumption of loop nodes is the product of the time consumption of a single iteration and the number of iterations, and the total time consumption of branch nodes is the sum of the products of the time consumption of each branch path node and the execution probability of that branch path. The time consumed by substructures of a function node is aggregated according to the calling relationship and the inclusion relationship. The total time consumed by a function node is the sum of the time consumed by each sub-node in the function body plus the calling overhead introduced by the function call itself.

7. The method according to claim 6, characterized in that, The overall performance is decomposed into multiple independently modelable performance components. These components are then fitted and calibrated based on the training data and extrapolated to the target prediction scale parameter, including: The overall performance is decomposed into computation time components, nearest neighbor communication time components, cross-rack remote communication time components, aggregate communication time components, synchronization overhead components, I / O overhead components, and system noise components, and the components satisfy an additive decomposition relationship. Based on the training data, each of the performance components is independently fitted and calibrated. The fitting process involves determining the model parameters of each component based on small-scale sampling data, and the calibration process involves comparing the fitted model output with the high-precision benchmark data obtained by deterministic replay and adjusting the parameters to reduce the deviation. After each performance component completes independent fitting and calibration, the value of each component under the target scale is calculated according to the scale variable on which each component depends in the target prediction scale parameter. Then, the components are combined and summed according to the decomposition relationship to obtain the overall performance prediction value extrapolated to the target prediction scale parameter.

8. An electronic device, characterized in that, Including processor and memory; The processor reads executable program code stored in the memory to run a program corresponding to the executable program code, so as to implement the method as described in any one of claims 1-7.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1-7.

Images