Performance prediction method and system for deep learning-based scientific computing application

Abstract

The present invention provides a performance prediction method and system for a deep learning-based scientific computing application. The method comprises: constructing a data set for training a deep learning model, wherein the training set comprises parameter data and performance data that are matched performance-wise, and forming a <parameter, performance> pair; providing a pre-trained generative large model, and using the <parameter, performance> pair to finely tune the pre-trained generative large model to obtain a finely-tuned model; and converting parameter configuration data of a scientific computing application to be predicted into a numerical vector by means of a specific encoding mode, inputting the numerical vector into the finely-tuned model, and performing feature extraction on inputted text data by means of the finely-tuned model to generate a corresponding performance prediction result. The present invention improves the prediction accuracy, reduces the computation overhead, improves the prediction effect, and enhances the generalization capability of the model.

Description

A Deep Learning-Based Method and System for Predicting the Performance of Scientific Computing Applications Technical Field

[0001] This invention relates to the field of computer science and the technology of performance prediction for scientific computing applications. Specifically, it relates to a method and system for predicting the performance of scientific computing applications based on deep learning, and also to a corresponding computer terminal and computer-readable storage medium. Background Technology

[0002] In recent years, deep learning-based scientific computing applications have been widely used in various high-performance computing tasks. Deep learning technology, with its outstanding performance in image processing, natural language processing, physics simulation, and bioinformatics, has become a core driving force for the advancement of scientific computing. These applications rely on the powerful representational capabilities of neural networks, achieving efficient solutions to complex problems through training on massive amounts of data. With the increasing penetration of deep learning in scientific computing, more and more research tasks are shifting towards relying on the training and inference of large-scale neural network models, which also brings new challenges and opportunities to high-performance computing (HPC).

[0003] To meet the enormous computing resource demands of deep learning-based scientific computing applications, supercomputing centers (HPC centers) have become the preferred operating environment. Supercomputing centers are typically equipped with massively parallel processing capabilities, including multiple nodes, multiple GPUs, and even hardware resources specifically designed for AI acceleration, such as NVIDIA A100 and Sugon DCUs. These computing resources are interconnected through high-speed networks, providing the ability to process massive amounts of data and complex models. However, despite the powerful computing capabilities offered by supercomputing centers, the actual performance of different deep learning tasks on these centers is still affected by various factors such as hardware configuration, network bandwidth, task size, and parallel computing efficiency.

[0004] As scientific research tasks become increasingly complex and large-scale, efficiently utilizing the resources of supercomputing centers has become a critical issue. Scientists need to understand the performance of different deep learning applications under varying computing configurations in order to allocate computing resources rationally, optimize task scheduling, and avoid resource waste. Furthermore, given the diversity of deep learning frameworks (such as TensorFlow and PyTorch) and model types (such as convolutional neural networks, recurrent neural networks, and generative adversarial networks), accurately predicting the performance of these applications not only helps improve resource utilization efficiency but also assists researchers in making more reasonable tuning and configuration choices before submitting tasks.

[0005] Predicting the performance of deep learning-based scientific computing applications is of great significance in multiple aspects, including resource allocation, improving research efficiency, controlling economic costs, and system design and optimization. However, existing methods for modeling and predicting the performance of scientific computing applications mainly rely on traditional methods based on numerical models or simulation systems, which have the following limitations:

[0006] Nonlinear relationships are difficult to model: The performance of scientific computing applications is affected by a variety of factors, such as hardware configuration, task size, and deep learning model structure. The relationships between these factors are complex, and traditional numerical models struggle to accurately capture these high-dimensional nonlinear relationships.

[0007] High computational overhead: Using traditional simulation systems to simulate performance under different parameter combinations requires a lot of computational resources and time, resulting in low efficiency.

[0008] Insufficient generalization ability: Traditional models can usually only provide effective predictions under specific hardware configurations or task types, making it difficult to adapt to the diverse needs of scientific computing applications.

[0009] Furthermore, in existing performance modeling and prediction processes, the parameter data used in scientific computing applications is mostly textual or numerical information, which is difficult to directly apply to complex deep learning models. At the same time, textual data is often too abstract to be directly processed by models, resulting in insufficient prediction accuracy.

[0010] Therefore, existing technologies have many limitations in predicting the performance of deep learning scientific computing applications, necessitating the development of new prediction methods and techniques to improve prediction accuracy, reduce computational overhead, and enhance the model's generalization ability. Currently, no descriptions or reports of technologies similar to this invention have been found, nor have similar domestic or international materials been collected. Summary of the Invention

[0011] To address the aforementioned shortcomings in the prior art, this invention provides a method and system for predicting the performance of scientific computing applications based on deep learning, along with a corresponding computer terminal and computer-readable storage medium.

[0012] According to one aspect of the present invention, a method for predicting the performance of scientific computing applications based on deep learning is provided, comprising:

[0013] Construct a dataset for training a deep learning model, wherein the training set includes parameter data and performance data corresponding to the performance, forming <parameter, performance> pairs;

[0014] Provide a pre-trained generative large model, and fine-tune the pre-trained generative large model using the <parameters, performance> to obtain the fine-tuned model;

[0015] The parameter configuration data of the scientific computing application to be predicted is converted into a numerical vector through a specific encoding method and input into the fine-tuned model. The fine-tuned model then extracts features from the input text data to generate corresponding performance prediction results.

[0016] Preferably, the dataset for training the deep learning model comprises:

[0017] Obtain performance data for scientific computing applications running under different HPC configurations;

[0018] The parameter data and performance data from the diverse performance data obtained are mapped to form <parameter, performance> text data, and a dataset is constructed.

[0019] Preferably, the performance data includes performance index parameters and performance parameters; wherein:

[0020] The performance index parameters include:

[0021] Training time parameter: The total time required to complete a scientific computing task or achieve a certain level of accuracy;

[0022] Resource utilization parameters: GPU / CPU utilization, memory consumption, and I / O bandwidth;

[0023] Convergence speed parameter: The speed at which a deep learning model reaches a predetermined accuracy or loss value during training;

[0024] Performance parameters obtained using the training time parameter, resource utilization parameter, and / or convergence speed parameter under different parameter combinations include:

[0025] Hardware parameters: GPU model, number of nodes, and computing resource allocation;

[0026] Deep learning model parameters: model type, number of layers, parameter size, and batch size;

[0027] Task configuration: dataset size, number of nodes for distributed training, and gradient synchronization method.

[0028] Preferably, the provision of a pre-trained generative large model, and the fine-tuning of the pre-trained generative large model using the <parameters, performance> to obtain the fine-tuned model, includes:

[0029] A pre-trained generative large model is provided. The parameters in the <parameter, performance> pair are used as parameters for distributed training of AI applications. The encoded text data of the parameters for distributed training of AI applications is used as the input of the model. The time required for distributed training of AI applications to converge to one epoch is used as the output of the model. The generative large model is fine-tuned to obtain the fine-tuned model.

[0030] The forward propagation path of the fine-tuned model is as follows: text input → embedding of discrete and continuous data → generating visualized image data → preprocessing image data → decoding into text.

[0031] Preferably, the process of converting the parameter configuration data of the scientific computing application to be predicted into a numerical vector using a specific encoding method includes:

[0032] By using a specific encoding method, discrete data variables in the parameter configuration data of the scientific computing application to be predicted are transformed into numerical vectors, and continuous numerical variables in the parameter configuration data of the scientific computing application to be predicted are directly processed into numerical vectors, thus constructing a unified vector representation space.

[0033] Preferably, the specific encoding method includes:

[0034] Discrete data variables are transformed into numerical vectors using word embedding or one-hot encoding methods; where:

[0035] The word vector embedding method includes: defining an embedding matrix E∈R^{n×d}, where n is the number of discrete data categories and d is the vector dimension; and obtaining the vector representation corresponding to each discrete data value.

[0036] The one-hot encoding method includes: representing each category as a sparse vector of length n, where n is the number of categories of the discrete variable, each category corresponds to one position of 1 and other positions of 0; and through mapping, using matrix multiplication or directly constructing a sparse vector, transforming each discrete variable into a sparse vector after one-hot encoding.

[0037] The Min-Max normalization method or the Z-score normalization method are used to directly process continuous numerical variables into numerical vectors;

[0038] The numerical vectors of discrete variables are concatenated with the numerical vectors of continuous variables to form a unified vector representation.

[0039] Preferably, the parameter configuration data of the scientific computing application to be predicted includes: HPC hardware configuration data and deep learning model configuration data.

[0040] According to another aspect of the present invention, a deep learning-based scientific computing application performance prediction system is provided, comprising:

[0041] The training dataset construction module is used to construct a dataset for training deep learning models. The training set includes parameter data and performance data corresponding to the performance, forming <parameter, performance> pairs.

[0042] The model training module provides a pre-trained generative large model, which is then fine-tuned using the <parameters, performance> to obtain the fine-tuned model.

[0043] The performance prediction module is used to convert the parameter configuration data of the scientific computing application to be predicted into a numerical vector, and input the numerical vector into the fine-tuned model through an encoder to extract features from the input text data and generate the corresponding performance prediction results.

[0044] According to a third aspect of the present invention, a computer terminal is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it can be used to perform the method described in any one of the above inventions, or to run the system described in the above inventions.

[0045] According to a fourth aspect of the present invention, a computer-readable storage medium is provided having a computer program stored thereon, characterized in that, when executed by a processor, the computer program can be used to perform the method described in any one of the above-described inventions, or to run the system described in the above-described inventions.

[0046] By adopting the above technical solution, the present invention has at least one of the following beneficial effects compared with the prior art:

[0047] Efficient nonlinear relationship modeling: The deep learning-based scientific computing application performance prediction method and system provided in this invention can effectively model multidimensional nonlinear relationships in scientific computing applications through the training and fine-tuning of deep learning models, which greatly improves the accuracy of performance prediction, especially in complex multi-parameter scenarios.

[0048] Improved computational efficiency: The deep learning-based scientific computing application performance prediction method and system provided by this invention, compared with traditional simulation systems, can predict performance results in real time after fine-tuning of its deep learning model, greatly reducing the consumption of computing resources and significantly improving prediction efficiency.

[0049] Strong generalization ability: The scientific computing application performance prediction method and system based on deep learning provided by this invention has good representation ability and adaptability based on generative large models. It can adapt to various hardware configurations and deep learning task types, and has wide applicability.

[0050] Improved Model Applicability: The deep learning-based scientific computing application performance prediction method and system provided in this invention expands the application of generative large models in the field of performance prediction by encoding text into images, enhancing the model's adaptability and versatility. Different types of scientific computing application data can be processed through a unified image format, simplifying cross-application prediction. By fully utilizing the powerful inherent capabilities of generative models, and by encoding text into images and then decoding images back into text, it solves the problem that pre-trained generative models cannot directly generate the performance text data to be predicted, effectively obtaining the predicted performance data through image decoding into text.

[0051] Improved accuracy: The deep learning-based scientific computing application performance prediction method and system provided by this invention, through image input, fully utilizes the strong representational capabilities of generative large models, improves the accuracy and stability of performance prediction, and enables complex parameter relationships to be better captured. Attached Figure Description

[0052] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0053] Figure 1 is a flowchart of the performance prediction method for scientific computing applications based on deep learning in a preferred embodiment of the present invention.

[0054] Figure 2 is an architecture diagram of the performance prediction model in a preferred embodiment of the present invention.

[0055] Figure 3 is a schematic diagram illustrating the performance of predicting scientific computing applications on future supercomputing platforms in a preferred embodiment of the present invention.

[0056] Figure 4 is a schematic diagram of the components of a deep learning-based scientific computing application performance prediction system according to a preferred embodiment of the present invention. Detailed Implementation

[0057] The embodiments of the present invention are described in detail below: These embodiments are implemented based on the technical solution of the present invention, and provide detailed implementation methods and specific operation processes. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present invention, and these all fall within the protection scope of the present invention.

[0058] Existing technologies have many limitations in predicting the performance of deep learning-based scientific computing applications. To overcome these shortcomings, one embodiment of this invention provides a deep learning-based method for predicting the performance of scientific computing applications. This method introduces deep learning technology and generative large models. First, it utilizes the powerful representational capabilities of large models to replace the complex process of traditional numerical modeling, enabling better capture of the nonlinear characteristics of performance under different configurations. Second, it uses the visual feature learning capabilities of generative large models for performance prediction, allowing complex parameter information to be input into the deep learning model in an intuitive and easily processed manner. Through vector encoding, it achieves efficient prediction of scientific computing application performance, improving prediction accuracy, reducing computational overhead, enhancing prediction results, and strengthening the model's generalization ability.

[0059] Specifically, as shown in Figure 1, the deep learning-based scientific computing application performance prediction method provided in this embodiment may include the following operations:

[0060] S1, Construct a dataset for training deep learning models. The training set includes parameter data and performance data corresponding to the performance, forming <parameter, performance> pairs;

[0061] S2 provides a pre-trained generative large model, and uses <parameters, performance> to fine-tune the pre-trained generative large model to obtain the fine-tuned model;

[0062] S3 converts the parameter configuration data of the scientific computing application to be predicted into a numerical vector through a specific encoding method and inputs it into the fine-tuned model. The fine-tuned model then extracts features from the input text data to generate the corresponding performance prediction results.

[0063] In some preferred embodiments, S1 above, which involves constructing a dataset for training a deep learning model, may further include the following operations:

[0064] S11, obtain performance data of scientific computing applications running under different HPC configurations;

[0065] S12, the parameter data and performance data in the obtained diverse performance data are matched to form <parameter, performance> pairs, and the dataset is constructed.

[0066] In some preferred embodiments, the performance data in S11 above may further include: performance index parameters and performance parameters; wherein:

[0067] Performance metrics parameters include:

[0068] Training time parameter: The total time required to complete a scientific computing task or achieve a certain level of accuracy;

[0069] Resource utilization parameters: GPU / CPU utilization, memory consumption, and I / O bandwidth;

[0070] Convergence speed parameter: The speed at which a deep learning model reaches a predetermined accuracy or loss value during training;

[0071] Performance parameters obtained using training time parameters, resource utilization parameters, and / or convergence speed parameters, under different parameter combinations, include:

[0072] Hardware parameters: GPU model, number of nodes, and computing resource allocation;

[0073] Deep learning model parameters: model type, number of layers, parameter size, and batch size;

[0074] Task configuration: dataset size, number of nodes for distributed training, and gradient synchronization method.

[0075] In some preferred embodiments, S2 above, which provides a pre-trained generative large model, fine-tunes the pre-trained generative large model using <parameters, performance> to obtain a fine-tuned model, may further include the following operations:

[0076] S21 provides a pre-trained generative large model (such as a diffusion model), and uses the parameters in the <parameter, performance> pair as parameters for distributed training of the AI ​​application; further, the parameters for distributed training of the AI ​​application may include: number of nodes, number of GPUs, I / O performance, CPU performance, GPU bandwidth, dataset size, batch size, etc.

[0077] S22, the encoded text data of the parameters trained by the AI ​​application in a distributed manner is used as the input of the model, and the time required for the AI ​​application in a distributed manner to converge to one epoch is used as the output of the model. The generative large model is then fine-tuned to obtain the fine-tuned model.

[0078] S23, the forward propagation path of the fine-tuned model is: text input → embedding of discrete and continuous data → generating visualized image data → preprocessing (e.g., normalization, cropping) of image data → decoding into text, and then obtaining performance indicators. As shown in Figure 2.

[0079] In S22 above, the encoded text data of the parameters trained by the AI ​​application in a distributed manner can further include the following operations:

[0080] Word embedding or one-hot encoding techniques are used to transform discrete data variables in <parameter, performance> pairs into numerical vectors, and continuous numerical variables in <parameter, performance> pairs are directly processed into numerical vectors, constructing a unified vector representation space to obtain text data. Specifically, a word embedding layer similar to that in language models is used, defining an embedding matrix E∈R^{n×d}, where n is the number of discrete data categories and d is the vector dimension. Each discrete data value obtains its corresponding vector representation by looking up a table. The embedding layer can be randomly initialized or pre-trained based on prior knowledge or historical performance data. Word embedding can capture the implicit relationships between discrete variables, helping the model to better handle differences between devices.

[0081] In some preferred embodiments, the encoding method used in S22 above may further include the following operations:

[0082] (1) Word embedding initialization based on prior knowledge. If discrete variables have prior semantic or performance information, they can be initialized using embeddings based on prior knowledge. For example, hardware variables such as GPU model and CPU model often have performance indicators, which can be used to initialize the embedding vector. This allows the model to have a certain feature discrimination ability in the early stages of training. The specific method is to use the actual performance data of the hardware (such as the computing power and bandwidth of the GPU) to assign initial embeddings to discrete variables. For example, based on the performance parameters of the device, an initial embedding table can be constructed, and the hardware performance data can be mapped to the embedding vector through linear regression or other regression methods.

[0083] (2) Utilize the interrelationships of hardware configurations in distributed training scenarios, such as communication bandwidth between nodes and the coupling performance between CPU and GPU. Adaptive attention mechanisms can be designed to address these hardware relationships, ensuring that the model can capture these specific dependencies.

[0084] In some preferred embodiments, S3 above, which converts the parameter configuration data of the scientific computing application to be predicted into a numerical vector through a specific encoding method, may further include the following operations:

[0085] S31, through a specific encoding method, converts discrete data variables in the parameter configuration data of the scientific computing application to be predicted into numerical vectors, and directly processes continuous numerical variables in the parameter configuration data of the scientific computing application to be predicted into numerical vectors, thus constructing a unified vector representation space.

[0086] In some preferred embodiments, the specific encoding method in S31 above may further include:

[0087] Discrete data variables are transformed into numerical vectors using word embedding or one-hot encoding methods; where:

[0088] The word vector embedding method includes: defining an embedding matrix E∈R^{n×d}, where n is the number of discrete data categories and d is the vector dimension; and obtaining the vector representation corresponding to each discrete data value.

[0089] The one-hot encoding method includes: representing each category as a sparse vector of length n, where n is the number of categories of the discrete variable, with one position of 1 for each category and other positions of 0; and transforming each discrete variable into a sparse vector after one-hot encoding by mapping, using matrix multiplication or directly constructing a sparse vector.

[0090] The Min-Max normalization method or the Z-score normalization method are used to directly process continuous numerical variables into numerical vectors;

[0091] The numerical vectors of discrete variables are concatenated with the numerical vectors of continuous variables to form a unified vector representation.

[0092] It should be noted that the encoding method of S31 described above can also be used to encode the parameters of distributed training of AI applications into text data.

[0093] In some preferred embodiments, the parameter configuration data of the scientific computing application to be predicted in S3 above may further include: HPC hardware configuration data and deep learning model configuration data.

[0094] The above embodiments mainly include the following parts and achieve corresponding innovations in performance prediction:

[0095] The data encoding and visualization mapping section converts abstract numerical or symbolic data such as hardware configurations and model parameters of scientific computing applications into text data through encoding. This encoding method not only better represents the high-dimensional parameter space but also enables generative large models to acquire performance prediction capabilities through feature learning. In this section, when converting parameter data into text data, discrete data variables in the <parameter, performance> pair are transformed into numerical vectors, while continuous numerical variables in the <parameter, performance> pair are directly processed into numerical vectors, constructing a unified vector representation space to facilitate recognition and learning by generative large models.

[0096] The fine-tuning part of the deep learning model uses a large-scale pre-trained model and fine-tunes it with parameter-performance data from scientific computing applications. This enables the model to generate corresponding performance predictions based on the input parameter graph. The model leverages its representational capabilities to capture complex nonlinear relationships between parameters. Specifically, this part uses a training set containing performance data from scientific computing applications under different parameter configurations to fine-tune the pre-trained model, improving its prediction accuracy for specific tasks.

[0097] The vectorization representation of text data first transforms the parameters of scientific computing applications (such as GPU type, model structure, task scale, etc.) into numerical vectors, representing discrete and continuous variables in vector form to facilitate subsequent data processing and image generation. In this part, techniques such as word embedding or one-hot encoding are used to convert discrete variables into numerical vectors, while continuous variables are directly quantified to construct a unified vector representation space.

[0098] In the training and inference part of the large-scale generative model, features are extracted from the input text data. Through fine-tuning, the model generates corresponding performance prediction results. In this part, the generative large-scale model employs deep learning architectures such as convolutional neural networks and Transformers, enabling it to extract rich feature information from the input image and output accurate performance prediction results through fine-tuning. The model's forward propagation path is: text input → embedding of discrete and continuous data → image generation → image normalization and cropping → decoding into text (performance metrics).

[0099] To better understand the technical solutions provided by the above embodiments of the present invention, the specific implementation steps of the present invention will be further described in detail below.

[0100] The deep learning-based scientific computing application performance prediction method provided in the above embodiments of the present invention utilizes deep learning technology, combined with generative pre-trained models (such as Stable Difference), to predict the performance of scientific computing applications in high-performance computing (HPC) environments. Its specific implementation steps include:

[0101] 1. Construct the training dataset

[0102] In this specific implementation step, the first step is to construct a dataset capable of training deep learning models. This dataset mainly consists of performance data from scientific computing applications running under different HPC configurations. It includes:

[0103] (1) Performance indicators and parameters may include, but are not limited to:

[0104] Training time: The total time required to complete a scientific computing task or achieve a certain level of accuracy;

[0105] Resource utilization: such as GPU / CPU utilization, memory consumption, I / O bandwidth, etc.;

[0106] Convergence speed: The speed at which a deep learning model reaches a predetermined accuracy or loss value during training.

[0107] (2) Performance parameters: In order to obtain these parameter data, users can collect performance data under different combinations of performance index parameters, including but not limited to the following dimensions:

[0108] Hardware parameters: GPU model (e.g., A100, V100, RTX4090, etc.), number of nodes, and computing resource allocation (GPU, CPU core count, etc.);

[0109] Deep learning model parameters: model type (e.g., convolutional neural network, recurrent neural network, Transformer, etc.), number of layers, parameter size, batch size;

[0110] Task configuration: dataset size, number of nodes for distributed training, gradient synchronization method, etc.

[0111] These performance metrics correspond to performance parameters, forming a <parameter, performance> pair, which constructs the training data. By collecting runtime data from diverse HPC configurations and deep learning tasks, sufficient samples can be provided for subsequent model training.

[0112] 2. Data Conversion

[0113] To leverage the power of pre-trained generative models (such as Stable Diffusion), this embodiment of the invention transforms the <parameter, performance> pair to make it suitable for visual model input. Specifically, the transformation method is as follows:

[0114] Data encoding: Converting HPC parameters (such as GPU type and number of nodes) and deep learning model configurations (such as model type and batch size) into numerical vectors or symbolic representations. Then, using specific encoding methods, transforming this information into a visual form.

[0115] Visualization Mapping: The numerical vectors are converted into RGB images using the fine-tuned large model. This process involves discretizing the parameters and mapping them to different regions of the image according to different parameter dimensions. For example, pixels of different colors can represent GPU models, and different brightness or textures can represent model parameters, task configurations, etc. This step is similar to converting text data into visual features, leveraging the strong representational capabilities of visual generative models.

[0116] To ensure that the pre-trained generative model can effectively handle this type of task, we fine-tune it using the dataset constructed in step 1. During fine-tuning, the model learns how to generate accurate performance predictions based on features in the images.

[0117] 3. Generate prediction results

[0118] After training and fine-tuning the model, the prediction system of this invention can generate performance prediction results based on the input scientific computing application parameters. The specific operation is as follows:

[0119] Users input the parameter configurations for scientific computing applications, such as HPC hardware configurations and deep learning model configurations.

[0120] The system uses an encoder to convert these parameters into the same image format as the training data.

[0121] The generative pre-trained model receives the image input and generates corresponding performance predictions, such as training time and resource utilization. These predictions, based on the model's learning under different configurations, provide users with performance estimates.

[0122] It is important to note that different scientific computing applications exhibit significantly different performance characteristics due to variations in their model structures and task complexities. Therefore, when using this invention for performance prediction, independent models and datasets should be established for specific applications to avoid mixing data from different types of tasks, thereby improving prediction accuracy.

[0123] The technical solution provided by the above embodiments of the present invention will be further described in detail below with reference to a specific application example.

[0124] In this specific application example, taking the application of deep learning-based scientific computing in image processing as an example, we predict its application performance. Specifically:

[0125] To predict the performance of image processing AI applications in future large-scale HPCs, this specific application example first collects runtime data of image processing AI applications on a small-scale HPC, and then fine-tunes and predicts the performance using the model provided in the above embodiments of the present invention. Next, when a large-scale HPC is built and available, a small amount of runtime data of image processing AI applications is collected on it. Using transfer learning, this data is used to further fine-tune the model provided in the above embodiments of the present invention, ultimately successfully predicting the performance of image processing AI applications on large-scale HPCs. Its working principle is shown in Figure 3.

[0126] Based on the same inventive concept, one embodiment of the present invention also provides a performance prediction system for scientific computing applications based on deep learning.

[0127] Specifically, as shown in Figure 4, the deep learning-based scientific computing application performance prediction system provided in this embodiment may include the following modules:

[0128] The training dataset building module is used to build a dataset for training deep learning models. The training set includes parameter data and performance data corresponding to the performance, forming <parameter, performance> pairs.

[0129] The model training module provides a pre-trained generative large model and fine-tunes the pre-trained generative large model using <parameters, performance> to obtain the fine-tuned model.

[0130] The performance prediction module is used to convert the parameter configuration data of the scientific computing application to be predicted into numerical vectors, and input the numerical vectors into the fine-tuned model through an encoder. It then extracts features from the input text data and generates the corresponding performance prediction results.

[0131] It should be noted that the steps in the method provided by the present invention can be implemented using corresponding modules, devices, units, etc. in the system. Those skilled in the art can refer to the technical solution of the method to realize the composition of the system. That is, the embodiments in the method can be understood as preferred examples for building the system, and will not be elaborated here.

[0132] An embodiment of the present invention also provides a computer terminal, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it can be used to execute the method of any of the above embodiments of the present invention, or to run the system of any of the above embodiments of the present invention.

[0133] Optionally, the memory is used to store programs; the memory may include volatile memory, such as random-access memory (RAM), such as static random-access memory (SRAM), double data rate synchronous dynamic random-access memory (DDR SDRAM), etc.; the memory may also include non-volatile memory, such as flash memory. The memory is used to store computer programs (such as application programs, functional modules, etc. that implement the above methods), computer instructions, etc., and the aforementioned computer programs, computer instructions, etc., can be partitioned and stored in one or more memories. Furthermore, the aforementioned computer programs, computer instructions, data, etc., can be accessed by the processor.

[0134] The aforementioned computer programs, computer instructions, etc., can be stored in partitions within one or more memory locations. Furthermore, the aforementioned computer programs, computer instructions, data, etc., can be accessed by a processor.

[0135] A processor is used to execute computer programs stored in memory to implement the various steps of the methods or various modules of the systems involved in the above embodiments. For details, please refer to the relevant descriptions in the preceding method and system embodiments.

[0136] The processor and memory can be separate structures or integrated structures. When the processor and memory are separate structures, they can be coupled together via a bus.

[0137] An embodiment of the present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, can be used to perform the method of any of the above embodiments of the present invention, or to run the system of any of the above embodiments of the present invention.

[0138] Computer-readable media include computer storage media and communication media, wherein communication media include any medium that facilitates the transfer of computer programs from one place to another. Storage media can be any available medium accessible to a general-purpose or special-purpose computer. An exemplary storage medium is coupled to a processor, enabling the processor to read information from and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and storage medium can reside in an ASIC. Alternatively, the ASIC can reside in a user device. Of course, the processor and storage medium can also exist as discrete components in a communication device.

[0139] The scientific computing application performance prediction method and system based on deep learning provided in the above embodiments of the present invention employ the following innovative means to ensure prediction accuracy and practicality:

[0140] Applications of generative pre-trained models: By converting <parameter, performance> pairs into image data and leveraging the powerful learning capabilities of generative models such as Stable Diffusion, visual performance prediction has been pioneered in the field of deep learning. This approach fully utilizes the large-scale parameter representation capabilities of pre-trained models and can be fine-tuned to adapt to specific scientific computing applications.

[0141] Encoding and Visualization of Multidimensional Data: This invention addresses the challenge of traditional numerical models capturing complex nonlinear relationships between parameters by encoding multidimensional HPC configurations and deep learning task parameters into image information. This method enables prediction models to effectively handle high-dimensional inputs.

[0142] Efficient Prediction Process: Compared to traditional performance prediction methods that rely on complex numerical models or simulation systems, this invention simplifies the prediction process through generative models. Users only need to input specific task parameters to obtain rapid performance prediction results through pre-trained models, greatly improving the real-time performance of predictions and the flexibility of applications.

[0143] Through the detailed description of this embodiment, the present invention proposes an innovative deep learning-based scientific computing application performance prediction scheme, which significantly simplifies the process while improving the accuracy of performance prediction and provides researchers with an efficient tool for task configuration.

[0144] The method and system for predicting the performance of scientific computing applications based on deep learning provided in the above embodiments of the present invention introduce deep learning technology and generative large models, and achieve efficient prediction of the performance of scientific computing applications through vector encoding. This scheme utilizes the powerful representational capabilities of large models to replace the complex process of traditional numerical modeling, enabling better capture of the nonlinear characteristics of performance under different configurations. Through large models, text data is first generated into images, then preprocessed and decoded to the predicted performance (text), allowing complex parameter information to be input into the deep learning model in an intuitive and easily processed manner, thereby improving prediction results.

[0145] Any matters not covered in the above embodiments of the present invention are well-known in the art.

[0146] The specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various modifications or variations within the scope of the claims, which do not affect the essence of the present invention.

Claims

1. A method for predicting the performance of scientific computing applications based on deep learning, characterized in that, include: Construct a dataset for training a deep learning model, wherein the training set includes parameter data and performance data corresponding to the performance, forming <parameter, performance> pairs; Provide a pre-trained generative large model, and fine-tune the pre-trained generative large model using the <parameters, performance> to obtain the fine-tuned model; The parameter configuration data of the scientific computing application to be predicted is converted into a numerical vector through a specific encoding method and input into the fine-tuned model. The fine-tuned model then extracts features from the input text data to generate corresponding performance prediction results.

2. The method for predicting the performance of scientific computing applications based on deep learning according to claim 1, characterized in that, The dataset used to train the deep learning model includes: Obtain performance data for scientific computing applications running under different HPC configurations; The parameter data and performance data from the diverse performance data obtained are mapped to form <parameter, performance> text data, and a dataset is constructed.

3. The method for predicting the performance of scientific computing applications based on deep learning according to claim 2, characterized in that, The performance data includes performance index parameters and performance parameters; wherein: The performance index parameters include: Training time parameter: The total time required to complete a scientific computing task or achieve a certain level of accuracy; Resource utilization parameters: GPU / CPU utilization, memory consumption, and I / O bandwidth; Convergence speed parameter: The speed at which a deep learning model reaches a predetermined accuracy or loss value during training; Performance parameters obtained using the training time parameter, resource utilization parameter, and / or convergence speed parameter under different parameter combinations include: Hardware parameters: GPU model, number of nodes, and computing resource allocation; Deep learning model parameters: model type, number of layers, parameter size, and batch size; Task configuration: dataset size, number of nodes for distributed training, and gradient synchronization method.

4. The method for predicting the performance of scientific computing applications based on deep learning according to claim 1, characterized in that, The provision of a pre-trained generative large model, and the fine-tuning of the pre-trained generative large model using the <parameters, performance> to obtain the fine-tuned model, includes: A pre-trained generative large model is provided. The parameters in the <parameter, performance> pair are used as parameters for distributed training of AI applications. The encoded text data of the parameters for distributed training of AI applications is used as the input of the model. The time required for distributed training of AI applications to converge to one epoch is used as the output of the model. The generative large model is fine-tuned to obtain the fine-tuned model. The forward propagation path of the fine-tuned model is as follows: text input → embedding of discrete and continuous data → generating visualized image data → preprocessing image data → decoding into text.

5. The method for predicting the performance of scientific computing applications based on deep learning according to claim 1, characterized in that, The process of converting the parameter configuration data of the scientific computing application to be predicted into a numerical vector using a specific encoding method includes: By using a specific encoding method, discrete data variables in the parameter configuration data of the scientific computing application to be predicted are transformed into numerical vectors, and continuous numerical variables in the parameter configuration data of the scientific computing application to be predicted are directly processed into numerical vectors, thus constructing a unified vector representation space.

6. The method for predicting the performance of scientific computing applications based on deep learning according to claim 5, characterized in that, The specific encoding method includes: Discrete data variables are transformed into numerical vectors using word embedding or one-hot encoding methods; where: The word vector embedding method includes: defining an embedding matrix E∈R^{n×d}, where n is the number of discrete data categories and d is the vector dimension; and obtaining the vector representation corresponding to each discrete data value. The one-hot encoding method includes: representing each category as a sparse vector of length n, where n is the number of categories of the discrete variable, each category corresponds to one position of 1 and other positions of 0; and through mapping, using matrix multiplication or directly constructing a sparse vector, transforming each discrete variable into a sparse vector after one-hot encoding. The Min-Max normalization method or the Z-score normalization method are used to directly process continuous numerical variables into numerical vectors; The numerical vectors of discrete variables are concatenated with the numerical vectors of continuous variables to form a unified vector representation.

7. The method for predicting the performance of scientific computing applications based on deep learning according to claim 1, characterized in that, The parameter configuration data for the scientific computing application to be predicted includes: HPC hardware configuration data and deep learning model configuration data.

8. A performance prediction system for scientific computing applications based on deep learning, characterized in that, include: The training dataset construction module is used to construct a dataset for training deep learning models. The training set includes parameter data and performance data corresponding to the performance, forming <parameter, performance> pairs. The model training module provides a pre-trained generative large model, which is then fine-tuned using the <parameters, performance> to obtain the fine-tuned model. The performance prediction module is used to convert the parameter configuration data of the scientific computing application to be predicted into a numerical vector, and input the numerical vector into the fine-tuned model through an encoder to extract features from the input text data and generate the corresponding performance prediction results.

9. A computer terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it can be used to perform the method of any one of claims 1-7, or to run the system of claim 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program can be used to perform the method of any one of claims 1-7, or to run the system of claim 8.

Images