A feature and knowledge driven optimal parallelism prediction method, system and medium

Abstract

The application discloses a feature and knowledge driven optimal parallelism prediction method, system and medium, the method comprises the following steps: obtaining the multivariate runtime features of a target parallel program and embedding and coding to obtain multivariate fusion features; inputting the multivariate fusion features into a pre-trained speedup ratio prediction model to predict the speedup ratios of the target parallel program under a plurality of given parallelisms, the speedup ratio prediction model is a neural network prediction model, and a mapping relationship between the multivariate fusion features of the parallel program and the speedup ratios under the plurality of given parallelisms is established by training the training sample of the multivariate fusion features of the parallel program sample and the speedup ratio prior knowledge under different parallelisms; selecting the parallelism with the maximum speedup ratio as the optimal parallelism obtained by prediction. The application aims to realize efficient, accurate and reliable prediction of the optimal parallelism of the program, improve the computing efficiency, reduce the system resource consumption and reduce the execution time of the program.

Description

Technical Field

[0001] This invention relates to scheduling optimization techniques for parallel programs, specifically to a feature- and knowledge-driven method, system, and medium for predicting optimal parallelism. Background Technology

[0002] With the continuous growth in computing power demands, the scale of parallel programs is increasing dramatically, leading to a surge in parallel computing capabilities. Numerous parallel applications are applied in cutting-edge fields such as climate simulation, fluid dynamics analysis, and deep learning training. Optimal parallelism, a key indicator determining the performance and scalability of a parallel program, refers to the parallel scale required for the program to complete its target task in the shortest possible time. When the parallel scale of a parallel program exceeds the optimal parallelism, issues such as increased communication overhead, intensified resource contention, and load imbalance can lead to longer execution times and wasted system resources. Existing methods for predicting the optimal parallelism of parallel programs rely on static modeling, but these methods suffer from problems such as insufficient feature representation, scarce training samples, and inadequate predictive reliability. Summary of the Invention

[0003] The technical problem to be solved by this invention is to provide a feature- and knowledge-driven optimal parallelism prediction method, system, and medium to address the above-mentioned problems in existing parallel program optimal parallelism prediction. This invention aims to solve the technical problems of insufficient feature representation, scarce training samples, and insufficient rationality and generalization ability of prediction results in existing parallel program optimal parallelism prediction, so as to achieve efficient, accurate, and reliable prediction of the optimal parallelism of the program, improve computational efficiency, reduce system resource consumption, and reduce program execution time.

[0004] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0005] A feature- and knowledge-driven method for predicting optimal parallelism includes the following steps:

[0006] S101, obtain the multi-dimensional runtime features of the target parallel program and embed them into the encoding to obtain multi-dimensional fusion features;

[0007] S102, input the multi-dimensional fusion features into the pre-trained speedup prediction model to predict the speedup of the target parallel program under given multiple degrees of parallelism. The speedup prediction model is a neural network prediction model, and the mapping relationship between the multi-dimensional fusion features of the parallel program and the speedup under given multiple degrees of parallelism is established by training samples constructed in advance using the multi-dimensional fusion features of the parallel program samples and the prior knowledge of the speedup under different degrees of parallelism.

[0008] S103, select the parallelism with the highest speedup ratio as the optimal parallelism obtained from the prediction.

[0009] Optionally, the multi-dimensional runtime features in step S101 include the computational features, memory access features, communication features, and program identifier of the target parallel program during runtime. The computational features are CPU-related data extracted from performance counter data during the parallel program's execution. The memory access features are memory access-related data during the parallel program's execution. The communication features are inter-process behavior data during the parallel program's execution. Furthermore, when embedding and encoding the program identifier, it includes determining whether the target parallel program is in a preset parallel program sample and determining the embedding and encoding result of the program identifier according to the following formula:

[0010] ;

[0011] in, Parallel Programs for Targets The embedded encoding result of the program identifier, This indicates that the target parallel program is among the preset parallel program samples. This indicates that the target parallel program is not in the preset parallel program sample. This is the maximum index value of the preset parallel program sample. The default embedding encoding result is used; the multi-dimensional fusion feature is obtained by concatenating the embedding encoding results of each runtime feature in the multi-dimensional runtime feature.

[0012] Optionally, the CPU-related data includes some or all of the cache miss count and branch prediction miss count; the memory access-related data includes some or all of the memory usage and read / write status; and the inter-process behavior data includes some or all of the data transfer volume and number of transfers.

[0013] Optionally, after step S101 and before step S102, the method further includes calculating the distance between each cluster in the clustering result of the multivariate fusion features of the target parallel program and the multivariate fusion features of the preset parallel program samples, finding the closest cluster, and using the multivariate fusion features corresponding to the cluster center of the closest cluster as the final multivariate fusion features of the target parallel program; the preset parallel program samples are the parallel program samples used when training the speedup prediction model.

[0014] Optionally, training the speedup prediction model includes:

[0015] S201: Obtain multi-dimensional runtime features of a batch of parallel program samples running at multiple parallelism levels within a given range and embed them into the encoding to obtain multi-dimensional fusion features.

[0016] S202, for each parallel program sample, generates multi-dimensional fusion features based on multiple parallelism runtimes within a given parallelism range. A combination of parallelism windows, Indicates from Select from 10 different elements The number of combinations of elements, where For the number of parallelisms within a given range, This is the size of the parallelism window, thus obtaining the size of each parallelism window. A training set is constructed from multi-factor fusion features with parallelism of 1, and a length of 1 is given. Within a given range of parallelism Construct training samples with varying degrees of parallelism;

[0017] S203, Construct a neural network prediction model as a speedup prediction model. The input of the neural network prediction model is the multi-dimensional fusion features of the parallel program, and the output is the speedup under different degrees of parallelism.

[0018] S204 optimizes the model parameters of the neural network prediction model using training samples, a preset loss function, and a Bayesian hyperparameter optimization algorithm, thereby obtaining a trained prediction model with a faster speedup.

[0019] Optionally, the preset loss function is the loss function with Amdahl's law constraint added, as shown in the following formula:

[0020] ;

[0021] in, Parallel program sample The loss function after adding Amdahl's law constraint, Parallel program sample The mean square error loss, The coefficient of the regularization error term is . This is a reference quantity in Amdahl's law that characterizes the proportion of a program that can be parallelized. To obtain the maximum value, Parallel program sample Given the j-th degree of parallelism within a given range, for The relative acceleration ratio The reciprocal of.

[0022] Optionally, the parallel program sample The expression for the mean squared error loss is:

[0023] ;

[0024] in, Parallel program sample The mean square error loss, For the number of parallelisms within a given range, For a given range of parallelism The actual relative acceleration ratio The reciprocal of; To speed up the prediction model for a given range of parallelism. Predicted relative acceleration ratio The reciprocal of; the relative acceleration ratio The expression for the computation function is:

[0025] ;

[0026] in, Parallel program sample The speedup performance when running at the j-th degree of parallelism within a given range of parallelism. Parallel program sample Acceleration performance when running at the minimum parallelism within a given range of parallelism.

[0027] The present invention also provides a feature and knowledge-driven optimal parallelism prediction system, comprising an interconnected microprocessor and a memory, wherein the microprocessor is programmed or configured to execute the feature and knowledge-driven optimal parallelism prediction method.

[0028] The present invention also provides a computer-readable storage medium storing a computer program or instructions that are programmed or configured to execute the feature- and knowledge-driven optimal parallelism prediction method by a processor.

[0029] To address the problems of insufficient feature representation, scarce training samples, and inadequate prediction results and generalization ability in existing technologies, this invention obtains multi-dimensional runtime features of the target parallel program and embeds them into encoding to obtain multi-dimensional fusion features. These multi-dimensional fusion features are then input into a pre-trained speedup prediction model to predict the speedup of the target parallel program under multiple given degrees of parallelism. The parallelism with the highest speedup is selected as the predicted optimal parallelism. The speedup prediction model is a neural network prediction model, and it is trained using training samples constructed from the multi-dimensional fusion features of parallel program samples and prior knowledge of speedup under different degrees of parallelism to establish a mapping relationship between the multi-dimensional fusion features of the parallel program and the speedup under multiple given degrees of parallelism. This achieves the following beneficial effects: 1. Significantly improved prediction accuracy: Through the modeling method of constructing multi-dimensional fusion features and constraining domain knowledge, the complex relationship between program performance and parallelism is comprehensively captured, effectively avoiding misjudgment of "local optima" and improving the accuracy of optimal parallelism prediction; 2. Optimized sample utilization: Without adding additional pre-run sampling, the sample expansion method using multiple local parallelism windows fully utilizes limited sampled data to generate high-quality training samples, reducing data dependence; 3. Enhanced generalization ability: By leveraging Bayesian hyperparameter optimization and implicit feature modeling, the model's adaptability to different types of parallel programs is improved, effectively addressing the prediction needs of unknown programs; 4. Increased result reliability: By introducing domain knowledge constraints, abnormal prediction results are avoided, ensuring that the prediction output conforms to the basic principles of parallel computing, thus enhancing the practical value of the solution. Therefore, this invention can improve the accuracy, generalization ability, and result reliability of optimal parallelism prediction for parallel programs, and is suitable for various scenarios relying on parallel computing, such as large-scale scientific computing and deep learning model training. Attached Figure Description

[0030] Figure 1 This is a schematic diagram of the basic process of the method in an embodiment of the present invention.

[0031] Figure 2 This is a schematic diagram of the training process of the speedup prediction model in an embodiment of the present invention. Detailed Implementation

[0032] To enable those skilled in the art to better understand the technical solutions of the present invention, the technical solutions of the present invention will be further described in detail below with reference to the accompanying drawings in the embodiments of the present invention.

[0033] like Figure 1 As shown, the feature- and knowledge-driven optimal parallelism prediction method in this embodiment includes the following steps:

[0034] S101, obtain the multi-dimensional runtime features of the target parallel program and embed them into the encoding to obtain multi-dimensional fusion features;

[0035] S102, input the multi-dimensional fusion features into the pre-trained speedup prediction model to predict the speedup of the target parallel program under given multiple degrees of parallelism. The speedup prediction model is a neural network prediction model, and the mapping relationship between the multi-dimensional fusion features of the parallel program and the speedup under given multiple degrees of parallelism is established by training samples constructed in advance using the multi-dimensional fusion features of the parallel program samples and the prior knowledge of the speedup under different degrees of parallelism.

[0036] S103, select the parallelism with the highest speedup ratio as the optimal parallelism obtained from the prediction.

[0037] Step S101 is the step of constructing multi-dimensional fusion features, which includes collecting multi-dimensional runtime features and encoding them through an embedding encoding mechanism, encoding implicit features such as program identifiers and behavior similarity clustering, generating a unified dimension fusion feature vector, realizing the fusion representation of explicit features and implicit features, and comprehensively depicting the program behavior pattern.

[0038] The multi-dimensional runtime features in step S101 of this embodiment include the computational features, memory access features, communication features, and program identifier of the target parallel program during runtime. The computational features are CPU-related data extracted from the performance counter (PMC) data during the parallel program's execution. The memory access features are memory access-related data during the parallel program's execution, which can be sampled using relevant commands (such as perf, top, etc.). The communication features are inter-process behavior data during the parallel program's execution, which can be obtained using relevant tools (such as mpiP, etc.). The CPU-related data includes cache miss counts, branch prediction miss counts, etc.; the memory access-related data includes memory usage, read / write activity (such as read / write volume, read / write rate, read / write bandwidth, etc.); and the inter-process behavior data includes the amount of data transferred, the number of transfers, etc. For the program... Its runtime feature vector Includes computational features Memory access characteristics and communication characteristics During training, further consideration needs to be given to relevant program configuration features. (Parallelism), the target parallelism sequence is:

[0039] ,

[0040] The feature vector of the target parallelism sequence can be obtained through encoding. .

[0041] The embedding encoding of program identifiers is used to represent the feature labels of known programs, so that when these programs are run again, a preliminary feature representation can be obtained. The embedding encoding result of the program identifier is not obtained through direct extraction, but through an embedding encoding layer. This embedding encoding layer can be used as a component of the speedup prediction model, dynamically updating its parameters during the training process. During model training, the embedding encoding result of the program identifier is continuously updated through backpropagation to achieve the goal of minimizing the speedup prediction error. In this embodiment, the embedding encoding of the program identifier includes determining whether the target parallel program is in a preset parallel program sample, and determining the embedding encoding result of the program identifier according to the following formula:

[0042] ;

[0043] in, Parallel Programs for Targets The embedded encoding result of the program identifier, This indicates that the target parallel program is among the preset parallel program samples. This indicates that the target parallel program is not in the preset parallel program sample. This is the maximum index value of the preset parallel program sample. The default embedding encoding result is used; the multi-dimensional fusion feature is obtained by concatenating the embedding encoding results of each runtime feature in the multi-dimensional runtime feature set. (Default embedding encoding result) Specifically, this refers to the program identifier embedding encoding vector assigned to programs not appearing in the training set. Its purpose is to ensure that the prediction model can output prediction results correctly during the speedup prediction process. (Default embedding encoding result) Instead of being obtained through feature extraction, it is generated through artificial settings, such as setting it as an all-zero vector or randomly assigning an existing program identifier embedding vector from the training set.

[0044] Program clustering is used to describe the similarity characteristics of parallel programs (with other known programs) in behavior. In this embodiment, after step S101 and before step S102, cluster embedding is also included, that is: calculating the distance between each cluster in the clustering result of the multivariate fusion features of the target parallel program and the multivariate fusion features of the preset parallel program samples, finding the nearest cluster, and using the multivariate fusion feature corresponding to the cluster center of the nearest cluster as the final multivariate fusion feature of the target parallel program; the preset parallel program samples are the parallel program samples used when training the speedup prediction model. For example, specifying the number of clusters. For known program assemblies Construct the initial cluster partitions And trained the program to belong to The eigenvector representation corresponding to the cluster In forecasting, for newly arriving procedures... The corresponding clusters are obtained based on the initial clustering partition. and return its feature vector representation. .

[0045] like Figure 2 As shown, the training of the speedup prediction model in this embodiment includes:

[0046] S201: Obtain multi-dimensional runtime features of a batch of parallel program samples running at multiple parallelism levels within a given range and embed them into the encoding to obtain multi-dimensional fusion features.

[0047] S202, for each parallel program sample, generates multi-dimensional fusion features based on multiple parallelism runtimes within a given parallelism range. A combination of parallelism windows, Indicates from Select from 10 different elements The number of combinations of elements, i.e.:

[0048] ;

[0049] in For the number of parallelisms within a given range, For the parallelism window size, " represents factorial; thus obtaining the window size for each degree of parallelism" A training set is constructed from multi-factor fusion features with parallelism of 1, and a length of 1 is given. Within a given range of parallelism Construct training samples with varying degrees of parallelism;

[0050] S203, Construct a neural network prediction model as a speedup prediction model. The input of the neural network prediction model is the multi-dimensional fusion features of the parallel program, and the output is the speedup under different degrees of parallelism.

[0051] S204 optimizes the model parameters of the neural network prediction model using training samples, a preset loss function, and a Bayesian hyperparameter optimization algorithm, thereby obtaining a trained prediction model with a faster speedup.

[0052] Step S202 is used for multi-window sample expansion. In this embodiment, based on the core idea of ​​"local prediction of global," a local parallelism window of a preset size is set to combine the sampling parallelism of a single program, generating multiple training samples covering different parallelism intervals. This expands the sample size while preserving the performance evolution correlation between parallelisms, thus improving the model's extrapolation capability. Specifically, a local parallelism window size can be set. For each sampling procedure If the program has a high degree of parallelism Run the following command to generate There are several parallelism window combinations, and the relevant features under each parallelism combination constitute a training set. Accordingly, for the program that needs to make predictions, the window size needs to be collected. Performance characteristics of program execution under parallelism, and also provide the length of the program. The prediction target parallelism sequence is used to predict the performance changes of the program under these parallelism levels.

[0053] In step S203, when constructing the neural network prediction model as the speedup prediction model, a multilayer learning machine (MLP) can be used. The relative speedup of the program under the target parallelism is used as the prediction target. Domain prior knowledge reflecting the reasonableness of the speedup is introduced to avoid the predicted speedup being too large, too small, or even negative, ensuring the reasonableness of the prediction results. During model training, a penalty term inspired by Amdahl's Law is added to the calculation of modeling error. Amdahl's Law is a law describing the speedup of parallel programs. It states that there is an upper limit to the speedup of parallel programs. When the speedup predicted by the model exceeds this upper limit, it indicates that the model prediction result is unreasonable, and a corresponding model penalty is imposed. Specifically, in this embodiment, the preset loss function is the loss function after adding the Amdahl's Law constraint, as shown in the following formula:

[0054] ;

[0055] in, Parallel program sample The loss function after adding Amdahl's law constraint, Parallel program sample The mean square error loss, The coefficient of the regularization error term (Amdahl's law penalty term) is denoted by . This is a reference quantity in Amdahl's law that characterizes the proportion of a program that can be parallelized. To obtain the maximum value, Parallel program sample Given the j-th degree of parallelism within a given range, for The relative acceleration ratio The reciprocal of the formula. That is, for each predicted speedup result, Amdahl's Law determines whether to accumulate the error. Amdahl's Law is a reference value characterizing the proportion of the program that can be parallelized. It is a component of the model's hyperparameters. It is the coefficient of the regularization error term and a component of the model's hyperparameters; its optimal value can be obtained through training.

[0056] For the program The model predicts the reciprocal of the relative speedup. According to Amdahl's law, the relative speedup of this program has an upper limit, meaning its reciprocal has a lower limit. Therefore, the parallel program sample in this embodiment The expression for the mean squared error loss is:

[0057] ;

[0058] in, Parallel program sample The mean square error loss, For the number of parallelisms within a given range, For a given range of parallelism The actual relative acceleration ratio The reciprocal of; To speed up the prediction model for a given range of parallelism. Predicted relative acceleration ratio The reciprocal of; the relative acceleration ratio The expression for the computation function is:

[0059] ;

[0060] in, Parallel program sample The speedup performance when running at the j-th degree of parallelism within a given range of parallelism. Parallel program sample Acceleration performance when running at the minimum parallelism within a given range of parallelism.

[0061] In step S204, when optimizing the model parameters of the neural network prediction model using training samples, a preset loss function, and the Bayesian hyperparameter optimization algorithm, to address the issues of model parameter sensitivity and low efficiency of manual tuning, the Bayesian hyperparameter optimization algorithm is used to iteratively search for key hyperparameters such as feature dimensions, network structure parameters, and constraint weights to obtain the optimal parameter configuration, thereby improving the model's prediction accuracy and generalization ability. It should be noted that the Bayesian hyperparameter optimization algorithm is a well-known method for optimizing model parameters in neural network prediction models; therefore, its implementation details will not be elaborated here. Based on the model being trained with various local parallelisms, for the parallel program to be predicted, feature data under local parallelism with a window size consistent with the training samples is collected, input into the Bayesian-optimized neural network model, and the program speedup ratio within the target parallelism range is output, thus determining the optimal parallelism of the program. In this embodiment, based on the neural network model of the feature and knowledge-driven optimal parallelism prediction method being trained with various local parallelisms, the running characteristics of the program to be predicted under partial parallelism are input, and the prediction speedup ratio within the target parallelism range is output, thus determining the optimal parallelism.

[0062] To verify the feature- and knowledge-driven optimal parallelism prediction method of this embodiment, this embodiment tested a set of known parallel program benchmark programs, including CG(C), CG(D), HPL-AI, EP(C), EP(D), BFS(20), BFS(21), BFS(22), IS(D), LU(C), LU(D), SMG2000, and HPCG. Based on the method of this embodiment, the optimal execution time (method ①), default (method ②), no embedding (program identifier embedding + cluster embedding, method ③), only program identifier embedding (method ④), only cluster embedding (method ⑤), no multi-window sample expansion (methods ⑥ and ⑦), no regularization error term (Amdahl's law penalty term, method ⑧), and Bayesian hyperparameter optimization (method ⑨) were tested. The final execution time test results under various methods are shown in Table 1.

[0063] Table 1: Execution time test results under various methods

[0064]

[0065] In Table 1, the optimal execution time (Method ①) refers to the execution time of the program under the actual optimal parallelism, while the default (Method ②) refers to the error under the default hyperparameter configuration. For easier understanding, a plus sign is added to indicate "how much longer (slower) the predicted optimal execution time is compared to the actual optimal time." In the methods without multi-window sample expansion (Methods ⑥ and ⑦), Method ⑥ (TrainGen WLeastP) means not using training set expansion, but only using the minimum parallelism to construct the training set (the number of parallelisms is the same as the local parallelism window size). Method ⑦ (TrainGen WMostP) means not using training set expansion, but only using the maximum parallelism to construct the training set (the number of parallelisms is the same as the local parallelism window size). As shown in Table 1, in this embodiment, program identifier embedding, cluster embedding, multi-window sample expansion, regularization error term (Amdahl's law penalty term), and Bayesian hyperparameter optimization can all reduce the gap between the predicted optimal parallelism and the actual optimal parallelism, improving program performance and enhancing the prediction effect of optimal parallelism.

[0066] Those skilled in the art will understand that the technical solutions provided by this invention can take the form of methods, systems, or computer program products. For example, this invention can provide a feature- and knowledge-driven optimal parallelism prediction system, including an interconnected microprocessor and a memory, wherein the microprocessor is programmed or configured to execute the feature- and knowledge-driven optimal parallelism prediction method. This invention can provide a computer-readable storage medium storing a computer program or instructions programmed or configured to execute the feature- and knowledge-driven optimal parallelism prediction method via a processor. This invention can provide a computer program product including a computer program or instructions programmed or configured to execute the feature- and knowledge-driven optimal parallelism prediction method via a processor. This invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this invention can take the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of a flowchart and / or block diagram, and combinations of blocks in a flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing device, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0067] The above description is merely a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should also be considered within the scope of protection of the present invention.

Claims

1. A feature and knowledge driven optimal parallelism prediction method, characterized in that, The method comprises the following steps: S101, obtaining the multi-element runtime features of a target parallel program and embedding the codes to obtain multi-element fusion features; S102, inputting the multi-element fusion features into a pre-trained speedup ratio prediction model to predict the speedup ratios of the target parallel program under given multiple parallel degrees, wherein the speedup ratio prediction model is a neural network prediction model, and the mapping relationship between the multi-element fusion features of the parallel program and the speedup ratios under the given multiple parallel degrees is established by training the speedup ratio prediction model with training samples pre-constructed by using the multi-element fusion features of parallel program samples and the speedup ratio prior knowledge under different parallel degrees; S103, selecting the parallel degree with the maximum speedup ratio as the predicted optimal parallel degree; The training of the speedup ratio prediction model comprises: S201, obtaining the multi-element runtime features of a batch of parallel program samples under multiple parallel degrees within a given parallel degree range and embedding the codes to obtain multi-element fusion features; S202, for each parallel program sample, generating a multi-element fusion feature according to the running of a plurality of parallel degrees in a given parallel degree range parallel degree window combinations, representing the number of combinations of selecting elements from different elements, wherein is the number of parallel degrees in a given parallel degree range, is the parallel degree window size, so as to obtain a plurality of parallel degree window sizes multi-element fusion features under each parallel degree constitute a set of training sets, and a given parallel degree range with a length of parallel degrees construct training samples;​ S203, constructing a neural network prediction model as the speedup ratio prediction model, wherein the input of the neural network prediction model is the multi-element fusion features of the parallel program, and the output is the speedup ratios under different parallel degrees; S204, optimizing the model parameters of the neural network prediction model by using the training samples, a pre-set loss function and a Bayesian hyperparameter optimization algorithm, so as to obtain the trained speedup ratio prediction model; The pre-set loss function is a loss function with Amdahl's law constraint as shown in the following formula: ； wherein, is the parallel program sample the loss function after adding the Amdahl's law constraint, is the parallel program sample the mean square error loss of is the coefficient of the regularization error term, is the reference quantity representing the proportion of the parallel part of the program in Amdahl's law, is the maximum value, is the parallel program sample the jth parallelism in the given parallelism range, is the relative speedup ratio under the inverse of The parallel program sample The computational function expression of the mean square error loss is: ； wherein is the mean squared error loss of a parallel program sample , is the number of parallelism degrees within a given parallelism degree range, is the relative speedup ratio of a parallel program sample with a given parallelism degree within a given parallelism degree range; is the inverse of the relative speedup ratio predicted by the speedup ratio prediction model for a given parallelism degree within a given parallelism degree range; is the relative speedup ratio predicted by the speedup ratio prediction model for a given parallelism degree within a given parallelism degree range; is the inverse of the relative speedup ratio ; and the computational function expression of the relative speedup ratio ； wherein, is the parallel program sample is the speedup performance when running at the jth parallelism within the given parallelism range, is the parallel program sample is the speedup performance when running at the minimum parallelism within the given parallelism range.

2. The feature and knowledge driven optimal parallelism prediction method of claim 1, wherein, The multi-element runtime features in step S101 comprise the computing features, memory access features, communication features and program identification of the target parallel program during runtime, the computing features are CPU-related data extracted from the performance counter data during the runtime of the parallel program, the memory access features are memory access-related data during the runtime of the parallel program, the communication features are inter-process behavior data during the runtime of the parallel program, and when embedding the codes of the program identification, it comprises judging whether the target parallel program is in the pre-set parallel program samples, and determining the embedding code result of the program identification according to the following formula: ； wherein, a program identification of the target parallel program an embedded encoding result of the program identification, indicates that the target parallel program is in the preset parallel program sample, indicates that the target parallel program is not in the preset parallel program sample, is a maximum index value of the preset parallel program sample, is a default embedded encoding result; the multi-element fusion feature is obtained by splicing embedded encoding results of each runtime feature in the multi-element runtime feature.

3. The feature and knowledge driven optimal parallelism prediction method of claim 2, wherein, The CPU-related data comprises part or all of cache miss counts and branch prediction miss counts; the memory access-related data comprises part or all of memory usage and read-write conditions; and the inter-process behavior data comprises part or all of transmission data volume and transmission times.

4. The feature and knowledge driven optimal parallelism prediction method of claim 1, wherein, After step S101 and before step S102, it further comprises calculating the distance between the multi-element fusion features of the target parallel program and the multi-element fusion features of each clustering cluster in the clustering result of the pre-set parallel program samples, finding the nearest clustering cluster, and taking the multi-element fusion features corresponding to the clustering center of the nearest clustering cluster as the final multi-element fusion features of the target parallel program; and the pre-set parallel program samples are the parallel program samples used for training the speedup ratio prediction model.

5. A feature and knowledge driven optimal parallelism prediction system comprising interconnected microprocessors and memory, characterized by, The microprocessor is programmed or configured to perform the features and the knowledge-driven optimal parallel degree prediction method in any one of claims 1-4.

6. A computer-readable storage medium having stored therein a computer program or instructions, characterized in that, The computer program or instructions are programmed or configured to perform the features and the knowledge-driven optimal parallel degree prediction method in any one of claims 1-4 by the processor.

Images