A method for predicting supercomputer job duration based on semantics and time series

Abstract

The present invention provides a method for predicting supercomputer job duration based on semantics and timing, which relates to the field of supercomputers and solves the problem of ignoring semantic information in the job path and timing information between jobs in supercomputer job runtime prediction. The method first obtains job log data, distinguishes different user types through data grouping, uses different data storage methods to store the user's job log data, and forms a user model training set through coarse-grained clustering. Subsequently, a prediction model for job runtime is constructed and model training is performed by improving the BERT architecture. During the training process, a timing prediction method is combined so that the prediction model predicts the timing of new jobs based on timing information. After the training is completed, the user submits the job path information of the new job, and the prediction model determines the job category of the new job and outputs the job runtime. The present invention improves the prediction accuracy of job runtime, facilitating subsequent backfill scheduling.

Description

Technical Field

[0001] The present invention relates to the field of supercomputer technology, and is applied to a job prediction process, and particularly relates to a supercomputer job duration prediction method based on semantics and time series. Background Art

[0002] In recent years, with the continuous increase in the scale and complexity of supercomputers, their computing speeds have reached billions or even teraflops per second. However, issues such as limited computing resources and wasteful scheduling system resources have hindered improvements in system throughput. Improving the efficiency of system resource utilization has been a hot topic of research. A job is a task or program submitted by a user to a supercomputer system for execution. These programs typically require significant computing resources. In supercomputer clusters, users can submit jobs through command lines, scripts, and other methods. Job scheduling can improve hardware resource utilization, optimize job throughput, and significantly reduce system overhead. Common job scheduling strategies include first-come, first-served, shortest job first, and time-slice round-robin, and considerable research is underway on various scheduling methods. Supercomputer clusters typically use a first-come, first-served scheduling strategy to ensure service fairness, but this can lead to waste of computing resources.

[0003] A large number of jobs are submitted to a supercomputer, and the job management system schedules these jobs and allocates computing resources. FCFS (First Come First Served) is a popular scheduling strategy, where jobs are queued and run in the order they are submitted. However, under the FCFS strategy, if the available free resources cannot meet the requirements of the job at the head of the queue, they cannot be allocated to other jobs, resulting in a waste of computing resources. Figure 1 As shown in the figure, the horizontal grid represents the job running time, and the vertical grid represents the system resources, such as the processors. Jobs 1 and 2 are running jobs, and jobs 3, 4, and 5 are waiting to be run, that is, waiting queues. Under the FCFS policy, Figure 1 The remaining idle resources of Job 2 cannot meet the needs of Job 3. The resources can only be released after Job 2 is completed and then allocated to Job 3, resulting in a waste of idle resources when Job 2 is running.

[0004] In order to optimize resource utilization, the following Figure 2The backfilling method shown here allocates the remaining idle compute nodes to a small, short-running, non-head job without delaying the original head job, allowing the subsequent waiting jobs to "interrupt" their execution and thus improve resource utilization. Specifically, when job 2 is running and the idle resources do not satisfy job 3, job 5 is "interrupted" to run in advance to improve resource utilization. The backfilling method requires obtaining the job's runtime in advance, while the job runtime is traditionally provided by the user. Typically, users provide an overestimated job runtime to prevent the job from timing out and being canceled by the system. An overestimated job runtime is detrimental to the system's backfill scheduling and still results in low resource utilization, limiting the widespread application of backfill scheduling. Therefore, an algorithm that can accurately predict the job runtime is needed to address this limitation.

[0005] The job path is the folder path where the code is located when a user submits a job. Users typically name the job path based on user names, software names, project names, algorithm names, and parameter information. Therefore, the job path contains rich semantic information about the job. That is, jobs with more similar job paths tend to have more similar content, and thus, more similar job runtimes. Many studies extract features from historical job logs and use machine learning algorithms to predict job runtimes. However, these methods can only input various numerical information, such as user serial number, submission time, and requested CPU number, into the model, ignoring the semantic information contained in the job path. This makes it difficult to further improve the accuracy of job runtime predictions.

[0006] In current practice, a large amount of historical log data is used to train a model. The model extracts various job characteristics from the job's "general job log information," such as the user sequence number, submission time, and number of requested processors (CPUs). This model then predicts the runtime of newly submitted jobs. The system then uses this predicted runtime to better schedule jobs. Existing methods for predicting job runtimes fall into two main categories. One involves analyzing the job's code files or compiled program files, predicting the runtime by analyzing the runtime and input / output (I / O) time of each function within the code. However, scheduling platforms at supercomputing centers typically lack access to user code files, so this method is more commonly used with the user's compiler or system-level tools. The other method starts with searching for similar jobs within historical jobs, as similar jobs typically have similar runtimes. Therefore, job runtime predictions at supercomputing centers are typically based on historical job analysis.

[0007] In the prior art, Zhou Longfang et al. published a paper titled "Hierarchical Clustering Algorithm for Predicting Job Runtimes" in the Journal of the National University of Defense Technology, Vol. 44, No. 5, October 2022. This paper proposes a hierarchical clustering algorithm for job names based on letters, structures, and numbers to better cluster jobs and thereby improve the accuracy of the prediction model. Specifically, job paths are composed of three types: letters, special characters, and numbers. The authors believe that the importance of these three types decreases gradually. Therefore, they first cluster the letters using a clustering algorithm. Then, they cluster the special characters within each category to obtain different "structures." Within each category, they then cluster the data based on the number of numbers to obtain the final categories. After three clustering steps, the data is divided into thousands of categories. Data within the same category is considered to have similar job paths. The category is then used as a new attribute and fed into the model along with other attributes to predict job runtimes. This technique uses clustering to convert job paths (strings), which are incomprehensible to the model, into category numbers (numbers), allowing the prediction model to understand the job path information.

[0008] In the prior art, the invention patent with patent number CN202210132077.9 discloses "a method for actively predicting supercomputer job failures based on application similarity". This method adds the job path and job name to other features of the job log, and also uses a machine learning algorithm to predict the job status; if the predicted job status is an unsuccessful state (abnormal state), corresponding measures are taken to remedy the situation in a timely manner to reduce resource waste. Specifically, this method calculates the similarity of the job name and job path using the longest common subsequence similarity and the Levenshtein distance similarity, respectively, and then clusters them using a clustering algorithm. The category information is then added to the data as a new attribute. After filtering out meaningless data, coarse-grained and fine-grained models are used to predict the job status respectively. The coarse-grained model inputs all data into the model for training, and the fine-grained model is a model independently established for each user, and each user's data is input into the corresponding user model.

[0009] It can be seen from this that the existing technology relies on machine learning models and clustering technology to predict job running time, and the prediction accuracy is difficult to continue to improve. Therefore, it is necessary to explore new job prediction methods. Summary of the Invention

[0010] Based on the current state of the art, this invention aims to address the problem of ignoring semantic information in job paths in supercomputer job runtime prediction. By exploring ways to use pre-trained models in natural language processing (particularly the Transformer-based bidirectional encoder BERT) to understand semantic information in job logs, this invention proposes a supercomputer job duration prediction method based on semantics and time series. This method first separates different batch jobs through clustering and BERT, then leverages BERT's semantic understanding capabilities to obtain coarse-grained job runtime intervals. Then, using techniques related to gated recurrent units (GRUs), it leverages the temporal nature of historical data to train new jobs, thereby enabling job runtime prediction. This invention further improves the accuracy of job runtime prediction, facilitating subsequent job backfill scheduling and increasing data utilization in supercomputer systems.

[0011] The present invention adopts the following technical solutions to achieve the purpose:

[0012] A method for predicting supercomputer job duration based on semantics and time series includes the following steps:

[0013] S1. Obtain historical job log data and perform data preprocessing. After preprocessing, group the job log data to distinguish different user types through data grouping. The job log data includes job path information.

[0014] S2. Use different data storage methods to store the user's job log data according to different user types; use coarse-grained clustering to filter and store different batches of batch processing jobs in the user's job log data as a model training set;

[0015] S3. Build a job runtime prediction model by improving the BERT architecture and train the prediction model using the model training set.

[0016] S4. In the prediction model training process, the time series prediction method is combined to extract the time series information of the existing jobs in the model training set, so that the prediction model can predict the time series of the new job based on the time series information;

[0017] S5. After the training of the prediction model is completed, the user submits the job path information of the new job to the prediction model. The prediction model determines the job category of the new job and outputs the corresponding job running time, completing the job prediction process.

[0018] Specifically, in step S1, the job log data of the historical records is cleaned up; the job log data includes multiple job scripts with different job statuses, and each job script includes a script file and at least one job; the job script with a completed job status is taken, the script file therein is deleted, and the job information therein is retained; then, based on the start time and end time recorded in the job information, the job running time of the job is calculated and stored in a list, and the list also stores the job ID, user ID, resource requirements, job path and job name of the job, thereby completing the preprocessing of the job log data.

[0019] Preferably, after preprocessing, the job log data is distinguished into different user types based on the number of jobs corresponding to each user ID; a first threshold and a second threshold of the number of jobs are preset, wherein the first threshold is less than the second threshold; users whose number of jobs is less than the first threshold are distinguished as first-type users, users whose number of jobs is greater than or equal to the first threshold and less than the second threshold are distinguished as second-type users, and users whose number of jobs is greater than or equal to the second threshold are distinguished as third-type users, thereby completing the distinction between user types.

[0020] Preferably, in step S2, for the first category of users, their job log data is deleted, and the model training set will not use the job log data of the first category of users; for the second category of users, the job log data of all the second category users are merged into a whole and stored as a whole, and the whole is used as a training part in the model training set; for the third category of users, the job log data of each specific user in the third category is stored separately and separately, and used as a training part in the model training set.

[0021] Furthermore, in the coarse-grained clustering process, for each training part in the model training set, the K-Means clustering algorithm is used to cluster the job runtime column stored in the list, and the column is scaled by natural logarithm during clustering; the absolute percentage error is used and the weighted absolute percentage error is further calculated to evaluate the clustering effect, and the selection of the evaluation indicator K value is determined accordingly; after clustering is completed, the clustering category of each job in each training part is obtained, that is, the category of batch processing jobs represented as different batches; the clustering category is stored together with the job path and job name corresponding to the job stored in the list to form a model training set.

[0022] Furthermore, in step S3, the constructed prediction model includes an input layer, a feature extraction layer, a position fusion layer, and an output layer;

[0023] The input layer receives the job path information input from the model training set and generates an embedding vector consisting of word vectors, segment vectors, and position vectors;

[0024] The embedding vector is processed by the feature extraction layer. By presetting the first encoder and the second encoder at different layers in the improved BERT architecture, the vector encoding information of syntactic features and deep semantic features are extracted from the embedding vector respectively.

[0025] The vector encoding information is input into the position fusion layer, and multi-dimensional semantic fusion is used to achieve feature enhancement and generate category probability distribution;

[0026] The output layer selects the category with the highest confidence from the category probability distribution as the final classification result of semantic fusion, and obtains the job category corresponding to the job path information; based on the obtained job category, the corresponding job running time is output.

[0027] Specifically, in the position fusion layer, the variable-length sequence of vector encoding information is first converted into a fixed-dimensional global feature vector by the pooling layer, which then performs a random neuron masking operation through the Dropout layer. The vector features are then mapped to the pre-clustered coarse-grained category space through the linear layer. After the output of the dual encoding layer is subjected to mean fusion, the category probability distribution is generated through the Softmax layer.

[0028] The output layer uses the argmax method to select the category with the highest confidence from the category probability distribution.

[0029] Preferably, in step S4, the timing prediction method uses a GRU network to perform timing prediction. For batch jobs of the same batch in the same user, when the prediction model predicts the job running time of a new job, the job categories of a preset number of predicted jobs before the new job are input into the GRU network, the running time of the batch jobs of the batch is extracted, and the timing information of the new job is predicted as a reference for predicting the job category and job running time of the new job.

[0030] Preferably, in step S5, based on the job log data of the historical records, a corresponding model test set is constructed in the same manner as steps S1 and S2; the prediction model is tested using the model test set, and the accuracy deviation between the job running time output by the prediction model and the actual time is calculated; when the accuracy deviation meets the preset training standard, the training of the prediction model is completed.

[0031] Preferably, an online learning mechanism is added to the prediction model after training is completed. In actual applications, whenever a user submits the job path information of a new job, the prediction model adjusts some of its own model parameters to adapt to the job path information based on the data characteristics corresponding to the job path information of the new job, and predicts the corresponding job running time.

[0032] In summary, due to the adoption of this technical solution, the beneficial effects of the present invention are as follows:

[0033] This invention is the first to use a large language model represented by BERT to understand the semantic information of the job path. Existing technical papers use a "letter-structure-number" clustering algorithm to obtain categories. It cannot be shown that the clustering algorithm can understand the semantic information of the job path only through the category number. Existing patents use similarity for clustering, add the clustering results to other attributes and put them into the prediction model for prediction. The job path is ultimately converted into the cluster category number, which is actually unable to understand the semantic information of the job path. The present invention is based on the model improvement of the BERT architecture. Its hierarchical fusion method can be applied to job paths of different types and lengths, converting the job path into coded information, and then understanding the semantic information of the job path.

[0034] The present invention not only studies and understands the semantic information in the job path, but also introduces GRU technology to capture the temporal relationship between jobs in response to the phenomenon that the running time of batch jobs is extremely similar. This allows the present invention to conduct a comprehensive evaluation from the two dimensions of temporal and semantic information, further improving the accuracy of job runtime prediction. In contrast, when using machine learning technology after clustering, the existing technology will shuffle the data before training, ignoring the previous and next correlations between jobs. Therefore, the present invention is more in-depth in extracting job semantics and has more advantages in understanding job timing information. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 Schematic diagram of resource waste caused by FCFS strategy in the existing technology;

[0036] Figure 2 A schematic diagram of optimizing resource utilization for a backfill method in the prior art;

[0037] Figure 3 Schematic diagram of the overall process of the method of the present invention;

[0038] Figure 4 Detailed schematic diagram of the operation process of the method of the present invention;

[0039] Figure 5 Schematic diagram of the structure of the job running time prediction model in the method of the present invention. DETAILED DESCRIPTION

[0040] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings herein can be arranged and designed in various different configurations.

[0041] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention as claimed, but rather merely represents selected embodiments of the present invention. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without creative effort shall fall within the scope of protection of the present invention.

[0042] Example

[0043] A method for predicting supercomputer job duration based on semantics and time series. Figure 3 The overall process of the method is briefly described in the following figure, which can be viewed simultaneously; the key steps of the method are summarized as follows:

[0044] S1. Obtain historical job log data and perform data preprocessing. After preprocessing, group the job log data to distinguish different user types through data grouping. The job log data includes job path information.

[0045] S2. Use different data storage methods to store the user's job log data according to different user types; use coarse-grained clustering to filter and store different batches of batch processing jobs in the user's job log data as a model training set;

[0046] S3. Build a job runtime prediction model by improving the BERT architecture and train the prediction model using the model training set.

[0047] S4. In the prediction model training process, the time series prediction method is combined to extract the time series information of the existing jobs in the model training set, so that the prediction model can predict the time series of the new job based on the time series information;

[0048] S5. After the training of the prediction model is completed, the user submits the job path information of the new job to the prediction model. The prediction model determines the job category of the new job and outputs the corresponding job running time, completing the job prediction process.

[0049] In this embodiment, the job runtime prediction model constructed by improving the BERT architecture is more suitable for job runtime prediction tasks. The BERT architecture itself uses a multi-head self-attention mechanism, which can effectively capture the dependencies of the input sequence and improve the depth and accuracy of language understanding. The self-attention mechanism calculates the relationship between each element in the input sequence and other elements to generate new representations, capture long-distance dependencies, and flexibly adjust the contextual feature representation. The multi-head self-attention mechanism, as an extension of the self-attention mechanism, is applied to each Transformer layer of the BERT architecture. The multi-layer structure aggregates semantic information at different levels to capture the rich relationships and potential meanings in the text.

[0050] This embodiment will now introduce in detail the details and preferred methods of each step in the execution of the method through more specific numerical examples. Figure 4 's hint.

[0051] First, in step S1, the historical job log data is cleaned. Unused job script information in the job log data is deleted, leaving only the most granular information about individual jobs. The job log data includes multiple job scripts with different job states, each of which consists of a script file and at least one job.

[0052] This embodiment retrieves job scripts with a completed job status (i.e., a status record of "Completed") and deletes any script files ending in .batch, retaining only one or more job information. The job's runtime is then calculated based on the start and end times recorded in the job information and stored in a list. The job runtime can be stored in a column named "Seconds" within the list. The list also stores the job's job ID, user ID, resource requirements (i.e., the number of CPUs required), job path, and job name. The job path and job name are concatenated and stored in a column named "Path." Thus, the column names in the list can be JobID (job ID), UID (user ID), ReqCPUs (resource requirements), Seconds, and Path, completing the preprocessing of the job log data.

[0053] The data grouping process aims to distinguish different user types: some users submit few jobs infrequently, resulting in high training and research costs and low returns; while others submit many jobs frequently. The job log data for these jobs can be separated to train the model independently, thereby better learning the job characteristics of these users. This embodiment distinguishes different user types based on the number of jobs corresponding to each user ID. Using the 311,421 valid data records from the training process as an example, a first threshold of 100 and a second threshold of 5000 are preset for the number of jobs. If the data volume varies in actual applications, the thresholds can be adjusted to better differentiate between user types.

[0054] In this example, users with fewer than 100 jobs are classified as "software developers," users with 100 or more and less than 5,000 jobs are classified as "scientific explorers," and users with 5,000 or more jobs are classified as "engineering applications." These three types of users correspond to different data processing measures, and this classification method, combined with practical engineering practices, is highly relevant. Once this classification is complete, the method proceeds to step S2.

[0055] For "software development" users, since their job data is very small, the prediction model is not sufficient to learn the relationship between the job path and the job runtime. Therefore, their job log data is deleted and the model training set will not use this part of the data.

[0056] For "scientific exploration" users, since their job data is relatively small, the job log data of all users of this type are merged into a whole and stored as a whole, which serves as a training part in the model training set.

[0057] For "engineering application type" users, the job log data of each specific user of this type is stored separately and used as a training part in the model training set.

[0058] Next, for each user's specific job, the distribution of job runtime is extremely uneven, making traditional density-based clustering methods ineffective. Therefore, in this embodiment, a coarse-grained clustering process is used to cluster the job runtime columns stored in the list for each training portion of the model training set using the K-Means clustering algorithm. This coarse-grained clustering process is used to screen batch jobs from different batches. Jobs in the same batch differ only in parameters and have very similar runtimes. However, jobs in different batches typically have significantly different runtimes due to differences in data and test tasks. By clustering job runtimes through the coarse-grained clustering process, different job categories and their approximate time intervals can be determined, facilitating subsequent prediction models to predict the runtimes of new jobs.

[0059] After preprocessing and storing the corresponding list of job log data in step S1, K-Means clustering can be performed on the "Seconds" column, and the user and job information corresponding to the same row can be adjusted. Because the data distribution of job runtimes recorded in the "Seconds" column is extremely uneven, this embodiment applies natural logarithmic scaling to this column during clustering to better facilitate the clustering process.

[0060] Then, the absolute percentage error And further calculate the weighted absolute percentage error The clustering effect is evaluated in this way, and the selection of the evaluation index K value is determined accordingly. The relevant formula is as follows:

[0061]

[0062]

[0063]

[0064] Where, Representative The predicted value at a time point, Representative The actual value at a time point, Representative The number of class assignments, Represents the total number of user jobs in a training session. Representative After clustering is complete, the cluster categories of each job in each training part are obtained, that is, the categories of batch jobs representing different batches. These can be saved in a column named "Class" in the list. Finally, the cluster category is stored together with the job path and job name corresponding to the job stored in the list to form the model training set, which includes the "Path" and "Class" columns. This is used as input into the job runtime prediction model. In addition to using K-Means clustering, other algorithms can also be selected according to actual needs while ensuring the corresponding clustering effect.

[0065] In step S3 of this embodiment, the prediction model, constructed using an improved BERT architecture, extracts information of different dimensions through encoders at different levels. The advantage of this layered fusion is that it can take into account the semantic extraction of job paths of varying lengths. The prediction model ultimately outputs the job category, which can be used to determine the job's runtime.

[0066] like Figure 5 As shown, the prediction model constructed in this embodiment includes an input layer, a feature extraction layer, a position fusion layer and an output layer.

[0067] The input layer receives the job path information in the model training set (which can be processed as an example to Input vector) input, generate an embedding vector composed of word vector Token Embeddings, segment vector Segment Embeddings and position vector PositionEmbeddings ( to ).

[0068] The embedding vector is processed by the feature extraction layer. By presetting the first encoder and the second encoder at different layers in the improved BERT architecture, the vector encoding information of the syntactic features and the deep semantic features are respectively extracted from the embedding vector. In this embodiment, the encoder Encoder 8 at the 8th layer and the encoder Encoder 12 at the 12th layer in the improved BERT architecture are respectively extracted. The model hierarchical information here can also be changed according to the naming method and length of the operation path, so the layer position is not fixed. If the data in training and actual application has a large number of long paths and medium paths, the encoders of the 8th layer and the 12th layer can be selected for processing according to the method of this embodiment.

[0069] The vector coding information is input into the position fusion layer, and multi-dimensional semantic fusion is performed to achieve feature enhancement and generate category probability distribution. In the position fusion layer of this embodiment, the variable-length sequence of the vector coding information is first converted into a fixed-dimensional global feature vector by the pooling layer, which then implements a random neuron shielding operation through the random inactivation layer Dropout to improve the generalization ability of the prediction model. Then, the vector features are mapped to the pre-clustered coarse-grained category space through the linear layer Linear. After the double coding layer outputs and performs mean fusion, the syntactic features and deep semantic features are respectively formed. to and to vector (the number of fused vectors is reduced from the original Became ), and finally generate the category probability distribution through the normalized exponential function layer, namely the Softmax layer.

[0070] The output layer uses the maximum indexing method (argmax method) to select the category with the highest confidence from the category probability distribution as the final classification result of semantic fusion, and obtains the job category corresponding to the job path information; based on the obtained job category, the corresponding job running time is output.

[0071] As a preferred embodiment of this embodiment, in step S4, the timing prediction method uses a GRU network to perform timing prediction. For batch jobs of the same batch in the same user, when the prediction model predicts the job running time of a new job, the job category of a preset number of predicted jobs before the new job (for example, the first n job-related data) is input into the GRU network, the running time of the batch jobs of the batch is extracted, and the timing information of the new job is predicted as a reference for predicting the job category and job running time of the new job.

[0072] Capturing time series data and long- and short-term dependencies is crucial in time series prediction. The GRU network introduces a gated recurrent unit (GRU), which includes an update gate and a reset gate. This gate structure effectively captures long-term dependencies. This embodiment utilizes the GRU network to selectively retain and update historical information, while deciding what to retain and update for new information. This feature enables the GRU network to better process long-sequence data, maintain stable information transmission, and improve model performance and generalization.

[0073] In addition to using GRU networks for time series prediction, networks that can obtain data time series information, such as long short-term memory networks (LSTMs), can also be used for time series prediction. However, compared to LSTM networks, GRU networks have a simpler structure, reduce the number of parameters, and improve computational efficiency, making them more suitable for processing large-scale data. This embodiment preferably uses the GRU network. By closely integrating the improved BERT architecture with the GRU network, without BERT-related methods, it is impossible to obtain pure and accurate data of the same batch of job types, and data from different batches may be mixed together. Without the time series prediction of the GRU network, it is difficult to connect the time series information of the same type (same batch) of job type data.

[0074] In step S5 of this embodiment, based on the job log data recorded in the history, a model test set is constructed in the same manner as in steps S1 and S2. The prediction model is tested using the model test set, and the accuracy deviation between the job running time output by the prediction model and the actual time is calculated. , as follows:

[0075]

[0076]

[0077] Where, Representative The accuracy deviation of each user, Represents the job running time predicted by the model, Represents the actual running time of the job. Represents the total number of users in the model test set. When the accuracy deviation meets the preset training standard, the prediction model training is completed.

[0078] As a preferred embodiment of this invention, an online learning mechanism is added to the prediction model after training is completed. In actual applications, whenever a user submits the job path information of a new job, the prediction model adjusts some of its own model parameters to adapt to the job path information based on the data characteristics corresponding to the job path information of the new job, and predicts the corresponding job running time.

[0079] The online learning mechanism enables the model to not only learn patterns from historical data during initial training, but also automatically adjust and improve itself based on newly available data after deployment. For the scenario of predicting job runtimes in this embodiment, when job types are constantly updated, the prediction model will not become outdated or inaccurate due to long periods of non-updates. Because the online learning method is fine-tuned based on the latest submitted job data, it can help the model capture short-term trends, such as changes in system resource usage within a specific time period, thereby providing more accurate job runtime predictions.

[0080] The online learning process can also be implemented concisely through a feedback loop. After each job runtime prediction is completed, the corresponding actual job runtime is also recorded synchronously and compared with the predicted result. The prediction model can adjust internal parameters by combining online gradient descent, passive attack learning, etc., thereby showing stronger adaptability in dynamic job environments.

Claims

1. A method for predicting supercomputer job duration based on semantics and time series, characterized by: The steps include: S1. Obtain historical job log data and perform data preprocessing. After preprocessing, group the job log data and distinguish different user types through data grouping. The job log data includes job path information; S2. Use different data storage methods to store the user's job log data according to different user types; use coarse-grained clustering to filter and store different batches of batch processing jobs in the user's job log data as a model training set; S3. Build a job runtime prediction model by improving the BERT architecture and train the prediction model using the model training set. S4. In the prediction model training process, the time series prediction method is combined to extract the time series information of the existing jobs in the model training set, so that the prediction model can predict the time series of the new job based on the time series information; S5. After the prediction model is trained, the user submits the job path information of the new job to the prediction model. The prediction model determines the job category of the new job and outputs the corresponding job running time, completing the job duration prediction process. In step S3, the constructed prediction model includes an input layer, a feature extraction layer, a position fusion layer, and an output layer; The input layer receives the job path information input from the model training set and generates an embedding vector consisting of word vectors, segment vectors, and position vectors; The embedding vector is processed by the feature extraction layer. By presetting the first encoder and the second encoder at different layers in the improved BERT architecture, the vector encoding information of syntactic features and deep semantic features are extracted from the embedding vector respectively. The vector encoding information is input into the position fusion layer, and multi-dimensional semantic fusion is used to achieve feature enhancement and generate category probability distribution; The output layer selects the category with the highest confidence from the category probability distribution as the final classification result of semantic fusion, and obtains the job category corresponding to the job path information. Based on the obtained job category, the corresponding job running time is output; In step S4, the time series prediction method uses the GRU network to perform time series prediction. For batch jobs of the same batch in the same user, when the prediction model predicts the job running time of the new job, the job categories of a preset number of predicted jobs before the new job are input into the GRU network, the running time of the batch jobs of the batch is extracted, and the timing information of the new job is predicted as a reference for predicting the job category and job running time of the new job.

2. The method for predicting supercomputer operation duration according to claim 1, characterized in that: In step S1, the job log data of the historical records is cleaned; the job log data includes multiple job scripts with different job statuses, and each job script includes a script file and at least one job; the job script with a completed job status is taken, the script file therein is deleted, and the job information therein is retained; then, based on the start time and end time recorded in the job information, the job running time of the job is calculated and stored in a list, and the list also stores the job ID, user ID, resource requirements, job path and job name of the job, thereby completing the preprocessing of the job log data.

3. The method for predicting supercomputer operation duration according to claim 2, characterized in that: After preprocessing the job log data, different user types are distinguished based on the number of jobs corresponding to each user ID; a first threshold and a second threshold for the number of jobs are preset, where the first threshold is less than the second threshold; users with a number of jobs less than the first threshold are distinguished as first-type users, users with a number of jobs greater than or equal to the first threshold and less than the second threshold are distinguished as second-type users, and users with a number of jobs greater than or equal to the second threshold are distinguished as third-type users, thus completing the distinction between user types.

4. The method for predicting supercomputer operation duration according to claim 3, characterized in that: In step S2, for the first type of users, their job log data will be deleted, and the model training set will not use the job log data of the first type of users; for the second type of users, the job log data of all the second type of users are merged into a whole and stored as a whole, and the whole is used as a training part in the model training set; for the third type of users, the job log data of each specific user in the third type of users are stored separately and used as a training part in the model training set respectively.

5. The method for predicting supercomputer operation duration according to claim 4, characterized in that: During the coarse-grained clustering process, for each training part in the model training set, the K-Means clustering algorithm is used to cluster the job runtime column stored in the list, and the column is scaled by natural logarithm during clustering; the clustering effect is evaluated by using the absolute percentage error and further calculating the weighted absolute percentage error, and the selection of the evaluation indicator K value is determined accordingly; after clustering is completed, the cluster category of each job in each training part is obtained, that is, the category of batch processing jobs represented by different batches; the cluster category is stored together with the job path and job name corresponding to the job stored in the list to form the model training set.

6. The method for predicting supercomputer operation duration according to claim 1, characterized in that: In the position fusion layer, the variable-length sequence of vector encoding information is first converted into a fixed-dimensional global feature vector by the pooling layer, which then performs a random neuron masking operation through the Dropout layer. The vector features are then mapped to the pre-clustered coarse-grained category space through the linear layer. After the output of the dual encoding layer is subjected to mean fusion, the category probability distribution is generated through the Softmax layer. The output layer uses the argmax method to select the category with the highest confidence from the category probability distribution.

7. The method for predicting supercomputer operation duration according to claim 1, characterized in that: In step S5, based on the job log data in the historical records, a model test set is constructed in the same manner as steps S1 and S2; the prediction model is tested using the model test set, and the accuracy deviation between the job running time output by the prediction model and the actual time is calculated; when the accuracy deviation meets the preset training standard, the training of the prediction model is completed.

8. The method for predicting supercomputer operation duration according to claim 7, characterized in that: An online learning mechanism is added to the prediction model after training. In actual applications, whenever a user submits the job path information of a new job, the prediction model adjusts some of its own model parameters to adapt to the job path information based on the data characteristics corresponding to the job path information of the new job, and predicts the corresponding job running time.

Images