Instruction semantic enhancement-based error elasticity prediction method for high-performance computing equipment

Abstract

The invention discloses an instruction semantic enhancement-based error elasticity prediction method for high-performance computing equipment. The method comprises the following steps of: obtaining an assembly instruction sequence and offset address information of a high-performance computing program; performing single event upset fault simulation to obtain a small number of high-performance computing equipment fault samples; generating instruction semantic embedding so as to encapsulate the semantics of the target high-performance computing program instruction; analyzing a control execution and data dependency relationship between instructions, constructing two instruction topological graphs, modeling the two instruction topological graphs by using a graph neural network, and extracting an error propagation mode; and fault embedding extracted from different graphs is combined to carry out error elasticity prediction. According to the method, instruction semantic representation is carried out on the high-performance computing program, the elasticity characteristic of the high-performance computing program can be better understood, accurate error elasticity prediction is achieved by modeling error propagation of the high-performance computing program at the bit level without a large amount of fault simulation, computing resources and expenditure are greatly saved, and the method is suitable for large-scale popularization and application. And the method can be flexibly applied to various high-performance computing programs running on the high-performance computing equipment.

Description

Technical Field

[0001] The present invention belongs to the technical field of hardware device error resilience assessment, and in particular to a high-performance computing device error resilience prediction method based on instruction semantic enhancement. Background Art

[0002] In modern high-performance computing (HPC) systems, graphics processing units are widely deployed due to their powerful computing capabilities. However, with the increase in the volume of applications and the decrease in operating voltage, the incidence of failures in computing elements has increased significantly. These failures can be divided into permanent failures or transient failures. Permanent failures originate from irreversible changes in hardware components and persist until direct hardware intervention is implemented to correct them. In contrast, transient failures, also known as soft errors, are mainly bit flips within computing units caused by electromagnetic interference. Bit flip failures can affect running HPC programs and lead to various results such as system crashes, run timeouts, etc. (HPC programs here refer to any program executed on high-performance computing devices, regardless of the program language). The biggest threat is silent data corruption (SDC), which severely undermines data integrity and produces incorrect output. In addition, SDC does not cause system crashes or obvious error messages, making it difficult to detect.

[0003] To mitigate the potential risk of SDC, most reliable systems adopt conservative schemes such as modular redundancy, which indiscriminately protect every part of the program executed on the hardware, even if some parts almost never trigger SDC. The huge overhead of these schemes poses a significant challenge to their efficiency and widespread adoption, especially for highly parallel HPC programs. Therefore, it is intuitively more effective to adopt adaptive strategies such as selective instruction redundancy, which allocates computing resources to different parts by considering different degrees of error resilience among assembly instructions.

[0004] The traditional approach is to intentionally introduce faults during program execution and compare the resulting output with predetermined criteria to determine the type of error that occurred (SDC, crash, etc.), namely fault injection (FI). Although this approach can provide relatively accurate estimates of error resilience, it usually requires executing the entire program tens of thousands of times to perform exhaustive FI activities. Obviously, this is very resource-intensive for HPC programs with high-density computing. In order to speed up the evaluation, researchers began to consider reducing the number of FI activities required while maintaining the accuracy of error resilience estimates. Existing work related to device resilience estimates the error resilience of the program by understanding the resilience characteristics or modeling error propagation. However, these works still have certain limitations, mainly because the features indicating resilience rely on manual extraction from simulations of numerous faults, and error propagation analysis cannot target fine-grained bit-level faults. Summary of the invention

[0005] The purpose of the present invention is to provide a method for predicting the error resilience of high-performance computing devices based on instruction semantics enhancement to address the problems of the above-mentioned prior art. By considering the contextual semantic features of instructions, simulating the error propagation process of bit-level faults, exploring the device resilience mechanism, and achieving accurate prediction of the error resilience of the fault point with only a small number of fault injection samples. In addition, the present invention has good versatility and is suitable for the resilience prediction of various programs running on high-performance computing devices.

[0006] The technical solution to achieve the purpose of the present invention is: a method for predicting error resilience of high-performance computing equipment based on instruction semantic enhancement, the method comprising the following steps:

[0007] Step 1: Perform secondary development on the open source high-performance computing device binary plug-in tool NVBit to obtain the assembly instruction sequence and offset address information of the high-performance computing program;

[0008] Step 2: Perform single-particle upset fault simulation at the bit level based on the fault injection tool NVBitFI to obtain high-performance computing device fault samples;

[0009] Step 3, generating instruction semantic embedding from the assembly instruction sequence of the high-performance computing program obtained in step 1 by adjusting a large language model with a large instruction sequence corpus, thereby encapsulating the semantics of the target high-performance computing program instructions;

[0010] Step 4: Analyze the control execution and data dependency between instructions, construct two instruction topology graphs to represent possible error propagation paths of bit-level faults, build a graph neural network to model the two instruction topology graphs, and extract error propagation patterns;

[0011] Step 5: Design a fusion method based on the multi-head attention mechanism to combine the fault embeddings extracted from different graphs to achieve error resilience prediction of the target fault point.

[0012] Furthermore, the assembly instruction sequence of the program in step 1 includes all static assembly instructions executed during program execution, as well as the type of each instruction and the text information of the register; the offset address of each instruction, that is, the relative address of each instruction in the corresponding kernel function.

[0013] Furthermore, in step 2, a single-particle upset fault simulation is performed based on NVBitFI to obtain the fault point of the high-performance computing device. Each fault point F site It is expressed as:

[0014] F site = j ,R q ,B o >​

[0015] In the formula, I j ,R q and B o They respectively indicate that the target fault occurs in the j-th instruction, at the o-th bit in the register numbered q.

[0016] Furthermore, each fault point F in step 2 site Each failure point is pre-encoded by encoding each element in its triplet in a one-hot manner and concatenating them as Where m is the total encoding dimension. The fault injection tool NVBitFI is used to inject fault points, obtain the error resilience labels of the fault points, and construct high-performance computing device fault samples.

[0017] Furthermore, in step 3, the assembly instruction sequence of the high-performance computing program obtained from step 1 is generated by adjusting a large language model with a large instruction sequence corpus to encapsulate the semantics of the target high-performance computing program instructions, specifically including:

[0018] Step 3-1, obtain the instruction sequence Φ of each program as:

[0019] Φ=[I 1 ,I 2 ,...,I n ],

[0020] Where n is the number of instructions that the program needs to implement a certain function, I n is the nth instruction;

[0021] Step 3-2, using the instruction sequence of the program obtained in step 3-1 for pre-training of a large language model, learning a bidirectional context representation, and obtaining a large language model with enhanced instruction semantics;

[0022] Step 3-3, based on the instruction semantics enhanced large language model obtained in step 3-2, encapsulate the instruction semantics of the program, and the instruction semantics embedding is represented as S:

[0023]

[0024] In the formula, d is the dimension of the embedding vector, c indicates that these semantic vectors are the embeddings corresponding to the [CLS] tags extracted from the large model, and n indicates the total number of instructions. Represents the semantic embedding representation of the nth instruction.

[0025] Furthermore, in step 4, the control execution and data dependency between instructions are analyzed to construct two instruction topology graphs, which represent possible error propagation paths of bit-level faults, specifically including:

[0026] Step 4-1: Analyze the control flow of the program, that is, the execution relationship of the instructions in the control flow; with instructions as nodes and control execution relationships as edges, the instruction control flow graph is defined as G c = {V,A c}, where V represents the node set consisting of instruction sequence Φ, A c represents the adjacency matrix specified by the control flow;

[0027]

[0028] In the formula, Yes A c The element in the i-th row and j-th column of V i and V j are two instruction nodes in V, → indicates instruction transmission from front to back; X, Y and Z are sets of different control relationship instruction nodes, including conditional branch, function call and synchronization;

[0029] Step 4-2: Analyze the data flow of the program, that is, the dependency relationship of the instructions in the data flow; with instructions as nodes and dependencies as edges, the instruction data flow graph is defined as G o = {V,A o}, A o Represents the adjacency matrix specified by the data flow:

[0030]

[0031] In the formula, It means that two instructions use the same register, that is, there is a dependency relationship; Yes A o The element at the i-th row and j-th column in .

[0032] Furthermore, in step 4, a graph neural network is constructed to model the two instruction topology graphs and extract the error propagation mode, specifically: a graph neural network is constructed to model the instruction control flow graph G c And instruction data flow graph G o Modeling is performed with the instruction semantic embedding S and the encoding of the fault point F e As input, extract instruction embeddings, specifically including:

[0033] Step 4-3, control flow graph G c Use GCN network modeling to get the embedding of control flow instructions

[0034] Step 4-4, for the instruction data flow graph G o Using GAT network modeling, we can embed the data flow instructions.

[0035] Furthermore, the fusion method based on the multi-head attention mechanism is designed in step 5 to combine the fault embeddings extracted from different graphs to achieve error resilience prediction of the target fault point, specifically including:

[0036] Step 5-1: For each fault point, extract the embedded control flow instructions and embedding of data flow instructions Extract the fault instructions and embed them separately and Using the fault point code F e Calculate the attention scores s for both 1 and 2 They are:

[0037]

[0038] In the formula, w a , W 1 and W 2 is a training parameter, τ(·) represents the nonlinear activation function Tanh;

[0039] Step 5-2: Use the attention score obtained in step 5-1 to calculate the fused fault embedding for:

[0040]

[0041] Step 5-3: Map the fault embedding fused in step 5-2 to the probability distribution p∈[0,1] of three possible results by adding a linear layer and a softmax function. 3 ;

[0042]

[0043] Where W p and b p are the training parameters of the linear layer.

[0044] Furthermore, in step 5, the error resilience prediction of the target fault point is realized, and the premise is to train the model, which specifically includes: using the high-performance computing device fault samples obtained in step 2 to train the model formed according to steps 3 to 5, and guiding the optimization of model parameters through the cross entropy loss function L:

[0045]

[0046] In the formula, |D| represents the size of the training sample data, and are the true label and predicted label of the i-th sample in the training sample set D.

[0047] On the other hand, a high-performance computing device error resilience prediction system based on instruction semantic enhancement is provided, the system comprising:

[0048] The first module is used to conduct secondary development of the open source high-performance computing device binary plug-in tool NVBit to obtain the assembly instruction sequence and offset address information of the high-performance computing program;

[0049] The second module is used to build a fault simulation platform, perform single-particle upset fault simulation at the bit level based on the fault injection tool NVBitFI, and obtain fault samples of high-performance computing equipment;

[0050] A third module is used to generate instruction semantic embedding from the high-performance computing program instruction sequence obtained from the first module by adjusting a large language model with a large instruction sequence corpus, thereby encapsulating the semantics of the target high-performance computing program instruction;

[0051] The fourth module is used to analyze the control execution and data dependencies between instructions, construct two instruction topology graphs to represent the possible error propagation paths of bit-level faults, and build a graph neural network to extract error propagation patterns.

[0052] The fifth module is used to fuse fault embedding features and design a multi-head attention mechanism to combine the fault embeddings extracted from different graphs to achieve error resilience prediction of the target fault point.

[0053] Compared with the prior art, the present invention has the following significant advantages:

[0054] 1) It solves the difficulties of characterizing HPC program instructions and modeling bit-level error propagation, and achieves accurate prediction of error resilience of high-performance computing devices.

[0055] 2) Contextual semantics are used to characterize assembly-level instructions, and a large language model-based method is proposed to automatically mine instruction semantic embeddings, avoiding manual heuristic feature design.

[0056] 3) Two topological graphs were constructed based on the control execution and data transmission between instructions, and the error propagation of bit-level faults was innovatively simulated by designing a graph neural network, so as to achieve the goal of efficiently and accurately predicting the error resilience of high-performance computing devices.

[0057] The present invention is further described in detail below in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 The present invention is a flow chart of a method for predicting error resilience of a high-performance computing device based on instruction semantic enhancement.

[0059] Figure 2 : is a graph showing the difference in performance of a model under different amounts of training data in one embodiment, and a comparison with an ablation model and a baseline method, wherein Figure 2 (a) to (e) show the differences in model performance under different amounts of training data on the NW program (BI: bioinformatics field), DWT program (DC: data compression field), KNN program (DM: data mining field), srad program (IP: image processing field), and LU program (LA: linear algebra field), as well as comparison charts with the ablation model and baseline method.

[0060] Figure 3 FIG. 1 is a schematic diagram showing the effect of a model identifying a key SDC error in an embodiment, wherein Figure 3 (a) is a visualization of the confusion matrix of the present invention predicting three elastic output results. Figure 3 (b) in the figure shows the performance difference between the present invention and the baseline in identifying SDC errors. DETAILED DESCRIPTION

[0061] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0062] It should be noted that if there are descriptions involving "first", "second", etc. in the embodiments of the present invention, the descriptions of "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In addition, the technical solutions between the various embodiments can be combined with each other, but they must be based on the ability of ordinary technicians in the field to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such a combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0063] In one embodiment, a method for predicting error resilience of a high-performance computing device based on instruction semantic enhancement is provided, the method comprising the following steps:

[0064] Step 1: Perform secondary development on NVBit, an open source binary instrumentation tool for high-performance computing devices, to obtain the assembly instruction sequence and offset address information of the HPC program;

[0065] Here, the assembly instruction sequence of the program includes all static assembly instructions executed during program execution, as well as the type of each instruction and the text information of the register; the offset address of each instruction, that is, the relative address of each instruction in the corresponding kernel function.

[0066] Step 2: Perform single-particle upset fault simulation at the bit level based on the fault injection tool NVBitFI to obtain a small number of high-performance computing device fault samples;

[0067] Here, a single-particle upset fault simulation is performed based on NVBitFI to obtain the device fault point. Each fault point F site It is expressed as:

[0068] F site = j ,R q ,B o >

[0069] In the formula, I j ,R q and B o They represent that the target fault occurs at the jth instruction, at the oth bit in the register numbered q. Each fault point F site Each failure point is pre-encoded by encoding each element in its triplet in a one-hot manner and concatenating them as Where m is the total encoding dimension. Use NVBitFI to inject the fault point, obtain the error resilience label of the fault point, and construct the device fault sample.

[0070] Step 3: Generate instruction semantic embedding from the device program instruction sequence obtained in step 1 by fine-tuning a large language model with a large instruction sequence corpus, thereby encapsulating the semantics of the target program instruction, specifically including:

[0071] Step 3-1, the instruction sequence Φ of each program:

[0072] Φ=[I 1 ,I 2 ,...,I n ],

[0073] Where n is the number of instructions that the target program needs to implement a certain function, I n is the nth instruction;

[0074] Step 3-2, according to step 3-1, obtain instruction sequences of a large number of programs for pre-training of a large language model, learn bidirectional context representation, and obtain a large language model with enhanced instruction semantics.

[0075] Step 3-3, according to step 3-2, a large language model with enhanced instruction semantics is obtained, which encapsulates the instruction semantics of the target program, and the semantic embedding representation S is:

[0076]

[0077] In the formula, ​d is the dimension of the embedding vector, c indicates that these semantic vectors are the embeddings corresponding to the [CLS] tags extracted from the large model, and n indicates the total number of instructions. Represents the semantic embedding representation of the nth instruction;

[0078] Step 4: Analyze the control execution and data dependencies between instructions and construct two instruction topology graphs to represent the possible error propagation paths of bit-level faults, including:

[0079] Step 4-1: Analyze the control flow of the program, that is, the execution relationship of the instructions in the control flow; with instructions as nodes and control execution relationships as edges, the instruction control flow graph is defined as G c = {V,A c}, where V represents the node set consisting of instruction sequence Φ, A c Represents the adjacency matrix specified by control flow:

[0080]

[0081] In the formula, Yes A c The element in the i-th row and j-th column of V i and V j are two instruction nodes in V, → indicates instruction transmission from front to back; X, Y and Z are sets of different control relationship instruction nodes, including conditional branch, function call and synchronization;

[0082] Step 4-2, analyze the data flow of the program, that is, the dependency relationship of the instructions in the data flow. With instructions as nodes and dependencies as edges, the instruction data flow graph is defined as G o = {V,A o},A o Represents the adjacency matrix specified by the data flow:

[0083]

[0084] In the formula, Indicates that the two instructions use the same register, that is, there is a dependency relationship. Yes A o The element at the i-th row and j-th column in .

[0085] Furthermore, a graph neural network (graph convolutional neural network GCN and graph attention network GAT) is constructed to control the instruction flow graph G c And instruction data flow graph G o Modeling is performed to extract error propagation patterns, including:

[0086] Step 4-3, control flow graph G cUse GCN network modeling to get the embedding of control flow instructions

[0087] Step 4-4, for the instruction data flow graph G o Using GAT network modeling, we can embed the data flow instructions.

[0088] Step 5: Design a fusion method based on the multi-head attention mechanism to combine the fault embeddings extracted from different graphs to achieve error resilience prediction of the target fault point. Specifically:

[0089] Step 5-1: For each fault point, embed the command obtained in step 4 and Extract the fault instructions and embed them separately and Use the fault point code F obtained in step 2 e Calculate the attention scores s for both 1 and 2 for:

[0090]

[0091] In the formula, w a , W 1 and W 2 is a training parameter, τ(·) represents the nonlinear activation function Tanh;

[0092] Step 5-2: Use the attention weights obtained in step 5-1 to calculate the fused fault embedding for:

[0093]

[0094] Step 5-3, the fault embedding fused in step 5-2 is mapped to the probability distribution p∈[0,1] of three possible results (error masked (Masked), detected but unrecoverable error (DUE), and silent data corruption (SDC)) by adding a linear layer and a softmax function. 3 :

[0095]

[0096] Where W p and b p are the training parameters of the linear layer.

[0097] Furthermore, according to the model constructed in steps 3, 4, and 5, a small number of labeled equipment failure point samples D obtained in step 2 are used to train the model, and the designed cross entropy loss function L is used to guide the optimization of the model parameters:

[0098]

[0099] In the formula, |D| represents the size of the training sample data, and are the true label and predicted label of the i-th sample in D. Finally, a model with elastic prediction capability of high-performance computing devices is obtained.

[0100] In one embodiment, a high performance computing device error resilience prediction system based on instruction semantic enhancement is provided, the system comprising:

[0101] The first module is used to conduct secondary development of the open source high-performance computing device binary plug-in tool NVBit to obtain the assembly instruction sequence and offset address information of the high-performance computing program;

[0102] The second module is used to build a fault simulation platform, perform single-particle upset fault simulation at the bit level based on the fault injection tool NVBitFI, and obtain fault samples of high-performance computing equipment;

[0103] A third module is used to generate instruction semantic embedding from the high-performance computing program instruction sequence obtained from the first module by adjusting a large language model with a large instruction sequence corpus, thereby encapsulating the semantics of the target high-performance computing program instruction;

[0104] The fourth module is used to analyze the control execution and data dependencies between instructions, build two instruction topology graphs to represent the possible error propagation paths of bit-level faults, and build a graph neural network to extract error propagation patterns;

[0105] The fifth module is used to fuse fault embedding features and design a multi-head attention mechanism to combine the fault embeddings extracted from different graphs to achieve error resilience prediction of the target fault point.

[0106] For the specific limitations of the high-performance computing device error resilience prediction system based on instruction semantics enhancement, please refer to the limitations of the high-performance computing device error resilience prediction method based on instruction semantics enhancement mentioned above, which will not be repeated here. Each module in the above-mentioned high-performance computing device error resilience prediction system based on instruction semantics enhancement can be implemented in whole or in part through software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can perform the operations corresponding to the above modules.

[0107] As a specific example, in one of the embodiments, the present invention is further verified and explained.

[0108] This embodiment uses programs in the Rodinia and PolyBench benchmark test sets to conduct experiments, including 19 HPC applications such as clustering algorithms (kmeans), Gaussian calculations (gaussian), and Huffman coding (huffman). This embodiment divides these programs into 10 areas according to their functions in order to view the performance of the method, and the statistics are shown in Table 1 below. 10,000 fault samples are randomly selected from each program as training data, and another 20,000 samples (i.e., test sets) are selected as the standard for evaluating model performance. The training code is written based on the Tensorflow framework, and the trained model is used to predict the most likely elasticity results, i.e., one of masked, crash or timeout (DUE), and SDC, with accuracy (Acc) and macro F1 score (F1) as performance indicators.

[0109] Table 1 Statistics of programs by field

[0110]

[0111]

[0112] This embodiment selects several currently popular advanced and relevant methods as performance references for the model of the present invention: G-SEPM and PROGRAML methods based on error recovery characteristics or program code representation; IVDETECT and AMPLE methods focusing on error pattern mining. The method of the present invention is denoted as InstrDGM. The prediction results of each method on the benchmark program are shown in Table 2 below.

[0113] Table 2 Performance comparison of different methods on various benchmark programs (presented by field), bold and underlined indicate the best and second best performance respectively

[0114]

[0115] It can be seen from Table 2 that the method proposed in the present invention (InstrDGM) performs better in predicting the error resilience of equipment failure points. For each program, InstrDGM achieved better performance than the baseline on all 19 programs. Compared with the most competitive baseline, InstrDGM improved the Acc indicator by 1.49%-5.73% and the F1 indicator by 3.79%-8.49%. In addition, InstrDGM exhibits excellent robustness and adaptability, achieving up to 86.73% Acc even in the worst-performing DC domain. In contrast, some baselines show relatively large performance differences in different fields. For example, G-SEPM, PROGRAML and IVDETECT achieved higher Acc in the BI field, but only reached about 75% in the PS field. These results show that the present invention can effectively predict bit-level error resilience and significantly improve the accuracy of resilience prediction.

[0116] In order to verify the effectiveness of the present invention, it is necessary not only to consider the detection rate, but also to evaluate the efficiency. Therefore, to verify how many fault samples InstrDGM needs to achieve robust performance, the model is trained multiple times under different data amounts and the performance is evaluated. A program is randomly selected in each field for experimentation to obtain more reliable conclusions. Specifically, for each program, a certain number of samples (1K, 2K, ..., 8K) are randomly selected from the training set to retrain the model and evaluate the accuracy. Figure 2 The variation of accuracy with the amount of training data is shown. It can be observed that the accuracy of InstrDGM is basically positively correlated with the amount of data, and the performance improvement gradually slows down after 3K in most programs. In addition, the same experiment was performed on the best performing baseline. Figure 2 The comparison results in show that InstrDGM always outperforms the baseline, and the performance advantage is more obvious with smaller data volumes. These results show that InstrDGM has high training efficiency and does not lose much performance even with a small number of fault samples (for example, the accuracy difference between 3K and 9K training data is 3.22% on average). Based on considerations of semantic importance, the high efficiency of the present invention stems from the instruction semantic representation module of InstrDGM. To verify this, the instruction semantics enhanced large language model in the original model is replaced with the basic large language model (i.e., Roberta) to eliminate the semantic enhancement effect, and the reconstructed model is denoted as InstrDGM (w / o SE). As Figure 2 As shown in FIG. 1 , compared with InstrDGM, the accuracy of InstrDGM (w / oSE) is much worse, fluctuates greatly, and is even lower than the baseline in several programs. This means that the high efficiency is mainly due to the large language model with enhanced instruction semantics designed by the present invention.

[0117] In order to evaluate the performance of the present invention in identifying critical errors, the confusion matrix predicted by the model is visualized, as Figure 3 As shown in (a) in . It can be observed that the darker areas are concentrated on the main diagonal of each matrix, which indicates that the present invention correctly distinguishes the vast majority of different errors in programs in these domains. In addition, since SDC errors are the most serious, it is necessary to pay more attention to the performance of the model in identifying the fault locations that cause SDC. To this end, the F1 score of the model in identifying SDC errors is calculated and compared with the best performing baseline in each domain. Figure 3 As shown in (b), the average F1 score of InstrDGM is 86.08%, which is an average improvement of 3.58% compared with the most competitive baseline. This shows that the present invention can effectively identify key SDC errors.

[0118] In summary, the high-performance computing device error resilience prediction method based on instruction semantics enhancement proposed in the present invention solves the difficulties in characterizing HPC program instructions and modeling bit-level error propagation, and realizes accurate high-performance computing device error resilience prediction. Technically, the present invention uses contextual semantics to characterize device assembly-level instructions, and proposes a method based on a large language model to automatically mine instruction semantic embeddings, avoiding manual heuristic feature design. In addition, the present invention constructs two topological graphs based on control execution and data transmission between instructions, and innovatively simulates the error propagation of bit-level faults therein by designing a graph neural network, thereby achieving the purpose of efficiently and accurately predicting the error resilience of high-performance computing devices.

[0119] The above shows and describes the basic principles, main features and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments, and the above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, and these changes and improvements all fall within the scope of the present invention to be protected.

Claims

1. A method for predicting error resilience of high-performance computing equipment based on instruction semantic enhancement, characterized in that: The method comprises the following steps: Step 1: Perform secondary development on the open source high-performance computing device binary plug-in tool NVBit to obtain the assembly instruction sequence and offset address information of the high-performance computing program; Step 2: Perform single-particle upset fault simulation at the bit level based on the fault injection tool NVBitFI to obtain high-performance computing device fault samples; Step 3, generating instruction semantic embedding from the assembly instruction sequence of the high-performance computing program obtained in step 1 by adjusting a large language model with a large instruction sequence corpus, thereby encapsulating the semantics of the target high-performance computing program instructions; Step 4: Analyze the control execution and data dependency between instructions, construct two instruction topology graphs to represent possible error propagation paths of bit-level faults, build a graph neural network to model the two instruction topology graphs, and extract error propagation patterns; Step 5: Design a fusion method based on the multi-head attention mechanism to combine the fault embeddings extracted from different graphs to achieve error resilience prediction of the target fault point.

2. The method for predicting error resilience of high-performance computing equipment based on instruction semantic enhancement according to claim 1, characterized in that: The assembly instruction sequence of the program described in step 1 includes all static assembly instructions executed during the program running process, as well as the type of each instruction and the text information of the register; the offset address of each instruction, that is, the relative address of each instruction in the corresponding kernel function.

3. The method for predicting error resilience of high-performance computing equipment based on instruction semantic enhancement according to claim 1, characterized in that: In step 2, a single-particle upset fault simulation is performed based on NVBitFI to obtain the fault point of the high-performance computing device. Each fault point F site It is expressed as: F site =<I j ,R q ,B o > In the formula, I j ,R q and B o They respectively indicate that the target fault occurs in the j-th instruction, at the o-th bit in the register numbered q.

4. The method for predicting error resilience of high-performance computing equipment based on instruction semantic enhancement according to claim 3, characterized in that: Each fault point F in step 2 site Each failure point is pre-encoded by encoding each element in its triplet in a one-hot manner and concatenating them as Where m is the total encoding dimension; Use the fault injection tool NVBitFI to inject fault points, obtain the error resilience labels of the fault points, and construct high-performance computing equipment fault samples.

5. The method for predicting error resilience of high-performance computing equipment based on instruction semantic enhancement according to claim 4, characterized in that: In step 3, the assembly instruction sequence of the high-performance computing program obtained from step 1 is generated by adjusting a large language model with a large instruction sequence corpus to encapsulate the semantics of the target high-performance computing program instructions, specifically including: Step 3-1, obtain the instruction sequence Φ of each program as: Φ=[I1,I2,...,I n ], Where n is the number of instructions that the program needs to implement a certain function, I n is the nth instruction; Step 3-2, using the instruction sequence of the program obtained in step 3-1 for pre-training of a large language model, learning a bidirectional context representation, and obtaining a large language model with enhanced instruction semantics; Step 3-3, based on the instruction semantics enhanced large language model obtained in step 3-2, encapsulate the instruction semantics of the program, and the instruction semantics embedding is represented as S: In the formula, d is the dimension of the embedding vector, c indicates that these semantic vectors are the embeddings corresponding to the [CLS] tags extracted from the large model, and n indicates the total number of instructions. Represents the semantic embedding representation of the nth instruction.

6. The method for predicting error resilience of high-performance computing equipment based on instruction semantic enhancement according to claim 5, characterized in that: Step 4 analyzes the control execution and data dependencies between instructions and constructs two instruction topology graphs to represent possible error propagation paths of bit-level faults, specifically including: Step 4-1: Analyze the control flow of the program, that is, the execution relationship of the instructions in the control flow; with instructions as nodes and control execution relationships as edges, the instruction control flow graph is defined as G c = {V,A c }, where V represents the node set consisting of instruction sequence Φ, A c represents the adjacency matrix specified by the control flow; In the formula, Yes A c The element in the i-th row and j-th column of V i and V j are two instruction nodes in V, → indicates instruction transmission from front to back; X, Y and Z are sets of different control relationship instruction nodes, including conditional branch, function call and synchronization; Step 4-2: Analyze the data flow of the program, that is, the dependency relationship of the instructions in the data flow; with instructions as nodes and dependencies as edges, the instruction data flow graph is defined as G o = {V,A o }, A o Represents the adjacency matrix specified by the data flow: In the formula, It means that two instructions use the same register, that is, there is a dependency relationship; Yes A o The element at the i-th row and j-th column in .

7. The method for predicting error resilience of high-performance computing equipment based on instruction semantic enhancement according to claim 6, characterized in that: In step 4, a graph neural network is built to model the two instruction topology graphs and extract the error propagation mode, specifically: a graph neural network is built to model the instruction control flow graph G c And instruction data flow graph G o Modeling is performed with the instruction semantic embedding S and the encoding of the fault point F e As input, extract instruction embeddings, specifically including: Step 4-3, control flow graph G c Use GCN network modeling to get the embedding of control flow instructions Step 4-4, for the instruction data flow graph G o Using GAT network modeling, we can embed the data flow instructions.

8. The method for predicting error resilience of high-performance computing equipment based on instruction semantic enhancement according to claim 7, characterized in that: Step 5 describes the design of a fusion method based on a multi-head attention mechanism, which combines the fault embeddings extracted from different graphs to achieve error resilience prediction of the target fault point, including: Step 5-1: For each fault point, extract the embedded control flow instructions and embedding of data flow instructions Extract the fault instructions and embed them separately and Using the fault point code F e Calculate the attention scores s1 and s2 of each of them respectively: In the formula, w a , W1 and W2 are training parameters, τ(·) represents the nonlinear activation function Tanh; Step 5-2: Use the attention score obtained in step 5-1 to calculate the fused fault embedding for: Step 5-3: Map the fault embedding fused in step 5-2 to the probability distribution p∈[0,1] of three possible results by adding a linear layer and a softmax function. 3 ; Where W p and b p are the training parameters of the linear layer.

9. The method for predicting error resilience of high-performance computing equipment based on instruction semantic enhancement according to claim 8, characterized in that: In step 5, the error resilience prediction of the target fault point is realized, and the premise is to train the model, which specifically includes: using the high-performance computing device fault samples obtained in step 2 to train the model formed according to steps 3 to 5, and guiding the optimization of model parameters through the cross entropy loss function L: In the formula, |D| represents the size of the training sample data, and are the true label and predicted label of the i-th sample in the training sample set D.

10. A high performance computing device error resilience prediction system based on instruction semantic enhancement based on the method according to any one of claims 1 to 9, characterized in that: The system includes sequentially executing: The first module is used to conduct secondary development of the open source high-performance computing device binary plug-in tool NVBit to obtain the assembly instruction sequence and offset address information of the high-performance computing program; The second module is used to build a fault simulation platform, perform single-particle upset fault simulation at the bit level based on the fault injection tool NVBitFI, and obtain fault samples of high-performance computing equipment; A third module is used to generate instruction semantic embedding from the high-performance computing program instruction sequence obtained from the first module by adjusting a large language model with a large instruction sequence corpus, thereby encapsulating the semantics of the target high-performance computing program instruction; The fourth module is used to analyze the control execution and data dependencies between instructions, construct two instruction topology graphs to represent the possible error propagation paths of bit-level faults, and build a graph neural network to extract error propagation patterns. The fifth module is used to fuse fault embedding features and design a multi-head attention mechanism to combine the fault embeddings extracted from different graphs to achieve error resilience prediction of the target fault point.

Images