Error resilience prediction method for high performance computing devices based on instruction semantic enhancement

Abstract

The application discloses a high-performance computing device error resilience prediction method based on instruction semantic enhancement, comprising the following steps: obtaining the assembly instruction sequence and offset address information of a high-performance computing program; performing single event upset fault simulation to obtain a small amount of high-performance computing device fault samples; generating instruction semantic embedding to encapsulate the semantics of the target high-performance computing program instruction; analyzing the control execution and data dependency relationship between instructions, constructing two instruction topology graphs, modeling the same by using a graph neural network, and extracting error propagation patterns; and combining the fault embedding extracted from different graphs to perform error resilience prediction. The application can better understand the resilience characteristics of the high-performance computing program by performing instruction semantic representation on the high-performance computing program, and can realize accurate error resilience prediction without a large amount of fault simulation by modeling the error propagation of the high-performance computing program at the bit level, thereby greatly saving the computing resources and cost, and being suitable for various high-performance computing programs running on the high-performance computing device.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of hardware device error resilience evaluation, and particularly relates to a high-performance computing device error resilience prediction method based on instruction semantic enhancement. BACKGROUND

[0002] In modern high-performance computing (HPC) systems, graphics processing units are widely deployed due to their powerful computing capabilities. However, as the size of application programs increases and the operating voltage decreases, the failure rate in computing elements significantly increases. These failures can be classified into permanent faults or transient faults. Permanent faults result from irreversible changes in hardware components and persist until direct hardware intervention is implemented to correct them. In contrast, transient faults, also known as soft errors, are mainly caused by bit flips within computing units due to electromagnetic interference. Bit flip faults affect running HPC programs and result in various outcomes, such as system crashes, runtime timeouts, etc. (here, the HPC program refers to any program executed on a high-performance computing device, regardless of the program language). The biggest threat is silent data corruption (SDC), which severely damages data integrity and produces incorrect outputs. Moreover, SDC does not cause system crashes or obvious error messages, making it difficult to detect.

[0003] To reduce the potential risk of SDC, most reliable systems employ conservative schemes, such as modular redundancy, which protect every part of the program executed on hardware without discrimination, even though some parts almost never trigger SDC. The significant overhead of these schemes poses a major challenge to their efficiency and widespread adoption, especially for highly parallel HPC programs. Therefore, it is more intuitive to employ adaptive strategies, such as selective instruction redundancy, which allocates computing resources to different parts by considering the varying degrees of error resilience between assembly instructions.

[0004] The traditional method is to deliberately introduce faults during program execution and compare the resulting output with predetermined standards to determine the type of error that occurred (SDC, crash, etc.), i.e., fault injection (FI). Although this method can provide relatively accurate error resilience estimates, it usually requires tens of thousands of executions of the entire program to conduct exhaustive FI activities. Obviously, this is very resource-intensive for HPC programs with high-density computing. To speed up the evaluation, researchers have begun to consider reducing the number of required FI activities while maintaining the accuracy of error resilience estimates. Existing device resilience-related work estimates the error resilience of programs by understanding resilience characteristics or modeling error propagation. However, these works still have certain limitations, mainly that the features indicating resilience are dependent on manual extraction from numerous fault simulations, and error propagation analysis cannot be targeted at fine-grained bit-level faults. SUMMARY

[0005] The present application aims at the problems existing in the prior art, and provides a high-performance computing device error resilience prediction method based on instruction semantic enhancement, which simulates the error propagation process of bit-level faults by considering the context semantic features of instructions, explores the device resilience mechanism, and realizes accurate fault point error resilience prediction under the condition of only a small number of fault injection samples. In addition, the present application has good universality and is suitable for resilience prediction of various programs running on high-performance computing devices.

[0006] The technical solution for achieving the object of the present application is: a high-performance computing device error resilience prediction method based on instruction semantic enhancement, comprising the following steps:

[0007] Step 1: secondary development of the open-source high-performance computing device binary instrumentation tool NVBit to obtain the assembly instruction sequence and offset address information of the high-performance computing program;

[0008] Step 2: single event upset fault simulation at the bit level based on the fault injection tool NVBitFI to obtain high-performance computing device fault samples;

[0009] Step 3: generating instruction semantic embeddings by adjusting a large instruction sequence corpus language model from the assembly instruction sequence of the high-performance computing program obtained in step 1, so as to encapsulate the semantics of the target high-performance computing program instructions;

[0010] Step 4: analyzing the control execution and data dependency relationship between instructions, constructing two instruction topology graphs to represent the possible error propagation paths of bit-level faults, building a graph neural network to model the two instruction topology graphs, and extracting error propagation patterns;

[0011] Step 5: designing a fusion method based on a multi-head attention mechanism to combine the fault embeddings extracted from different graphs, and realizing error resilience prediction of the target fault point.

[0012] Further, the assembly instruction sequence of the program in step 1 includes all static assembly instructions executed during program execution, as well as the type and register text information of each instruction; the offset address of each instruction, i.e. the relative address of each instruction within the corresponding kernel function.

[0013] Further, in step 2, single event upset fault simulation is performed based on NVBitFI to obtain high-performance computing device fault points, each fault point F site is represented as:

[0014] F site =<I j ,R q ,B o >

[0015] where I j ,R q and B o represent the target fault occurs at the jth instruction, the oth bit in the register numbered q.

[0016] Further, each fault point F site Each fault point is pre-encoded as where m is the total encoding dimension; the fault point is injected using the fault injection tool NVBitFI, the error resilience label of the fault point is obtained, and the fault sample of the high-performance computing device is constructed.

[0017] Further, the assembly instruction sequence of the high-performance computing program obtained in step 1 is adjusted to generate instruction semantic embedding by adjusting the large language model with a large instruction sequence corpus, so as to encapsulate the semantics of the target high-performance computing program instruction, specifically including:

[0018] Step 3-1, obtaining the instruction sequence Φ of each program:

[0019] Φ=[I1,I2,...,In] n ],

[0020] where n is the number of instructions for the program to implement a certain function, I n is the nth instruction;

[0021] Step 3-2, according to the instruction sequence of the program obtained in step 3-1, the pre-training of the large language model is used to learn the bidirectional context representation, and the instruction semantic enhanced large language model is obtained;

[0022] Step 3-3, according to the instruction semantic enhanced large language model obtained in step 3-2, the instruction semantic of the program is encapsulated, and the instruction semantic embedding is represented as S:

[0023]

[0024] where d is the dimension of the embedding vector, c represents the embedding corresponding to the [CLS] mark extracted in the large model, n represents the total number of instructions, represents the semantic embedding representation of the nth instruction.

[0025] Further, step 4 analyzes the control execution and data dependency relationship between instructions, and constructs two instruction topology graphs to represent the possible error propagation path of the bit-level fault, specifically including:

[0026] Step 4-1, analyze the control flow of the program, that is, the execution relationship of instructions in the control flow; define the instruction control flow graph as G c ={V,A c},where V represents a node set consisting of the instruction sequence Φ, and A c represents an adjacency matrix specified by the control flow;

[0027]

[0028] In the formula, is the element in the i-th row and the j-th column of A c , V i and V j are two instruction nodes in V, and → represents an instruction transmission from the front to the back; X, Y, and Z are different sets of control relationship instruction nodes, including conditional branches, function calls, and synchronization instruction nodes;

[0029] Step 4-2, analyze the data flow of the program, that is, the dependency relationship of instructions in the data flow; define the instruction data flow graph as G o ={V,A o},A o represents an adjacency matrix specified by the data flow:

[0030]

[0031] In the formula, represents that two instructions use the same register, that is, there is a dependency relationship; is the element in the i-th row and the j-th column of A o .

[0032] Further, in step 4, a graph neural network is built to model the two instruction topological graphs and extract error propagation patterns, specifically: a graph neural network is built to model the instruction control flow graph G c and the instruction data flow graph G o , taking the instruction semantic embedding S and the encoding F e of the fault point as input, extracting instruction embedding, specifically including:

[0033] Step 4-3, model the instruction control flow graph G c using a GCN network to obtain the embedding of the control flow instruction

[0034] Step 4-4, model the instruction data flow graph G o using a GAT network to obtain the embedding of the data flow instruction

[0035] Further, step 5 describes a fusion method based on the multi-head attention mechanism, which combines the fault embeddings extracted from different graphs to realize error-resilient prediction of the target fault point, specifically including:

[0036] Step 5-1, for each fault point, according to the extracted embedding of the control flow instruction and the embedding of the data flow instruction extract the fault instruction embedding and from them respectively e Calculate the respective attention scores s1 and s2 of the two as follows:

[0037]

[0038] where w a , W1 and W2 are training parameters, and τ(·) represents a nonlinear activation function Tanh;

[0039] Step 5-2, using the attention scores obtained in step 5-1, calculate the fused fault embedding as follows:

[0040]

[0041] Step 5-3, map the fused fault embedding in step 5-2 to the probability distribution p∈[0,1] 3 of the three possible results by adding a linear layer and a softmax function.

[0042]

[0043] where W p and b p are training parameters of the linear layer.

[0044] Further, step 5 realizes error-resilient prediction of the target fault point, which is premised on training the model, specifically including: using the high-performance computing device fault samples obtained in step 2 to train the model formed according to steps 3 to 5, and guiding the optimization of model parameters through a cross-entropy loss function L:

[0045]

[0046] where |D| represents the size of the training sample data, and is the true label and predicted label of the i-th sample in the training sample set D.

[0047] On the other hand, a high-performance computing device error-resilient prediction system based on instruction semantic enhancement is provided, which includes the following steps executed in sequence:

[0048] The first module is used for secondary development of an open-source high-performance computing device binary instrumentation tool NVBit, so as to obtain an assembly instruction sequence and offset address information of a high-performance computing program.

[0049] The second module is used for constructing a fault simulation platform, and performing single event upset fault simulation at a bit level based on a fault injection tool NVBitFI, so as to obtain a high-performance computing device fault sample.

[0050] The third module is used for obtaining the high-performance computing program instruction sequence from the first module, generating instruction semantic embedding by adjusting a large language model with a large instruction sequence corpus, and encapsulating semantics of a target high-performance computing program instruction.

[0051] The fourth module is used for analyzing control execution and data dependency relationships between instructions, constructing two instruction topology graphs to represent possible error propagation paths of bit-level faults, and building a graph neural network to extract error propagation patterns.

[0052] The fifth module is used for fusing fault embedding features, designing a multi-head attention mechanism, combining fault embeddings extracted from different graphs, and realizing error resilience prediction of a target fault point.

[0053] Compared with the prior art, the present application has the following advantages:

[0054] 1) The difficulty of representing HPC program instructions and modeling bit-level error propagation is solved, and accurate high-performance computing device error resilience prediction is realized.

[0055] 2) Context semantics is used to represent assembly-level instructions, and a method based on a large language model is proposed to automatically mine instruction semantic embedding, avoiding manual heuristic feature design.

[0056] 3) Two topology graphs are constructed based on control execution and data transmission between instructions, and a graph neural network is designed to innovatively simulate error propagation of bit-level faults, so as to achieve the purpose of efficiently and accurately predicting high-performance computing device error resilience.

[0057] The present application will be described in further detail below with reference to the accompanying drawings. BRIEF DESCRIPTION OF DRAWINGS

[0058] Figure 1 A flowchart of the present application based on instruction semantic enhancement for high-performance computing device error resilience prediction method.

[0059] Figure 2 A difference in performance of a model in an embodiment under different training data amounts, and a comparison with an ablation model and a baseline method. Figure 2(a)-(e) in FIG. 1 are respectively: the difference of model performance under different training data volume on NW program (BI: bioinformatics field), DWT program (DC: data compression field), KNN program (DM: data mining field), srad program (IP: image processing field), and LU program (LA: linear algebra field), and comparison with ablation model and baseline method.

[0060] Figure 3 FIG. 2 is a schematic diagram of the effect of model identifying key SDC errors in an embodiment, wherein Figure 3 (a) in FIG. 3 is a confusion matrix visualization of the three elastic output results predicted by the application, Figure 3 (b) in FIG. 3 is the performance difference of the application and the baseline in identifying SDC errors. DETAILED DESCRIPTION

[0061] In order to make the purposes, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0062] It should be noted that if the present application has a description of "first", "second", etc. in the embodiments, the description of "first", "second", etc. is only for the purpose of description, and cannot be understood as indicating or implying the relative importance of the indicated technical features or implicitly indicating the number of the indicated technical features. Therefore, the features with "first", "second" can explicitly or implicitly include at least one of the features. In addition, the technical solutions of each embodiment can be combined with each other, but it must be based on the realization of ordinary skilled in the art, when the combination of technical solutions appears contradictory or unachievable, it should be considered that the combination of technical solutions does not exist, nor within the scope of protection claimed by the present application.

[0063] In one embodiment, a high-performance computing device error resilience prediction method based on instruction semantic enhancement is provided, and the method comprises the following steps:

[0064] Step 1, the open source binary instrumentation tool NVBit of high-performance computing device is developed again to obtain the assembly instruction sequence and offset address information of HPC program;

[0065] Here, the assembly instruction sequence of the program includes all static assembly instructions executed during program execution, as well as the type and register text information of each instruction; the offset address of each instruction, i.e. the relative address of each instruction within the kernel function.

[0066] Step 2, based on the fault injection tool NVBitFI, single event upset fault simulation is performed at the bit level to obtain a small number of high-performance computing device fault samples;

[0067] Here, a single-event upset fault simulation is performed based on NVBitFI to obtain the device fault points, and each fault point F site Represented as:

[0068] F site = j ,R q B o >

[0069] In the formula, I j ,R q and B o These represent the target fault occurring in the o-th bit of register q, which is located at instruction j. Each fault point F... site Each element in the triplet is encoded and concatenated using a single-heat method, precoding each fault point as follows: Where m is the total encoding dimension. NVBitFI is used to inject fault points, obtain the error resilience labels of the fault points, and construct device fault samples.

[0070] Step 3: From the device program instruction sequence obtained in Step 1, generate instruction semantic embeddings by fine-tuning a large language model with a large corpus of instruction sequences, thereby encapsulating the semantics of the target program instructions. Specifically, this includes:

[0071] Step 3-1, the instruction sequence Φ for each program:

[0072] Φ=[I1,I2,...,I n ],

[0073] In the formula, n is the number of instructions that the target program needs to implement a certain function, and I n It is the nth instruction;

[0074] Step 3-2: Based on the instruction sequences of a large number of programs obtained in Step 3-1, use them for pre-training of the large language model, learn bidirectional contextual representations, and obtain a large language model with enhanced instruction semantics.

[0075] Step 3-3: Based on the instruction semantic enhancement large language model obtained in Step 3-2, encapsulate the instruction semantics of the target program, and the semantic embedding representation S:

[0076]

[0077] In the formula, d is the dimension of the embedding vectors, c indicates that these semantic vectors are the embeddings corresponding to the [CLS] tags extracted from the large model, and n represents the total number of instructions. The semantic embedding representation of the nth instruction;

[0078] ​Step 4, analyze the control execution and data dependency relationship between instructions, build two instruction topological graphs to represent the possible error propagation paths of bit-level faults, including:

[0079] Step 4-1, analyze the control flow of the program, that is, the execution relationship of the instructions in the control flow; define the instruction control flow graph G c ={V, A c}, where V represents the node set composed of the instruction sequence Φ, and A c represents the adjacency matrix specified by the control flow:

[0080]

[0081] In the formula, is the element in the i-th row and j-th column of A c , V i and V j are two instruction nodes in V, → represents the instruction transmission with the front pointing to the back; X, Y and Z are three instruction node sets of different control relationships, including conditional branching, function calling and synchronization;

[0082] Step 4-2, analyze the data flow of the program, that is, the dependency relationship of the instructions in the data flow. Define the instruction data flow graph G o ={V, A o}, A o represents the adjacency matrix specified by the data flow:

[0083]

[0084] In the formula, represents that two instructions use the same register, that is, there is a dependency relationship, is the element in the i-th row and j-th column of A o .

[0085] Further, build a graph neural network (graph convolutional neural network GCN and graph attention network GAT) to model the instruction control flow graph G c and the instruction data flow graph G o , extract the error propagation pattern, including:

[0086] Step 4-3, model the instruction control flow graph G c using the GCN network to obtain the embedding of the control flow instruction

[0087] Step 4-4, model the instruction data flow graph G o using the GAT network to obtain the embedding of the data flow instruction

[0088] Step 5, a fusion method based on multi-head attention mechanism is designed to combine the fault embeddings extracted from different graphs to realize error resilient prediction of the target fault point, specifically:

[0089] Step 5-1, for each fault point, the instruction embedding obtained in step 4 is and extract the fault instruction embedding and from them respectively, and use the fault point encoding F obtained in step 2 e Calculate the attention scores s1 and s2 of the two respectively as:

[0090]

[0091] In the formula, w a , W1 and W2 are training parameters, and τ(·) represents a nonlinear activation function Tanh.

[0092] Step 5-2, using the attention weights obtained in step 5-1, calculate the fused fault embedding as:

[0093]

[0094] Step 5-3, the fused fault embedding in step 5-2 is mapped to the probability distribution p ∈ [0, 1] of the three possible results (error masked (Masked), detected but unrecoverable error (DUE), and silent data corruption (SDC)) by adding a linear layer and a softmax function 3 :

[0095]

[0096] In the formula, W p and b p are training parameters of the linear layer.

[0097] Further, according to the model constructed in steps 3, 4, and 5, use the small number of labeled device fault point samples D obtained in step 2 to train the model, and use the designed cross-entropy loss function L to guide the optimization of the model parameters:

[0098]

[0099] In the formula, |D| represents the size of the training sample data, and are the true label and predicted label of the i-th sample in D. Finally, a model with high-performance computing device resilient prediction capability is obtained.

[0100] In one embodiment, a high-performance computing device error resilience prediction system based on instruction semantic enhancement is provided, and the system comprises the following sequentially executed modules:

[0101] A first module for secondary development of an open-source high-performance computing device binary instrumentation tool NVBit to obtain assembly instruction sequences and offset address information of a high-performance computing program;

[0102] A second module for constructing a fault simulation platform to simulate single event upset faults at the bit level based on a fault injection tool NVBitFI to obtain high-performance computing device fault samples;

[0103] A third module for obtaining high-performance computing program instruction sequences from the first module, generating instruction semantic embeddings by adjusting a large language model with a large instruction sequence corpus, and encapsulating the semantics of target high-performance computing program instructions;

[0104] A fourth module for analyzing control execution and data dependency relationships between instructions, constructing two instruction topology graphs to represent possible error propagation paths of bit-level faults, and building a graph neural network to extract error propagation patterns;

[0105] A fifth module for fusing fault embedding features, designing a multi-head attention mechanism to combine fault embeddings extracted from different graphs, and realizing error resilience prediction for target fault points.

[0106] For specific limitations of the high-performance computing device error resilience prediction system based on instruction semantic enhancement, refer to the limitations of the high-performance computing device error resilience prediction method based on instruction semantic enhancement in the above, which will not be repeated here. Each module in the above high-performance computing device error resilience prediction system based on instruction semantic enhancement can be realized by software, hardware, and combinations thereof, in whole or in part. The above modules can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory of the computer device in software form, so that the processor executes the operations corresponding to each module.

[0107] As a specific example, in one embodiment, the present application is further verified and described.

[0108] This example uses programs in the Rodinia and PolyBench benchmark suites for experiments, including 19 HPC applications such as clustering algorithm (kmeans), Gaussian computation (gaussian), Huffman encoding (huffman), etc. This example divides these programs into 10 domains according to their functions in order to view the performance of the method, as shown in Table 1 below. Randomly select 10,000 fault samples in each program as training data, and select another 20,000 samples (i.e. test set) as a standard for evaluating the performance of the model. The training code is written based on the Tensorflow framework, and the trained model is used to predict the most likely resilience result, i.e. one of Masked, Crash or Timeout (DUE) and SDC, and accuracy (Acc) and macro F1 score (F1) as performance indicators.

[0109] Table 1 Statistics of program division by domain

[0110]

[0111]

[0112] This example selects several advanced and related methods that are currently popular as performance references for the model of the present application: G-SEPM and PROGRAML methods based on error recovery characteristics or program code representation; IVDETECT and AMPLE methods focusing on error pattern mining. The method of the present application is denoted as InstrDGM. The prediction results of each method on the benchmark program are shown in Table 2 below.

[0113] Table 2 Performance comparison of different methods on each benchmark program (presented by domain), bold and underlined respectively indicating the best and second best performance

[0114]

[0115] As can be seen from Table 2, the method (InstrDGM) proposed by the present application performs better in error resilience prediction at the device failure point. For each program, InstrDGM achieves better performance than the baseline on all 19 programs. Compared with the most competitive baseline, InstrDGM improves the Acc index by 1.49%-5.73% and the F1 index by 3.79%-8.49%. In addition, InstrDGM shows excellent robustness and adaptability, achieving up to 86.73% Acc even in the worst-performing DC domain. In contrast, some baselines show relatively large performance differences in different domains, for example, G-SEPM, PROGRAML and IVDETECT achieve higher Acc in the BI domain, but only about 75% in the PS domain. These results show that the present application can effectively predict bit-level error resilience, significantly improving the accuracy of resilience prediction.

[0116] To verify the effectiveness of the present application, not only the detection rate needs to be considered, but also the efficiency needs to be evaluated. Therefore, to verify how many fault samples InstrDGM needs to achieve robust performance, the model is trained multiple times under different data quantities and the performance is evaluated. One program is randomly selected in each domain for experiments to obtain more reliable conclusions. Specifically, for each program, a certain number of samples (1K, 2K, …, 8K) are randomly extracted from the training set to retrain the model and evaluate the accuracy. Figure 2 The accuracy changes with the amount of training data are shown. It can be observed that the accuracy of InstrDGM is basically positively correlated with the data quantity, and in most programs, the performance improves gradually slowly after 3K. In addition, the same experiment is also performed on the best-performing baseline, Figure 2 The comparison results in Table 3 show that InstrDGM is always better than the baseline, and the performance advantage is more obvious under smaller data quantities. These results show that InstrDGM has high training efficiency, and does not lose much performance even with a small number of fault samples (for example, the accuracy difference between 3K and 9K training data is an average of 3.22%). Based on the consideration of semantic importance, the high efficiency of the present application is derived from the instruction semantic representation module of InstrDGM. To verify this, the original model is replaced with a basic large language model (i.e. Roberta) to eliminate the semantic enhancement effect, and the rebuilt model is denoted as InstrDGM(w / o SE). As shown in Table 4, Figure 2 Compared with InstrDGM, the accuracy of InstrDGM(w / o SE) is much worse and fluctuates greatly, and even lower than the baseline on several programs. This means that this high efficiency is mainly due to the instruction semantic enhancement large language model designed by the present application.

[0117] To evaluate the performance of the present application in identifying critical errors, the confusion matrix of the model predictions is visualized as shown in (a) of Figure 3 It can be observed that the darker regions are concentrated on the main diagonal of each matrix, which indicates that the present application correctly distinguished the vast majority of different errors of the programs in these domains. Moreover, since SDC errors are the most severe, more attention needs to be paid to the performance of the model in identifying the location of faults leading to SDC. To this end, the F1 score of the model in identifying SDC errors is calculated and compared with the best-performing baseline in each domain. As shown in (b) of Figure 3 , where the average F1 score of InstrDGM is 86.08%, which is an average improvement of 3.58% compared with the most competitive baseline. This shows that the present application can effectively identify critical SDC errors.

[0118] In summary, the present application proposes a high-performance computing device error resilience prediction method based on instruction semantic enhancement, which solves the difficulty of characterizing HPC program instructions and modeling bit-level error propagation, and realizes accurate high-performance computing device error resilience prediction. Technically, the present application uses context semantics to characterize device assembly-level instructions, and at the same time proposes a method based on large language models to automatically mine instruction semantic embeddings, avoiding manual heuristic feature design. In addition, the present application constructs two topology graphs based on the control execution and data transmission between instructions, and innovatively simulates the error propagation of bit-level faults through the design of graph neural networks, thereby achieving the purpose of efficiently and accurately predicting the error resilience of high-performance computing devices.

[0119] The basic principles, main features and advantages of the present application are shown and described above. It should be understood by those skilled in the art that the present application is not limited by the above examples, and the above examples and descriptions in the specification are only to illustrate the principles of the present application. Without departing from the spirit and scope of the present application, various changes and improvements can be made to the present application, and these changes and improvements all fall within the scope of the claimed present application.

Claims

1. A method for error resilience prediction of high-performance computing devices based on instruction semantic enhancement, characterized in that, The method includes the following steps: Step 1: Perform secondary development on the open-source high-performance computing device binary instrumentation tool NVBit to obtain the assembly instruction sequence and offset address information of the high-performance computing program; Step 2: Based on the fault injection tool NVBitFI, perform single-event flip fault simulation at the bit level to obtain fault samples of high-performance computing devices; Step 3: The assembly instruction sequence of the high-performance computing program obtained in Step 1 is used to generate instruction semantic embedding by adjusting a large language model with a large instruction sequence corpus, thereby encapsulating the semantics of the target high-performance computing program instructions. Step 4: Analyze the control execution and data dependencies between instructions, construct two instruction topology graphs to represent possible error propagation paths of bit-level faults, build a graph neural network to model the two instruction topology graphs, and extract error propagation patterns; Step 5: Design a fusion method based on multi-head attention mechanism to combine fault embeddings extracted from different graphs to achieve error elasticity prediction of target fault points; Step 4 analyzes the control execution and data dependencies between instructions, constructs two instruction topology graphs, and represents possible error propagation paths for bit-level faults, specifically including: Step 4-1: Analyze the program's control flow, that is, the execution relationships of instructions within the control flow; with instructions as nodes and control execution relationships as edges, the instruction control flow graph is defined as follows: ,in Indicates a sequence of instructions The set of nodes formed This represents the adjacency matrix specified by the control flow; ； In the formula, yes The element in the i-th row and j-th column, and yes The two instruction nodes in the diagram, →, indicate instruction transmission from front to back; X, Y, and Z are sets of instruction nodes with different control relationships, including conditional branch, function call, and synchronization instruction nodes. Step 4-2: Analyze the program's data flow, specifically the dependencies between instructions within the data flow; using instructions as nodes and dependencies as edges, the instruction data flow graph is defined as follows: , The adjacency matrix specified by the data stream: ； In the formula, This indicates that two instructions use the same register, meaning there is a dependency between them; yes The element in the i-th row and j-th column; In step 4, a graph neural network is built to model the two instruction topology graphs and extract error propagation patterns. Specifically, this involves building a graph neural network to model the instruction control flow graph. and Modeling is performed to embed the instruction semantics into S and the encoding of the fault point. As input, the instruction embedding is extracted, specifically including: Step 4-3, perform an instruction control flow graph Using GCN network modeling, the embedding of control flow instructions is obtained. ; Step 4-4, perform an instruction data flow diagram Using GAT network modeling, the embedding of data flow instructions is obtained. ; Step 5 describes a fusion method based on a multi-head attention mechanism, which combines fault embeddings extracted from different graphs to achieve resilient error prediction of the target fault point. Specifically, this includes: Step 5-1: For each fault point, based on the embedded control flow instructions... and Extract the fault instructions and embed them separately. and Using the code of the fault point Calculate the attention scores for each. and They are respectively: ； ； In the formula, , and These are training parameters. Tanh represents the nonlinear activation function; Step 5-2: Using the attention score obtained in Step 5-1, calculate the fused fault embedding. for: ； Step 5-3: The fault embedding fused in Step 5-2 is mapped to the probability distribution of the three possible outcomes by adding a linear layer and a softmax function. ; ； In the formula, and These are the training parameters for the linear layer.

2. The high-performance computing device error resilience prediction method based on instruction semantic enhancement according to claim 1, characterized in that, The assembly instruction sequence of the program described in step 1 includes all static assembly instructions executed during program execution, as well as the type and register text information of each instruction; the offset address of each instruction, that is, the relative address of each instruction within its kernel function.

3. The high-performance computing device error resilience prediction method based on instruction semantic enhancement according to claim 1, characterized in that, In step 2, a single-event upset fault simulation is performed based on NVBitFI to obtain the fault points of the high-performance computing device. Each fault point... Represented as: ； In the formula, and These represent the target fault occurring in register q of instruction j. The o-th bit in.

4. The error resilience prediction method for high-performance computing devices based on instruction semantic enhancement according to claim 3, characterized in that, Each fault point in step 2 Each element in the triplet is encoded and concatenated using a single-heat method, precoding each fault point as follows: , where m is the total encoding dimension; The fault injection tool NVBitFI is used to inject faults into the fault points, obtain the error resilience labels of the fault points, and construct fault samples of high-performance computing devices.

5. The high-performance computing device error resilience prediction method based on instruction semantic enhancement according to claim 4, characterized in that, Step 3 involves obtaining the assembly instruction sequence of the high-performance computing program from Step 1, and then generating instruction semantic embeddings by adjusting a large language model with a large instruction sequence corpus to encapsulate the semantics of the target high-performance computing program instructions. Specifically, this includes: Step 3-1: Obtain the instruction sequence for each program. for: ； In the formula, n is the number of instructions the program needs to implement a certain function. It is the nth instruction; Step 3-2: Based on the instruction sequence of the program obtained in Step 3-1, use it for pre-training of the large language model to learn bidirectional contextual representation and obtain a large language model with enhanced instruction semantics. Step 3-3: Based on the instruction semantic enhancement large language model obtained in Step 3-2, encapsulate the instruction semantics of the program. The instruction semantic embedding representation is S: ； In the formula, d is the dimension of the embedding vector. This indicates that these semantic vectors are embeddings corresponding to the [CLS] tags extracted from the large model. Indicates the total number of instructions. Indicates the first Semantic embedding representation of instructions.

6. The high-performance computing device error resilience prediction method based on instruction semantic enhancement according to claim 5, characterized in that, Step 5 achieves error resilience prediction for the target fault point, which requires training the model. Specifically, this includes training the model formed in steps 3 to 5 using the high-performance computing equipment fault samples obtained in step 2, and then applying the cross-entropy loss function. Guiding the optimization of model parameters: ； In the formula, Indicates the size of the training sample data. and Training sample set The true label and predicted label of the i-th sample.

7. A high-performance computing device error resilience prediction system based on instruction semantic enhancement, using the method described in any one of claims 1 to 6, characterized in that, The system includes sequential execution of: The first module is used for secondary development of the open-source high-performance computing device binary instrumentation tool NVBit to obtain the assembly instruction sequence and offset address information of the high-performance computing program. The second module is used to build a fault simulation platform, which uses the fault injection tool NVBitFI to perform single-event flip fault simulation at the bit level and obtain fault samples of high-performance computing devices. The third module is used to encapsulate the semantics of the target high-performance computing program instructions by adjusting the large language model with a large corpus of instruction sequences obtained from the first module to generate instruction semantic embeddings. The fourth module is used to analyze the control execution and data dependencies between instructions, construct two instruction topology graphs to represent possible error propagation paths for bit-level faults, and build a graph neural network to extract error propagation patterns. The fifth module is used to fuse fault embedding features and design a multi-head attention mechanism to combine fault embeddings extracted from different graphs to achieve erroneous elastic prediction of target fault points.

Images