Multi-architecture industrial control software reverse analysis method based on machine language large model

Abstract

The application relates to a multi-architecture industrial control software reverse analysis method based on a machine language large model. The method comprises the following steps: acquiring machine language codes of industrial control software of ARM architecture and x86 architecture, and converting the machine language codes into unified intermediate representations; encoding instruction sequences of the unified intermediate representations by using a Transformer-based encoder to obtain sequence representations of functions; training a model through a contrast learning framework to enhance semantic representation capabilities of the sequence representations, and realizing function function identification of the machine language codes of the industrial control software based on the enhanced semantic representations; constructing a control flow graph of the functions, processing the control flow graph by using a graph neural network to obtain graph representations of the functions; fusing the sequence representations and the graph representations to form fusion representations of the functions, and performing code similarity matching based on the fusion representations. The method improves semantic understanding accuracy in industrial control software vulnerability mining, cross-platform migration and other scenes.

Description

Technical Field

[0001] This application relates to the field of industrial control system software security analysis technology, and in particular to a reverse analysis method for multi-architecture industrial control software based on a large machine language model. Background Technology

[0002] Industrial control systems play a crucial role in key sectors such as energy, power, and manufacturing, making their security paramount. With the development of information technology, industrial control software faces increasingly severe security threats, such as malware intrusion and vulnerability exploitation. Reverse engineering, as an important means of ensuring the security of industrial control software, helps security personnel gain a deeper understanding of software functionality and discover potential security vulnerabilities. However, because industrial control software often runs on various hardware architectures (such as ARM and x86 architectures), and the instruction sets of these different architectures vary significantly, the reverse engineering process is complex and inefficient.

[0003] Traditional reverse engineering methods rely on manual interpretation of machine language code across different architectures. This is not only time-consuming and labor-intensive but also demands a high level of expertise and experience from analysts, making it difficult to handle large-scale and complex industrial control software analysis tasks. Furthermore, existing methods lack accuracy in key areas such as function identification and code similarity matching, failing to meet the practical needs of industrial control software security analysis. Therefore, there is an urgent need for an efficient and accurate reverse engineering method for multi-architecture industrial control software to improve the security and stability of industrial control systems.

[0004] Therefore, there is an urgent need in related technologies for a way to improve the security and stability of industrial control systems and to perform efficient and accurate reverse engineering of multi-architecture industrial control software. Summary of the Invention

[0005] Therefore, it is necessary to provide a reverse engineering method for multi-architecture industrial control software based on a large machine language model to address the aforementioned technical problems.

[0006] Firstly, this application provides a reverse engineering method for multi-architecture industrial control software based on a large machine language model. The method includes: Obtain the machine language code of industrial control software based on ARM and x86 architectures, and convert the machine language code into a unified intermediate representation; The instruction sequence of the unified intermediate representation is encoded using a Transformer-based encoder to obtain a sequence representation of the function; The model is trained by a contrastive learning framework to enhance the semantic representation capability of the sequence representation, and the function recognition of machine language code in industrial control software is realized based on the enhanced semantic representation. Construct the control flow graph of the function, and process the control flow graph using a graph neural network to obtain a graph representation of the function; The sequence representation and the graph representation are fused to form a fused representation of the function, and code similarity matching is performed based on the fused representation.

[0007] Optionally, in one embodiment of this application, the Transformer-based encoder includes L stacked Transformer Blocks, each Transformer Block containing a multi-head self-attention mechanism and a feedforward neural network, and the input and output of each Transformer Block are processed by LayerNorm; the feedforward neural network includes two fully connected layers, and the two fully connected layers use ReLU or GELU activation functions. The Transformer-based encoder is pre-trained using a masked language model task. 15% of the instruction tokens in the input instruction sequence are randomly masked, and the training model predicts the masked instructions based on the context.

[0008] Optionally, in one embodiment of this application, training the model using a contrastive learning framework to enhance the semantic representation capability of the sequence representation includes: Construct positive sample pairs and negative sample pairs. The positive sample pairs are constructed through compiler variant generation, semantic equivalence transformation or multiple implementations with the same function. The negative sample pairs are constructed using an in-batch negative sample strategy. In a training batch, all samples except the positive samples of the anchor samples are used as negative samples. The InfoNCE loss is used as the loss function for contrastive learning, and a hard negative sample mining mechanism is introduced to select negative samples with similarity within a preset range to participate in the loss calculation.

[0009] Optionally, in one embodiment of this application, training the model using a contrastive learning framework to enhance the semantic representation capability of the sequence representation further includes: Multi-task learning is performed using a joint function function classification task. The function function classification task uses cross-entropy loss as the loss function. The total loss of multi-task learning is the weighted sum of InfoNCE loss and cross-entropy loss. The relative importance of the two losses is controlled by balancing hyperparameters.

[0010] Optionally, in one embodiment of this application, the function recognition of industrial control software machine language code based on enhanced semantic representation includes: A dual-tower retrieval architecture is adopted, which includes a code encoder and a query encoder. The code encoder is the Transformer-based encoder, and the query encoder processes natural language queries to obtain query vectors. The dual-tower retrieval architecture is trained for cross-modal alignment by constructing paired data. Zero-shot classification is used to expand functional categories. A text description is written for each functional category. The text description is converted into a category prototype vector by a query encoder. The similarity between the semantic representation of the code to be classified and the prototype vector of each category is calculated. The category with the highest similarity is taken as the functional category of the code to be classified.

[0011] Optionally, in one embodiment of this application, the control flow graph for constructing the function includes: Divide the function into basic blocks. The first instruction of the basic block is the program entry point, the jump target, or the instruction immediately following the jump instruction. The last instruction of the basic block is the jump instruction, the function return instruction, or the instruction immediately preceding the jump target. The control flow graph is formed by connecting basic blocks according to the type of jump instruction. Unconditional jump instructions are connected to the target basic block, conditional jump instructions are connected to the target basic block and the basic block corresponding to the next instruction, function call instructions are connected to the entry and return address of the called function, and function return instructions are connected to the call point. The instruction sequence features of each basic block are extracted using the Transformer-based encoder, and the instruction sequence features are aggregated by average pooling or attention pooling to obtain the basic block features.

[0012] Optionally, in one embodiment of this application, the graph neural network adopts an architecture combining graph convolutional networks and graph attention networks, wherein: The graph convolutional network updates node features through neighborhood aggregation, calculates the weighted sum of the features of a node and its neighboring nodes, and processes it through an activation function. The graph attention network calculates the attention weights of a node to its neighboring nodes using an attention function, and updates the node features based on these attention weights. A graph-level readout operation is performed using an attention-based readout method. The attention weight of each node is calculated, and the features of all nodes are aggregated based on the attention weights to obtain a graph representation of the function.

[0013] Optionally, in one embodiment of this application, the fusion of the sequence representation and the graph representation employs a gating fusion mechanism, including: The gating coefficients are calculated by performing a linear transformation on the concatenation result of the sequence representation and the graph representation, followed by a sigmoid activation function. The gating coefficient is multiplied by the sequence representation using element-wise multiplication. The result of subtracting the gating coefficient from 1 is multiplied by the graph representation. The two multiplication results are then added together to obtain the fused representation of the function.

[0014] Secondly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the steps of the methods described in the various embodiments above.

[0015] Thirdly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of the methods described in the various embodiments above.

[0016] Compared with the prior art, the present invention has the following beneficial effects: First, by converting the machine language code of industrial control software with different architectures into a unified intermediate representation, the architectural differences are masked. Combined with the powerful Transformer encoder, this greatly improves the processing efficiency of multi-architecture code and reduces the workload and difficulty of manual analysis. For example, when processing large-scale industrial control software code libraries, compared with traditional methods, preliminary analysis can be completed in a shorter time, and key code areas can be quickly located.

[0017] Second, by utilizing a contrastive learning framework and a multi-task learning strategy, the model's ability to understand the semantic representation of functions is effectively enhanced, significantly improving the accuracy of function function identification. The function function identification method of this invention can more accurately determine the actual function of a function in industrial control software, providing a reliable basis for in-depth analysis of software behavior.

[0018] Third, by fusing the sequential and graphical representations of the fusion function and comprehensively considering the semantic and structural information of the code, code similarity matching becomes more accurate. This helps to discover potential security vulnerabilities in the code, such as by identifying segments similar to known vulnerable code, thus preventing security risks in advance. In practical applications, it can effectively detect issues such as code plagiarism and vulnerability reuse, providing strong support for the security protection of industrial control software. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating a reverse engineering method for multi-architecture industrial control software based on a large machine language model in one embodiment. Figure 2 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0021] In one embodiment, such as Figure 1As shown, a reverse engineering method for multi-architecture industrial control software based on a large machine language model is provided, including the following steps: S101: Obtain the machine language code of industrial control software based on ARM and x86 architectures, and convert the machine language code into a unified intermediate representation.

[0022] In this application, traditional binary analysis tools develop analysis modules independently for each architecture, resulting in a lack of knowledge sharing and transfer. This application designs an architecture-independent Unified Intermediate Representation (UIR), which sits between raw machine code and high-level semantic understanding, serving as a "common language" for different architectures. The UIR design follows these principles: architectural abstraction (eliminating hardware details specific to a particular architecture, such as register naming and instruction encoding formats); semantic integrity (preserving the essential semantics of program execution, including key information such as data operations, control flow transfers, and memory access); and scalability (supporting rapid integration with new architectures without requiring a complete system refactoring).

[0023] Specifically, UIR uses an operation-operand quadruple representation. Each IR instruction can be represented as... ,in, Opcodes abstract semantically identical operations across different architectures, such as... , , , wait; For the destination operand; and The source operand. Operands are uniformly represented using virtual registers, through... The formal numbering eliminates the differences between ARM's R0-R15 registers and x86's EAX, EBX, and other architecture-related registers. For memory access operations, abstract address expressions are used, such as... or This unifies the addressing methods for different architectures.

[0024] The lifting process is crucial for converting raw machine code into UIR (Unified Instruction Markup). This process is based on disassembly and instruction semantic mapping. For the ARM architecture, the lifting module first identifies the instruction format (ARM mode or Thumb mode), parses the opcode and operands, and then converts the ARM instruction into the corresponding UIR instruction sequence according to a predefined mapping table. For example, the ARM instruction ADD R0, R1, R2 is lifted to UIR. , including virtual registers Corresponding to physical registers For the x86 architecture, due to its CISC characteristics, a single instruction may correspond to multiple UIR instructions. For example, the x86 instruction ADD [EAX], EBX needs to be promoted to a LOAD operation to load data into memory, then the ADD operation is performed, and finally the data is stored back into memory, i.e., a sequence. , , .

[0025] S102: The instruction sequence of the unified intermediate representation is encoded using a Transformer-based encoder to obtain the sequence representation of the function.

[0026] In this embodiment, a neural network encoder based on the Transformer architecture is used to perform deep semantic modeling of the instruction sequence on top of the UIR. The Transformer model, due to its powerful sequence modeling capabilities and parallel computing advantages, has achieved significant success in the fields of natural language processing and code understanding; this application applies it to machine language understanding. The UIR instruction sequence is first processed by token embedding and positional encoding. Each UIR instruction is treated as a token, and through a learnable embedding matrix... Mapped to A dimensional vector space, where Size of the UIR instruction vocabulary.

[0027] For instruction sequences Its initial embedding is represented as

[0028] in The position encoding matrix uses sine-cosine position encoding or learnable position encoding to preserve the order information of instructions.

[0029] In one embodiment of this application, the Transformer-based encoder includes L stacked Transformer Blocks, each Transformer Block containing a multi-head self-attention mechanism and a feedforward neural network, and the input and output of each Transformer Block are processed by LayerNorm; the feedforward neural network includes two fully connected layers, and the two fully connected layers use ReLU or GELU activation functions; The Transformer-based encoder is pre-trained using a masked language model task. 15% of the instruction tokens in the input instruction sequence are randomly masked, and the training model predicts the masked instructions based on the context.

[0030] In one embodiment of this application, the Transformer encoder is... It consists of stacked Transformer Blocks, each layer containing a multi-head self-attention mechanism and a feed-forward network. The calculation process of the layer is as follows

[0031]

[0032] The core computation of the multi-head self-attention mechanism is...

[0033] The Query, Key, and Value matrices are obtained from the input through a linear transformation, i.e. , , ,in The parameter matrix is ​​learnable. The multi-head mechanism is achieved through... The attention heads are computed in parallel, the first... The output of the head is The final output is

[0034] in The output projection matrix is ​​used. The self-attention mechanism enables the representation of each instruction to incorporate information from all other instructions in the sequence, thereby capturing long-range dependencies, which is crucial for understanding data flow and control flow in machine language.

[0035] A feedforward neural network consists of two fully connected layers, with ReLU or GELU activation functions used in between. The calculation formula is as follows:

[0036] in , This is the weight matrix. Typically set to The LayerNorm layer normalizes the features at each location, accelerating training convergence and improving model stability.

[0037] go through After layer Transformer encoding, the final representation of the instruction sequence is as follows: ,in For the first The contextualized representation of instructions incorporates global information about the entire function. For function-level representations, pooling operations are used to aggregate the representations of all instructions; common methods include average pooling. Or use a special CLS token to represent .

[0038] The model is pre-trained using a Masked Language Model (MLM) task. 15% of the instruction tokens in the input sequence are randomly masked. The model is trained to predict the masked instructions based on the context, with a loss function of [missing information].

[0039] in The set of masked positions. Indicates except position Other instructions, These are the model parameters. The predicted probability is calculated by applying a softmax classifier to the final hidden layer output, i.e. ,in These are used for classification weights. Through pre-training with large-scale UIR instruction sequences, the model learns the syntactic rules and semantic patterns of machine language, providing powerful feature extraction capabilities for subsequent tasks.

[0040] S103: The model is trained by a contrastive learning framework to enhance the semantic representation capability of the sequence representation, and the function recognition of the machine language code of the industrial control software is realized based on the enhanced semantic representation.

[0041] In this embodiment, a contrastive learning framework is used to train a model to learn semantically invariant code representations, so that codes with the same function but different syntax are close in distance in the vector space, thereby achieving high-precision function recognition and semantic search.

[0042] In one embodiment of this application, training the model using a contrastive learning framework to enhance the semantic representation capability of the sequence representation includes: S201: Construct positive sample pairs and negative sample pairs. The positive sample pairs are constructed through compiler variant generation, semantic equivalence transformation, or multiple implementations with the same functionality. The negative sample pairs are constructed using an in-batch negative sample strategy. In a training batch, all samples except the positive samples of the anchor samples are used as negative samples.

[0043] S202: InfoNCE loss is used as the loss function for contrastive learning, and a hard negative sample mining mechanism is introduced to select negative samples with similarity within a preset range to participate in the loss calculation.

[0044] In one embodiment of this application, contrastive learning is a self-supervised learning paradigm. Its core idea is to train the model to learn discriminative representations by constructing positive and negative sample pairs. In machine language understanding tasks, positive sample pairs consist of semantically identical but differently implemented code functions, while negative sample pairs consist of semantically unrelated code functions. By maximizing the similarity of positive sample pairs and minimizing the similarity of negative sample pairs, the model is forced to learn to ignore surface syntactic differences and capture deep semantic invariance.

[0045] Constructing positive sample pairs is crucial for successful contrastive learning. This application employs multiple strategies to generate high-quality positive sample pairs. The first strategy is compiler variant generation, which involves compiling the same C / C++ source code using different compilers (GCC, Clang, MSVC, etc.), different optimization levels (O0, O1, O2, O3), and different target architectures (ARM, x86), resulting in multiple binary function variants. These variants differ significantly at the machine language level but implement the exact same functionality. For example, with O3 optimization enabled, the compiler applies aggressive optimizations such as loop unrolling, function inlining, and instruction reordering, generating code with a completely different structure than the O0 version, but semantically equivalent. The second strategy is semantically equivalent transformation, which involves applying semantically preserving transformations to the binary code, such as register renaming, instruction substitution (e.g., replacing ADD with a negative SUB), insertion of useless instructions, and code block reordering. The third strategy is to have multiple implementations with the same functionality. This involves collecting different open-source implementations that perform the same functions (such as AES encryption, SHA256 hashing, and Modbus protocol parsing). These implementations are written by different developers and have different coding styles and algorithm details, but the core functions are the same.

[0046] Constructing negative sample pairs is relatively simple; it involves randomly sampling functionally unrelated function pairs from the dataset. To improve training efficiency, an in-batch negatives strategy is adopted. This means that within a training batch, all samples except the positive anchor samples are treated as negative samples, effectively increasing the number of negative samples. For anchor samples... Its positive samples are Others within the batch Sample As a negative sample.

[0047] The objective function for contrastive learning uses the InfoNCE loss (also known as the NT-Xent loss), which takes the form of:

[0048] in and These are the feature vectors extracted by the encoder from the anchor sample and its positive sample, respectively. For similarity measurement functions, cosine similarity is usually used. , The temperature hyperparameter controls the smoothness of the distribution. InfoNCE loss is essentially a multi-class classification problem, with the goal of... 1 candidate sample (1 positive sample and 1 positive sample) The model correctly identifies positive samples from a pool of negative samples. By minimizing this loss, the similarity of positive sample pairs is increased, while the similarity of negative sample pairs is decreased, and the model learns a semantic discriminative representation.

[0049] To further improve representation quality, a hard negative sample mining mechanism is introduced. Not all negative samples contribute equally to training; "hard negative samples" that are semantically similar to anchor samples but functionally different are more valuable for model learning. A semi-hard negative sample mining strategy is adopted, that is, in each training batch, samples with similarity between the anchor and anchor samples are selected. Negative samples within the interval are used in the loss calculation, where These are the boundary hyperparameters. These samples are neither too simple (negative samples with very low similarity are easily distinguishable by the model) nor too difficult ("false negative samples" with higher similarity than positive samples), effectively improving training efficiency and model performance.

[0050] The loss function of contrastive learning can be extended to the form of triplet loss, explicitly modeling the relationship between anchor points, positive samples, and negative samples.

[0051] in For negative samples, The margin hyperparameter requires that the similarity of positive samples be at least higher than that of negative samples. This loss function directly optimizes the relative distance relationships between samples, making the training process more stable.

[0052] In one embodiment of this application, the step of training the model using a contrastive learning framework to enhance the semantic representation capability of the sequence representation further includes: Multi-task learning is performed using a joint function function classification task. The function function classification task uses cross-entropy loss as the loss function. The total loss of multi-task learning is the weighted sum of InfoNCE loss and cross-entropy loss. The relative importance of the two losses is controlled by balancing hyperparameters.

[0053] In one embodiment of this application, a multi-task learning mechanism is further introduced based on contrastive learning, jointly training the function classification task and the contrastive learning task. The function classification task divides the function into predefined function categories, such as authentication function, encryption / decryption function, protocol parsing function, etc. The loss function of the classification task is cross-entropy loss:

[0054] in For the number of functional categories, One-hot encoding of the real label. The class probabilities predicted by the model are obtained by applying the classification head to the functional representation, i.e. The total loss for multi-task learning is

[0055] in To balance hyperparameters and control the relative importance of the two tasks, the contrastive learning task drives the model to learn semantically invariant general representations, while the classification task provides supervision signals to guide the model to focus on discriminative features of specific functional categories. The two tasks mutually promote each other and jointly improve model performance.

[0056] In one embodiment of this application, the function recognition of industrial control software machine language code based on enhanced semantic representation includes: S301: A dual-tower retrieval architecture is adopted, which includes a code encoder and a query encoder. The code encoder is the Transformer-based encoder, and the query encoder processes natural language queries to obtain query vectors. The dual-tower retrieval architecture is trained for cross-modal alignment by constructing paired data.

[0057] S302: Zero-shot classification is used to expand the functional categories. A text description is written for each functional category. The text description is converted into a category prototype vector by a query encoder. The similarity between the semantic representation of the code to be classified and the prototype vector of each category is calculated. The category with the highest similarity is taken as the functional category of the code to be classified.

[0058] In one embodiment of this application, a two-tower retrieval architecture is employed. This architecture comprises two independent modules: a code encoder and a query encoder. The code encoder, namely the aforementioned Transformer-based encoder, encodes binary functions into vectors. The query encoder processes natural language queries, encoding user-input functional descriptions (such as "functions that implement AES encryption" or "dangerous functions that receive user input") into query vectors. The query encoder uses a pre-trained text encoding model, such as BERT or RoBERTa, and trains through cross-modal alignment to make semantically related code and queries close in the vector space.

[0059] Cross-modal alignment is achieved by constructing <text description, code> pairing data. Data sources include: pairing source code comments with corresponding functions, pairing function names with function bodies, and pairing manually annotated function descriptions with code. The training objective is to maximize the similarity of positive pairings.

[0060] in This is a positive pairing. Through this training, the model learns to map natural language queries to the code semantic space, enabling cross-modal retrieval. During the inference phase, the user inputs a natural language query, the query encoder generates a query vector, and by calculating the similarity between the query vector and all function vectors in the codebase, the most relevant function is retrieved, thus achieving intelligent search by "inputting a function description and finding the code that implements that function".

[0061] For seven (or more) predefined functional categories, a zero-shot classification paradigm can be used for flexible expansion. Standard text descriptions are written for each functional category, such as "This function implements user authentication, verifying the correctness of the username and password," or "This function implements data encryption, using symmetric or asymmetric algorithms to protect data confidentiality." These descriptions are then converted into category prototype vectors by a query encoder. The category of the function to be classified is determined by calculating the similarity between its vector and all category prototypes.

[0062] This method does not require collecting a large number of labeled samples for each category; classification can be achieved simply by defining category descriptions. It has strong scalability and flexibility and can quickly adapt to new functional category requirements.

[0063] S104: Construct the control flow graph of the function, and process the control flow graph using a graph neural network to obtain a graph representation of the function.

[0064] In one embodiment of this application, the control flow graph for constructing the function includes: S401: Divide the function into basic blocks. The first instruction of the basic block is the program entry point, jump target, or instruction immediately following the jump instruction. The last instruction of the basic block is the jump instruction, function return instruction, or instruction immediately preceding the jump target.

[0065] S402: Connect basic blocks according to the jump instruction type to form edges of the control flow graph. Unconditional jump instructions are connected to the target basic block, conditional jump instructions are connected to the target basic block and the basic block corresponding to the next instruction, function call instructions are connected to the entry and return address of the called function, and function return instructions are connected to the calling point.

[0066] S403: The instruction sequence features of each basic block are extracted using the Transformer-based encoder, and the instruction sequence features are aggregated by average pooling or attention pooling to obtain basic block features.

[0067] In one embodiment of this application, the inherent characteristics of code determine that it simultaneously possesses both sequence and graph structure attributes. From a sequence perspective, code is an ordered sequence of instructions, and the sequential relationships between instructions are crucial to program semantics. From a graph structure perspective, code contains rich structural relationships, including control flow relationships, data flow relationships, and call graph relationships. While relying solely on sequence models (such as Transformer) can capture local dependencies, its modeling of global structural features is limited; relying solely on graph models may ignore the importance of instruction order. Therefore, by jointly modeling sequence and graphs, we can fully utilize the multi-dimensional information of code to achieve a more comprehensive and in-depth understanding of the code.

[0068] A Control Flow Graph (CFG) is the core representation of code structure. An CFG divides a program into basic blocks, each a sequential sequence of instructions without branching or jumps. Basic blocks are connected by directed edges, representing the transitions in control flow. Formally, an CFG can be represented as a directed graph. , where the node set For the basic block set and edge set To control the flow edge. If there exists a flow edge from the basic block. arrive If the control flow transitions (such as conditional jumps, unconditional jumps, function call returns, etc.), then there exists a directed edge. .

[0069] The process of constructing a CFG involves two steps: basic block partitioning and edge connection. Basic block partitioning follows these rules: the first instruction of a basic block is the program entry point, jump target, or the instruction immediately following a jump instruction; the last instruction of a basic block is a jump instruction, function return instruction, or the instruction immediately preceding a jump target. Basic block partitioning is completed by traversing the disassembled code and identifying all jump instructions and jump targets. Edge connection is determined based on the type of jump instruction: unconditional jumps (such as...) , ) connects to the target block of the jump; conditional jump (such as , The jump target and the next instruction are connected to the two basic blocks corresponding to the jump target and the next instruction; the function call instruction is connected to the entry and return address of the called function; the function return instruction is connected to the call point.

[0070] Building upon CFG, each node (basic block) needs to be endowed with rich semantic features. A basic block is essentially a sequence of instructions, whose sequence representation can be extracted using the aforementioned Transformer encoder. Let the basic block... Includes instruction sequence ,in This represents the number of instructions. The sequence is processed by a Transformer encoder to obtain a contextualized representation of each instruction. The overall representation of the basic blocks is obtained through aggregation operations, commonly including average pooling, max pooling, or attention pooling. Average pooling is calculated as follows:

[0071] Attention pooling assigns different weights to different positions.

[0072] in This represents a learnable attention weight vector. Through feature empowerment, each node in the CFG possesses rich semantic information, laying the foundation for subsequent graph neural network processing.

[0073] In one embodiment of this application, the graph neural network adopts an architecture combining graph convolutional networks and graph attention networks, wherein: S501: The graph convolutional network updates node features through neighborhood aggregation, calculates the weighted sum of the features of a node and its neighboring nodes, and processes it through an activation function.

[0074] S502: The graph attention network calculates the attention weights of a node to its neighboring nodes through an attention function, and updates the node features based on the attention weights.

[0075] S503: Graph-level readout is performed using an attention-based readout method. The attention weight of each node is calculated, and the features of all nodes are aggregated based on the attention weights to obtain the graph representation of the function.

[0076] In one embodiment of this application, a Graph Neural Network (GNN) learns high-level representations of nodes and the graph by propagating and aggregating information on the graph structure. It employs an architecture combining a Graph Convolutional Network (GCN) and a Graph Attention Network (GAT). The core idea of ​​GCN is to perform neighborhood aggregation on node features. Layer nodes Feature update to

[0077] in For nodes The set of neighboring nodes, and They are nodes and The degree, For the first The learnable weight matrix of the layer, The activation function is ReLU. By stacking multiple layers of GCN, the representation of each node gradually incorporates information from its multi-hop neighborhood, thereby capturing global structural features.

[0078] GAT introduces an attention mechanism, assigning different importance weights to different neighboring nodes. For neighboring nodes The attention weights are calculated as follows:

[0079] in For the attention function, a single-layer feedforward neural network is often used, such as... , This indicates a splicing operation. This is a learnable parameter vector. Node features are updated as follows:

[0080] GAT's attention mechanism enables the model to adaptively focus on important control flow paths, assigning differentiated weights to different neighbor nodes, thereby improving the model's expressive power and its ability to model complex graph structures.

[0081] Building upon GNNs, a graph-level readout operation is employed to generate a graph representation of the entire function. Commonly used readout functions include global average pooling, global max pooling, and set aggregation. Global average pooling is calculated as follows:

[0082] in The number of layers in the GNN. For nodes The representation in the last layer. Preferred readout methods, such as Differentiable Pooling and Attention Readout, can assign different weights to different nodes, allowing for more flexible information aggregation. Attention Readout takes the form of...

[0083] in This is a learnable readout weight vector.

[0084] S105: The sequence representation and the graph representation are fused to form a fused representation of the function, and code similarity matching is performed based on the fused representation.

[0085] In one embodiment of this application, the fusion of the sequence representation and the graph representation employs a gating fusion mechanism, including: S601: Calculate the gating coefficients, which are obtained by performing a linear transformation on the concatenation result of the sequence representation and the graph representation and then applying the sigmoid activation function.

[0086] S602: The gating coefficient is multiplied by the sequence representation using element-wise multiplication, the result of subtracting the gating coefficient from 1 is multiplied by the graph representation, and the two multiplication results are added together to obtain the fused representation of the function.

[0087] In one embodiment of this application, the final function representation is obtained by fusing the sequence representation and the graph representation. Let the function sequence representation generated by the Transformer encoder be... The function graph generated by GNN is represented as follows: Fusion can employ splicing, weighted summation, or gating mechanisms. The splicing method is as follows: The weighted sum method is as follows , where weight and This can be learned. The gating fusion mechanism is more flexible, and the gating coefficient is calculated.

[0088] Fusion is represented as

[0089] in This represents element-wise multiplication. This is the sigmoid function. Gating mechanisms allow the model to dynamically adjust the contributions of sequence and graph representations based on specific circumstances, enhancing the model's adaptability.

[0090] Code similarity detection is performed based on fused representations. For the query function... and candidate function libraries First, extract the fused representation for all functions. Then, the similarity between the query function and each candidate function is calculated:

[0091] Sort by similarity in descending order, the top-K candidate functions are the search results. To improve search efficiency, an Approximate Nearest Neighbor (ANN) algorithm, such as HNSW (Hierarchical Navigable Small World) or the Faiss library, is used, which can achieve millisecond-level retrieval in millions or even tens of millions of codebases.

[0092] In one embodiment of this application, training employs a metric learning paradigm. The training objective is to learn a representation space such that similar codes are close to each other (less than a threshold) within that space. Dissimilar codes are far apart (greater than a threshold) The training loss function employs the aforementioned contrastive loss or triplet loss, which, through learning from large-scale training data, accurately distinguishes between similar and dissimilar codes. Evaluation metrics include accuracy, recall, F1 score, and Mean Average Precision (MAP). Accuracy is defined as the proportion of similar functions among the Top-K search results, and recall is defined as the proportion of successfully retrieved true similar functions. By fine-tuning the model parameters and similarity threshold on the validation set, a detection accuracy of ≥70% is ensured with a candidate function scale of 10,000.

[0093] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0094] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 2As shown, the computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When the computer program is executed by the processor, it implements a multi-architecture industrial control software reverse engineering method based on a large machine language model. The display screen can be an LCD screen or an e-ink display screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0095] Those skilled in the art will understand that Figure 2 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0096] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0097] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0098] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0099] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0100] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0101] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0102] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A reverse engineering method for multi-architecture industrial control software based on a large machine language model, characterized in that, The method includes: Obtain the machine language code of industrial control software based on ARM and x86 architectures, and convert the machine language code into a unified intermediate representation; The instruction sequence of the unified intermediate representation is encoded using a Transformer-based encoder to obtain a sequence representation of the function; The model is trained by a contrastive learning framework to enhance the semantic representation capability of the sequence representation, and the function recognition of machine language code in industrial control software is realized based on the enhanced semantic representation. Construct the control flow graph of the function, and process the control flow graph using a graph neural network to obtain a graph representation of the function; The sequence representation and the graph representation are fused to form a fused representation of the function, and code similarity matching is performed based on the fused representation.

2. The reverse engineering method for multi-architecture industrial control software based on a large machine language model according to claim 1, characterized in that, The Transformer-based encoder comprises L stacked Transformer Blocks, each containing a multi-head self-attention mechanism and a feedforward neural network, with both the input and output of each Transformer Block processed by LayerNorm; the feedforward neural network comprises two fully connected layers, with ReLU or GELU activation functions used between the two fully connected layers. The Transformer-based encoder is pre-trained using a masked language model task. 15% of the instruction tokens in the input instruction sequence are randomly masked, and the training model predicts the masked instructions based on the context.

3. The reverse engineering method for multi-architecture industrial control software based on a large machine language model according to claim 1, characterized in that, The step of training the model using a contrastive learning framework to enhance the semantic representation capability of the sequence representation includes: Construct positive sample pairs and negative sample pairs. The positive sample pairs are constructed through compiler variant generation, semantic equivalence transformation or multiple implementations with the same function. The negative sample pairs are constructed using an in-batch negative sample strategy. In a training batch, all samples except the positive samples of the anchor samples are used as negative samples. The InfoNCE loss is used as the loss function for contrastive learning, and a hard negative sample mining mechanism is introduced to select negative samples with similarity within a preset range to participate in the loss calculation.

4. The reverse engineering method for multi-architecture industrial control software based on a large machine language model according to claim 3, characterized in that, The step of training the model using a contrastive learning framework to enhance the semantic representation capability of the sequence representation also includes: Multi-task learning is performed using a joint function function classification task. The function function classification task uses cross-entropy loss as the loss function. The total loss of multi-task learning is the weighted sum of InfoNCE loss and cross-entropy loss. The relative importance of the two losses is controlled by balancing hyperparameters.

5. The reverse engineering method for multi-architecture industrial control software based on a large machine language model according to claim 1, characterized in that, The function recognition of industrial control software machine language code based on enhanced semantic representation includes: A dual-tower retrieval architecture is adopted, which includes a code encoder and a query encoder. The code encoder is the Transformer-based encoder, and the query encoder processes natural language queries to obtain query vectors. The dual-tower retrieval architecture is trained for cross-modal alignment by constructing paired data. Zero-shot classification is used to expand functional categories. A text description is written for each functional category. The text description is converted into a category prototype vector by a query encoder. The similarity between the semantic representation of the code to be classified and the prototype vector of each category is calculated. The category with the highest similarity is taken as the functional category of the code to be classified.

6. The reverse engineering method for multi-architecture industrial control software based on a large machine language model according to claim 1, characterized in that, The control flow graph for constructing the function includes: Divide the function into basic blocks. The first instruction of the basic block is the program entry point, the jump target, or the instruction immediately following the jump instruction. The last instruction of the basic block is the jump instruction, the function return instruction, or the instruction immediately preceding the jump target. The control flow graph is formed by connecting basic blocks according to the type of jump instruction. Unconditional jump instructions are connected to the target basic block, conditional jump instructions are connected to the target basic block and the basic block corresponding to the next instruction, function call instructions are connected to the entry and return address of the called function, and function return instructions are connected to the call point. The instruction sequence features of each basic block are extracted using the Transformer-based encoder, and the instruction sequence features are aggregated by average pooling or attention pooling to obtain the basic block features.

7. The reverse engineering method for multi-architecture industrial control software based on a large machine language model according to claim 1, characterized in that, The graph neural network adopts an architecture that combines graph convolutional networks and graph attention networks, wherein: The graph convolutional network updates node features through neighborhood aggregation, calculates the weighted sum of the features of a node and its neighboring nodes, and processes it through an activation function. The graph attention network calculates the attention weights of a node to its neighboring nodes using an attention function, and updates the node features based on these attention weights. A graph-level readout operation is performed using an attention-based readout method. The attention weight of each node is calculated, and the features of all nodes are aggregated based on the attention weights to obtain a graph representation of the function.

8. The reverse engineering method for multi-architecture industrial control software based on a large machine language model according to claim 1, characterized in that, The fusion of the sequence representation and the graph representation employs a gated fusion mechanism, including: The gating coefficients are calculated by performing a linear transformation on the concatenation result of the sequence representation and the graph representation, followed by a sigmoid activation function. The gating coefficient is multiplied by the sequence representation using element-wise multiplication. The result of subtracting the gating coefficient from 1 is multiplied by the graph representation. The two multiplication results are then added together to obtain the fused representation of the function.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 8.

Images