A binary vulnerability detection method and system based on deep learning

Abstract

This application discloses a binary vulnerability detection method and system based on deep learning, belonging to the field of security vulnerability detection technology. The method includes: parsing the binary program to be tested and extracting the pure instruction sequence; segmenting the instruction sequence into overlapping subsequences; mapping them to multi-dimensional vectors of the neural network input matrix; fusing contextual features to generate attention-enhanced features; outputting the vulnerability probability of each subsequence through a multilayer perceptron; and locating its offset address in the original binary program. The system parses the binary program through a preprocessing module, and the data is input to the model inference module; the classification and localization module calculates the vulnerability probability and locates the high-risk code address based on this; and the deployment and integration module integrates it into a domestic operating system. This application can adapt to the differences in binary generated by multiple compilers and hardware platforms, has higher robustness and compatibility, effectively performs detection, and improves the detection capability of binary vulnerabilities.

Description

Technical Field

[0001] This application belongs to the field of security vulnerability detection technology, and in particular relates to a binary vulnerability detection method and system based on deep learning. Background Technology

[0002] Binary vulnerabilities in modern software systems have become one of the major threats to cybersecurity. Traditional binary vulnerability detection methods mainly rely on expert-defined rules or pattern matching. These methods require manual feature construction and are prone to high false positives and false negatives. In addition, the same source code can generate binary code with different compilers, optimization options, or hardware platforms, resulting in various differences such as register reordering, instruction substitution, and control flow changes. Detection methods based on static patterns are difficult to adapt to the changes brought about by the diversity of compilers and platforms.

[0003] Existing binary vulnerability detection methods include rule / pattern matching, fuzzing, symbolic execution / taint analysis, and cross-domain model comparison. However, these methods have limitations in vulnerability detection. For example, rule / pattern matching is sensitive to compiler optimizations (instruction reordering, register allocation) and obfuscation (UPX compression, compression shells), and signatures are easily corrupted, leading to high false negatives / false positives. Fuzzing suffers from insufficient path coverage and high environment setup costs, and it also significantly misses logic and timing vulnerabilities. Symbolic execution / taint analysis faces severe path explosion and struggles to simulate firmware hardware interactions and BIOS interrupts, resulting in incomplete environment models. Cross-domain model comparison methods focus on traffic prediction and path evaluation, and are not suitable for binary vulnerability feature extraction.

[0004] In summary, existing vulnerability detection methods are not effective in detecting binary vulnerabilities and have significant shortcomings in terms of adaptability to compilation diversity, generalization ability to unknown vulnerabilities, and compatibility with domestic systems. Summary of the Invention

[0005] This application aims to solve the technical problem of effectively detecting binary vulnerabilities. To this end, this application provides a binary vulnerability detection method and system based on deep learning, which can adapt to the differences in binary generated by multiple compilers and hardware platforms, has higher robustness and compatibility, effectively detects vulnerabilities, and improves the detection capability of binary vulnerabilities.

[0006] In a first aspect, embodiments of this application provide a deep learning-based binary vulnerability detection method for testing binary programs, comprising: The binary program under test is parsed to remove non-code data and extract instruction sequences. The instruction sequence is segmented using a sliding window to generate overlapping subsequences; Each instruction in each subsequence is mapped to a D-dimensional vector to form the neural network input matrix; The input matrix is ​​fed into a pre-trained neural network model, which sequentially captures the context fusion features of the instruction sequence through a bidirectional long short-term memory network layer, and weights the context fusion features through a multi-head self-attention mechanism to generate attention-enhanced features that highlight key instruction subsequences. The attention-enhanced features are input into a multilayer perceptron to obtain the vulnerability probability of each subsequence. For subsequences whose vulnerability probability exceeds a preset threshold, their offset address in the original binary program is located based on the attention weights generated by the multi-head self-attention mechanism.

[0007] In some implementations, the binary program to be tested is parsed, including: dynamically identifying and parsing the private section layout and signature area offset of the domestic operating system according to a predefined configuration file, so as to strip the signature area and extract the executable section.

[0008] In some implementations, after generating overlapping subsequences, the method further includes: Establish an address backtracking table to record the mapping relationship between the starting address of each subsequence and the offset of the original binary file; when debugging information exists, further map it to the file name and line number of the source code.

[0009] In some implementations, the sliding window, whose size and step size are dynamically adjusted according to the hardware architecture and / or memory region type to which the currently processed instruction sequence belongs.

[0010] In some implementations, dynamic adjustment includes: When the instruction sequence is identified as belonging to the stack memory operation area, the first window size and the first step length are used; When the instruction sequence is identified as belonging to the heap memory operation area, the second window size and the second step size are used; The size of the first window is smaller than the size of the second window, and the step size is smaller than the step size.

[0011] In some implementations, the neural network model is trained using a sample set that includes multiple compilers, multiple optimization levels and multiple hardware architectures; the bidirectional long short-term memory network layer of the neural network model is a three-layer stacked bidirectional structure with 200-300 hidden units in each layer, which is used to capture local, mid-range and global instruction sequence dependencies in a hierarchical manner.

[0012] In some implementations, the neural network model is trained using one or more of the following methods during training: random insertion of no-operation instructions, safe replacement of register names, and code obfuscation; or trained using an Adam optimizer with early stopping and dynamic learning rate decay.

[0013] In some implementations, generating attention-enhanced features includes: The feature sequences output by the bidirectional long short-term memory network layer are mapped to query matrix, key matrix and value matrix respectively; Calculate the attention weight for each attention head and perform weighted aggregation on the value matrix; The outputs of multiple attention heads are concatenated and then linearly transformed to obtain attention-enhanced features.

[0014] In some implementations, the method also includes: exporting the neural network model as a configuration file corresponding to the domestic operating system, dynamically loading the configuration file, and identifying and parsing the binary program pre-stored in the domestic operating system.

[0015] Secondly, embodiments of this application provide a binary vulnerability detection system based on deep learning, comprising: The preprocessing module is used to parse the binary program under test, strip away non-code data, and extract instruction sequences; it uses a sliding window to segment the instruction sequences, generating overlapping subsequences; and it maps each instruction in each subsequence to a D-dimensional vector to form the neural network input matrix. The model inference module loads a pre-trained neural network model to capture the context fusion features of the instruction sequence, and weights the context fusion features through a multi-head self-attention mechanism to generate attention-enhanced features that highlight key instruction subsequences. The classification and localization module is used to input attention-enhanced features into the multilayer perceptron to obtain the vulnerability probability of each subsequence; for subsequences whose vulnerability probability exceeds a preset threshold, the offset address of the subsequence in the original binary program is located according to the attention weight generated by the multi-head self-attention mechanism. Deployment integration module, used to encapsulate the model inference module and classification and localization module as services, and integrate them into the continuous integration / continuous delivery pipeline of the domestic operating system for binary vulnerability scanning.

[0016] As can be seen from the above technical solution, the beneficial effects of this application are as follows: This application's method, through binary program parsing, extracts useful data and improves the accuracy of subsequent analysis, effectively removing non-code interference data such as signature areas and debugging information. By using a sliding window to segment the continuous instruction stream, it transforms it into units that the model can process and retain local semantics, allowing for more refined adaptation to different scenarios and enhancing the model's sensitivity and adaptability in capturing vulnerability patterns. By mapping to D-dimensional vectors to form a neural network input matrix, it transforms symbolic assembly instructions into numerical, semantically rich feature representations that deep learning models can process. Through a bidirectional long short-term memory network layer, it captures contextual fusion features and generates attention-enhanced features through a multi-head self-attention mechanism, capturing deep-level bidirectional contextual dependencies in the instruction sequence and understanding the data flow and control flow logic between instructions. The attention-enhanced features are input into a classifier to obtain vulnerability probabilities, and the offset addresses are located based on attention weights. Finally, the probability of each code segment containing a vulnerability is determined, completing vulnerability identification and localization. Thus, this application can adapt to the differences in binary generated by multiple compilers and hardware platforms, exhibiting higher robustness and compatibility, effectively performing detection, improving the detection capability of binary vulnerabilities, and ensuring terminal protection and IoT security.

[0017] The system described in this application uses a preprocessing module to parse and convert binary programs into parameters for a neural network model to learn; a model inference module to capture deep, bidirectional contextual dependencies between instructions using a Bidirectional Long Short-Term Memory (Bi-LSTM) network; a classification and localization module to make a final probability judgment on the existence of vulnerabilities based on features; and a deployment and integration module to encapsulate, deploy, and integrate the detected model into a real production environment. This makes the system suitable for domestic operating systems, eliminating the need for secondary training, and enabling automated identification and localization of binary vulnerabilities to improve endpoint protection and ensure IoT security. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced one by one below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other embodiments and drawings can be obtained based on these drawings without creative effort. Various schematic diagrams according to the embodiments of this application are shown in the accompanying drawings. These drawings are not necessarily drawn to scale. For the purpose of clarity, some details have been enlarged and some details may have been omitted.

[0019] Figure 1 A schematic flowchart of an embodiment of the binary vulnerability detection method based on deep learning of the present invention is shown; Figure 2A schematic diagram of an embodiment of the Bi-LSTM-Attention neural network model of the present invention is shown; Figure 3 A schematic diagram of an embodiment of the binary vulnerability detection system based on deep learning of the present invention is shown. Detailed Implementation

[0020] The technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. The detailed description of the embodiments of this application provided in the drawings is not intended to limit the scope of the claimed application. The described embodiments are only a part of the embodiments of this application, not all of them. Based on the embodiments in this application, they can be arranged and designed in various different configurations. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0021] This application is described below with reference to the accompanying drawings and specific embodiments: Please refer to Figure 1 The first aspect of this application provides a deep learning-based binary vulnerability detection method for testing binary programs, comprising: S1. Parse the binary program under test, strip away non-code data, and extract the instruction sequence. This step is an adaptive preprocessing and sliding window segmentation step. Parse the PE / COFF and ELF section headers, locate the executable section according to the section table pointer in the file header, extract the .text and .init / .fini sections, and identify and strip the signature area (such as PKCS#7 signature, SM2 signature), debugging information (such as .debug section), and compression shell (such as UPX).

[0022] S2. Segment the instruction sequence using a sliding window to generate overlapping subsequences. This step is a preprocessing step for the instruction sequence, where the window length is N, the step size is S, and the instruction sequence is the one stripped as described above. Apply a sliding window of size N and step size S to the instruction stream to generate overlapping subsequences (as the basic unit of input in step S3) to ensure the integrity of cross-boundary features; for example, set the window length N=60 (for ARM64) or N=40 (for x86) and the step size S=10. Segment the instruction sequence using a sliding window to generate overlapping subsequences. For example, if the instruction sequence is [I1, I2, ..., I100], then the first window is [I1-I60], the second window is [I11-I70], and so on.

[0023] S3. Map each instruction in each subsequence to a D-dimensional vector to form the neural network input matrix. This step is the instruction vectorization step. For example, the assembly instructions or pseudocode tokens of each subsequence are mapped to a D-dimensional vector sequence through the Token2Vec embedding layer. After Token2Vec embedding, the D-dimensional vector (D=256) will serve as key input features and be fully processed by the subsequent Bi-LSTM-Attention model. This step mathematizes semantic information, specifically mapping each instruction or pseudocode statement to a D-dimensional vector (D=128~512) to capture semantic features. Each instruction string is converted into a D-dimensional vector through a pre-trained Word2Vec model (dimension D=256). For example, the instruction "mov r0, #0" is mapped to a 256-dimensional vector [v1, v2, ..., v256]. The input of this step is an instruction sequence of length N, and the output is an N×D matrix. All instruction vectors constitute the sequence data input to the Bi-LSTM.

[0024] S4. Input the input matrix into a pre-trained neural network model. This model sequentially captures the contextual fusion features of the instruction sequence through bidirectional long short-term memory (LSTM) network layers, and weights these features using a multi-head self-attention mechanism to generate attention-enhanced features that highlight key instruction subsequences. This step involves bidirectional context modeling and attention enhancement to understand the contextual dependencies between instructions. For example, inputting a vector sequence into a neural network model with at least three layers of bidirectional LSTM units (256 hidden units per layer) and a multi-head attention mechanism layer (4 heads) effectively handles compiler optimizations and instruction reordering while maintaining semantic consistency. In the neural network model, through Bi... The LSTM layer computes the forward hidden state \(\overrightarrow{h_t}\) and the reverse hidden state \(\overleftarrow{h_t}\) of the sequence respectively, and concatenates them to generate the context fusion feature \(h_t=[\overrightarrow{h_t};\overleftarrow{h_t}]\); a multi-head attention layer is then applied to the Bi... The LSTM output features are weighted to highlight key instruction subsequences and generate attention-enhanced features.

[0025] S5. Input the attention-enhanced features into a multilayer perceptron (MLP, 512 hidden units, ReLU) to obtain the vulnerability probability of each subsequence (output via the Softmax function). For subsequences with vulnerability probabilities exceeding a preset threshold, locate their offset addresses in the original binary program based on the attention weights generated by the multi-head self-attention mechanism. This step is the classification and localization step. The multilayer perceptron (e.g., MLP, 512 hidden units, ReLU) and the Softmax / Sigmoid output layer output K-class vulnerability probabilities. The output probabilities are judged according to a preset threshold T, high-risk subsequences are marked, and the vulnerability code offsets are located using the attention weights. Associating the offset addresses located by the model with source code information greatly improves vulnerability auditing efficiency.

[0026] In some implementations, step S1 above involves parsing the binary program under test, including: dynamically identifying and parsing the private section layout and signature area offset of the domestic operating system according to a predefined configuration file, in order to strip the signature area and extract the executable section. This step is to adapt to domestic operating systems. For domestic systems (Kylin OS, Kylin OS), predefined private section name resolution rules (such as identifying .smu, .init_code, .sign, etc.) are automatically loaded. For example, based on the customized firmware standards of domestic systems such as Kylin OS and Kylin OS, private section name resolution rules (such as .smu, .init_code, .sign) are automatically loaded to ensure that the model can identify the operating system-specific section layout. During the extraction, the instruction stream is converted into text form, and disassemblers such as Capstone or Ghidra are used to generate assembly instructions or decompile pseudocode; for ARM64 architecture, capstone.CS_ARCH_ARM64 is used; for x86, capstone.CS_ARCH_X86 is used. The output format is a list of assembly instruction strings, such as ["mov r0, #0", "add r1, r0, #1", ...].

[0027] The above private section rules are as follows: For the Kylin system, .smu (security module), .init_code (driver initialization), and .sign (signature area); for the Kylin OS system, modify the configuration to .sec_smu, .drv_init, and .sec_sign. It also includes signature offset calculation, reading the ELF section header table, and retrieving the sh_offset value (e.g., 0x1200) of the section specified by signature_section (e.g., .sign). If the section does not exist, it falls back to fallback_offset (default 0x1000). An address backtracking table is established, such as: signature offset → binary offset (e.g., 0x1200); mapped to source code line numbers (e.g., driver.c:25) through the ELF symbol table and debugging information (-g compilation); if no debugging information is available, only the offset is retained. Seamless integration with vulnerability detection: preprocessing automatically generates the backtracking table → vulnerability detection directly outputs the source code line number (e.g., driver.c:25), eliminating the need for manual location.

[0028] In some implementations, after generating overlapping subsequences in step S2 above, the method further includes: Establish an address backtracking table to record the mapping relationship between the starting address of each subsequence and the offset of the original binary file; when debugging information exists, further map it to the file name and line number of the source code.

[0029] When establishing the address backtracking table, a binary segmentation window (a continuous instruction sequence fragment divided into segments of the original binary file (such as UEFI firmware or driver) according to fixed rules) and a backtracking table of the original firmware / driver file are created to map the original binary offset and file line number of each segment. A mapping table recording the position of each segmentation window in the original file is also created. The segmentation window is a "small fragment" for model analysis, and the backtracking table is the location of the small fragment to the source code. Together, they achieve accurate vulnerability detection.

[0030] The address backtracking table is established as follows: S201. Record the original offset during segmentation. When segmenting the instruction sequence in the sliding window, the starting offset of each slice is automatically calculated. Slice starting offset = original instruction sequence starting offset + (window index × step size). For example, instruction sequence starting offset = 0x1000, step size = 10, the starting offset of the 3rd slice = 0x1000 + (2 × 10) = 0x1020.

[0031] S202. Mapping to source code line numbers. Check if the ELF file contains debugging information (add the -g option during compilation). If debugging information is present, use a tool (such as addr2line) to convert the offset to a line number (e.g., driver.c:45). If no debugging information is present, the line number is marked as N / A. Line numbers can only be mapped if debugging information is included during compilation.

[0032] S203. Generate a backtracking table. Each slice stores: {"offset": 0x1020, "line": "driver.c:45"}, such as slice 1: {offset: 0x1000, line: "driver.c:20"}; slice 2: {offset: 0x1010, line: "driver.c:22"}. Creating a backtracking table automatically maps vulnerability detection results (binary segmented windows) to the original source code location (filename + line number), achieving precise vulnerability location without manual searching. After creating the backtracking table, for example, if the detected vulnerability slice offset is 0x1200, the backtracking table will directly display: 0x1200 → driver.c:45. Security personnel can then directly open the file and modify the code after seeing driver.c:45, improving efficiency.

[0033] In some implementations, the sliding window in step S2 above has its size and step size dynamically adjusted according to the hardware architecture and / or memory region type of the currently processed instruction sequence. By dynamically adjusting the window parameters based on the architecture and memory region, the model can more precisely capture vulnerability features in different scenarios, resulting in better robustness and higher detection accuracy.

[0034] The above dynamic adjustments are triggered as follows: The system identifies the region type in real time based on instruction sequence characteristics (e.g., push / pop → stack, malloc / free → heap); upon encountering a stack operation instruction, it automatically switches to N=30, S=5; upon encountering a heap operation instruction, it automatically switches to N=50, S=10. Without manual intervention, the system automatically completes parameter switching during the preprocessing stage, working in conjunction with a high-density partitioning strategy within functions (e.g., prioritizing stack parameters for the entry function).

[0035] In some implementations, the dynamic adjustment in step S2 above includes: When the instruction sequence is identified as belonging to the stack memory operation area, the first window size and the first step length are used; When the instruction sequence is identified as belonging to the heap memory operation area, the second window size and the second step size are used; The size of the first window is smaller than the size of the second window, and the step size is smaller than the step size.

[0036] Specifically, for the stack or heap regions, dynamic adjustment of the stride and window is used to enhance the detection of stack overflows or illegal memory accesses. Stack region (short lifespan): window size N=30 (smaller than the default 60); stride S=5 (smaller than the default 10); stack operations (such as push / pop) are intensive and change rapidly, requiring high-density segmentation to capture fast instruction sequences (such as consecutive instructions before a stack overflow). In the tested data, the stack overflow vulnerability detection rate improved by 12.7%. Heap region (long lifespan): window size N=50 (slightly smaller than the default 60, but larger than the stack); stride S=10 (keeping the default value); heap operations (such as malloc / free) are relatively sparse. An excessively large window can easily mix in irrelevant instructions, but it needs to cover a longer memory operation chain. In the tested data, the illegal heap access detection accuracy improved by 8.3%.

[0037] Please refer to Figure 2 In some implementations, in step S4 above, the neural network model is trained using a sample set that includes multiple compilers, multiple optimization levels and multiple hardware architectures; the bidirectional long short-term memory network layer of the neural network model is a three-layer stacked bidirectional structure, with 200-300 hidden units in each layer, used to capture local, mid-range and global instruction sequence dependencies in a hierarchical manner.

[0038] The aforementioned neural network model, such as a three-layer Bi-LSTM network with 256 hidden units per layer, is constructed based on deep sequence modeling and performance-complexity balance considerations: the first layer captures local instruction relationships (e.g., the semantics of a single instruction, "MOV R0, #0x10"); the second layer captures mid-range dependencies (e.g., instruction sequence patterns, ["MOV", "ADD", "STR"]); and the third layer captures the global context (e.g., vulnerability patterns of the entire window, such as instruction combinations causing buffer overflows). The input is an N×D vector sequence (N=60, D=256) generated by Token2Vec, and the processing is as follows: Input sequence processing: This involves processing the D-dimensional vector sequence of N instructions within the window (X = [x1, x2, ..., x...). ], where x ∈R (x1) as input; bidirectional parallel computation, forward LSTM: from x1→x Process sequentially to generate forward hidden states →h1, →h2,..., →h ; Inverse LSTM, from x →x1 is processed in reverse order to generate the reverse hidden state ←h , ←h -1, ..., ←h1; Concatenate at time step t, concatenating the forward and backward hidden states into h for each time step t. = [→h ; ←h ] ∈ R 512 Three-layer deep modeling: Layer 1, input dimension 256 → output dimension 512 (capturing local instruction relationships); Layer 2, input dimension 512 → output dimension 512 (capturing mid-range instruction dependencies); Layer 3, input dimension 512 → output dimension 512 (capturing global vulnerability patterns).

[0039] Through multi-faceted verification of the above model, when dealing with instruction reordering and compiler optimizations that change the instruction order (e.g., ["MOV", "ADD", "STR"] → ["ADD", "MOV", "STR"]), Bi-LSTM can still identify the semantic association between MOV and STR through bidirectional context. Experimental data supports this, showing that on the ARM64 instruction set, the three-layer Bi-LSTM achieves a vulnerability detection accuracy of 86.9% in O2-optimized code (compared to 87.3% in unoptimized code), a 4.1% improvement over the two-layer Bi-LSTM (82.8%). The key mechanism is bidirectional processing, which automatically captures data dependencies (e.g., instruction B depends on the result of instruction A) and control dependencies between instructions, conforming to the compiler's "as-if-serial" semantic constraints. Connecting with subsequent steps, the Bi-LSTM outputs N×512 context features (H = [h1, h2, ..., h...). This is directly used as input to the multi-head attention mechanism to generate query (Q), key (K), and value (V) through linear transformation; calculate attention weights to highlight instruction subsequences related to vulnerability patterns; and generate attention-enhanced features (N×256) for use by the classifier (multilayer perceptron). Here, Q represents the query: the vulnerability feature query vector of the current instruction; K represents the key: the semantic identifier vector of all instructions in the sequence; and V represents the value: the content vector of all instructions in the sequence.

[0040] In step S4 above, the multi-head self-attention mechanism extracts key features. To identify the instruction or instruction combination in an instruction sequence that contributes most to "vulnerability determination," a weight \[A^{(i)} =\mathrm{softmax}\bigl(\tfrac{Q^{(i)}K^{(i)T}}{\sqrt{d_k}}\bigr)\] is calculated for each attention head and weighted to generate the head output. The multi-head outputs are concatenated and linearly mapped to obtain the final attention feature \(Z\), highlighting high-risk instruction subsequences. The multi-head self-attention mechanism achieves automatic highlighting of high-risk instructions through the following steps: concatenating multi-head outputs, horizontally concatenating the outputs of the four attention heads (64-dimensional features per head) to form a 256-dimensional feature vector; linearly mapping to generate the final feature, where, for key instructions STR: - First output: [0.9, 0.95]; - First 2 outputs: [0.85, 0.9]; - First 3 outputs: [0.92, 0.95]; - First 4 outputs: [0.88, 0.92]; After splicing: [0.9, 0.95, 0.85, 0.9, 0.92, 0.95, 0.88, 0.92].

[0041] The concatenated features are transformed using a learnable weight matrix W^O, significantly amplifying the features of high-risk instructions: Z_{text{STR}} = text{concatenated features} times W^O; the eigenvalue of the STR instruction increases from 0.95 to 3.2 (MOV instructions increase from 0.3 to 1.1), automatically highlighting high-risk instructions. High-risk instructions (such as STR in buffer overflows) have the highest eigenvalues ​​in Z, while the eigenvalues ​​of low-risk instructions (such as RET) are relatively suppressed; each time step of Z directly corresponds to the original instruction sequence, and the instruction with the highest eigenvalue is the critical instruction for the vulnerability.

[0042] In the vulnerability detection process, the mapping relationship between the D-dimensional vector (D=256) and the Bi-LSTM output sequence H is a continuous chain of input-processing-output, specifically as follows: The D-dimensional vector is the semantic mathematical representation of a single instruction by Token2Vec. For example, the instruction "MOV R0, #0x10" is converted into a 256-dimensional vector. This vector is only used as the input feature of the Bi-LSTM network. When processing a sliding window containing N instructions (N=60), the D-dimensional vectors of all instructions constitute a 60×256 input matrix. The Bi-LSTM network (three-layer structure) performs deep processing on this input sequence and outputs a context feature sequence H (60×512 dimensions), where each time step h_t (512 dimensions) of H integrates the global semantic information of the instruction sequence.

[0043] In some implementations, during step S4 above, the neural network model is trained by: performing data augmentation using one or more of the following methods: random insertion of no-operation instructions, safe replacement of register names, and code obfuscation; and training with an Adam optimizer featuring early stopping and dynamic learning rate decay.

[0044] During the training process described above, sample data containing both vulnerabilities and non-vulnerabilities were constructed: positive and negative samples generated by GCC, Clang, and domestic compilers at optimization levels O0-O3 and on architectures such as x86, ARM, LoongArch, Phytium, and Kunpeng. Positive sample (vulnerability) construction: test programs were written using vulnerable code (e.g., improperly used C code with strcpy); compiled using GCC / Clang / domestic compilers at optimization levels O0-O3; the offset of the vulnerable instruction was automatically located using a binary analysis tool (IDA Pro) (e.g., the position of STR R1, [R2]); an instruction sequence was automatically generated (e.g., ["MOV R0, #0x10", "ADD R1, R0, #1", "STR R1, [R2]"]); and labeled as a positive sample (vulnerability probability = 1.0). This instruction sequence is the input to Token2Vec, which generates a D-dimensional vector, then processes it using Bi-LSTM, then applies attention features Z, and finally outputs a high vulnerability probability using an MLP. Negative sample (normal) construction involves extracting vulnerability-free code from secure open-source projects (Linux kernel, OpenSSL): selecting secure code snippets (e.g., MOV→ADD→RET sequence); compiling them using the same compiler / optimization level / architecture; automatically verifying vulnerability-free status using static analysis tools (e.g., Clang Static Analyzer); extracting instruction sequences; and labeling them as negative samples (vulnerability probability = 0.0). This instruction sequence serves as input to Token2Vec, generating a D-dimensional vector, which is then processed by Bi-LSTM, followed by attention features Z, and finally output by an MLP with a low vulnerability probability.

[0045] The training process also includes data augmentation and diversity enhancement, such as random NOP insertion (inserting no-ops between instructions to simulate compiler optimization), register substitution (replacing R0 with R1 to simulate different compiler register allocations), and code obfuscation (replacing instructions with operations like XOR to simulate malicious code obfuscation). Generating multiple variants of the same vulnerable code under different compilers / architectures enhances model robustness. This also includes seamless integration with the vulnerability detection process. In some implementations, steps S4 and S5 above, when generating attention-enhanced features, include: generating training data: vulnerable code → compiler → instruction sequence → Token2Vec → D-dimensional vector; secure code → compiler → instruction sequence → Token2Vec → D-dimensional vector. Model training is then performed, inputting a D-dimensional vector sequence (the positive / negative samples mentioned above); outputting vulnerability probabilities (positive samples → high probability, negative samples → low probability). The loss function is set to be the same as that used in conventional models, such as cross-entropy loss, automatically learning to distinguish between positive and negative samples. Finally, inference is performed, inputting a new instruction sequence (from the binary code to be detected), and outputting the vulnerability probability and vulnerability location (visualized through attention weights).

[0046] In some implementations, step S4 above, when generating attention-enhancing features, includes: The feature sequences output by the bidirectional long short-term memory network layer are mapped to query matrix, key matrix and value matrix respectively; Calculate the attention weight for each attention head and perform weighted aggregation on the value matrix; The outputs of multiple attention heads are concatenated and then linearly transformed to obtain attention-enhanced features.

[0047] In step S5 above, the multilayer perceptron is the final classification layer for vulnerability detection, converting the attention feature Z into vulnerability probability. A threshold T is set, and subsequences with probabilities ≥ T are marked as vulnerabilities. Attention weight visualization accurately locates vulnerability offsets, aiding in auditing. Attention weight visualization is the direct basis for vulnerability localization, and its process is closely linked to the preceding steps: S51. Calculation process (connecting to the multi-head attention mechanism in the above steps) Input window instruction sequence (e.g., [MOV, ADD, STR]); calculate attention weight matrix A = mathrm{softmax}left(frac{QK^T}{sqrt{64}}right), e.g., A_{text{STR, MOV}} = 0.92 (similarity weight between STR and MOV).

[0048] S52, Visual Implementation Convert the weight matrix A into a heatmap, with the horizontal axis representing the instruction sequence and the vertical axis representing the instruction sequence; high-weight regions (such as STR and MOV, ADD with weights > 0.8) are highlighted in red; low-weight regions (such as STR and RET with weights = 0.35) are represented in blue.

[0049] S53, Relationship with Classification When the MLP outputs a vulnerability probability ≥ 0.5 (e.g., 95%), attention visualization is automatically triggered; the visualization result directly displays the vulnerability instruction offset (e.g., STR in the 3rd position in the window → original offset = 0x1234); no additional steps are required, visualization is a natural output of the attention mechanism, and it shares the same feature calculation process as classification.

[0050] Through the above training process, positive and negative samples are automatically constructed (instead of manual selection). The model can learn the differences in code generation from different compilers (GCC / Clang / domestic), the impact of different optimization levels (O0~O3) on vulnerability patterns, and the instruction semantics of different architectures (x86 / ARM / LoongArch).

[0051] Please refer to Figure 3 In some implementations, the method further includes: S6, exporting the neural network model to a configuration file corresponding to the domestic operating system, dynamically loading the configuration file to identify and parse the pre-stored binary program in the domestic operating system. To adapt to the domestic operating system, this step can specifically encapsulate the detection system as a RESTful / CLI service, specifically exporting the model to ONNX / TorchScript and encapsulating it as a RESTful / CLI interface; integrating it into the CI / CD pipeline and firmware update process of the domestic operating system, specifically deploying it to the domestic OS security framework, and integrating firmware update and driver signing CI / CD processes. It can parse the private section layout and signature area offset according to the customized configuration of Kylin OS; dynamically adjust the segmentation strategy, and establish an address backtracking table mapping to the original offset and file line number.

[0052] In step S6 above, the header parsing does not directly read from the firmware standard, but instead loads rules through a predefined configuration file. The rules are derived from private section names (e.g., .sign) predefined by the developers based on the official firmware standard documents of Kylin / Kylin OS (e.g., the "Kylin V10 Firmware Specification"). These rules are written to the configuration file (kylin_config.json), for example: {"signature_section": ".sign"} / / Kylin standard. Runtime flow: The preprocessing module automatically loads this configuration file upon startup; it reads the signature_section field (.sign) and directly uses it to parse the ELF header table; it searches for the sh_offset (e.g., 0x1200) of the .sign section in the ELF file, without scanning the firmware content. During signature area parsing, the file parser identifies and strips away encrypted signatures and debugging information by matching the file signature (e.g., PKCS#7 or SM2 format). The remaining part is clean executable code, which is then input into the model inference.

[0053] In step S6 above, the dynamic segmentation strategy includes: architecture identification, reading the e_machine field of the ELF header of the binary file (EM_AARCH64 (183) → ARM64 architecture; EM_386 (3) → x86 architecture; EM_LOONGARCH (258) → LoongArch architecture). The window size is dynamically set as follows: ARM64 (fixed instruction length 4 bytes), N = 60 (larger window, capturing longer vulnerability context); x86 (instruction length variable 1-15 bytes), N = 40 (smaller window, avoiding redundant instruction interference); LoongArch (fixed instruction length 4 bytes), N = 50 (between ARM64 and x86), and the step size S of all architectures is unified to 10 to ensure consistent overlapping of sliding windows.

[0054] The platform architecture is entirely based on automatic identification of the ELF file header of the binary file, requiring no user input. The system reads the architecture identifier in the file header (e.g., 183 for ARM64, 3 for x86) to directly determine the platform type, and then dynamically adjusts the sliding window size: N=60 for ARM64, N=40 for x86, and N=50 for LoongArch, with a uniform step size S of 10. This process is completed automatically during the preprocessing stage, ensuring compatibility with the architectural characteristics of domestic system firmware and 100% avoiding manual configuration errors.

[0055] In this application, the training of the aforementioned neural network model includes the following configurations: cross-entropy loss, using the Adam optimizer (learning rate lr=1e-4), batch size of 64, and 50 training epochs. If the results on the validation dataset no longer improve accuracy, early stopping and learning rate decay are used. Details are as follows: 1. Learning rate decay Triggering condition: The validation set accuracy has decreased for 3 consecutive rounds (i.e., the current round's validation accuracy is less than or equal to the highest historical accuracy, and has not improved for 3 consecutive rounds).

[0056] Decay operation: Multiply the learning rate by 0.5 (e.g., the initial learning rate of 1e-4 decays to 5e-5).

[0057] Execution logic: When it is detected that the validation accuracy has not reached the historical high for 3 consecutive rounds, the learning rate is immediately updated and the continuous decline counter is reset. The new learning rate is used in the next round of training.

[0058] 2. Early shutdown mechanism Triggering condition: The validation set accuracy has decreased for 5 consecutive rounds (i.e., the current round's validation accuracy is less than or equal to the highest historical accuracy, and has not improved for 5 consecutive rounds).

[0059] Termination action: Immediately stop training and retain the model weights corresponding to the highest historical accuracy.

[0060] Key design: Early stop is triggered only when the learning rate fails to increase after decay, thus avoiding premature termination of training.

[0061] 3. Some training results are as follows: Round 10: Validation accuracy reached 86.5% (highest ever), model saved.

[0062] Round 11: 86.4% (↓1 round) → No decay triggered.

[0063] Round 12: 86.3% (↓2 rounds) → Trigger decay, learning rate adjusted from 1e-4 to 5e-5.

[0064] Round 13: 86.2% (↓3 rounds) → Trigger decay, learning rate adjusted to 2.5e-5.

[0065] Round 14: 86.1% (↓4 rounds) → Continue training.

[0066] Round 15: 86.0% (↓5 rounds) → Triggers early stop, terminates training and retains the model from round 10 (86.5%).

[0067] In the above training results, due to the use of a decay mechanism, the convergence rounds were shortened from 50 to 25 (stopping 50% earlier); due to the use of an early stopping mechanism, decay (3 rounds) provides the model with an opportunity for fine-tuning, and early stopping (5 rounds) avoids misjudgments due to short-term fluctuations, forming a gradient optimization chain. This strategy is a standard practice in deep learning (such as PyTorch's ReduceLROnPlateau) and has been verified to be effective in detecting vulnerabilities in domestic systems.

[0068] A second aspect of this application provides a binary vulnerability detection system based on deep learning, comprising: The preprocessing module, corresponding to steps S1 and S2 above, is used to parse the binary program under test, strip away non-code data, and extract instruction sequences; it uses a sliding window to segment the instruction sequences to generate overlapping subsequences; and it maps each instruction in each subsequence to a D-dimensional vector to form the neural network input matrix.

[0069] The model inference module, corresponding to steps S3 and S4 above, loads a pre-trained neural network model to capture the context fusion features of the instruction sequence, and weights the context fusion features through a multi-head self-attention mechanism to generate attention-enhanced features that highlight key instruction subsequences. The classification and localization module, corresponding to step S5 above, is used to input the attention-enhanced features into the multilayer perceptron to obtain the vulnerability probability of each subsequence; for subsequences whose vulnerability probability exceeds a preset threshold, the offset address of the subsequence in the original binary program is located according to the attention weight generated by the multi-head self-attention mechanism.

[0070] The deployment integration module, corresponding to the above-mentioned adaptation to domestic operating systems, is used to encapsulate the model inference module and classification and localization module as services and integrate them into the continuous integration / continuous delivery pipeline of the domestic operating system for binary vulnerability scanning.

[0071] This application employs the aforementioned method or system, which can promptly detect IoT or Industrial IoT vulnerabilities, thus serving as the foundation for endpoint protection and ensuring endpoint security. Based on vulnerability detection, this application has the following advantages: 1. Improved accuracy and recall: On cross-compiler and cross-platform test sets, the recall rate for known vulnerabilities is ≥97%, and the recall rate for unknown vulnerabilities is improved by ≥20%.

[0072] 2. Reduced false alarm rate: Attention dynamically focuses on key instructions, with a false alarm rate of ≤5%, which is approximately 15% lower than the model without Attention.

[0073] 3. High interpretability: Attention weights are visualized, vulnerability location accuracy is ≥90%, and line number and function name backtracking is supported.

[0074] 4. Strong robustness: copes with variations such as instruction reordering, register allocation, and obfuscation; compatible with domestic systems: no secondary training required, direct deployment of Galaxy Kylin 5.0 and Kylin OS 6.0 with only a ≤2% decrease in detection accuracy; high performance and automation: supports parallel sliding window inference with an average inference time of <0.1s per subsequence; after integrating CI / CD, firmware / driver updates trigger detection, improving performance by ≥30%.

[0075] Regarding the specific implementation methods of this application, it should be noted that: In the description of this application, unless otherwise expressly specified and limited, the terms "connection," "fixed," "connected," etc., should be interpreted broadly. For example, "fixed" can refer to a fixed connection, a detachable connection, or an integral molding; "connection" can refer to a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium; it can refer to the internal connection of two components or the interaction between two components, unless otherwise expressly limited; "connected" can refer to the internal connection of two parts and the connection between two parts, or the spatial connection between them, whereby the two parts are directly or indirectly connected through the part forming the space. The terms "set," "installed," "equipped with," "configured," etc., should also be interpreted broadly. For those skilled in the art, the specific meaning of the above terms in this application can be understood according to the specific circumstances.

[0076] In the description of this application, the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," and "counterclockwise," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are used only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the system or component referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application. All directional indications are only used to explain the relative positional relationship and movement between components in a specific orientation. If the specific orientation changes, the directional indication will also change accordingly.

[0077] In the description of this application, the use of terms such as "some embodiments," "optional embodiments," "example," "specific example," "optional example," or "optional embodiment," etc., indicates that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application, but does not imply that these embodiments illustrate and describe all possible forms of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Furthermore, those skilled in the art can combine and integrate the different embodiments or examples described in this specification.

[0078] The present invention has been described in detail above with reference to specific embodiments and exemplary examples. The above description is exemplary and not exhaustive, and is not limited to the disclosed embodiments; the above description should not be construed as a limitation of the present invention. Technical solutions between various embodiments can be combined with each other, but must be based on the ability of those skilled in the art to implement them. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the scope of protection claimed in this application. Although embodiments of the present application have been shown and described, various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present application. Those skilled in the art will understand that various other specific changes and combinations of embodiments based on the technical teachings disclosed in this application, without departing from the essence of the present application, are still within the scope of protection defined by the claims of the present invention and their equivalent technical solutions.

Claims

1. A deep learning-based binary vulnerability detection method for testing binary programs, characterized in that, include: The binary program under test is parsed to remove non-code data and extract instruction sequences. The instruction sequence is segmented using a sliding window to generate overlapping subsequences; Each instruction in each subsequence is mapped to a D-dimensional vector to form the neural network input matrix; The input matrix is ​​input into a pre-trained neural network model, which sequentially captures the context fusion features of the instruction sequence through a bidirectional long short-term memory network layer, and weights the context fusion features through a multi-head self-attention mechanism to generate attention-enhanced features that highlight key instruction subsequences. The attention-enhanced features are input into a multilayer perceptron to obtain the vulnerability probability of each subsequence; for subsequences whose vulnerability probability exceeds a preset threshold, their offset address in the original binary program is located according to the attention weights generated by the multi-head self-attention mechanism.

2. The binary vulnerability detection method based on deep learning according to claim 1, characterized in that, The parsing of the binary program under test includes: dynamically identifying and parsing the private section layout and signature area offset of the domestic operating system according to a predefined configuration file, so as to strip the signature area and extract the executable section.

3. The binary vulnerability detection method based on deep learning according to claim 1, characterized in that, After generating the overlapping subsequences, the method further includes: An address backtracking table is established to record the mapping relationship between the starting address of each subsequence and the offset of the original binary file; when debugging information exists, it is further mapped to the file name and line number of the source code.

4. The binary vulnerability detection method based on deep learning according to claim 1, characterized in that, The sliding window's size and step size are dynamically adjusted based on the hardware architecture and / or memory region type of the currently processed instruction sequence.

5. The binary vulnerability detection method based on deep learning according to claim 4, characterized in that, The dynamic adjustment includes: When the instruction sequence is identified as belonging to the stack memory operation area, the first window size and the first step length are used; When the instruction sequence is identified as belonging to the heap memory operation region, the second window size and the second step size are used; Wherein, the size of the first window is smaller than the size of the second window, and the length of the first step is smaller than the length of the second step.

6. The binary vulnerability detection method based on deep learning according to claim 1, characterized in that, The neural network model is trained using a sample set that includes multiple compilers, multiple optimization levels, and multiple hardware architectures. The bidirectional long short-term memory network layer of the neural network model is a three-layer stacked bidirectional structure with 200-300 hidden units in each layer, which is used to capture local, mid-range, and global instruction sequence dependencies in a hierarchical manner.

7. The binary vulnerability detection method based on deep learning according to claim 6, characterized in that, During training, the neural network model employs data augmentation using one or more of the following methods: random insertion of no-operation instructions, safe replacement of register names, and code obfuscation; and is trained using the Adam optimizer with early stopping mechanism and dynamic learning rate decay.

8. The binary vulnerability detection method based on deep learning according to claim 1, characterized in that, Generating the attention-enhanced features includes: The feature sequences output by the bidirectional long short-term memory network layer are mapped into a query matrix, a key matrix, and a value matrix, respectively. Calculate the attention weight for each attention head and perform weighted aggregation on the value matrix; The outputs of multiple attention heads are concatenated and the attention enhancement feature is obtained through a linear transformation.

9. The binary vulnerability detection method based on deep learning according to claim 1, characterized in that, Also includes: The neural network model is exported as a configuration file corresponding to the domestic operating system, and the configuration file is dynamically loaded to identify and parse the pre-stored binary program in the domestic operating system.

10. A binary vulnerability detection system based on deep learning, characterized in that, include: The preprocessing module is used to parse the binary program under test, strip away non-code data, and extract instruction sequences; The instruction sequence is segmented using a sliding window to generate overlapping subsequences; each instruction in each subsequence is mapped to a D-dimensional vector to form the neural network input matrix. The model inference module loads a pre-trained neural network model to capture the context fusion features of the instruction sequence, and weights the context fusion features through a multi-head self-attention mechanism to generate attention-enhanced features that highlight key instruction subsequences. The classification and localization module is used to input the attention-enhanced features into the multilayer perceptron to obtain the vulnerability probability of each subsequence; for subsequences whose vulnerability probability exceeds a preset threshold, the offset address of the subsequence in the original binary program is located according to the attention weight generated by the multi-head self-attention mechanism. An integration module is deployed to encapsulate the model inference module and the classification and localization module as services and integrate them into the continuous integration / continuous delivery pipeline of the domestic operating system for binary vulnerability scanning.

Images