Binary component vulnerability identification method based on large language model

Abstract

The invention belongs to the technical field of computer system security, and particularly relates to a binary component vulnerability recognition method based on a large language model. The method solves the problems of various equipment types, various instruction set architectures, different compilers, different compiling optimization levels and the like in the current Internet of Things binary component vulnerability recognition, and comprises the following steps: S100: collecting various vulnerability description information of an Internet of Things component, extracting data from the vulnerability description information, and constructing a text information data set; s200, performing text segmentation processing on a long text in the text information data set; s300, inputting the constructed text information data set into a large language model for training and optimization; and S400, detecting the binary component by using the trained vulnerability recognition large model to obtain a binary component vulnerability detection result report.

Description

Technical Field

[0001] The present invention belongs to the technical field of computer system security, and specifically is a binary component vulnerability identification method based on a large language model. Background Art

[0002] At present, due to the popularity and ease of use of components, IoT manufacturers often ignore component vulnerability detection when integrating and using these components to develop firmware. As a result, in the IoT software supply chain, malicious attackers can attack IoT devices by contaminating components. Therefore, the research on vulnerability identification of binary components in IoT firmware has become an important part of IoT firmware security protection. Binary code similarity is the main method for vulnerability identification research. Chandramohan et al. used selective inlining technology to capture the complete semantics of functions and introduced variable-length partial traces to model binary functions, completing cross-architecture and cross-platform binary function search tasks. In 2017, Xu et al. proposed the first method to generate binary function CFG embedding based on neural network. In a system called Gemini, they used the improved Structure2vec model to construct a Siamese network, embedded the CFG of the function into a high-dimensional vector, and measured the similarity between functions by calculating the cosine distance of the vector. Liu et al. used three semantic features to solve the challenge of cross-version binary code similarity. First, they used a deep neural network (DNN) to extract the intra-function features of each binary function, further analyzed the function call graph of each binary file that was relatively stable in the cross-version binary files, and extracted inter-function and inter-module features from it. Then, the distance calculated based on these three features is used for binary code similarity detection. Ding et al. implemented a representation learning model called Asm2vec, which does not require additional information and only uses assembly code as input to achieve detection tasks. After training, the model can represent assembly instructions in function units as feature vectors. Luo et al. proposed a cross-architecture binary code search model VulHawk, which uses an entropy adapter to transfer different file environments to the same environment for binary function embedding, thereby realizing cross-architecture binary vulnerability function search. When the above work is applied to vulnerability identification of IoT components, due to the large differences in binary components under different architectures, different compilers and different optimization levels, this will interfere with the accuracy of the detection results. It is necessary to study vulnerability identification technology suitable for IoT components. Summary of the invention

[0003] In order to solve the problems of various types of devices, diverse instruction set architectures, different compilers and compilation optimization levels in the current identification of binary component vulnerabilities in the Internet of Things, the present invention provides a binary component vulnerability identification method based on a large language model.

[0004] The present invention adopts the following technical solution: a binary component vulnerability identification method based on a large language model, comprising: S100: Collects various vulnerability descriptions of IoT components, extracts data from them and constructs a text information dataset; S200: Segmentation of long text in the text information data set; S300: inputting the constructed text information data set into the large language model for training and optimization; S400: Detect binary components using the trained vulnerability identification large model to obtain a binary component vulnerability detection result report.

[0005] In some embodiments, step S100 includes: S101: Obtain binary components, open source components, and vulnerability text information of IoT components; S102: Decompile the binary component into intermediate code; For open source components, compile and then decompile to generate intermediate code; Preprocess the vulnerability text information, including deduplication, standardization, and associating it with components; S103: Input the representation vector of each paragraph and the global context vector into the global attention mechanism, and output the paragraph representation that integrates the global information.

[0006] In some embodiments, step S103 includes: Encode the paragraphs: Use the pre-trained language model to encode each paragraph, convert it into a set of representation vectors, assign global vectors to the key positions of each paragraph, and use these vectors together with the paragraph representation vector for subsequent attention calculations; Calculate cross-paragraph attention: Extract query, key, and value vectors from the representation vectors of the current paragraph and other paragraphs; then calculate the cross-paragraph attention weight by calculating the similarity between the query vector of the current paragraph and the key vector of other paragraphs; Integrate global information: Use these attention weights to perform weighted summation on the global context vector and the Value vectors of other paragraphs to obtain a paragraph representation that integrates global information; Information fusion and update: The obtained global context representation is fused with the local representation of the current paragraph (usually through concatenation or addition operations) to obtain an updated paragraph representation; The fused representation not only contains the detailed information of the current paragraph, but also reflects the global structure of the text and the semantic associations across paragraphs.

[0007] In some embodiments, step S300 includes: S301: preprocessing the text information in the training sample set; S302: Use the training sample set to train the large language model to obtain a binary component vulnerability recognition model. The model determines whether there is a vulnerability and the type of vulnerability based on the input features. S303: Use the expert review system to conduct manual review, optimize the model based on the evaluation results, and further improve the accuracy and performance of the model.

[0008] In some embodiments, step S301 includes: Preprocess the input text information through natural language processing tools; Embed the preprocessed text into continuous word vectors; For each word, the word position information is embedded into the word embedding vector. The position encoding is calculated based on the word position information in the text sequence and added to the word embedding vector as the input of the model.

[0009] In some embodiments, in step S302, the large language model includes: The position encoding layer adds the position information of the input text to the embedding vector to help the model recognize the sequence order; A plurality of encoder layers connected in series, each of the encoder layers comprising a multi-head attention mechanism layer, a layer normalization and residual connection layer, a feedforward neural network, and a layer normalization and residual connection layer connected in series in sequence; A plurality of decoder layers connected in series, each of the decoder layers comprising a multi-head attention mechanism layer, a layer normalization and residual connection layer, a feedforward neural network, and a layer normalization and residual connection layer connected in series in sequence; The multi-head self-attention mechanism captures the dependencies between different parts of the input text through different attention heads; The feedforward neural network is used to perform nonlinear transformation on the hidden vector; The layer normalization and residual connection layers are used to accelerate model training and improve model stability.

[0010] In some embodiments, step S303 includes: Error classification and statistics; Network structure optimization based on error classification.

[0011] In some embodiments, error classification and statistics specifically include: Step 1: Collect error data: After evaluating the large language model, collect error data in the output of the large language model; Step 2: Manual classification: The expert team manually reviews the error output of the large language model and classifies it by error type; Step 3: Statistics and analysis: Statistical error frequency and error distribution are performed for each error type.

[0012] In some embodiments, performing network structure optimization based on error classification includes: Step 1: Identify bottlenecks in the large language model structure: Identify bottlenecks or weak links in the structure of large language models based on the statistics of misclassifications; Step 2: Optimize the model: Optimize the large language model based on the identified bottlenecks; Step 3: Iterative optimization and verification: Repeatedly adjust the network structure and verify it on new data sets to observe whether the optimized effect is improved.

[0013] In some embodiments, step S200 includes: According to the maximum length limit of the model, the initial window size is set, and the window size and sliding step size are dynamically adjusted according to the text length; If the text is long, choose a larger window to capture more context; If the text is short, reduce the window size to reduce the amount of computation; The text is divided into multiple smaller windows. The content of each window includes the maximum number of tokens that the model can handle, and slides according to the set sliding step size to cover the entire text.

[0014] Compared with the prior art, the present invention has the following beneficial effects: 1. The present invention introduces a large language model, which learns a lot of relevant knowledge during the training process, and can better understand the text content, including vulnerability descriptions and codes, so as to more accurately understand the semantics and contextual information in the text and improve the accuracy of vulnerability identification.

[0015] 2. The present invention uses a large language model to learn the language patterns and structures in the text, thereby realizing automatic extraction and learning of vulnerability features, reducing manual intervention; 3. The present invention improves the credibility of vulnerability identification results and repair efficiency by generating explanatory text related to the vulnerability and generating vulnerability repair suggestions for users. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 A binary component vulnerability identification flow chart provided by an embodiment of the present invention; Figure 2A flow chart of a method for constructing a training data set provided by an embodiment of the present invention; Figure 3 Schematic diagram of the architecture of a binary component vulnerability identification software system for large language models; Figure 4 This is the specific structure of the large language model. DETAILED DESCRIPTION

[0017] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0018] This embodiment provides a system example for identifying binary component vulnerabilities using a large language model. As shown in Figure X, the system mainly includes: A training sample set generator obtains a training sample set, wherein each training sample includes a processed intermediate code representation of a binary component and non-code text description information.

[0019] The large model trainer is used to train the large language model using the training sample set to obtain a binary component vulnerability identification function model.

[0020] Component verifier is used to analyze the internal structure of binary components and verify whether the binary components contain known vulnerabilities and potential vulnerabilities.

[0021] Vulnerability Analyzer, used to describe the specific characteristics of vulnerabilities and provide solutions to the vulnerabilities.

[0022] like Figure 1 As shown, a binary component vulnerability identification method based on a large language model includes: S100: Collect various vulnerability description information of IoT components, extract data from them and build a text information dataset.

[0023] Step S100 includes: S101: Obtain binary components, open source components, and vulnerability text information of IoT components; S102: Decompile the binary component into intermediate code; For open source components, compile and then decompile to generate intermediate code; Preprocess the vulnerability text information, including deduplication, standardization, and associating it with components; S103: Use a cross-paragraph global attention mechanism to enable the model to refer to local information and global information of the entire text when processing the current paragraph; the global attention mechanism can help the model consider the semantic structure of the entire text when processing each paragraph by adding a global context vector to the representation of each segment. Through the cross-paragraph attention mechanism, the model can establish semantic associations between different paragraphs. Global attention refers to adding global context vectors to each paragraph or specified position (such as the beginning of a paragraph or a key sentence), and these global vectors can interact with the content of other paragraphs. The global attention mechanism enables the model to consider the overall structure and context of the text, thereby better understanding the semantic associations between paragraphs.

[0024] Its main structure includes: Paragraph encoding: Use the pre-trained language model RoBERTa to encode each paragraph and convert it into a set of representation vectors. Global vectors are assigned to the key positions of each paragraph, and these vectors are used together with the paragraph representation vector for subsequent attention calculations. The RoBERTa model converts each word or token in a paragraph into a table (embedding), and these vectors capture the semantic information of the word. In this process, global context vectors are also assigned to the key positions in the paragraph. These vectors represent the global semantic background of the paragraph.

[0025] Cross-paragraph attention calculation: First, the Query, Key, and Value vectors are extracted from the representation vectors of the current paragraph and other paragraphs. Then the cross-paragraph attention weights are calculated. The cross-paragraph attention weights are obtained by calculating the similarity between the Query of the current paragraph and the Key vectors of other paragraphs. The model extracts the Query, Key, and Value vectors from the representation vectors of the current paragraph and other paragraphs respectively. The Query vector represents the semantic features of the current paragraph, while the Key and Value vectors represent the semantic features and content information of other paragraphs. Next, the similarity between the Query vector of the current paragraph and the Key vectors of other paragraphs is calculated to generate cross-paragraph attention weights. These weights are obtained by dot product calculation and converted into probability distributions by the softmax function, indicating the degree to which the model should pay attention to other paragraphs when processing the current paragraph. Finally, these attention weights are used to perform weighted summation on the Value vectors of other paragraphs to obtain a paragraph representation vector that integrates global semantic information. This vector contains the semantic association between the current paragraph and other paragraphs, providing a basis for subsequent global information integration and paragraph representation updates.

[0026] Global information integration: Use these attention weights to perform weighted summation on the global context vector and the Value vectors of other paragraphs to obtain a paragraph representation that integrates global information. This step ensures that the representation of the current paragraph retains its own detailed information while reflecting the global semantic structure of the entire text, and outputs an updated paragraph representation that contains local detailed information and global semantic associations.

[0027] Information fusion and update: The obtained global context representation is fused with the local representation of the current paragraph (usually through concatenation or addition operations) to obtain an updated paragraph representation.

[0028] The fused representation contains not only the detailed information of the current paragraph, but also reflects the global structure of the text and the semantic associations across paragraphs. These paragraph representations not only reflect the content of the paragraph itself, but also contain the semantic associations with other paragraphs. The model can use these updated paragraph representations for downstream tasks (such as classification, generation, etc.), thereby improving the performance of the task.

[0029] Specifically, for open source components, multiple versions of source code can be obtained from various code hosting platforms, source code in multiple languages ​​can be cloned on machines with multiple architectures and compiled using multiple types of compilers, and multiple optimization levels can be set during compilation, such as compiling on computers with multiple architectures such as X64, ARM, MIPS, RISC-V, etc., and multiple compilers can be installed according to the CPU architecture supported by the specific compiler, such as GCC, Clang, and MSVC compilers. The following optimization levels can be set during compilation: no optimization (-O0), low optimization (-O1), medium optimization (-O2), high optimization (-O3), performance optimization (-Of), debug mode (-g), etc. The compiled binary code is converted into an intermediate code in text format using a decompilation tool; for components that have no source code but only binary executable files, a textual intermediate code representation is also obtained through decompilation; the collected component version number and function description information are associated with the decompiled component intermediate code, and the above data is collected and organized using an automated script tool to form a training sample set for a large language model.

[0030] S200: Segmentation of long text in the text information dataset.

[0031] For a large language model, the length of the input text may be limited by the model. The large language model has a maximum limit on the input length. When the length exceeds this limit, the text needs to be specially processed. In an embodiment of the present invention, it is proposed to divide the long text into multiple smaller windows through a dynamic sliding window mechanism, and input the windows into the model for processing in turn, so as to solve the problem of a single input text exceeding the maximum length limit of the model.

[0032] To process very long texts and solve the problem of incoherent contexts through a dynamic sliding window mechanism, first, set the initial window size according to the maximum length limit of the model. Dynamically adjust the window size and sliding step size according to the length of the text. If the text is long, you can choose a larger window to capture more context; if the text is short, you can reduce the window size to reduce the amount of calculation. Split the text into multiple smaller windows. The content of each window should include the maximum amount of text information that the model can handle, and slide according to the set sliding step size to cover the entire text. The length of each window is determined by the current window size, and the position of each window is determined by the sliding step size. For example, for a text with a length of 2000 tokens, the window size is set to 512 and the step size is set to 256, then the window positions will be [0, 511], [256, 767], [512, 1023] in turn. Input each window into the model for processing. Record the output of the model for each window. After processing all windows, the results need to be merged. The outputs of each window are connected, and then the context incoherence problem is dealt with. If the context incoherence problem is serious, the overlapping window strategy is used to ensure that more context information is included in the window by increasing the overlap between windows.

[0033] S300: Input the constructed text information dataset into the large language model for training and optimization.

[0034] Step S300 includes: S301: Preprocess the text information in the training sample set, including text segmentation, tokenization, removal of stop words, and conversion of text information into word embeddings.

[0035] Step S301 includes: first, preprocessing the input text information through a natural language processing (NLP) tool, and using the Word2Vec method to represent the preprocessed text as a continuous word embedding vector; for each word, the word position information needs to be embedded into the word embedding vector, and the position encoding is calculated according to the word position information in the text sequence and added to the word embedding vector as the input of the model.

[0036] S302: Use the training sample set to train the large language model to obtain a binary component vulnerability identification model. The model will determine whether there is a vulnerability and the type of vulnerability based on the input features.

[0037] like Figure 4 As shown, the large language model includes: The position encoding layer adds the position information of the input text to the embedding vector to help the model recognize the sequence order; A plurality of encoder layers connected in series, each of the encoder layers comprising a multi-head attention mechanism layer, a layer normalization and residual connection layer, a feedforward neural network, and a layer normalization and residual connection layer connected in series in sequence; A plurality of decoder layers connected in series, each of the decoder layers comprising a multi-head attention mechanism layer, a layer normalization and residual connection layer, a feedforward neural network, and a layer normalization and residual connection layer connected in series in sequence.

[0038] The encoder layer is responsible for encoding the input text into a hidden vector, and the decoder layer decodes the hidden vector into the target output.

[0039] Multi-head self-attention mechanism: In each encoder and decoder layer, the multi-head self-attention mechanism captures the dependencies between different parts of the input text through different attention heads, enhancing the model's ability to understand the context.

[0040] Feedforward Neural Network: Each encoder and decoder layer also contains a feedforward neural network, which is used to perform nonlinear transformations on the hidden vectors to increase the expressive power of the model.

[0041] Position encoding: Position encoding adds the position information of the input text to the embedding vector to help the model recognize the sequence order.

[0042] Layer normalization and residual connection: In each encoder and decoder layer, layer normalization and residual connection techniques are used to help accelerate model training and improve model stability.

[0043] This embodiment adopts an encoder-decoder structure, takes text as an input sequence, and finally outputs a specific description of the vulnerability function, which mainly includes whether the binary component to be tested contains known vulnerabilities and potential vulnerabilities, the name and version of the vulnerability component, and possible solutions; the core part of the encoder is a multi-head attention module. The model used in the present invention adopts multiple attention heads, and each attention head will generate an attention matrix, which is composed of three types of matrices, namely, Q matrix (query matrix), K (key matrix) matrix and V matrix (value matrix); Q matrix mainly contains query information, K matrix contains information to be checked, and the query vector is generated by using the input vector in the self-attention mechanism, and the result of the inner product of Q matrix and K matrix is ​​divided by Calculate the similarity between the query and the key, where Represents the dimension of the key, which can smooth the probability distribution; the matrix obtained after the Softmax normalization operation is called the attention weight matrix A, which determines the size of the attention distribution; the V matrix stores the actual numerical information in the input sequence, and then the attention weight matrix A is multiplied by the V matrix to obtain the output feature.

[0044] The advantage of the attention mechanism is that it can calculate the relevant information between text information in parallel, avoiding the limitation of distance.

[0045] The present invention adopts a multi-head attention mechanism to split the Q, K, and V matrices and calculate the attention of each of them, so as to calculate the correlation between different fields, thereby improving the information extraction ability of the model; in order to improve the reasoning speed of the large language model in the present invention, speculative decoding is used to improve the parallelism of reasoning.

[0046] First, a smaller language model is built into the large language model. During the reasoning process, the smaller language model first predicts the text information, and the obtained multiple text information is put into the large language model for reasoning and verification. If the verification result is correct, the small language model is used to infer the subsequent text information. Otherwise, the large language model corrects the text information, and the small language model continues to infer the subsequent text information based on the correction result of the large language model.

[0047] The small language model is between the input preprocessing module and the large language model. The small language model receives the preprocessed characters, makes preliminary predictions, generates possible subsequent characters, and performs confidence assessment. By evaluating the confidence of the small model's predictions, it determines whether the large model needs to be called for verification.

[0048] S303: Use the expert review system to conduct manual review, optimize the model based on the evaluation results, and further improve the accuracy and performance of the model.

[0049] To classify and count the errors in the model evaluation and optimize the network structure, the following steps need to be carried out in detail: 1. Error classification and statistics Step 1: Collect error data After evaluating the model, collect error data in the model output. The evaluation usually includes the following indicators: Accuracy: The proportion of correct predictions made by the model.

[0050] Recall: The proportion of all true positive examples that the model can identify.

[0051] Precision: The proportion of true positive examples among the samples predicted by the model to be positive examples.

[0052] F1 Score value: The harmonic mean of precision and recall, used to comprehensively evaluate the performance of the model. During the evaluation process, record which instances are incorrectly predicted by the model, including missed detections, false positives, and type recognition errors.

[0053] Step 2: Manual classification The expert team manually reviews the model's erroneous outputs and classifies them by error type: False Negative: The model fails to detect a vulnerability that actually exists.

[0054] False Positive: The model mistakenly identifies a non-vulnerability as a vulnerability.

[0055] Misclassification: The model incorrectly identifies the vulnerability type, such as misidentifying a buffer overflow as a SQL injection.

[0056] Boundary issues: Errors in the model’s handling of boundary conditions of input data, such as failure to identify cross-boundary buffer overflows.

[0057] Step 3: Statistics and Analysis Statistics are collected for each error type to analyze its distribution and impact: Error frequency: Count the frequency of each error type.

[0058] Error distribution: Analyze whether these errors are concentrated on specific input patterns or vulnerability types. Context association: Consider whether the errors are related to some specific context or input data characteristics (such as specific code structure, coding style, etc.).

[0059] 2. Network structure optimization based on error classification Step 1: Identify bottlenecks in the model structure Based on the statistical results of the misclassification, the bottlenecks or weak links in the model structure are identified. For example, insufficient feature extraction: Some vulnerability type identification errors may be caused by insufficient model feature extraction, which cannot capture the key features of specific vulnerabilities.

[0060] Insufficient context understanding: The model makes mistakes when processing long sequences or complex dependencies, possibly due to limitations of the network layer (such as the attention mechanism or RNN structure).

[0061] Poor generalization ability: The model may overfit certain patterns, resulting in poor performance on unseen inputs.

[0062] Step 2: Specific methods for optimizing the model Specific network structure optimization methods are used for the identified bottlenecks: Enhance the feature extraction layer: Add feature channels to capture more dimensions of information. Add more levels of self-attention mechanisms to the model so that the model can better understand the global dependencies of the input data. Optimize the network hierarchy: If the model performs poorly when processing complex contexts, you can increase the number of layers or introduce deeper Transformer layers to enhance the model's contextual understanding ability. Design specialized processing modules for specific error types. For example, for boundary problems, you can introduce a hierarchy or preprocessing module that specifically handles boundary conditions into the model.

[0063] Adjust the training process: Introduce an error feedback loop so that the model pays special attention to error-prone cases during training. The robustness of the model can be enhanced by sample weighting (giving higher weights to error-prone samples) or adversarial training. Generate more training samples related to errors through data augmentation techniques to improve the model's ability to recognize these samples. Fine-tuning for specific domains: Fine-tune the model for specific domains and retrain it on specialized vulnerability datasets to improve the model's ability to recognize specific domains.

[0064] Step 3: Iterative optimization and verification Iterative adjustment: Based on the above optimization measures, repeatedly adjust the network structure and verify it on a new data set to observe whether the effect of optimization is significantly improved.

[0065] Feedback loop: Establish a continuous feedback mechanism to feed the results of each evaluation and optimization into the next round of model improvement, gradually reducing the error rate of the model.

[0066] Through detailed manual review of the model's preliminary output by the expert system, the focus is on its performance in the vulnerability identification task, including key indicators such as accuracy, recall, and F1 value. During the review process, the experts systematically marked and classified the model's erroneous output, analyzed the false positives and false negatives, and explored the root causes of these errors, such as insufficient semantic understanding or data set bias. According to the experts' specific improvement suggestions, the model was adjusted and optimized in many aspects: optimizing feature engineering to enhance the ability to extract vulnerability features, adjusting the model's hyperparameters such as learning rate and batch size to optimize performance indicators, and even redesigning the loss function to better adapt to the special needs of the vulnerability identification task. Subsequently, through multiple iterations and cross-validation, the effect of each improvement was verified to ensure the robustness and generalization ability of the model on different data subsets. At the same time, advanced explanatory techniques (such as SHAP value or LIME) were used to explain the model's decision-making process, helping experts to deeply understand the basis for the model's classification of vulnerabilities and enhance the acceptability and implementation effect of improvement suggestions. Finally, we established a continuous monitoring mechanism and feedback loop to regularly evaluate the performance of the model in actual applications, and continuously optimize and fine-tune the model based on actual feedback to ensure its continued effective application and superior performance in key tasks such as vulnerability identification. These comprehensive measures and fine-tuning enable the large language model to fully utilize the feedback from the expert system and continuously improve and optimize its application capabilities and accuracy in complex tasks.

[0067] S400: Detect binary components using the trained vulnerability identification large model to obtain a binary component vulnerability detection result report.

[0068] In an embodiment of the present invention, text information is trained to learn different forms of binary component vulnerability features. The vulnerability analyzer can output text information based on the prediction results of the large language model. The text information mainly includes the vulnerability type and vulnerability location in the binary component and the recommended solution to the vulnerability. An expert review system is set up in the system. When the expert obtains the analysis result of the vulnerability analyzer, he can conduct a manual review and feed the review result back to the large language model, and adjust the large language model according to the feedback. After obtaining the text information, the user can also correct the output of the large language model in the form of questions and answers, so as to help the user to repair the vulnerability.

Claims

1. A binary component vulnerability identification method based on a large language model, characterized in that: include: S100: Collects various vulnerability descriptions of IoT components, extracts data from them and constructs a text information dataset; S200: Segmentation of long text in the text information data set; S3 00: Input the constructed text information dataset into the large language model for training and optimization; S400: Detect binary components using the trained vulnerability identification large model to obtain a binary component vulnerability detection result report.

2. The binary component vulnerability identification method based on a large language model according to claim 1 is characterized in that: The step S100 includes: S101: Obtain binary components, open source components, and vulnerability text information of IoT components; S102: Decompile the binary component into intermediate code; For open source components, compile and then decompile to generate intermediate code; Preprocess the vulnerability text information, including deduplication, standardization, and associating it with components; S103: Input the representation vector of each paragraph and the global context vector into the global attention mechanism, and output the paragraph representation that integrates the global information.

3. The binary component vulnerability identification method based on a large language model according to claim 2 is characterized in that: The step S103 includes: Encode the paragraphs: Use the pre-trained language model to encode each paragraph, convert it into a set of representation vectors, assign global vectors to the key positions of each paragraph, and use these vectors together with the paragraph representation vector for subsequent attention calculations; Calculate cross-paragraph attention: Extract query, key, and value vectors from the representation vectors of the current paragraph and other paragraphs; then calculate the cross-paragraph attention weight by calculating the similarity between the query vector of the current paragraph and the key vector of other paragraphs; Integrate global information: Use these attention weights to perform weighted summation on the global context vector and the Value vectors of other paragraphs to obtain a paragraph representation that integrates global information; Information fusion and update: The obtained global context representation is fused with the local representation of the current paragraph to obtain an updated paragraph representation.

4. The binary component vulnerability identification method based on a large language model according to claim 1 is characterized in that: The step S300 includes: S301: preprocessing the text information in the training sample set; S302: Use the training sample set to train the large language model to obtain a binary component vulnerability recognition model. The model determines whether there is a vulnerability and the type of vulnerability based on the input features. S303: Use the expert review system to conduct manual review, optimize the model based on the evaluation results, and further improve the accuracy and performance of the model.

5. The binary component vulnerability identification method based on a large language model according to claim 4 is characterized in that: The step S301 includes: Preprocess the input text information through natural language processing tools; Embed the preprocessed text into continuous word vectors; For each word, the word position information is embedded into the word embedding vector. The position encoding is calculated based on the word position information in the text sequence and added to the word embedding vector as the input of the model.

6. The binary component vulnerability identification method based on a large language model according to claim 4 is characterized in that: In step S302, the large language model includes: The position encoding layer adds the position information of the input text to the embedding vector to help the model recognize the sequence order; A plurality of encoder layers connected in series, each of the encoder layers comprising a multi-head attention mechanism layer, a layer normalization and residual connection layer, a feedforward neural network, and a layer normalization and residual connection layer connected in series in sequence; A plurality of decoder layers connected in series, each of the decoder layers comprising a multi-head attention mechanism layer, a layer normalization and residual connection layer, a feedforward neural network, and a layer normalization and residual connection layer connected in series in sequence; The multi-head self-attention mechanism captures the dependencies between different parts of the input text through different attention heads; The feedforward neural network is used to perform nonlinear transformation on the hidden vector; The layer normalization and residual connection layers are used to accelerate model training and improve model stability.

7. The binary component vulnerability identification method based on a large language model according to claim 4 is characterized in that: The step S303 includes: Error classification and statistics; Network structure optimization based on error classification.

8. The binary component vulnerability identification method based on a large language model according to claim 7 is characterized in that: Error classification and statistics specifically include: Step 1: Collect error data: After evaluating the large language model, collect error data in the output of the large language model; Step 2: Manual classification: The expert team manually reviews the error output of the large language model and classifies it by error type; Step 3: Statistics and analysis: Statistical error frequency and error distribution are performed for each error type.

9. The binary component vulnerability identification method based on a large language model according to claim 7 is characterized in that: Network structure optimization based on error classification includes: Step 1: Identify bottlenecks in the large language model structure: Identify bottlenecks or weak links in the structure of large language models based on the statistics of misclassifications; Step 2: Optimize the model: Optimize the large language model based on the identified bottlenecks; Step 3: Iterative optimization and verification: Repeatedly adjust the network structure and verify it on new data sets to observe whether the optimized effect is improved.

10. The binary component vulnerability identification method based on a large language model according to claim 1, characterized in that: The step S200 includes: According to the maximum length limit of the model, the initial window size is set, and the window size and sliding step size are dynamically adjusted according to the text length; If the text is long, choose a larger window to capture more context; If the text is short, reduce the window size to reduce the amount of computation; The text is divided into multiple smaller windows. The content of each window includes the maximum number of tokens that the model can handle, and slides according to the set sliding step size to cover the entire text.

Images