Online Internet of Things equipment firmware identification method based on hierarchical multi-mode fusion

Abstract

The invention relates to a firmware identification method of IoT (Internet of Things) equipment, in particular to an online firmware identification method of the IoT equipment based on hierarchical multi-modal fusion, which comprises the following steps of: firstly, respectively extracting features from a DOM (Document Object Model) tree, text information and a function code in an embedded webpage of the IoT equipment, and performing embedded representation; then, a cross-modal fusion layer formed by a designed shallow cross-modal attention mechanism and deep cross-modal fusion is used, and the generated embedding is subjected to modal feature fusion from shallow to deep; and finally, realizing intra-class compactness and inter-class separation by using a designed HNMCL loss function, and improving the ability of the model to distinguish hard negative samples. According to the method, efficient classification of firmware versions is realized, negative samples difficult to distinguish can be focused, and the distinguishing capability of the model in a complex multi-modal task is enhanced. According to the method, the feature precision and the fusion depth are both considered, the firmware iteration page change is effectively adapted, and a solution is provided for multi-modal analysis and version identification.

Description

Technical Field

[0001] The present invention relates to a firmware identification method for an IoT device, and in particular to an online IoT device firmware identification method based on layered multimodal fusion. Background Art

[0002] In recent years, with the rapid development of the Internet of Things (IoT), a large number of IoT devices are connected to the cyberspace. Its rapid growth has also caused a series of security issues, because many devices are running outdated firmware with errors and vulnerabilities. These vulnerabilities are usually associated with specific firmware versions, so identifying the firmware version is crucial for device security.

[0003] Existing firmware version identification work can be divided into two categories: offline identification and online identification. Offline identification is usually based on a complete firmware image and is more suitable for in-depth analysis in the field of security research and vulnerability mining. Online identification has the advantages of low complexity and easy deployment, and is more suitable for real-time management and large-scale monitoring scenarios. However, there are several shortcomings in existing online identification methods. First, devices of different manufacturers and models use different firmware formats and version control mechanisms, which makes it extremely difficult to build a comprehensive and universal version fingerprint due to the high heterogeneity of firmware. Second, the device features (such as traffic, slogans, etc.) used by traditional device identification methods have no direct connection with the firmware. The granularity of traditional device identification is too coarse (only staying at the manufacturer, device type and model), and cannot cover the firmware version. At the same time, embedded web pages are usually composed of multiple types of modal information, including visual DOM structure, text content and function code. Manufacturers usually retain and inherit pages and codes during the firmware iteration process, resulting in highly similar modal information between different versions, which limits the expressiveness of single-modal features in capturing subtle differences. In addition, the widespread file reuse phenomenon exacerbates the complexity of version differentiation. The similarity of reused files of different versions is much higher than that of other files under the same version, which creates the problem of hard negative samples.

[0004] In summary, the defects of the existing technology mainly include: the heterogeneity of firmware makes it difficult to develop a general firmware identification method, the subjective bias introduced by static fingerprints does not conform to the firmware update rules, the unimodal information inhibits the firmware identification ability, and the file reuse leads to identification difficulties. It is difficult to cope with the above challenges using traditional machine learning tasks. Blindly selecting complex changes or representations for feature extraction often leads to time-consuming and subtle differences. The hand-crafted fusion strategy is not always compatible with the deep extracted features, which inevitably hinders the improvement of fusion performance. In order to adapt to the above-mentioned manufacturer diversity design and fine-grained text differentiation tasks, the model should not only have a strong difference capture ability in feature extraction and fusion, but also have the generalization ability to adapt to different design patterns. These defects show that there is an urgent need to develop more accurate and efficient firmware version identification technology to improve the accuracy and reliability of IoT device security assessment. Summary of the invention

[0005] In order to solve the problems of vendor diversity design and coarse-grained recognition of firmware, the present invention provides an online IoT device firmware recognition method based on hierarchical multimodal fusion.

[0006] The present invention is implemented by adopting the following technical scheme: an online IoT device firmware identification method based on hierarchical multimodal fusion, a hierarchical multimodal attention network is proposed, the hierarchical multimodal attention network includes a multimodal feature extraction module, a multimodal feature fusion module and a version classification recognition module; the multimodal feature extraction module further includes a structural modal feature extraction module, a textual modal feature extraction module and a sequence modal feature extraction module, the structural modal feature extraction module, the textual modal feature extraction module and the sequence modal feature extraction module respectively extract fine-grained features of embedded web pages and generate embedded representations; the multimodal feature fusion module includes an initial feature processing layer and a cross-modal fusion layer, the embedded representations of three modalities are input into the initial feature processing layer, thereby obtaining three modal preliminary features, and the cross-modal fusion layer realizes the fusion of modal preliminary features; the version classification recognition module combines a loss function and realizes efficient classification of firmware versions through a fully connected classifier.

[0007] In the above-mentioned online IoT device firmware recognition method based on hierarchical multimodal fusion, the structural modal feature extraction module uses the Node2Vec algorithm to extract the structural modal features of the embedded web page and generate the embedded representation W DOM ; The text modality feature extraction module uses the FastText algorithm to extract the text modality features of embedded web pages and generate an embedded representation W Text ; The sequential modal feature extraction module uses the CodeLlama algorithm to extract the sequential modal features of embedded web pages and generate an embedded representation W Seq .

[0008] In the above-mentioned online IoT device firmware recognition method based on hierarchical multimodal fusion, the initial feature processing layer includes a one-dimensional convolution layer, a batch normalization layer, a sigmoid activation function and a maximum pooling layer to achieve preliminary feature extraction and dimensionality unification, thereby obtaining preliminary feature representations of the three modalities E DOM 、E Text and E Seq .

[0009] The above-mentioned online IoT device firmware identification method based on hierarchical multimodal fusion deepens the preliminary features of the fusion modality layer by layer by cascading and stacking four cross-modal fusion layers CMFL. Each CMFL includes a pair of shallow cross-modal attention mechanisms SCMA and a deep cross-modal fusion mechanism DCMF; E DOM 、E Text As the input of the first cross-modal fusion layer CMFL, the output of the first cross-modal fusion layer CMFL and E Text As the input of the second cross-modal fusion layer CMFL, the output of the second cross-modal fusion layer CMFL and E Seq As the input of the third cross-modal fusion layer CMFL, the output of the third cross-modal fusion layer CMFL and E Seq As the input of the fourth cross-modal fusion layer CMFL.

[0010] The above-mentioned online IoT device firmware identification method based on hierarchical multimodal fusion, the loss function includes HNMCL loss function and cross entropy loss function HNMCL loss function is used to achieve intra-class compactness and inter-class separation. Anchor samples x are selected in the embedded web page. i and its positive sample pair x j , select the negative sample x that is most similar to the anchor web page file k* , in, Contains all i For web page files with the same name but different categories, the HNMCL loss function can be expressed as: Where θ is the cosine similarity between two sample embeddings, f(·) represents the fused features output by the multimodal feature fusion module, τ is the temperature parameter, N is the number of samples, and the total loss is defined as follows A hyperparameter λ is set to balance the HNMCL loss and the cross entropy loss to calculate the total loss.

[0011] This paper proposes a firmware version identification framework for IoT devices named HMAN, which introduces machine learning technology into the field of firmware identification for the first time. Compared with other technologies, it has the following significant advantages and effects:

[0012] 1. Improved multimodal processing capabilities for embedded web pages. Traditional technologies usually focus on single modal features and cannot fully reflect the complexity of embedded web pages. This method extracts structural, textual, and sequence modal features through independent encoders, and fully captures DOM tree relationships, textual semantics, and code logic. The joint processing of the three modalities significantly improves the integrity and precision of feature representation.

[0013] 2. Innovative cross-modal feature fusion mechanism. Compared with general linear or shallow fusion methods, this technology gradually extracts the interaction relationship and high-order characteristics between modalities through the combination of shallow cross-modal attention mechanism (SCMA) and deep fusion (DCMF). This hierarchical fusion method effectively captures the nonlinear relationship between modal features and greatly improves the model's ability to handle complex feature interactions.

[0014] 3. Adaptable encoder design. The model encoder does not need fine-tuning and can be replaced with the latest pre-trained model to adapt to the evolving technology ecosystem. Compared with the fixed feature extraction framework, this method has significant advantages in flexibility and scalability.

[0015] 4. Joint optimization strategy to comprehensively improve classification results. Compared with the technology that only uses a single optimization objective, this method combines HNMCL and cross entropy loss, and through hyperparameter balancing, comprehensively optimizes the learning effect of the model, and ultimately shows higher robustness and accuracy in the multimodal analysis of embedded web pages and firmware version classification tasks.

[0016] In the experiment, HMAN significantly outperforms the existing most advanced solutions in firmware identification. The present invention collected 1,000 firmware images and obtained 444,195 valid embedded web interface files as test data. It can be seen that the firmware identification accuracy of the HMAN framework is extremely superior and can significantly enhance the discrimination of features extracted from web files. At the same time, a generalization test was performed to verify the effectiveness of the present invention in practical applications. In the multimodal fusion process, the performance of the HMAN framework surpassed all baselines, and CMFL outperformed other single-modal models and fingerprint schemes in performance, indicating that the introduction of multimodal features significantly improved the firmware recognition performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 This is an overview diagram of HMAN. DETAILED DESCRIPTION

[0018] The present invention introduces a machine learning model in the field of firmware recognition for the first time, proposes a Transformer-based hierarchical multimodal attention network (Hierarchical Multimodal Attention Network, HMAN) to perform deep fusion feature extraction on embedded pages, and designs a hard negative mining contrast loss function (Hard Negative Mining Contrastive Loss, HNMCL) to optimize the training of the model to solve the hard sample problem caused by the page reuse technology used in firmware iteration. Specifically, the present invention first extracts features from the DOM tree (structural modality), text information (text modality) and function code (sequence modality) in the embedded web page of the Internet of Things device and embeds them for representation. Then, the cross-modal fusion layer (Cross-Modal Fusion Layer, CMFL) composed of the designed shallow cross-modal attention mechanism (Shallow Cross-Modal Attention, SCMA) and deep cross-modal fusion (Deep Cross-Modal Fusion, DCMF) is used to realize the fusion of modal features from shallow to deep in the above-generated embedding. Finally, the designed HNMCL loss function is used to achieve intra-class compactness and inter-class separation, and improve the model's ability to distinguish hard negative samples. The present invention is described in detail below with reference to the accompanying drawings.

[0019] The present invention includes three main modules: a multimodal feature extraction module, a multimodal feature fusion module and a version classification and recognition module. Figure 1 shown.

[0020] 1. The multimodal feature extraction module mainly contains different pre-trained encoders for three modal information, which are used to extract fine-grained features of embedded web pages and generate effective embedding representations. These encoders are not fine-tuned during training, so they can be replaced with the latest pre-trained models. It mainly consists of the following three modules: structural modal feature extraction module, textual modal feature extraction module and sequence modal feature extraction module.

[0021] (1) The structural modal feature extraction module mainly processes the DOM tree of the web page file. The complex relationship structure between the nodes in the DOM tree can reflect the hierarchy and semantic organization of the web page. In order to retain the structural similarity between nodes, Node2Vec random walk is used to sample between nodes, and the relationship between nodes and neighbor distribution information are combined to generate an effectively preserved structural modal embedding W. DOM .

[0022] (2) The text modality feature extraction module mainly focuses on explicit information such as device model, production date, and comments made by developers in the code. To address the challenge of limited text content that can be extracted without using a password to log in, TF-IDF has the advantage of being sensitive to the amount of information contained in a word. Therefore, we generate the text modality embedding W by using TF-IDF weighted FastText. Text , which can improve the discrimination of word importance in embedding.

[0023] (3) The sequence modal feature extraction module processes the function code, which not only has text characteristics but also contains significant temporal and dependency structures. The pre-trained CodeLlama is used to capture the syntax, logic and temporal dependency of the code snippet, and finally generates the sequence modal embedding W Seq .

[0024] 2. The multimodal feature fusion module mainly includes two steps: the initial feature processing layer and the cross-modal fusion layer. First, the embedding representations extracted by the three pre-trained encoders are preliminarily processed, and then the modal feature fusion is realized from shallow to deep. The initial feature processing layer includes a one-dimensional convolution layer with a convolution kernel size of 3, a batch normalization layer, a sigmoid activation function, and a maximum pooling layer to achieve initial feature extraction and dimensional unification, thereby obtaining the initial feature representation E of the three modalities. DOM 、E Text and E Seq . Then, the fusion features are deepened layer by layer through 4 cascaded stacks of multiple cross-modal fusion layers (CMFL), where the output of each CMFL is one of the inputs of the next CMFL, and each CMFL contains a pair of shallow cross-modal attention mechanisms (SCMA) and a deep cross-modal fusion mechanism (DCMF). The design of SCMA and DCMF still follows the feature fusion strategy of "from shallow to deep". SCMA captures the preliminary feature complementary relationship between different modalities through a shallow attention mechanism. DCMF extracts deep interaction features between modalities through more complex bilinear pooling and high-order attention mechanisms. The combination of the two achieves more detailed and nonlinear feature fusion. The calculation process of the four CMFLs is shown below. The calculation process of the first CMFL is as follows:

[0025]

[0026] The second CMFL calculation process is as follows:

[0027]

[0028] The third CMFL calculation process is as follows:

[0029]

[0030] The fourth CMFL calculation process is as follows:

[0031]

[0032] in, and Represents the shallow interaction information between different modalities in the i-th CMFL layer; It is the high-order correlation information of deep cross-modal fusion features. SCMA and DCMF are introduced in detail below.

[0033] (1) Shallow Cross-Modal Attention Mechanism (SCMA)

[0034] In web pages, text modality can supplement the semantic information and specific content of each element in the DOM, while sequence modality supplements the presentation level of text information by providing interaction and dynamic changes for text modality. SCMA introduces a cross-modal attention mechanism to extract local interaction information of multiple modal features of web pages to construct preliminary cross-modal complementary information. Specifically, the feature representation of each modality is first calculated separately through the Transformer's multi-head self-attention mechanism (MSA):

[0035]

[0036] Z i =HultiHead(E i )=Concat(head1,head2,...,head h )W O ,

[0037] Among them, W j Q , W j V and W O is the weight matrix, E i represents the feature representation of the i-th mode, is a scaling factor used to prevent numerical instability caused by excessive dot products, Z i Represents the internal association information of the modality. Subsequently, a cross-modal attention mechanism is adopted to make one modality interact in the context of another modality to obtain the common information of the modalities. The present invention takes the text modality and DOM modality as examples to calculate the cross-modal interaction process:

[0038] Z cross-Text =MSA(Z Text ,Z DOM ,Z DOM ),

[0039] Z co-srMDOD =MSA(ZDOM ,Z Text ,Z Text ).

[0040] In order to further integrate the interactive information of the two modalities, the residual connection is used to transform the interactive feature Z cross-DO and Z cross-Text And their respective self-attention features are injected into the fusion layer to obtain complementary information after preliminary fusion. The process is as follows:

[0041] Z combined-S =Linear(Concat(Z cross-Text ,Z cross-DOM )),

[0042] Z S =LayerNorm(Z combined-S +Z DOM +Z Text ),

[0043]

[0044] Among them, the fully connected feedforward network consists of two linear transformations, and the ReLU activation function is used in the middle to improve the nonlinear expression ability. It is the preliminary cross-modal complementary information obtained after fusion, which is used as the output of SCMA to lay the foundation for subsequent deep fusion.

[0045] (2) Deep Cross-Modal Fusion Mechanism (DCMF)

[0046] Although SCMA can extract complementary information between modalities, it fails to fully capture the high-order nonlinear interaction between modalities. This relationship is crucial to understanding the subtle changes in the content of the firmware page, which makes it difficult for the shallow features of SCMA to capture more complex dependencies between modalities. DCMF can make up for the shortcomings of SCMA to capture dynamic characteristics and behaviors in web pages and enhance the perception of multimodal high-order pattern differences. Specifically, the two SCMA output values ​​are first processed using bilinear pooling to obtain K and V, and then the nonlinear interaction between the two modalities is captured. The specific process is as follows:

[0047]

[0048] in, represents the outer product operation, W b is the dimension reduction matrix, and f(·) is the sigmoid nonlinear activation function. Then, Z b The output of the initial SCMA and Perform cross-modal attention fusion again to capture high-order correlation information between modalities. The process is as follows:

[0049]

[0050] Next, the residual connection is used to combine the fused high-order correlation information Z cross-S1 and Z cross-S2 Inject it into the complementary features of SCMA output for deep modal fusion, and then pass through the same fully connected feedforward network to obtain the final deep feature fusion representation. The process is as follows:

[0051] Z combined-D =Linear(Concat(Z cross-S1 ,Z cross-S2 )),

[0052]

[0053]

[0054] 3. Version classification identification module

[0055] Iteratively updated firmware under the same model generally uses page reuse, which results in the similarity of web pages with the same name being much greater than other web pages of the same type. In order to further improve the model's ability to represent files with the same name in different versions, the present invention designs the HNMCL loss function to achieve intra-class compactness and inter-class separation. Specifically, the anchor sample x is selected in the embedded web page. i and its positive sample pair x j , select the negative sample x that is most similar to the anchor web page file k* :

[0056]

[0057] in, Contains all i Web page files with the same name but different categories. Selecting the InfoNCE estimator makes the model focus more on negative samples that are difficult to distinguish, and improves the model's discrimination ability in complex multimodal tasks. The HNMCL loss can be expressed as:

[0058]

[0059] in, is the cosine similarity between two sample embeddings, f(·) represents the HMAN network, and τ is the temperature parameter used to adjust the smoothness of the similarity.

[0060] 4. Training and Optimization

[0061] In order to convert the output of the multimodal feature fusion module into category probability and realize the classification of firmware versions, the present invention adopts a fully connected layer classifier (with a RELU activation function between two fully connected layers) and HNMCL joint training. At the same time, the cross entropy loss function is used to calculate the gap between the model prediction value and the true label, which is defined as follows:

[0062]

[0063] Where N is the number of samples, C is the number of categories, and y i,c is the true label of the i-th sample belonging to the c-th class, p i,c is the probability that the model predicts that the i-th sample belongs to the c-th class. Next, a hyperparameter λ is set to balance HNMCL and cross entropy loss to calculate the total loss, which is defined as follows:

[0064]

[0065] In summary, the present invention provides a firmware version identification method for embedded web IoT devices, and its workflow includes: first, a pre-trained encoder is used in a multimodal feature extraction module to process three modal information, a structural modal module uses Node2Vec to extract DOM tree embedding, retains the web page hierarchy and semantic organization, a text modal module combines TF-IDF and FastText to extract device information and improve the discrimination of text embedding, and a sequence modal module captures the grammatical logic and temporal dependency of function codes based on CodeLlama to generate sequence embedding; then, the three modal dimensions are unified through initial feature processing in a multimodal feature fusion module, and then features are fused layer by layer through a cross-modal fusion layer (CMFL). Each CMFL contains a shallow cross-modal attention mechanism (SCMA) and a deep cross-modal fusion (DCMF). SCMA extracts preliminary complementary information, and DCMF uses bilinear pooling and high-order attention mechanisms to capture deep modal interaction features and improve the fusion effect; then, in the version classification and recognition module, the HNMCL loss function and cross entropy loss are combined to achieve efficient classification of firmware versions through a fully connected classifier, which can focus on difficult-to-distinguish negative samples and enhance the model's discrimination ability in complex multimodal tasks. This method takes into account both feature accuracy and fusion depth, effectively adapts to changes in firmware iteration pages, and provides a solution for multimodal analysis and version recognition.

Claims

1. An online IoT device firmware identification method based on hierarchical multimodal fusion, characterized in that: A hierarchical multimodal attention network is proposed, which includes a multimodal feature extraction module, a multimodal feature fusion module and a version classification and recognition module; the multimodal feature extraction module includes a structural modal feature extraction module, a textual modal feature extraction module and a sequence modal feature extraction module, which respectively extract fine-grained features of embedded web pages and generate embedded representations; the multimodal feature fusion module includes an initial feature processing layer and a cross-modal fusion layer, the embedded representations of the three modalities are input into the initial feature processing layer to obtain preliminary features of the three modalities, and the cross-modal fusion layer realizes the fusion of preliminary modal features; the version classification and recognition module combines the loss function and realizes efficient classification of firmware versions through a fully connected classifier.

2. According to claim 1, an online IoT device firmware identification method based on hierarchical multimodal fusion is characterized in that: The structural modal feature extraction module uses the Node2Vec algorithm to extract the structural modal features of the embedded web page and generate the embedded representation W DOM ; The text modality feature extraction module uses the FastText algorithm to extract the text modality features of embedded web pages and generate an embedded representation W Text ; The sequential modal feature extraction module uses the CodeLlama algorithm to extract the sequential modal features of embedded web pages and generate an embedded representation W Seq .

3. According to claim 2, an online IoT device firmware identification method based on hierarchical multimodal fusion is characterized in that: The initial feature processing layer includes a one-dimensional convolution layer, a batch normalization layer, a sigmoid activation function, and a maximum pooling layer to achieve preliminary feature extraction and dimension unification, thereby obtaining the preliminary feature representation E of the three modalities. DOM 、E Text and E Seq .

4. The method for online IoT device firmware identification based on hierarchical multimodal fusion according to claim 3 is characterized in that: By cascading and stacking four cross-modal fusion layers CMFL, the preliminary features of the fusion modality are gradually deepened. Each CMFL includes a pair of shallow cross-modal attention mechanisms SCMA and a deep cross-modal fusion mechanism DCMF; DOM 、E Text As the input of the first cross-modal fusion layer CMFL, the output of the first cross-modal fusion layer CMFL and E TexT As the input of the second cross-modal fusion layer CMFL, the output of the second cross-modal fusion layer CMFL and E Seq As the input of the third cross-modal fusion layer CMFL, the output of the third cross-modal fusion layer CMFL and E Seq As the input of the fourth cross-modal fusion layer CMFL.

5. The method for online IoT device firmware identification based on hierarchical multimodal fusion according to claim 4 is characterized in that: The loss function includes HNMCL loss function and cross entropy loss function HNMCL loss function is used to achieve intra-class compactness and inter-class separation. Anchor samples x are selected in the embedded web page. I and its positive sample pair x j , select the negative sample x that is most similar to the anchor web page file k* , in, Contains all i For web page files with the same name but different categories, the HNMCL loss function can be expressed as: Where θ is the cosine similarity between two sample embeddings, f(·) represents the fused features output by the multimodal feature fusion module, τ is the temperature parameter, N is the number of samples, and the total loss is defined as follows A hyperparameter λ is set to balance the HNMCL loss and the cross entropy loss to calculate the total loss.

Images