Iot lightweight rule detection method and system based on multi-granularity attention distillation

Abstract

A lightweight rule detection method and system for IoT based on multi-granularity attention distillation, relating to the field of IoT security, addresses the problems of existing high-performance rule detection models having insufficient detection accuracy and large size, making them unsuitable for deployment on resource-constrained edge devices. The method includes: constructing and training a gated attention-enhanced rule detection model and a multi-granularity attention distillation mechanism; transferring the multi-layered attention knowledge learned by the gated attention-enhanced rule detection model during security judgment to an ALBERT student model, which is then deployed on IoT edge devices; when the IoT edge device receives updated TAP rules, the student model performs real-time security risk detection and outputs the risk category; the detection results of the lightweight ALBERT student model and the original rule text are input into a structured prompt template combined with a large language model to generate a natural language explanation containing the risk category, potential harm, and modification suggestions.

Description

Technical Field

[0001] This invention relates to the field of Internet of Things (IoT) security technology, specifically to a lightweight rule detection method and system for IoT based on multi-granularity attention distillation. Background Technology

[0002] With the rapid development of Internet of Things (IoT) technology, smart home platforms such as IFTTT and SmartThings widely adopt the "IF-THEN" trigger-action programming (TAP) rule, enabling users to achieve automated linkage across devices and services without programming experience, such as "if the door lock is opened, then turn on the security camera".

[0003] However, this low-barrier-to-entry automation method introduces serious security and privacy risks. First, TAP rules may suffer from "excessive privilege," allowing attackers to exploit seemingly harmless rules to steal complete control of devices. Second, many user rules pose a semantic risk of privacy breaches or damage to the physical / network environment, and ordinary users often fail to notice such semantic ambiguities and logical vulnerabilities.

[0004] Existing detection methods for TAP rule security are mainly divided into two categories: 1. Static analysis and information flow control-based methods: These methods track information flow through preset strategies or lattice models and can identify simple privacy leaks, but they are difficult to handle complex semantic dependencies in user-defined natural language rules.

[0005] 2. Deep Learning and Natural Language Processing (NLP) Based Approaches: These methods utilize pre-trained large language models such as BERT or heterogeneous graph neural networks (GNNs) to capture the deep semantics of rules. However, these high-performance models have a huge number of parameters and high computational overhead, making them difficult to deploy for real-time inference on IoT edge devices (such as smart home gateways) where computation, memory, and power consumption are strictly limited. Furthermore, existing methods are mostly "black box" judgments, only outputting classification results and lacking intuitive risk explanations, making it difficult for users to understand potential vulnerabilities and correct configurations.

[0006] Therefore, the main shortcomings of existing technologies are: high-performance rule detection models have insufficient detection accuracy and are bulky, making them unsuitable for deployment on resource-constrained edge devices; while lightweight models struggle to maintain their ability to distinguish complex semantics and long-tailed risk samples; and they lack user-friendly and interpretable output. Summary of the Invention

[0007] This invention addresses the shortcomings of existing high-performance rule detection models, such as insufficient detection accuracy, large size, and inability to deploy on resource-constrained edge devices; lightweight models struggle to maintain the ability to discriminate complex semantics and long-tailed risk samples; and the lack of user-friendly and interpretable output. To resolve these issues, this invention proposes a lightweight rule detection method and system for the Internet of Things (IoT) based on multi-granularity attention distillation. The invention achieves these technical problems through the following technical solutions: Option 1: This invention proposes a lightweight rule detection method for the Internet of Things based on multi-granularity attention distillation, the method comprising the following steps: Step 1: Build and train a gated attention-enhanced rule detection model, which takes user-configured TAP rule text as input and outputs the corresponding security risk category prediction and internal attention features. Step 2: Construct a multi-granularity attention distillation mechanism to transfer the multi-level attention knowledge learned by the gated attention enhancement rule detection model constructed in Step 1 during the security discrimination process to the ALBERT student model; Step 3: Deploy the lightweight ALBERT student model trained in Step 2 to an IoT edge device. When the IoT edge device receives updated TAP rules, the student model performs real-time security risk detection and outputs the risk category. Step 4: Input the detection results of the lightweight ALBERT student model described in Step 3 and the original rule text into the structured prompt template combined with the large language model to generate a natural language explanation containing risk category, potential harm and modification suggestions, thus completing the lightweight rule detection for IoT based on multi-granularity attention distillation.

[0008] Furthermore, a preferred embodiment is provided, wherein the gating attention enhancement rule detection model described in step 1 includes: The feature encoding layer is used to convert the input TAP rule text into a sequence of word vectors rich in contextual semantics using a pre-trained BERT model, and to extract the attention output matrix of BERT. A bidirectional temporal modeling layer is used to input the word vector sequence into a bidirectional long short-term memory network to extract bidirectional temporal features and capture the time-dimensional dependency between triggering events and response actions. The self-attention and cross-attention fusion layer is used to fuse the global semantic features output by BERT with the temporal features output by the self-attention mechanism to form a fused feature representation. A gated attention mechanism layer is used to dynamically suppress noise and enhance key semantics.

[0009] Furthermore, a preferred embodiment is provided, wherein the gated attention mechanism layer uses the following method to dynamically suppress noise and enhance key semantics: The fused feature representations are linearly projected to generate a query vector Q, a key vector K, and a value vector V. The initial attention output vector is obtained by scaling the dot product attention calculation. The query vector Q is concatenated with the initial attention output vector and sent to the end of the multilayer perceptron (MLP) network to learn a gating vector g. The gating vector g is multiplied element-wise with the initial attention output vector to obtain the gated context representation. A residual gating fusion strategy is adopted to add the gated modulated context representation to the original query vector Q to obtain the final output feature.

[0010] Furthermore, a preferred embodiment is provided, wherein the construction of the multi-granularity attention distillation mechanism in step 2 includes: Global semantic distillation is used to calculate the KL divergence loss between the attention distributions of the CLS tags of the teacher model and the student model at each position of the input sequence. Local key feature distillation is used to aggregate all attention head weights of the teacher model, calculate the average attention score for each input label, dynamically select the top K key labels with the highest scores in each training sample, and finally calculate the KL divergence loss between the attention distributions of the teacher and student models on the key labels. Microscopic multi-head attention distillation is used to flatten the complete attention matrix of each attention head of the teacher model and the student model, and calculate the average KL divergence loss of both over all heads. Hard label distillation is used to calculate the loss between the student model's predictions and the true labels using a weighted cross-entropy loss function.

[0011] Furthermore, in a preferred embodiment, the multi-particle size attention distillation loss in step 2 is:

[0012] in, Let B be the number of heads in multi-head attention, and B be the number of samples. The top k high-weight attention positions are selected for each sample. For the SLC loss of the h-th attention head, The loss for the j-th top-k selected position, Let be the head loss of the h-th attention head.

[0013] Furthermore, in a preferred embodiment, when the weighted cross-entropy loss function is used to calculate the hard label distillation, the class weights are set according to the training set distribution as: w=[0.8,1.2,2.5,3.0].

[0014] Furthermore, a preferred embodiment is provided in which the size of the lightweight ALBERT student model described in step 3 is 44.5 MB.

[0015] Option 2: A lightweight rule detection system for the Internet of Things based on multi-granularity attention distillation, the system comprising: The Gated Attention Enhancement Rule Detection Model Module is used to build and train the Gated Attention Enhancement Rule Detection Model. It receives user-configured TAP rule text as input and outputs the corresponding security risk category prediction and internal attention features. The multi-granularity attention distillation mechanism module is used to construct a multi-granularity attention distillation mechanism, which transfers the multi-level attention knowledge learned by the gated attention enhancement rule detection model in the security discrimination process constructed by the gated attention enhancement rule detection model module to the ALBERT student model. The risk detection module is used to deploy the lightweight ALBERT student model trained by the multi-granularity attention distillation mechanism module in IoT edge devices. When the IoT edge device receives updated TAP rules, the student model performs real-time security risk detection and outputs the risk category. The rule detection module is used to input the detection results of the lightweight ALBERT student model described in the risk detection module and the original rule text into a structured prompt template combined with a large language model to generate a natural language explanation containing risk categories, potential hazards and modification suggestions, thus completing the lightweight rule detection of IoT based on multi-granularity attention distillation.

[0016] Option 3: A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the method described in Option 1.

[0017] Option 4: A computer device, including a memory and a processor, wherein the memory stores a computer program, and when the processor runs the computer program stored in the memory, the processor executes the method described in Option 1.

[0018] The advantages of this invention are: The lightweight rule detection method for IoT based on multi-granularity attention distillation described in this invention, and the teacher model GA-RDM constructed in the system, dynamically suppress noise in the rule text by introducing a gated attention mechanism, ensuring the capture of key semantics. Experimental results show that on the four-class threat detection task, GA-RDM achieves an accuracy of 88.36%, which is superior to baseline models such as SAFE-TAP (87.54%) and BERT (79.72%). Especially in difficult categories such as physical environmental damage, its F1 score is improved by 0.5% to 5.75% compared to the baseline model.

[0019] The lightweight rule detection method and system for IoT based on multi-granularity attention distillation described in this invention significantly outperforms similar models without distillation in terms of detection performance through the multi-granularity attention distillation mechanism. Its accuracy increases from 77.37% to 81.65%, surpassing the baseline BERT model. Simultaneously, the model size is drastically compressed from 442MB for the teacher model (based on BERT-base) to 44.5MB, a compression of nearly ten times, enabling it to run smoothly on resource-constrained IoT edge devices and achieve real-time detection.

[0020] The lightweight rule detection method for IoT based on multi-granularity attention distillation described in this invention utilizes the MGAD distillation mechanism at three granularities—global, local, and micro—to align attention, enabling the student model to effectively inherit the teacher model's discrimination logic for harmful rules (such as physical damage) with a small sample size. After distillation, the student's recall rate for category 2 (physical environmental damage) increased from 37.92% to 71.46%, and the F1 score increased from 52.31% to 76.01%, significantly reducing the missed detections of high-risk rules.

[0021] The lightweight rule detection method and system for IoT based on multi-granularity attention distillation described in this invention combines detection results with LLM's structured prompting strategy. The system can transform numerical classification results into natural language reports containing risk reasoning and specific optimization suggestions, helping non-professional users understand the root causes of risks and safely correct rule configurations, thus making up for the shortcomings of existing black-box detection solutions.

[0022] This invention is also applicable to applications such as a lightweight deep learning method and system for detecting potential hazards of Internet of Things trigger-action programming (TAP) rules. Attached Figure Description

[0023] Figure 1 This is a flowchart illustrating the lightweight rule detection method for the Internet of Things based on multi-granularity attention distillation as described in Implementation Method 1.

[0024] Figure 2This is a schematic diagram of the structure of the gated attention enhancement rule detection model described in Implementation Method 1.

[0025] Figure 3 This is a schematic diagram illustrating the principle of the multi-granularity attention distillation mechanism described in Implementation Method 1. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them.

[0027] Implementation Method 1, see [link] Figures 1 to 3 This embodiment proposes a lightweight rule detection method for the Internet of Things (IoT) based on multi-granularity attention distillation. The method constructs a closed-loop technology of "high-performance teacher model—knowledge distillation—lightweight student model—interpretable output," compressing the model size by nearly ten times while maintaining high detection accuracy. This allows for deployment on IoT edge devices and the generation of user-understandable risk warnings. The method specifically includes the following steps: Step 1: Construct and train a Gate-Attention-enhanced Rule Detection Model (GA-RDM) as the teacher model. This model receives user-configured TAP rule text as input and outputs the corresponding security risk category prediction and internal attention features.

[0028] Step 2: Construct a multi-granularity attention distillation (MGAD) mechanism to transfer the multi-level attention knowledge learned by the teacher model in the security discrimination process described in Step 1 to an ALBERT student model with a very small number of parameters, which is suitable for edge deployment.

[0029] Step 3: Deploy the trained lightweight ALBERT student model on IoT edge devices. When a device receives a new TAP rule, the student model performs real-time security risk detection and outputs the risk category.

[0030] Step 4: Input the detection results (risk category) of the student model and the original rule text into a structured prompt template that combines a large language model (LLM) to generate a natural language explanation containing risk category, potential harm and modification suggestions, and present it to the user.

[0031] I. The gated attention enhancement rule detection model (teacher model GA-RDM) described in this embodiment is as follows: To address the problem that conventional models struggle to dynamically assess the importance of different triggering conditions, this invention designs a GA-RDM model, whose internal processing flow is as follows: Feature encoding layer: Using a pre-trained BERT model, the input TAP rule text (containing trigger conditions and action descriptions) is converted into a sequence of word vectors rich in contextual semantics, and the attention output matrix of BERT is extracted.

[0032]

[0033]

[0034] in, Given a sequence of text as input, Let h be the query matrix of the h-th attention head. Let h be the key matrix of the h-th attention head. The dimension of a single attention head.

[0035] Bidirectional temporal modeling layer: The word vector sequence is input into a bidirectional long short-term memory network (BiLSTM) to extract bidirectional temporal features and capture the time-dimensional dependency between triggering events and response actions.

[0036]

[0037]

[0038]

[0039] in, The output context hidden state. For the learnable weight vector of the attention layer, Let be the attention weight coefficient for the t-th token. is the output hidden state of the bidirectional LSTM, C is the global context vector obtained by attention weighting, and L is the total length of the tokens in the input text sequence.

[0040] Self-attention and cross-attention fusion layer: First, a self-attention mechanism is applied to the output of BiLSTM to highlight key temporal information in the sequence. Then, a cross-attention mechanism is used to fuse the global semantic features output by BERT with the temporal features output by the self-attention mechanism to form a comprehensive feature representation.

[0041]

[0042]

[0043]

[0044] in, To map BERT hidden states to cross-attention Q, To map the global vector C to the key K, To map the global vector C to the value V, For the query matrix of cross-attention, The key vector for cross-attention. Let d be the value vector of the cross-attention, and d be the dimension of the attention head. This is the cross-attention weight matrix. This represents the output of cross-attention.

[0045] Gated attention mechanism layer: This layer is crucial to the model, used to dynamically suppress noise and enhance key semantics. Specifically, it operates as follows: The fused features are linearly projected to generate a query vector (Q), a key vector (K), and a value vector (V).

[0046]

[0047] By scaling the dot product attention calculation, an initial attention output vector is obtained, which reflects the degree of correlation between the trigger and the action semantics.

[0048]

[0049]

[0050] in, Output feature sequences for the self-attention module. V is the self-attention weight matrix, and V is the Value matrix.

[0051] The original query vector Q is concatenated with the initial attention output vector and fed into a multilayer perceptron network (MLP) to learn a gating vector g. This gating vector represents the importance weight of each semantic channel in the current context.

[0052]

[0053]

[0054] The gating vector g is multiplied element-wise with the initial attention output vector to obtain the gated and modulated context representation. This process selectively amplifies feature channels beneficial for security discrimination while suppressing noise channels.

[0055]

[0056] A residual gating fusion strategy is adopted to add the gated modulated context representation to the original query vector Q to obtain the final output feature, and the model prediction is based on this feature.

[0057]

[0058]

[0059]

[0060] in, For learnable linear weight matrices, To output a feature sequence for self-attention, It is a fully connected linear layer.

[0061] II. The multi-granularity attention distillation mechanism (MGAD) described in this embodiment is as follows: To address the model compression issue, this invention proposes the MGAD mechanism, which transfers knowledge from the teacher model (GA-RDM) to the ALBERT student model. The core of this mechanism is to force the student's attention distribution to approximate the teacher's attention distribution at different granularities, rather than simply mimicking the final output. The total loss function is:

[0062] in, For attention branch loss, for The corresponding loss weight hyperparameter, The weight hyperparameters for the main task loss. The main task's hard label loss.

[0063] The total loss from attention distillation consists of the following three levels of loss:

[0064] in, Let B be the number of heads in multi-head attention, and let B be the number of samples. The top k high-weight attention positions are selected for each sample. For the SLC loss of the h-th attention head, The loss for the j-th top-k selected position, Let be the head loss of the h-th attention head.

[0065] Global Semantic Distillation: Calculate the KL divergence loss between the attention distributions of the teacher and student models at various positions in the input sequence using their CLS tags (representing global semantics). This aims to allow the student model to inherit the teacher model's ability to judge the overall safety intent of the rules.

[0066]

[0067] in, The original attention score matrix for the k-th head of the student model. This is the original attention score matrix of the k-th head of the teacher model.

[0068] Local key feature distillation: First, aggregate all attention head weights of the teacher model and calculate the average attention score for each input label. Then, dynamically select the top K key labels with the highest scores in each training sample. Finally, calculate the KL divergence loss between the attention distributions of the teacher and student models on these key labels. This forces the student model to focus on a few core fields that play a decisive role in risk assessment (such as action verbs and key equipment), ignoring irrelevant noise.

[0069]

[0070]

[0071]

[0072] in, For the student model, the attention score on the top-k position subset selected for the i-th sample. This represents the attention score of the teacher model on the top-k position subset.

[0073] Micro-level multi-head attention distillation: Flatten the complete attention matrix of each attention head in both the teacher and student models, and calculate the average KL divergence loss across all heads. This aims to preserve the diverse structural knowledge in multi-head attention mechanisms that captures different syntactic and semantic relationships (cross-field dependencies).

[0074]

[0075]

[0076] in, Let h be the flattened vector of the h-th attention head in the student model. This is the flattened vector of the h-th attention head in the teacher model.

[0077] Hard label distillation: A weighted cross-entropy loss function is used to calculate the loss between the student model's predictions and the true labels. By assigning higher weights to minority classes (harmful rules), the bias caused by the long-tail distribution of data is mitigated.

[0078]

[0079] in, For category weights, For the true keyword labels of the i-th sample, The output for the student model is C, where C is the total number of keyword categories and c is the category traversal index.

[0080] III. Large Language Model Hint Strategies: To address the "black box" problem, this invention designs a multi-level, progressively structured prompt template to guide LLM in generating interpretations. The template includes an input information area and an output structure area. The output structure area pre-sets three levels of analysis requirements. The following is the structured prompt template used: Table 1 Prompt Word Templates

[0081] Example 1: This example provides a process for real-time security detection and prompting of user-created TAP rules in a smart home gateway scenario.

[0082] 1. Preparation Phase (Offline Training) First, we constructed and trained the teacher model GA-RDM. This model is based on BERT-base and subsequently incorporates BiLSTM layers, cross-attention layers, and the gated attention layer of this invention. It was trained using a dataset containing tens of thousands of labeled TAP rules, with label categories including: harmless, privacy breach, physical environment damage, and network environment damage. Simultaneously, we initialized an ALBERT-base model with approximately one-tenth the number of parameters as the student model.

[0083] 2. Knowledge Distillation Stage The parameters of the pre-trained GA-RDM teacher model are fixed. The same training data is simultaneously input into both the teacher and student models. When updating the student model parameters during backpropagation, instead of directly calculating the loss using its output and the true labels, a combined loss is calculated according to the formula (see point 3 of the invention description): The total loss from attention distillation comprises three parts: global, local, and micro. For example, for the rule "If no one is home, turn off all power," the teacher model, under gated attention control, will focus heavily on "turn off all power" rather than "no one is home." During local key feature distillation, we directly instruct the student model to mimic the teacher's attention distribution to these core action phrases, thereby quickly learning the risk discrimination pattern.

[0084] Hard-label distillation loss: Since some samples have a very small number of samples, when calculating this loss, we set the class weights according to the training set distribution experience as: w=[0.8,1.2,2.5,3.0].

[0085] After multiple rounds of iterative training, the student model ALBERT continuously approximates the attention pattern of the teacher model, and finally completes the knowledge transfer.

[0086] 3. Deployment and Reasoning Phase We deployed the final trained, lightweight ALBERT student model, which is only 44.5MB in size, into the firmware of the smart home gateway.

[0087] A user creates a new rule via a mobile app: "Turn off all power if no one is home." This rule text is sent to the gateway, where the ALBERT model immediately infers its meaning. The model predicts the rule to be "Category 2: Physical Environmental Damage." The entire inference process is completed within milliseconds, causing no perceptible delay for the user.

[0088] 4. Interpretable Output Stage The gateway packages the detection results (Category 2) and the original rule text, sends them to a large language model interface in the cloud, and includes a pre-designed structured prompt template: Table 2 Structured Hint Template

[0089] In conclusion, after seeing this clear explanation and suggestion, users can immediately understand the potential harm of the original rule and adopt the suggestion to make corrections, thereby eliminating security risks during the configuration phase.

[0090] Those skilled in the art will understand that the above description is merely a preferred embodiment of the present invention, and the features described in the various embodiments and / or technical solutions of this disclosure can be combined or combined in various ways, even if such combinations or combinations are not explicitly described in this disclosure. This is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

[0091] Although preferred embodiments of the present invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended technical solutions are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the present invention. Clearly, those skilled in the art can make various modifications and variations to the present invention without departing from its spirit and scope. Thus, if these modifications and variations of the present invention fall within the scope of the present invention and its equivalents, the present invention also intends to include these modifications and variations.

Claims

1. A lightweight rule detection method for the Internet of Things based on multi-granularity attention distillation, characterized in that, The method includes the following steps: Step 1: Build and train a gated attention-enhanced rule detection model, which takes user-configured TAP rule text as input and outputs the corresponding security risk category prediction and internal attention features. Step 2: Construct a multi-granularity attention distillation mechanism to transfer the multi-level attention knowledge learned by the gated attention enhancement rule detection model constructed in Step 1 during the security discrimination process to the ALBERT student model; Step 3: Deploy the lightweight ALBERT student model trained in Step 2 to an IoT edge device. When the IoT edge device receives updated TAP rules, the student model performs real-time security risk detection and outputs the risk category. Step 4: Input the detection results of the lightweight ALBERT student model described in Step 3 and the original rule text into the structured prompt template combined with the large language model to generate a natural language explanation containing risk category, potential harm and modification suggestions, thus completing the lightweight rule detection for IoT based on multi-granularity attention distillation.

2. The lightweight rule detection method for IoT based on multi-granularity attention distillation according to claim 1, characterized in that, The gated attention enhancement rule detection model described in step 1 includes: The feature encoding layer is used to convert the input TAP rule text into a sequence of word vectors rich in contextual semantics using a pre-trained BERT model, and to extract the attention output matrix of BERT. A bidirectional temporal modeling layer is used to input the word vector sequence into a bidirectional long short-term memory network to extract bidirectional temporal features and capture the time-dimensional dependency between triggering events and response actions. The self-attention and cross-attention fusion layer is used to fuse the global semantic features output by BERT with the temporal features output by the self-attention mechanism to form a fused feature representation. A gated attention mechanism layer is used to dynamically suppress noise and enhance key semantics.

3. The lightweight rule detection method for the Internet of Things based on multi-granularity attention distillation according to claim 2, characterized in that, The gated attention mechanism layer uses the following method to dynamically suppress noise and enhance key semantics: The fused feature representations are linearly projected to generate a query vector Q, a key vector K, and a value vector V. The initial attention output vector is obtained by scaling the dot product attention calculation. The query vector Q is concatenated with the initial attention output vector and sent to the end of the multilayer perceptron (MLP) network to learn a gating vector g. The gating vector g is multiplied element-wise with the initial attention output vector to obtain the gated context representation. A residual gating fusion strategy is adopted to add the gated modulated context representation to the original query vector Q to obtain the final output feature.

4. The lightweight rule detection method for IoT based on multi-granularity attention distillation according to claim 1, characterized in that, Step 2 involves constructing a multi-granularity attention distillation mechanism, including: Global semantic distillation is used to calculate the KL divergence loss between the attention distributions of the CLS tags of the teacher model and the student model at each position of the input sequence. Local key feature distillation is used to aggregate all attention head weights of the teacher model, calculate the average attention score for each input label, dynamically select the top K key labels with the highest scores in each training sample, and finally calculate the KL divergence loss between the attention distributions of the teacher and student models on the key labels. Microscopic multi-head attention distillation is used to flatten the complete attention matrix of each attention head of the teacher model and the student model, and calculate the average KL divergence loss of both over all heads. Hard label distillation is used to calculate the loss between the student model's predictions and the true labels using a weighted cross-entropy loss function.

5. The lightweight rule detection method for IoT based on multi-granularity attention distillation according to claim 4, characterized in that, The multi-particle size attention distillation loss mentioned in step 2 is: in, Let B be the number of heads in multi-head attention, and B be the number of samples. The top k high-weight attention positions are selected for each sample. For the SLC loss of the h-th attention head, The loss for the j-th top-k selected position, Let be the head loss of the h-th attention head.

6. The lightweight rule detection method for IoT based on multi-granularity attention distillation according to claim 4, characterized in that, When using the weighted cross-entropy loss function in the hard-label distillation, the class weights are set according to the training set distribution as: w=[0.8,1.2,2.5,3.0].

7. The lightweight rule detection method for IoT based on multi-granularity attention distillation according to claim 1, characterized in that, The lightweight ALBERT student model described in step 3 has a size of 44.5 MB.

8. A lightweight rule detection system for the Internet of Things based on multi-granularity attention distillation, characterized in that, The system includes: The Gated Attention Enhancement Rule Detection Model Module is used to build and train the Gated Attention Enhancement Rule Detection Model. It receives user-configured TAP rule text as input and outputs the corresponding security risk category prediction and internal attention features. The multi-granularity attention distillation mechanism module is used to construct a multi-granularity attention distillation mechanism, which transfers the multi-level attention knowledge learned by the gated attention enhancement rule detection model in the security discrimination process constructed by the gated attention enhancement rule detection model module to the ALBERT student model. The risk detection module is used to deploy the lightweight ALBERT student model trained by the multi-granularity attention distillation mechanism module in IoT edge devices. When the IoT edge device receives updated TAP rules, the student model performs real-time security risk detection and outputs the risk category. The rule detection module is used to input the detection results of the lightweight ALBERT student model described in the risk detection module and the original rule text into a structured prompt template combined with a large language model to generate a natural language explanation containing risk categories, potential hazards and modification suggestions, thus completing the lightweight rule detection of IoT based on multi-granularity attention distillation.

9. A computer storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1-7.

10. A computer device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, the processor executing the program to implement the method of any one of claims 1-7.

Images