Intelligent phishing mail identification method and system based on BERT and active learning

Abstract

The invention discloses an intelligent phishing mail identification method and system based on BERT and active learning, and the method comprises the steps: local model training: carrying out the independent model training of each device through employing local data, employing a pre-trained BERT model as a feature extractor, employing preset mail features, and applying an attention mechanism and an active learning strategy; model parameter updating: based on a federated learning technology, each device only sends updating information of the model parameters to the central server after local model training is completed; global model updating: the central server collects updating information of the model parameters from all the devices and aggregates the updating information to obtain updating of the global model; and model synchronization: after all devices receive the global model, synchronizing the local model and the global model, and identifying phishing mails and legal mails through the trained model. According to the invention, the technical performance is improved, and the user experience and the transparency of the system are also considered.

Description

Technical Field

[0001] The present invention relates to the field of information security technology, and in particular to an intelligent phishing email identification method and system based on BERT and active learning. Background Art

[0002] Currently, phishing email detection mainly focuses on machine learning methods. These technologies usually involve extracting features from email content and using these features to train models to distinguish phishing emails from legitimate emails. The following are several common existing technical solutions:

[0003] (1) Machine learning methods based on content analysis

[0004] In the field of phishing email detection, machine learning methods based on content analysis are a widely used technology. The core of this type of method is to extract various features from emails, such as text content, structural features, metadata (such as sender, receiver, email subject), etc., and use these features to train machine learning models. Feature extraction is a crucial step in this process, which directly affects the performance of the model.

[0005] Analysis of text content usually involves natural language processing techniques, such as using bag-of-words models, TF-IDF (term frequency-inverse document frequency) and other methods to transform text data. These techniques can extract keywords and phrases from the email text to help identify typical phishing email features, such as deceptive language or threatening words. In addition, structural feature analysis may include email format, HTML tags used, etc., while metadata analysis involves sending and receiving information about emails.

[0006] After feature extraction is completed, these features are used to train different types of machine learning models, such as support vector machines (SVM), decision trees, random forests, neural networks, etc. These models distinguish phishing emails from legitimate emails by analyzing the patterns of email features. For example, a neural network can learn complex nonlinear relationships through its multi-layer structure, thereby improving its ability to identify phishing emails.

[0007] The advantage of these methods is that they usually provide high accuracy, especially when there is sufficient training data. However, they also face some challenges, such as the need for a large amount of labeled training data, limited adaptability to new phishing methods, and potential privacy issues when processing data. In addition, these methods may require large computing resources, especially when using complex models such as deep neural networks.

[0008] (2) Analysis based on user behavior

[0009] User behavior-based analysis is a relatively new approach that considers not only the content and features of the email itself, but also how the user interacts with the email. The core idea of ​​this approach is to identify potential signs of phishing emails by analyzing how users interact with emails. For example, the user's behavior of clicking on links after receiving an email, the frequency of replying to emails, etc., can all be used as a basis for identifying phishing emails.

[0010] This method usually involves collecting and analyzing a large amount of user behavior data, including click-through rate, reading time, response mode, etc. Through this data, the present invention can learn what kind of user behavior patterns may be related to phishing emails. For example, if a user often clicks on links immediately after receiving emails from unknown senders, this may indicate that these emails are phishing attempts.

[0011] To implement this approach, complex algorithms and models are usually needed to process and analyze large amounts of data. These models may include clustering algorithms, anomaly detection algorithms, etc., which can identify behaviors that do not conform to regular patterns. In addition, this approach may also be combined with traditional content analysis techniques to improve the accuracy of detection.

[0012] The advantage of the user behavior-based analysis method is that it can identify phishing emails that are difficult to detect through conventional content analysis. However, this method also has some challenges, such as the need to process and analyze a large amount of user data, which may raise concerns about privacy protection. In addition, the effectiveness of this method depends heavily on the quality and quantity of available user behavior data.

[0013] (3) Combining NLP methods

[0014] In recent years, methods combining natural language processing (NLP) have become increasingly popular in phishing email detection. The core of these methods is to deeply analyze the semantic content of the email text, not just the surface text features. By using NLP techniques such as sentiment analysis, semantic role labeling, language models, etc., the present invention can more accurately understand the intent and context of the email content.

[0015] For example, sentiment analysis can be used to identify misleading or threatening language in emails, which is a common tactic used in phishing emails. By analyzing the sentiment of email text, the present invention can identify emails that attempt to manipulate the recipient's emotions. In addition, semantic role labeling can help the present invention understand the semantic roles of each component in the email, such as who is the initiator of the action, who is the object of the action, etc., so as to better understand the true intention of the email content.

[0016] Such methods usually rely on advanced deep learning models, such as recurrent neural networks (RNNs), long short-term memory networks (LSTMs), and transformer models, which are able to process complex language structures and learn deep semantic information from them.

[0017] Methods combined with NLP have significant advantages in understanding the complexity of email content, especially when dealing with complex phishing emails that are difficult to identify through traditional methods. However, these methods also face some challenges, such as requiring a lot of computing resources, having high requirements on the quality and diversity of data, and having difficulties in dealing with the diversity and complexity of languages.

[0018] In general, these existing technologies have their own advantages and limitations. Machine learning methods based on content analysis work well when there is sufficient training data, but may lack flexibility; analysis methods based on user behavior can identify phishing emails that are difficult to detect with traditional methods, but may involve privacy issues; and methods combined with NLP have advantages in understanding the complexity of email content, but have high computational costs and high requirements on data quality. These challenges have prompted researchers and developers to seek new methods, such as the method proposed in this invention that combines federated learning and deep learning, to provide a more effective and secure phishing email detection solution.

[0019] In summary, the existing phishing email detection technology has the following problems:

[0020] Problem 1: Insufficient privacy protection

[0021] In the modern Internet environment, privacy protection has become a key issue. Especially in the field of phishing email detection, how to effectively identify and prevent phishing attacks without infringing user privacy is a major challenge. Existing phishing email identification technologies often require the analysis of a large amount of personal or sensitive information, which may lead to user privacy leakage. For example, traditional machine learning methods based on content analysis process the content of users' emails, which directly involves user privacy.

[0022] Problem 2: Lack of adaptability to new phishing tactics

[0023] As phishing attacks continue to evolve and diversify, how to quickly adapt to new phishing strategies has become a major challenge. Existing methods have obvious deficiencies in adaptability and flexibility. On the one hand, methods that rely on pre-labeled large-scale data sets for learning are relatively slow in adapting to new attack strategies. On the other hand, due to the diversity and complexity of phishing strategies, the quality and quantity of data annotations are high, which also limits the model's ability to adapt to new strategies.

[0024] Problem 3: Dependence on large amounts of labeled data and insufficient model interpretability

[0025] Interpretability has always been a challenge in complex machine learning models, especially deep learning models. Users and system administrators often find it difficult to understand the inner workings of the model, which leads to distrust in the model's decisions. Especially in security-related fields, model transparency and interpretability are particularly important, because wrong judgments can have serious consequences. Summary of the invention

[0026] In order to solve the above problems, the present invention proposes an intelligent phishing email identification method and system based on BERT and active learning, which uses a pre-trained BERT model as a feature extractor to effectively capture the complex semantics and contextual information of email text. By designing specific email features, applying self-attention mechanisms and active learning strategies, the model's predictive performance and adaptability to new phishing email strategies are significantly improved, the model's interpretability is enhanced, and it helps users understand and trust the model's decision-making process. Most importantly, the use of federated learning allows model training to be performed using data distributed on multiple devices while protecting data privacy. These features together constitute an efficient and secure phishing email detection technology with significant technical advantages and application value.

[0027] Specifically, in response to the above-mentioned problem one, the present invention adopts federated learning technology, which can independently learn on each node, and only send the learned model update information to the central server for aggregation, without involving the exchange of original data. This method effectively solves the problem of privacy leakage. Federated learning allows the model to be trained and updated locally, thereby avoiding the centralized storage and transmission of sensitive data. Therefore, even in an environment with a high risk of data leakage, the privacy of users can be effectively protected. In addition, the present invention can also improve the adaptability and efficiency of the model in different environments through intelligent learning and adaptive algorithms, further reducing the potential threat to user privacy. In this way, it not only improves the accuracy of phishing email identification, but also protects user privacy and data security, providing an innovative technical path for solving the thorny problem of privacy protection.

[0028] In response to the second problem mentioned above, the present invention greatly improves the model's ability to adapt to new phishing strategies by introducing an active learning mechanism. Active learning is an intelligent learning method that allows the model to proactively identify data samples that are most valuable for improving its own performance and request manual labeling. This method significantly reduces the reliance on large amounts of pre-labeled data and speeds up the model's adaptation to new attack methods. In this way, the model can quickly identify and learn the characteristics of newly emerging phishing emails, improving the accuracy and timeliness of detection. The introduction of active learning not only improves the learning efficiency of the model, but also increases the model's ability to adapt to emerging phishing strategies. This enables the model to remain efficient and agile in an ever-changing network security environment, and to achieve a wider range of security protection at a lower cost.

[0029] In response to the third problem mentioned above, the present invention significantly improves the interpretability of the model by combining BERT (an advanced natural language processing model) and the attention mechanism. The attention mechanism allows the model to significantly highlight those parts that are most important for the final judgment when processing input data, which makes the model decision-making process more transparent. Users and administrators can understand why the model makes specific judgments by observing the parts that the model focuses on. This approach not only improves the accuracy of the model, but also increases the user's trust in the system. Through this innovative combination, the present invention not only improves the accuracy of phishing email identification, but also makes significant improvements in the interpretability of the model. This is of great significance for improving user trust in the system, reducing false alarm rates, and improving the overall level of network security protection.

[0030] In general, compared with the existing technologies, this invention proposes innovative and effective solutions in terms of privacy protection, adapting to new phishing strategies, and improving model interpretability. These solutions not only improve technical performance, but also take into account user experience and system transparency, demonstrating their advancement and practicality in the field of intelligent phishing email identification technology.

[0031] Specifically, the technical solution adopted by the present invention is as follows:

[0032] An intelligent phishing email recognition method based on BERT and active learning, comprising:

[0033] Local model training: Each device uses local data to independently train the model, using the pre-trained BERT model as a feature extractor to capture the semantics and contextual information of the email text; and by using preset email features, applying attention mechanisms and active learning strategies, the model's predictive performance and adaptability to new phishing email strategies are enhanced;

[0034] Model parameter update: Based on federated learning technology, each device only sends the update information of model parameters to the central server after completing local model training;

[0035] Global model update: The central server collects and aggregates the updated information of model parameters from all devices to obtain the updated global model. After the aggregation is completed, the central server distributes the updated global model to all devices.

[0036] Model synchronization: After all devices receive the global model, they synchronize the local model with the global model, that is, use the global model to replace the local model; after the model is synchronized, each device can continue to use the new data to train the model locally, update the model parameters and update the global model, and can identify phishing emails and legitimate emails through the trained model.

[0037] Furthermore, in the local model training, the preset email features include metadata information of the email and email text content extraction features, the metadata information includes sender and receiver information, sending time and email subject, and the email text content extraction features include text length, language used, and set words or phrases.

[0038] Furthermore, in the local model training, the attention mechanism is applied to enable the model to highlight the most critical features when making phishing email judgments, making the model decision-making process more transparent.

[0039] Furthermore, during the local model training, an active learning strategy is used to enable the model to proactively identify data samples that are most valuable for improving its own performance and request labeling, thereby reducing dependence on a large amount of pre-labeled data.

[0040] Furthermore, in the updating of the model parameters, each device is enabled to learn independently through the federated learning technology, and only the learned model update information is sent to the central server for aggregation without involving the exchange of original data, thereby avoiding the centralized storage and transmission of sensitive data.

[0041] An intelligent phishing email recognition system based on BERT and active learning, including:

[0042] Several local devices, each of which uses local data to independently train the model, using the pre-trained BERT model as a feature extractor to capture the semantics and contextual information of the email text; and by using preset email features, applying attention mechanisms and active learning strategies, the model's predictive performance and adaptability to new phishing email strategies are enhanced; based on federated learning technology, each device only sends the update information of the model parameters to the central server after the local model training is completed; after all devices receive the global model, they synchronize the local model with the global model, that is, use the global model to replace the local model; after the model is synchronized, each device can continue to use new data to train the model locally, update the model parameters and update the global model, and can identify phishing emails and legitimate emails through the trained model;

[0043] The central server is configured to collect and aggregate the update information of model parameters from all devices to obtain the update of the global model; after the aggregation is completed, the central server distributes the updated global model to all devices.

[0044] Furthermore, the preset email features include metadata information of the email and email text content extraction features, the metadata information includes sender and receiver information, sending time and email subject, and the email text content extraction features include text length, language used, and set words or phrases.

[0045] Furthermore, by applying the attention mechanism, the model highlights the most critical features when making phishing email judgments, making the model decision-making process more transparent.

[0046] Furthermore, through the active learning strategy, the model actively identifies the data samples that are most valuable for improving its own performance and requests annotation, thereby reducing the dependence on a large amount of pre-annotated data.

[0047] Furthermore, federated learning technology enables each device to learn independently, and only sends the learned model update information to the central server for aggregation without involving the exchange of original data, thereby avoiding the centralized storage and transmission of sensitive data.

[0048] The beneficial effects of the present invention are:

[0049] 1. Enhanced privacy protection: This invention uses federated learning technology to enable data processing to be completed locally without uploading the original data to the cloud or server. This processing method effectively prevents the risk of data leakage during transmission and protects the privacy of users. It is particularly suitable for enterprises or individuals involving sensitive information, providing users with a safer data processing environment.

[0050] 2. Improved semantic understanding: Using the advanced BERT model as a feature extractor, this patent effectively improves the ability to understand complex semantics and context in email text. The BERT model can deeply explore the implicit meaning in the text, thereby more accurately identifying fraudulent content in phishing emails, greatly improving the accuracy and efficiency of detection compared to traditional keyword matching methods.

[0051] 3. Application of active learning strategy: Through the active learning strategy, the present invention can intelligently select the data samples that are most helpful for model training for labeling. This not only reduces the demand for a large number of pre-labeled data sets and reduces the cost of manual labeling, but also enables the model to quickly adapt to new threats and improves the ability to detect new phishing emails.

[0052] 4. Enhanced model interpretability: Using technologies such as attention mechanisms, this patent enhances the interpretability of the model, allowing users to understand how the model makes decisions. This transparency is critical to increasing user trust in the system, especially in the security field, where users need to know why a certain email is marked as a phishing email.

[0053] 5. Improved adaptability and flexibility: This patent significantly improves the model's adaptability and flexibility to new phishing strategies by combining the specific features of emails and the self-attention mechanism. This means that even if phishing attackers change their strategies or methods, the technology proposed in this invention can quickly adapt and effectively identify new threats, maintaining the efficiency and effectiveness of the system.

[0054] In summary, the above advantages together constitute the core competitiveness of this patented technology, enabling it to perform outstandingly in the field of phishing email detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 It is a flow chart of the intelligent phishing email identification method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0056] In order to have a clearer understanding of the technical features, purposes and effects of the present invention, the specific implementation methods of the present invention are now described. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention, that is, the embodiments described are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present invention.

[0057] The existing technology has limitations in phishing email detection, including reliance on a large amount of labeled data, insufficient privacy protection, insufficient adaptability to new phishing methods, and insufficient model interpretability. Based on this, this embodiment provides an intelligent phishing email identification method and system based on BERT and active learning, aiming to improve the effect of phishing email detection by combining federated learning and deep learning technology while protecting user privacy. The intelligent phishing email identification system of this embodiment includes a central server and several local devices; Figure 1 As shown, the intelligent phishing email identification method includes:

[0058] Local model training: Each device uses local data to independently train the model, using the pre-trained BERT model as a feature extractor to capture the semantics and contextual information of the email text; and through the preset email features, application attention mechanism and active learning strategy, the model's predictive performance and adaptability to new phishing email strategies are enhanced. Specifically, for a simple logistic regression model, local updates can use the gradient descent method to calculate the gradient of the loss function based on local data, and then update the model parameters. In this step, there is no need to exchange data between devices, and each device performs local training independently.

[0059] Model parameter update: Based on federated learning technology, each device only sends the update information of the model parameters to the central server after the local model training is completed. This update is generally expressed as the gradient of the model parameters. The device only needs to send the gradient information, not the original data, thus protecting the user's privacy. For those devices with limited computing resources or network bandwidth, you can choose to send only some model parameter updates.

[0060] Global model update: The central server collects and aggregates the updated information of model parameters from all devices to obtain the update of the global model; after the aggregation is completed, the central server distributes the updated global model to all devices. Specifically, the aggregation method can choose a simple averaging algorithm, that is, averaging the updates of all devices. For more complex situations, such as uneven data distribution between devices, more complex aggregation algorithms may be required. After the aggregation is completed, the server distributes the updated global model to all devices.

[0061] Model synchronization: After all devices receive the global model, they synchronize their local models with the global model, that is, replace the local model with the global model. In this way, the models of all devices will be consistent and contain the learning information of all devices. After the model is synchronized, the device can continue to use the new data to train the model locally, and then repeat the above steps to update the model parameters, update the global model, and synchronize the model.

[0062] This embodiment uses a federated learning approach to enhance privacy protection. Federated learning is a distributed machine learning approach that allows multiple participants to jointly train a shared model without sharing their original data. This means that the content of an individual user's email does not leave their device, greatly enhancing privacy protection. The use of federated learning is critical to collecting and utilizing distributed data to improve detection accuracy while protecting privacy.

[0063] This embodiment also uses deep learning technology, especially the latest advances in the field of natural language processing (NLP), such as the BERT (Bidirectional Encoder Representations from Transformers) model. The introduction of the BERT model enables the present invention to more effectively understand and analyze the complex semantics of email text. Compared with traditional rule-based or simple machine learning models, BERT can capture more detailed and deeper language features, thereby more accurately identifying various forms of phishing emails, including those using clever disguises and new strategies. This ability, especially the advantage in understanding text semantics, is crucial to improving the accuracy of phishing email detection.

[0064] In view of the reliance of the prior art on a large amount of labeled data, this embodiment adopts an active learning strategy. Active learning is a machine learning paradigm that allows the model to actively identify and request the labeling of data samples that are most valuable for improving its performance. This approach enables the detection system to effectively utilize unlabeled data, especially in phishing email scenarios, where new attacks appear frequently and diversely. Through active learning, the present invention can quickly adapt to new phishing methods, reducing the reliance on a large amount of pre-labeled data, while improving the adaptability and effectiveness of the model.

[0065] Considering the importance of model interpretability to user trust and system transparency, this embodiment places special emphasis on improving the interpretability of the model in its design. By using technologies such as the attention mechanism, the model can highlight the most critical features when making phishing email judgments, making it easier for users and system administrators to understand the model's decision-making process. This not only enhances user trust in the system, but also provides experts with important information for adjusting and optimizing the model.

[0066] On the whole, the technical solution proposed in this embodiment comprehensively considers multiple aspects of phishing email detection, including data privacy protection, detection accuracy, adaptability to new attacks, and model interpretability. Through federated learning, the introduction of the BERT model, the application of active learning strategies, and the emphasis on model interpretability, the present invention provides a new, efficient and secure phishing email detection technology for the field of network security, which has significant technical advantages and practical application value. In addition, the technical solution of the present invention also solves some major challenges in traditional phishing email detection methods, such as insufficient privacy protection, insufficient adaptability to new phishing methods, and reliance on a large amount of pre-labeled data, bringing new ideas and possibilities to the field of network security.

[0067] Specifically, the key technologies used in this embodiment are described as follows:

[0068] 1. Using BERT as a feature extractor

[0069] The basis of the BERT model is the Transformer model, whose calculation is based on the attention mechanism. The basic attention mechanism can be described as:

[0070]

[0071] Among them, Q, K and V represent query, key and value respectively, that is, query vector, key vector and value vector. This formula is the Scaled Dot-Product Attention mechanism, and its result is the weighted sum of value, and the weight is the softmax of the dot product of query and key.

[0072] First, you need to download the pre-trained BERT model and its accompanying tokenizer. The HuggingFace library provides many pre-trained models and tokenizers. Using Python, you can do this:

[0073]

[0074] We need to convert each email into a format that BERT can process. Assuming emails is a list containing the email text, we can process it like this:

[0075]

[0076] Then, we input the preprocessed email text into the BERT model to obtain the feature representation of the email:

[0077]

[0078] 2. Design specific features

[0079] The specific feature design depends on the content and metadata of the email. Generally speaking, we can use regular expressions, TF-IDF, one-hot encoding and other methods to extract and encode features. For example, if we want to extract whether the email contains a link as a feature, we can do this:

[0080]

[0081] 3. Apply the attention mechanism

[0082] As mentioned above, the basic formula of the attention mechanism is:

[0083]

[0084] Among them, Q, K and V represent the query vector, key vector and value vector respectively.

[0085] The specific implementation of the attention mechanism depends on the model framework used. In PyTorch, we can use torch.nn.MultiheadAttention to implement the attention mechanism.

[0086] 4. Use Active Learning

[0087] In active learning, a commonly used uncertainty measure is the entropy of the prediction probability, which is formulated as:

[0088]

[0089] Among them, p i is the model’s predicted probability for class i.

[0090] The specific implementation depends on the data annotation process and equipment. Generally speaking, we need to establish a feedback mechanism so that the model can send uncertain prediction results to human annotators. One possible implementation is:

[0091]

[0092] 5. Increase the interpretability of the model

[0093] In PyTorch, we can use the torch.nn.functional.softmax function to calculate the attention weights. We can then visualize these weights, for example using the matplotlib library:

[0094]

[0095]

[0096] 6. Federated Learning

[0097] The core of federated learning is to calculate the gradient locally and then aggregate these gradients on the central server. Assuming n is the total number of devices, w_i is the model parameter of device i, and L_i is the loss function of device i, then we can update the global model like this:

[0098]

[0099] Where η is the learning rate.

[0100] The specific implementation depends on the device and network configuration. Generally speaking, we need to build a distributed computing network so that each device can train the model locally and send model updates to the server. This may require the use of a specialized distributed computing library, such as PyTorch's torch.distributed.

[0101] In summary, the intelligent phishing email identification method of this embodiment has the following characteristics:

[0102] 1. Combining federated learning and deep learning technologies: This embodiment combines federated learning with deep learning technologies to improve the detection of phishing emails while protecting user privacy. Federated learning is a distributed machine learning method that allows multiple participants to jointly train a shared model without sharing the original data. This method enables each participating device to train the model locally and only send the model updates to the central server for aggregation, thereby ensuring that the email content of individual users does not leave their devices, significantly enhancing privacy protection.

[0103] 2. Use of BERT model: The patent adopts the latest progress in the field of natural language processing, especially the BERT (Bidirectional Encoder Representations from Transformers) model. The introduction of the BERT model enables the present invention to more effectively understand and analyze the complex semantics of email text, capture more detailed and deeper language features, and thus more accurately identify various forms of phishing emails. Compared with traditional rule-based or simple machine learning models, BERT's advantage in understanding text semantics is crucial to improving the accuracy of phishing email detection.

[0104] 3. Active learning strategy: In response to the reliance of existing technologies on large amounts of labeled data, this embodiment adopts an active learning strategy. Active learning is a machine learning paradigm that allows the model to actively identify and request labeling of data samples that are most valuable for improving its performance. This approach enables the detection system to effectively utilize unlabeled data, especially in phishing email scenarios, where new attacks appear frequently and diversely. Through active learning, the present invention can quickly adapt to new phishing methods, reduce reliance on large amounts of pre-labeled data, and improve the adaptability and effectiveness of the model.

[0105] 4. Model interpretability: This embodiment places special emphasis on improving the interpretability of the model, considering the importance of model interpretability for user trust and system transparency. Interpretability is an important aspect of modern machine learning models, especially when applied to sensitive areas such as network security. By improving the interpretability of the model, users and system administrators can more easily understand and trust the decision-making process of the model.

[0106] 5. Designing specific features and using BERT as a feature extractor: This embodiment introduces specific features based on the BERT feature vector to enhance the prediction performance of the model. These features include metadata information of the email (such as sender and receiver information, sending time, email subject) and features extracted from the email text content (such as text length, language used, presence of specific words or phrases). The combination of this feature design method and the BERT model can effectively capture the complex semantics and contextual information of the email text, thereby improving the detection efficiency of phishing emails.

[0107] In summary, the key points of the method in this embodiment are to use federated learning to protect privacy, use the BERT model to enhance semantic understanding, apply active learning strategies to improve the adaptability and effectiveness of the model, focus on the interpretability of the model, and design targeted features to enhance prediction performance. These features together constitute an efficient and secure phishing email detection technology.

[0108] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity of description, they are expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

Claims

1. An intelligent phishing email identification method based on BERT and active learning, characterized in that: include: Local model training: Each device uses local data to independently train the model, using the pre-trained BERT model as a feature extractor to capture the semantics and contextual information of the email text; and by using preset email features, applying attention mechanisms and active learning strategies, the model's predictive performance and adaptability to new phishing email strategies are enhanced; Model parameter update: Based on federated learning technology, each device only sends the update information of model parameters to the central server after completing local model training; Global model update: The central server collects and aggregates the updated information of model parameters from all devices to obtain the updated global model. After the aggregation is completed, the central server distributes the updated global model to all devices. Model synchronization: After all devices receive the global model, they synchronize the local model with the global model, that is, use the global model to replace the local model; after the model is synchronized, each device can continue to use the new data to train the model locally, update the model parameters and update the global model, and can identify phishing emails and legitimate emails through the trained model.

2. According to claim 1, a method for intelligent phishing email identification based on BERT and active learning is characterized in that: In the local model training, the preset email features include metadata information of the email and email text content extraction features, the metadata information includes sender and receiver information, sending time and email subject, and the email text content extraction features include text length, language used, and set words or phrases.

3. According to claim 1, a method for intelligent phishing email identification based on BERT and active learning is characterized in that: In the local model training, the attention mechanism is applied to enable the model to highlight the most critical features when making phishing email judgments, making the model decision-making process more transparent.

4. According to claim 1, a method for intelligent phishing email identification based on BERT and active learning is characterized in that: During the local model training, an active learning strategy is used to enable the model to proactively identify the data samples that are most valuable for improving its own performance and request annotations, thereby reducing dependence on a large amount of pre-annotated data.

5. According to claim 1, a method for intelligent phishing email identification based on BERT and active learning, characterized in that: In the model parameter update, each device is enabled to learn independently through federated learning technology, and only the learned model update information is sent to the central server for aggregation without involving the exchange of original data, thereby avoiding centralized storage and transmission of sensitive data.

6. An intelligent phishing email identification system based on BERT and active learning, characterized in that: include: Several local devices, each of which uses local data to independently train the model, using the pre-trained BERT model as a feature extractor to capture the semantics and contextual information of the email text; and by using preset email features, applying attention mechanisms and active learning strategies, the model's predictive performance and adaptability to new phishing email strategies are enhanced; Based on federated learning technology, each device only sends the updated information of model parameters to the central server after the local model training is completed; after all devices receive the global model, they synchronize the local model with the global model, that is, use the global model to replace the local model; after the model is synchronized, each device can continue to use the new data to train the model locally, update the model parameters and update the global model, and can identify phishing emails and legitimate emails through the trained model; The central server is configured to collect and aggregate the update information of model parameters from all devices to obtain the update of the global model; after the aggregation is completed, the central server distributes the updated global model to all devices.

7. The intelligent phishing email identification system based on BERT and active learning according to claim 6, characterized in that: The preset email features include metadata information of the email and email text content extraction features, the metadata information includes sender and receiver information, sending time and email subject, and the email text content extraction features include text length, language used, and set words or phrases.

8. The intelligent phishing email identification system based on BERT and active learning according to claim 6, characterized in that: By applying the attention mechanism, the model highlights the most critical features when making phishing email judgments, making the model decision-making process more transparent.

9. The intelligent phishing email identification system based on BERT and active learning according to claim 6, characterized in that: Through active learning strategies, the model proactively identifies the data samples that are most valuable for improving its own performance and requests annotations, thereby reducing dependence on large amounts of pre-annotated data.

10. The intelligent phishing email identification system based on BERT and active learning according to claim 6, characterized in that: Federated learning technology enables each device to learn independently, and only sends the learned model update information to the central server for aggregation without involving the exchange of original data, thereby avoiding the centralized storage and transmission of sensitive data.

Images