Multi-modal fusion feature-based phishing detection method and system

Abstract

The invention relates to the technical field of phishing detection, in particular to a phishing detection method and system based on multi-modal fusion features, and the method comprises the following steps: S1, constructing a phishing sample based on multi-modal simulation and data generation; according to the method, the multi-modal simulation and data generation technology is utilized, phishing samples with high diversity and authenticity are created, a solid foundation is provided for the model, diversified interactive behavior data are generated through the user behavior simulator designed by the reinforcement learning model, the system can simulate and understand behaviors of different types of users, and the user experience is improved. The system improves the detection precision and efficiency through a non-traditionally designed multi-mode fusion deep detection model in combination with a multi-branch neural network and multi-task learning, realizes the real-time adaptive detection and defense capability through adversarial training and federated learning, greatly improves the accuracy and robustness of phishing detection, and improves the detection accuracy and efficiency. And comprehensive protection is provided for network security.

Description

Technical Field

[0001] The present invention relates to the technical field of website phishing detection, and in particular to a network phishing detection method and system based on multimodal fusion features. Background Art

[0002] With the widespread use of the Internet and the increase in online transactions, phishing attacks have become one of the main means of threatening network security. Phishing attacks usually induce users to disclose sensitive information such as usernames, passwords, and bank accounts by forging legitimate websites, emails, or social media messages. Traditional phishing website detection methods mainly rely on feature matching and rule analysis, usually based on website domain names, page content, or URL analysis. However, these methods are often unable to cope with increasingly complex phishing attacks, especially for some highly simulated and concealed phishing attacks, the accuracy and robustness of traditional detection technologies are insufficient. In order to more effectively identify and defend against phishing attacks, more and more studies have begun to explore multimodal data analysis and deep learning technologies.

[0003] Although the existing phishing website detection technology has improved the ability to identify phishing attacks to a certain extent, it still has many shortcomings when facing the ever-changing attack methods and attack environments. Traditional methods mostly rely on static feature extraction, which is difficult to cope with the continuous evolution and diversification of phishing attack methods. Existing detection models often ignore the impact of user behavior and cannot effectively simulate and understand the user's behavior patterns in actual interactions. The detection accuracy and efficiency of traditional models are still limited by algorithm design and training data, and lack adaptability and real-time defense capabilities. In these cases, the accuracy and robustness of existing phishing detection systems are still difficult to meet the security requirements in complex network environments. Therefore, we provide a phishing detection method and system based on multimodal fusion features. Summary of the invention

[0004] In view of the above-mentioned shortcomings of the prior art, the first object of the present invention is to provide a phishing detection method and system based on multimodal fusion features to solve the problems in the above-mentioned background technology.

[0005] To achieve the above object, the present invention provides the following technical solutions: A phishing detection method and system based on multimodal fusion features, comprising the following steps: S1. Construction of phishing samples based on multimodal simulation and data generation; S2, cross-modal dynamic relationship network modeling and feature conflict mining; S3. Non-traditional design and optimization of multi-modal fusion deep detection model; S4. Real-time adaptive phishing detection and countermeasure defense ecosystem construction.

[0006] The present invention is further configured as follows: in the step S1, in constructing a phishing sample based on multimodal simulation and data generation: S1.1. Build a containerized phishing webpage generation environment through Docker and Kubernetes clusters, combine Selenium and Puppeteer automation tools to generate phishing webpages, use GAN (StyleGAN) to dynamically generate a mixture of real and phishing webpages, adjust style parameters (fonts, colors, and layouts), configure PhantomJS simulation rendering environment, ensure that the generated pages are consistent with the real scene, and integrate real-time traffic analysis tools (Wireshark) for testing; S1.2. Design a user behavior simulator using the Reinforcement Learning model (DQN), perform mouse trajectories, scroll operations, and click behaviors in the browser environment, use the WebRTC interface to collect response data of simulated behaviors, record the interactive behaviors in the MongoDB database, set user type parameters (beginners, ordinary users, and expert users), and adjust operation details (click delay, error rate) to enhance diversity; S1.3. Use the Tcpreplay tool to replay the real captured phishing attack traffic (DNS requests generated by DGA domain names) to the network, use SuricataIDS to monitor network layer behavior, use the Scapy module to dynamically insert forged traffic (abnormal HTTP / HTTPS requests), test the robustness of the detection model to abnormal patterns, and set the replay rate to 10Gbps to match the high-traffic environment; S1.4. Use AdversarialPatch technology to generate visual interference elements, inject small perturbations (pixel modification of brand logos) into phishing pages, use NLP data enhancement technology (WordEmbeddings replacement) to fine-tune the page text content while maintaining semantic integrity, deploy OpenCV and PyTorch for adversarial image generation, and extract text features through Spacy to achieve controllable deformation; S1.5. Use Elasticsearch to configure a time series database to record the time series behavior of users visiting web pages. Analyze changes in user behavior patterns based on the ARIMA model and LSTM network, detect abnormal access frequency or behavior switching (suddenly clicking on phishing links in large numbers), and realize real-time time series data transmission through the Kafka streaming processing framework.

[0007] The present invention is further configured as follows: in the step S2, cross-modal dynamic relationship network modeling and feature conflict mining: S2.1. Use KnowledgeGraph to build a semantic knowledge graph of web page content (DBpediaAPI extracts entities and relationships), and use SpaCy and GloVe models to capture semantic conflicts between text and images. For example, detect whether there are conflicts in keywords related to shopping in the "Bank Official Website" page, and use Neo4j database to store and query graph relationships. S2.2. Construct a sparse relationship matrix, sparse the visual features (image hash values), text features (keyword vectors), and interaction features (mouse tracks) into a unified matrix, use Graph Convolutional Network to learn the dependencies of feature nodes, optimize the expression of features through edge weights (modal relevance scores), and configure TensorFlow and PyTorch environments to run the GCN model; S2.3. Introduce the SimCLR self-supervised learning model, train the similarity detector between modalities, classify the correlation between modalities through the contrastive loss function, define the pseudo correlation as low-confidence correlation (similarity threshold is less than 0.7), and set the NVIDIA A100 GPU cluster to run large-scale sample training; S2.4. Design a dynamic feature fuser based on Transformer to embed modal features into a unified space. Dynamically adjust the weight distribution of each modality (0.4 for visual modality and 0.3 for text modality) through the policy network of Reinforcement Learning. Use the distributed deep learning framework provided by AWS SageMaker to accelerate training. S2.5. Use the RandomWalk algorithm to generate relationship chains between modalities, use the PyKEEN framework to visualize the relationship chains, calculate the confidence scores of abnormal paths, set the walk step length to 10, and the number of samples to 1000 to ensure sufficient data to support anomaly mining.

[0008] The present invention is further configured as follows: in the step S3, the non-traditional design and optimization of the multi-modal fusion depth detection model: S3.1. Imitating human multimodal information processing, a multi-branch neural network (BranchCNN) is used to perform specialized processing on image, text, and behavior features respectively. Combined with the SoftAttention mechanism, key modal information is passed to the fusion layer. Model training is based on ImageNet and IMDB datasets for transfer learning. S3.2. Design a modality selection module to dynamically decide whether to enable a modality through modality credibility indicators (data integrity, noise ratio). For example, when network traffic data is missing, only text and visual data are used to complete the detection. Use PyTorch to configure the dynamic Masking layer to achieve modality selection. S3.3. Train the contrast detector through Siamese network, compare the input modality with the target modality, output the similarity score, optimize the contrast loss function to distinguish normal pages from phishing pages, and use TPU (TensorProcessingUnit) to improve the model's processing ability for large-scale training samples; S3.4. Build a multi-task learning model that shares underlying features and optimizes phishing detection, malicious domain classification, and spam filtering tasks simultaneously, using a Transformer model with shared weights and trained using the HuggingFaceTransformers framework. S3.5. Asynchronous distributed training is implemented through Horovod. Separate training steps are set for vision, text, and interactive behaviors (the step size for visual modality is 0.01, and that for text modality is 0.05). BatchNormalization is used to calibrate the feature distribution differences between modalities.

[0009] The present invention is further configured as follows: in the step S4, in the construction of real-time adaptive phishing detection and countermeasure defense ecosystem: S4.1. Use the CleverHans tool to generate adversarial samples (blurred images, URL splicing perturbations), train the detection model based on adversarial training, and achieve a detection rate of more than 95% for adversarial attack samples. Configure the RTX3090 graphics card to run large-scale adversarial sample generation. S4.2. Use Amazon SageMaker to achieve online learning capabilities, input real-time streaming data to update model weights, and use the federated learning framework (TensorFlowFederated) to collaboratively update the model to ensure data privacy; S4.3. Deploy Apache Kafka to implement real-time data processing, and combine the risk scoring algorithm (Logistic Regression) to generate a risk score of 0-1 for each web page. Set the score threshold to 0.7. Pages above this value trigger in-depth analysis. S4.4. Use the Honeypot system to capture attacker behavior, return fake pages (fake forms) to suspected attackers, monitor attacker traffic patterns through Splunk and predict the next attack method; S4.5. Build an ecosystem based on Threat Intelligence to collect multimodal threat information, use VirusTotalAPI to regularly update the malicious domain name blacklist, and visualize the threat situation through ElasticStack (Elasticsearch, Kibana).

[0010] The present invention is further configured to include a comprehensive data generation simulation module, a multi-source information relationship construction module, an innovative detection model design module, and a real-time monitoring defense mechanism module.

[0011] The present invention is further configured as follows: the comprehensive data generation simulation module includes an environment simulation page creation module, a user behavior reproduction module, a network activity playback module, an adversarial element injection module, and a behavior pattern recording module; The multi-source information relationship construction module includes a heterogeneous data modeling module, an information fusion mechanism module, a data integrity screening module, a similarity analysis mechanism module and a comprehensive task training module.

[0012] The present invention is further configured as follows: the innovative detection model design module includes a parallel processing architecture module, a dynamic data selection module, a difference detection architecture module, a joint task optimization module and a distributed learning strategy module; The real-time monitoring defense mechanism module includes an anti-interference training module, a dynamic model updating module, an instant risk assessment module, an attack behavior tracking module and a threat intelligence integration module.

[0013] The present invention is further configured as follows: the heterogeneous data modeling module includes a similarity calculation module, a feature extraction and preprocessing module, a contrast learning module, a dynamic network construction module, an anomaly detection module and a result integration feedback module.

[0014] Beneficial Effects Compared with the known public technology, the technical solution provided by the present invention has the following beneficial effects: 1. The present invention utilizes multimodal simulation and data generation technology to create phishing samples with high diversity and authenticity, providing a solid foundation for the model. The user behavior simulator designed by the reinforcement learning model generates diversified interactive behavior data, so that the system can simulate and understand the behavior of different types of users. The system improves detection accuracy and efficiency through a non-traditional multimodal fusion deep detection model combined with multi-branch neural networks and multi-task learning. Through adversarial training and federated learning, the system realizes real-time adaptive detection and defense capabilities, greatly improves the accuracy and robustness of phishing detection, and provides comprehensive protection for network security.

[0015] 2. The present invention consists of multiple modules that work together to ensure full process coverage from data generation to threat response. The comprehensive data generation simulation module creates rich training samples and enhances the model's perception ability through highly simulated web page environment and user behavior reproduction. The multi-source information relationship construction module ensures that the system can effectively integrate and understand information from different channels through data modeling and fusion. The innovative detection model design module provides advanced parallel processing and dynamic data selection functions to make the detection process more efficient. The real-time monitoring defense mechanism module ensures that the system can quickly adapt to and respond to emerging threats through anti-interference training and dynamic updates. Overall, the system significantly improves the accuracy and efficiency of phishing attack detection through multi-level data processing and real-time defense strategies.

[0016] 3. The present invention extracts the features of input data through convolutional neural networks, and these features are further passed to the similarity calculation module, which measures the similarity of data by calculating the cosine similarity between feature vectors. This process optimizes the feature representation, so that the contrastive learning module can more effectively distinguish similar and dissimilar samples. The dynamic network pruning module simplifies the neural network structure through the pruning algorithm, thereby improving the operating efficiency and simplicity of the model. The anomaly detection module uses the isolation forest algorithm to identify potential abnormal samples, ensuring that phishing attacks can be effectively detected and intercepted. The final result integration and feedback module integrates the detection results through a voting mechanism, thereby improving the accuracy and consistency of the system output. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A schematic diagram of a process of a phishing detection method based on multimodal fusion features of the present invention; Figure 2 A schematic diagram of system modules of a phishing detection system based on multimodal fusion features of the present invention; Figure 3 A schematic diagram of a heterogeneous data modeling module of a phishing detection system based on multimodal fusion features of the present invention. DETAILED DESCRIPTION

[0018] In order to make the technical problems, technical solutions and beneficial effects to be solved by the present invention clearer, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the described embodiments are only part of the embodiments of the present application, rather than all the embodiments, and the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of this application.

[0019] It should be further explained that the drawings and implementation modes of the present invention mainly describe the concept of the present invention. On the basis of this concept, the specific forms and settings of some connection relationships, positional relationships, power mechanisms, power supply systems, hydraulic systems and control systems may not be fully described. However, on the premise that those skilled in the art understand the concept of the present invention, those skilled in the art can implement the above-mentioned specific forms and settings in a well-known manner.

[0020] When an element is referred to as being "fixed to" or "disposed on" another element, it may be directly on the other element or indirectly on the other element. When an element is referred to as being "connected to" another element, it may be directly connected to the other element or indirectly connected to the other element.

[0021] The directional words "inside and outside" refer to the inside and outside relative to the outline of each component itself. The directions or positional relationships indicated by the terms "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside" and "outside" are based on the directions or positional relationships shown in the drawings and are only for the convenience of describing the present invention and simplifying the description. They do not indicate or imply that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore should not be understood as limiting the present invention.

[0022] For ease of description, spatially relative terms such as "above", "above", "on the upper surface of", "above", etc. may be used here to describe the spatial positional relationship between a device or feature and other devices or features as shown in the figure. It should be understood that spatially relative terms are intended to include different orientations of the device in use or operation in addition to the orientation described in the figure. For example, if the device in the accompanying drawings is inverted, the device described as "above other devices or structures" or "above other devices or structures" will be positioned as "below other devices or structures" or "below other devices or structures". Thus, the exemplary term "above" may include both "above" and "below". The device may also be positioned in other different ways, and the spatially relative descriptions used here are interpreted accordingly.

[0023] The terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, "multiple" means two or more, and "several" means one or more, unless otherwise clearly and specifically defined.

[0024] A phishing detection method and system based on multimodal fusion features provided by the present invention are now described.

[0025] Example 1 like Figure 1 As shown, the technical solution provided by the present invention is: a phishing detection method based on multimodal fusion features, comprising the following steps: S1. Construction of phishing samples based on multimodal simulation and data generation; S2, cross-modal dynamic relationship network modeling and feature conflict mining; S3. Non-traditional design and optimization of multi-modal fusion deep detection model; S4. Real-time adaptive phishing detection and defense ecosystem construction; In the step S1, constructing a phishing sample based on multimodal simulation and data generation: S1.1. Build a containerized phishing webpage generation environment through Docker and Kubernetes clusters, combine Selenium and Puppeteer automation tools to generate phishing webpages, use GAN (StyleGAN) to dynamically generate a mixture of real and phishing webpages, adjust style parameters (fonts, colors, and layouts), configure PhantomJS simulation rendering environment, ensure that the generated pages are consistent with the real scene, and integrate real-time traffic analysis tools (Wireshark) for testing; S1.2. Design a user behavior simulator using the Reinforcement Learning model (DQN), perform mouse trajectories, scroll operations, and click behaviors in the browser environment, use the WebRTC interface to collect response data of simulated behaviors, record the interactive behaviors in the MongoDB database, set user type parameters (beginners, ordinary users, and expert users), and adjust operation details (click delay, error rate) to enhance diversity; S1.3. Use the Tcpreplay tool to replay the real captured phishing attack traffic (DNS requests generated by DGA domain names) to the network, use SuricataIDS to monitor network layer behavior, use the Scapy module to dynamically insert forged traffic (abnormal HTTP / HTTPS requests), test the robustness of the detection model to abnormal patterns, and set the replay rate to 10Gbps to match the high-traffic environment; S1.4. Use AdversarialPatch technology to generate visual interference elements, inject small perturbations (pixel modification of brand logos) into phishing pages, use NLP data enhancement technology (WordEmbeddings replacement) to fine-tune the page text content while maintaining semantic integrity, deploy OpenCV and PyTorch for adversarial image generation, and extract text features through Spacy to achieve controllable deformation; S1.5. Use Elasticsearch to configure a time series database, record the time series behavior of users visiting web pages, analyze changes in user behavior patterns based on the ARIMA model and LSTM network, detect abnormal access frequency or behavior switching (suddenly clicking on phishing links in large numbers), and implement real-time time series data transmission through the Kafka streaming processing framework; In the step S1, constructing a phishing sample based on multimodal simulation and data generation: S1.1. Build a containerized phishing webpage generation environment through Docker and Kubernetes clusters, combine Selenium and Puppeteer automation tools to generate phishing webpages, use GAN (StyleGAN) to dynamically generate a mixture of real and phishing webpages, adjust style parameters (fonts, colors, and layouts), configure PhantomJS simulation rendering environment, ensure that the generated pages are consistent with the real scene, and integrate real-time traffic analysis tools (Wireshark) for testing; S1.2. Design a user behavior simulator using the Reinforcement Learning model (DQN), perform mouse trajectories, scroll operations, and click behaviors in the browser environment, use the WebRTC interface to collect response data of simulated behaviors, record the interactive behaviors in the MongoDB database, set user type parameters (beginners, ordinary users, and expert users), and adjust operation details (click delay, error rate) to enhance diversity; S1.3. Use the Tcpreplay tool to replay the real captured phishing attack traffic (DNS requests generated by DGA domain names) to the network, use SuricataIDS to monitor network layer behavior, use the Scapy module to dynamically insert forged traffic (abnormal HTTP / HTTPS requests), test the robustness of the detection model to abnormal patterns, and set the replay rate to 10Gbps to match the high-traffic environment; S1.4. Use AdversarialPatch technology to generate visual interference elements, inject small perturbations (pixel modification of brand logos) into phishing pages, use NLP data enhancement technology (WordEmbeddings replacement) to fine-tune the page text content while maintaining semantic integrity, deploy OpenCV and PyTorch for adversarial image generation, and extract text features through Spacy to achieve controllable deformation; S1.5. Use Elasticsearch to configure a time series database, record the time series behavior of users visiting web pages, analyze changes in user behavior patterns based on the ARIMA model and LSTM network, detect abnormal access frequency or behavior switching (suddenly clicking on phishing links in large numbers), and implement real-time time series data transmission through the Kafka streaming processing framework; In the step S3, non-traditional design and optimization of the multi-modal fusion depth detection model: S3.1. Imitating human multimodal information processing, a multi-branch neural network (BranchCNN) is used to perform specialized processing on image, text, and behavior features respectively. Combined with the SoftAttention mechanism, key modal information is passed to the fusion layer. Model training is based on ImageNet and IMDB datasets for transfer learning. S3.2. Design a modality selection module to dynamically decide whether to enable a modality through modality credibility indicators (data integrity, noise ratio). For example, when network traffic data is missing, only text and visual data are used to complete the detection. Use PyTorch to configure the dynamic Masking layer to achieve modality selection. S3.3. Train the contrast detector through Siamese network, compare the input modality with the target modality, output the similarity score, optimize the contrast loss function to distinguish normal pages from phishing pages, and use TPU (TensorProcessingUnit) to improve the model's processing ability for large-scale training samples; S3.4. Build a multi-task learning model that shares underlying features and optimizes phishing detection, malicious domain classification, and spam filtering tasks simultaneously, using a Transformer model with shared weights and trained using the HuggingFaceTransformers framework. S3.5. Asynchronous distributed training is implemented through Horovod. Separate training steps are set for vision, text, and interactive behaviors (the step size for vision modality is 0.01, and the step size for text modality is 0.05). BatchNormalization is used to calibrate the feature distribution differences between modalities. In the step S4, real-time adaptive phishing detection and countermeasure defense ecosystem construction: S4.1. Use the CleverHans tool to generate adversarial samples (blurred images, URL splicing perturbations), train the detection model based on adversarial training, and achieve a detection rate of more than 95% for adversarial attack samples. Configure the RTX3090 graphics card to run large-scale adversarial sample generation. S4.2. Use Amazon SageMaker to achieve online learning capabilities, input real-time streaming data to update model weights, and use the federated learning framework (TensorFlowFederated) to collaboratively update the model to ensure data privacy; S4.3. Deploy Apache Kafka to implement real-time data processing, and combine the risk scoring algorithm (Logistic Regression) to generate a risk score of 0-1 for each web page. Set the score threshold to 0.7. Pages above this value trigger in-depth analysis. S4.4. Use the Honeypot system to capture attacker behavior, return fake pages (fake forms) to suspected attackers, monitor attacker traffic patterns through Splunk and predict the next attack method; S4.5. Build an ecosystem based on Threat Intelligence to collect multimodal threat information, use VirusTotalAPI to regularly update the malicious domain name blacklist, and visualize the threat situation through ElasticStack (Elasticsearch, Kibana).

[0026] In this embodiment, a containerized environment is built through Docker and Kubernetes to generate phishing web pages, and a hybrid page is created in combination with GAN technology to ensure the authenticity and diversity of the samples. A user behavior simulator is designed using a reinforcement learning model, and various user interaction behaviors are recorded to enrich the data set. Tools such as Tcpreplay and SuricataIDS are used to reproduce and monitor phishing traffic, test the robustness of the system to abnormal patterns, and use AdversarialPatch and NLP technology to enhance data to challenge the detection model. Time series databases and streaming processing frameworks are used to monitor user behavior in real time and detect abnormal patterns. Then, multimodal fusion detection is achieved through deep learning architectures such as multi-branch neural networks and Siamese networks, and model performance is optimized using modal selection and multi-task learning. At the same time, Horovod is used to implement asynchronous distributed training to improve efficiency.

[0027] In the real-time detection and defense stage, adversarial training is used to enhance the model's anti-attack capabilities, combined with federated learning to achieve online updates, and real-time risk assessment and attack defense are performed through the Kafka and Honeypot systems. The threat intelligence ecosystem improves the understanding and response capabilities of threats by continuously updating blacklists and visualization tools. This method significantly improves the accuracy, efficiency and robustness of phishing detection through a systematic process of data generation, feature fusion, model optimization and real-time defense, providing comprehensive protection for network security.

[0028] Example 2 like Figure 1-2 As shown, the technical solution provided by the present invention is: a phishing detection system based on multimodal fusion features, including a comprehensive data generation simulation module for creating diverse and realistic training samples to support phishing detection, a multi-source information relationship construction module for integrating data from different sources, an innovative detection model design module for developing and optimizing detection models, and a real-time monitoring defense mechanism module for providing immediate detection and countermeasures to prevent phishing attacks; The comprehensive data generation simulation module includes an environment simulation page creation module for generating simulated web pages using virtualization technology and automation tools, a user behavior reproduction module for simulating the interactive behaviors of different users to capture diversified data, a network activity playback module for reproducing captured network traffic to test the system's abnormality recognition ability, an adversarial element injection module for adding visual and textual perturbations to evaluate the system's robustness, and a behavior pattern recording module for recording and analyzing time series changes in user access behaviors; The multi-source information relationship construction module includes a heterogeneous data modeling module for constructing a network for analyzing multi-source information relationships, an information fusion mechanism module for integrating different data forms into unified features using deep learning, a data integrity screening module for dynamically selecting suitable data information, a similarity analysis mechanism module for distinguishing normal and abnormal data patterns through comparative learning, and a comprehensive task training module for simultaneously optimizing multiple security-related tasks; The innovative detection model design module includes a parallel processing architecture module for implementing a parallel processing structure, a dynamic data selection module for adopting a flexible data selection mechanism to cope with data missing, a difference detection architecture module for introducing contrastive learning technology to enhance phishing page detection, a joint task optimization module for improving the collaborative efficiency of different security tasks using a joint learning model, and a distributed learning strategy module for deploying a distributed training method to accelerate model development and deployment; The real-time monitoring defense mechanism module includes an anti-interference training module for improving the model's ability to resist attacks through interference sample training, a dynamic model update module for realizing continuous online updating of the model to adapt to new threats, an instant risk assessment module for quickly evaluating potential threats using a real-time processing system, an attack behavior tracking module for identifying and analyzing attacker behavior patterns through a trapping system, and a threat intelligence integration module for establishing an intelligence sharing platform to continuously update the threat information library.

[0029] In this embodiment, a comprehensive data generation simulation module is used to create diverse and realistic training samples. Among them, the environment simulation page creation module generates simulated web pages to provide a platform for the user behavior reproduction module to capture diverse data interactions. Then the network activity playback module reproduces these interactive behaviors and adds disturbances in combination with the adversarial element injection module to enhance the robustness of the system. Finally, the behavior pattern recording module records the access behavior time series and analyzes the system's response to disturbances. The multi-source information relationship construction module integrates multi-source information through heterogeneous data modeling and information fusion mechanism modules, selects the best data through the data integrity screening module, and uses the similarity analysis mechanism module to distinguish normal and abnormal data patterns. In addition, multiple security tasks are optimized in the comprehensive task training module, and the innovative detection model design module is used. The block provides parallel processing and dynamic data selection capabilities, strengthens phishing detection through the difference detection architecture module, improves task collaboration efficiency through the joint task optimization module, and accelerates model development and deployment through the distributed learning strategy module. The real-time monitoring defense mechanism module improves defense capabilities through anti-interference training, the dynamic model update module adapts to new threats, the instant risk assessment module quickly responds to potential threats, the attack behavior tracking module identifies attacker behavior, and the threat intelligence integration module continuously updates the threat information library. The system improves detection accuracy, efficiency and robustness through multi-level feature generation, data fusion, model optimization and real-time defense, providing strong support for network security.

[0030] Among them: the environment simulation page creation module generates a simulated web page environment to provide a realistic test platform for the user behavior reproduction module, so that the user behavior reproduction can capture diverse data interactions on the simulated web page; the user behavior reproduction module simulates the interactive behaviors of different users and records these interactions, and the network activity playback module uses these records to reproduce network traffic in order to test the system's abnormal recognition ability; when the network activity playback module reproduces the traffic, the adversarial element injection module can add visual and text disturbances on this basis to evaluate the system's robustness and performance under different disturbances; during the disturbance test of the adversarial element injection module, the behavior pattern recording module records the user's access behavior time series under different disturbances to analyze the system's response to the disturbance; The heterogeneous data modeling module constructs a relational network for analyzing multi-source information, and the information fusion mechanism module uses these relational networks to integrate different data forms into unified features through deep learning; the data features integrated by the information fusion mechanism module are passed to the data integrity screening module, which dynamically selects the best data to ensure the accuracy and completeness of model training; the screening data provided by the data integrity screening module is input into the similarity analysis mechanism module, which further distinguishes normal and abnormal data patterns through comparative learning to improve the detection accuracy of the system; the distinction model optimized by the similarity analysis mechanism module is handed over to the comprehensive task training module, which further optimizes on multiple security tasks to improve the synergistic effectiveness of the overall model.

[0031] Among them: the parallel processing architecture module provides an efficient data processing environment, and the dynamic data selection module flexibly selects data in this environment to deal with data missing and imbalance problems; the flexible data provided by the dynamic data selection module is passed to the difference detection architecture module, which enhances the detection ability of phishing pages through comparative learning technology; the detection results optimized by the difference detection architecture module are further handed over to the joint task optimization module to improve the collaborative efficiency of different security tasks and achieve more comprehensive security detection; the task model optimized by the joint task optimization module is deployed in the distributed learning strategy module, and the model development and deployment process is accelerated through the distributed training method to improve the system response speed; The heterogeneous data modeling module constructs a relational network for analyzing multi-source information, and the information fusion mechanism module uses these relational networks to integrate different data forms into unified features through deep learning; the data features integrated by the information fusion mechanism module are passed to the data integrity screening module, which dynamically selects the best data to ensure the accuracy and integrity of model training; the screening data provided by the data integrity screening module is input into the similarity analysis mechanism module, which further distinguishes normal and abnormal data patterns through comparative learning to improve the detection accuracy of the system; the distinction model optimized by the similarity analysis mechanism module is handed over to the comprehensive task training module, which is further optimized on multiple security tasks to improve the synergistic effectiveness of the overall model; The anti-interference training module improves the model's anti-attack capability, and the dynamic model update module can update the model in time after obtaining new threat intelligence and attack samples to maintain adaptability to new threats; the latest model provided by the dynamic model update module is used in the instant risk assessment module to evaluate potential threats in real time and reduce potential losses through rapid response; after the instant risk assessment module detects a potential threat, the attack behavior tracking module further identifies and analyzes the attacker's behavior pattern to provide a reference for threat response; the attacker behavior information collected by the attack behavior tracking module is passed to the threat intelligence integration module, which integrates and updates the intelligence library to continuously improve the system's threat detection capability.

[0032] Example 3 like Figure 1-3 As shown, the technical solution provided by the present invention is: a phishing detection system based on multimodal fusion features, the heterogeneous data modeling module includes a function for extracting features through a convolutional neural network, using PyTorch to extract features (F) from input data (X) using a convolution kernel (K) through the formula Implement the feature extraction and preprocessing module of the convolution layer, and use NumPy's built-in function to calculate the cosine similarity (S) formula for two feature vectors (A) and (B). A similarity calculation module for performing calculations, a contrastive learning module for introducing a contrastive loss function to optimize feature representation, a dynamic network pruning module for reducing unimportant neuron connections through a pruning algorithm, an anomaly detection module for detecting abnormal samples based on an isolation forest method, and a result integration and feedback module for integrating the detection results through a voting mechanism; The feature extraction and preprocessing module uses a convolutional neural network to perform preliminary processing and feature extraction on the input data to generate a feature vector and pass it to the similarity calculation module, which uses NumPy's built-in function to calculate the cosine similarity between the feature vectors to measure the similarity between the input data; the similarity calculation module uses the calculated similarity information to optimize the feature representation and passes it to the contrastive learning module to introduce a contrast loss function to enhance the model's ability to distinguish similar and dissimilar samples; the contrastive learning module generates a simplified model after optimizing the feature representation and passes it to the dynamic network pruning module to reduce unimportant neuron connections in the neural network through a pruning algorithm to improve the efficiency and simplicity of the model; the dynamic network pruning module outputs the optimized model after reducing the neuron connections and passes it to the anomaly detection module to detect abnormal samples using an isolation forest-based method to identify potential phishing attacks; the anomaly detection module identifies potential abnormal samples and passes the results to the result integration and feedback module to integrate the detection results through a voting mechanism to improve the accuracy and consistency of the final output.

[0033] In this embodiment, the feature extraction and preprocessing module uses a convolutional neural network (CNN) to perform preliminary processing on the input data and extract valid feature vectors, which are then passed to the similarity calculation module, and the cosine similarity between the feature vectors is calculated by NumPy to preliminarily evaluate the similarity between the data. The contrastive learning module optimizes the feature representation by introducing a contrastive loss function to enhance the model's ability to distinguish between similar and dissimilar samples. The optimized model enters the dynamic network pruning module, and the pruning algorithm is used to reduce unimportant neuronal connections, thereby improving the efficiency and simplicity of the model. The anomaly detection module uses the isolation forest method to identify potential abnormal samples and effectively detect phishing attacks. The result integration and feedback module integrates the detection results through a voting mechanism to ensure the accuracy and consistency of the final output.

[0034] Among them: The calculation formula of the contrast loss function is .

[0035] in: is the Euclidean distance between samples, is a label (0 for similarity, 1 for dissimilarity), is the boundary distance.

[0036] How it works like Figure 1-3 As shown, in actual use, the present invention provides a comprehensive solution for phishing detection through multimodal simulation and data generation technology. First, the system generates hybrid pages by utilizing a containerized environment built with Docker and Kubernetes, as well as GAN technology. This method ensures the authenticity and diversity of samples, providing a solid foundation for subsequent detection. Next, the system uses a reinforcement learning model to design a user behavior simulator to record diverse user interaction behaviors. Using Tcpreplay and SuricataIDS tools, the system is able to reproduce and monitor phishing traffic, and enhance the challenge of the data through AdversarialPatch and NLP technology, thereby improving the robustness of the model under different interferences.

[0037] Through the environment simulation page creation module, the system generates a highly simulated web page environment, providing a realistic and credible platform for the user behavior reproduction module. This enables the user behavior reproduction module to effectively capture diverse data interactions. Subsequently, these interactive behaviors are reproduced by the network activity playback module, and combined with the adversarial element injection module, the robustness of the system is further enhanced through visual and text perturbations. Finally, the behavior pattern recording module records the time series of user access behaviors and analyzes the system's response performance under different perturbation conditions.

[0038] In terms of information integration, the multi-source information relationship construction module successfully integrates information from different sources through heterogeneous data modeling and information fusion mechanism modules. In addition to information integration and consolidation, the data integrity screening module selects the best data to ensure the accuracy and reliability of the information. The similarity analysis mechanism module uses these screened data to accurately distinguish between normal and abnormal data patterns, enabling the system to make correct judgments in complex and changing data environments. The comprehensive task training module optimizes a variety of security tasks to improve the overall collaborative efficiency of the detection model.

[0039] The innovative detection model design module provides technical support for the efficient operation of the system. This module enhances the efficiency of the system in data processing and analysis through parallel processing and dynamic data selection capabilities. The difference detection architecture module strengthens the system's ability to detect phishing pages, while the joint task optimization module improves the collaborative efficiency of different security tasks, ensuring that the system can still operate efficiently in a multi-task environment. The distributed learning strategy module uses modern distributed technology to accelerate the development and deployment of models, which significantly improves the system's response speed and adaptability.

[0040] In terms of defense mechanisms, the real-time monitoring defense mechanism module provides instant response and adaptability. The anti-interference training module enhances the model's ability to resist multiple attacks by introducing interference samples. The dynamic model update module ensures that the system can continuously adapt to emerging threats, while the real-time risk assessment module quickly identifies and responds to potential threats. The attack behavior tracking module deeply analyzes the attacker's behavior and cooperates with the threat intelligence integration module to continuously update and enrich the threat information library.

[0041] In the heterogeneous data modeling module, the feature extraction and preprocessing module extracts features from the input data through a convolutional neural network, generates feature vectors and passes them to the similarity calculation module. NumPy is used to calculate the cosine similarity between these vectors to measure the similarity of the data. The optimized feature representation is passed to the contrastive learning module, which introduces a contrastive loss function to enhance the model's ability to distinguish between similar and dissimilar samples. The optimized model enters the dynamic network pruning module, which uses a pruning algorithm to reduce unimportant neuronal connections and improve the efficiency and simplicity of the model. Finally, the anomaly detection module uses the isolation forest method to identify potential anomaly samples to ensure that phishing attacks can be effectively detected. The result integration and feedback module integrates the detection results through a voting mechanism to improve the accuracy and consistency of the output results.

[0042] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

[0043] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0044] Unless otherwise specifically stated, the relative arrangement, numerical expressions and numerical values ​​of the parts and steps described in these embodiments do not limit the scope of the present application. At the same time, it should be understood that, for ease of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship. The technology, methods and equipment known to ordinary technicians in the relevant field may not be discussed in detail, but in appropriate cases, the technology, methods and equipment should be regarded as a part of the authorization specification. In all examples shown and discussed here, any specific value should be interpreted as being merely exemplary, rather than as a limitation. Therefore, other examples of exemplary embodiments may have different values. It should be noted that similar reference numerals and letters represent similar items in the following drawings, so once a certain item is defined in an accompanying drawing, it does not need to be further discussed in subsequent drawings.

Claims

1. A phishing detection method based on multimodal fusion features, characterized in that: The following steps are involved: S1. Construction of phishing samples based on multimodal simulation and data generation; S2, cross-modal dynamic relationship network modeling and feature conflict mining; S3. Non-traditional design and optimization of multi-modal fusion deep detection model; S4. Real-time adaptive phishing detection and countermeasure defense ecosystem construction.

2. The phishing detection method based on multimodal fusion features according to claim 1 is characterized by: In the step S1, the phishing sample is constructed based on multimodal simulation and data generation: S1.

1. Build a containerized phishing webpage generation environment through Docker and Kubernetes clusters, combine Selenium and Puppeteer automation tools to generate phishing webpages, and use GAN to dynamically generate a mixed page of real and phishing webpages to adjust style parameters; S1.

2. Design a user behavior simulator using the ReinforcementLearning model, execute mouse tracks, scrolling operations, and click behaviors in the browser environment, and use the WebRTC interface to collect response data of simulated behaviors and record interactive behaviors in the MongoDB database; S1.

3. Use the Tcpreplay tool to replay the real captured phishing attack traffic to the network, use SuricataIDS to monitor the network layer behavior, and use the Scapy module to dynamically insert fake traffic to test the robustness of the detection model to abnormal patterns. S1.4, using AdversarialPatch technology to generate visual interference elements, injecting small perturbations into phishing pages, using NLP data augmentation technology to fine-tune the page text content while maintaining semantic integrity, deploying OpenCV and PyTorch for adversarial image generation and extracting text features through Spacy to achieve controllable deformation; S1.

5. Use Elasticsearch to configure a time series database to record the time series behavior of users visiting web pages. Analyze changes in user behavior patterns based on the ARIMA model and LSTM network to detect abnormal access frequency or behavior switching, and realize real-time time series data transmission through the Kafka streaming processing framework.

3. The phishing detection method based on multimodal fusion features according to claim 1, characterized in that: In the step S2 of cross-modal dynamic relationship network modeling and feature conflict mining: S2.

1. Use KnowledgeGraph to build a semantic knowledge graph of web page content and capture the semantic conflict between text and image through SpaCy and GloVe models. S2.2, construct a sparse relationship matrix, sparse the visual features, text features and interaction features into a unified matrix, and use Graph Convolutional Network to learn the dependency relationship of feature nodes; S2.3, introduce the SimCLR self-supervised learning model, train the similarity detector between modalities, classify the correlation between modalities through the contrast loss function, define the pseudo correlation as low-confidence correlation, and set the NVIDIA A100 GPU cluster to run large-scale sample training; S2.

4. Design a dynamic feature fuser based on Transformer to embed modal features into a unified space, dynamically adjust the weight distribution of each modality through the Reinforcement Learning policy network, and use the distributed deep learning framework provided by AWS SageMaker to accelerate training; S2.

5. Use the random walk algorithm to generate relationship chains between modalities, use the PyKEEN framework to visualize the relationship chains, and calculate the confidence scores of abnormal paths.

4. The phishing detection method based on multimodal fusion features according to claim 1, characterized in that: In the non-traditional design and optimization of the multi-modal fusion depth detection model in step S3: S3.1, imitate human multimodal information processing, use multi-branch neural network to specialize image, text and behavior features respectively, combine with SoftAttention mechanism, and pass key modal information to the fusion layer; S3.

2. Design a modality selection module to dynamically decide whether to enable a modality based on the modality credibility index, and use PyTorch to configure a dynamic Masking layer to implement modality selection; S3.3, train the contrast detector through Siamese network, compare the input modality with the target modality, output the similarity score, and use TPU to improve the model's processing ability for large-scale training samples; S3.

4. Build a multi-task learning model that shares underlying features and optimizes phishing detection, malicious domain classification, and spam filtering tasks simultaneously, using a Transformer model with shared weights and trained using the HuggingFaceTransformers framework. S3.

5. Implement asynchronous distributed training through Horovod, set separate training steps for vision, text, and interactive behaviors, and use BatchNormalization to calibrate the feature distribution differences between modalities.

5. The phishing detection method based on multimodal fusion features according to claim 1, characterized in that: In step S4, real-time adaptive phishing detection and countermeasure defense ecosystem construction: S4.

1. Use the CleverHans tool to generate adversarial samples and train the detection model based on adversarial training. The detection rate of adversarial attack samples reaches more than 95%; S4.

2. Implement online learning capabilities through Amazon SageMaker, input real-time streaming data to update model weights, and use a federated learning framework to collaboratively update the model; S4.

3. Deploy Apache Kafka to implement real-time data processing and combine the risk scoring algorithm to generate a risk score of 0-1 for each web page; S4.

4. Use the Honeypot system to capture attacker behavior and return the forged page to the suspected attacker. Use Splunk to monitor the attacker's traffic pattern and predict the next attack method. S4.

5. Build an ecosystem based on Threat Intelligence, collect multimodal threat information, use VirusTotalAPI to regularly update the malicious domain name blacklist, and visualize the threat situation through ElasticStack.

6. A phishing detection method based on multimodal fusion features according to any one of claims 1 to 5, characterized in that: The phishing detection system includes a comprehensive data generation simulation module, a multi-source information relationship construction module, an innovative detection model design module, and a real-time monitoring defense mechanism module.

7. The phishing detection system based on multimodal fusion features according to claim 6, characterized in that: The comprehensive data generation simulation module includes an environment simulation page creation module, a user behavior reproduction module, a network activity playback module, an adversarial element injection module, and a behavior pattern recording module; The multi-source information relationship construction module includes a heterogeneous data modeling module, an information fusion mechanism module, a data integrity screening module, a similarity analysis mechanism module and a comprehensive task training module.

8. The phishing detection system based on multimodal fusion features according to claim 6, characterized in that: The innovative detection model design module includes a parallel processing architecture module, a dynamic data selection module, a difference detection architecture module, a joint task optimization module and a distributed learning strategy module; The real-time monitoring defense mechanism module includes an anti-interference training module, a dynamic model updating module, an instant risk assessment module, an attack behavior tracking module and a threat intelligence integration module.

9. The phishing detection system based on multimodal fusion features according to claim 7, characterized in that: The heterogeneous data modeling module includes a similarity calculation module, a feature extraction and preprocessing module, a contrastive learning module, a dynamic network construction module, an anomaly detection module and a result integration feedback module.

Images