Phishing email dynamic defense decision method and system based on reinforcement learning

Abstract

The present invention provides a phishing email dynamic defense decision method and system based on reinforcement learning, which relates to the field of network security technology, including: obtaining historical phishing email data, performing feature extraction, generating multi-dimensional features and constructing a high-quality phishing email feature data set, constructing a phishing email dynamic defense decision model and training it, encapsulating the trained phishing email dynamic defense decision model into a standardized interface for integration, obtaining newly received emails, extracting initial email features, performing initial judgment, performing deep feature extraction, performing secondary screening, generating accurate recognition results, and if it is a phishing email, generating a phishing email attack portrait and an active defense strategy; outputting defense action instructions, sending them to defense execution nodes, obtaining user feedback and behavior data in real time, evaluating the global phishing email attack situation and defense effect, and updating the phishing email dynamic defense decision model in combination with analysis of feedback opinions.

Description

Technical Field

[0001] The present invention relates to the field of network security technology, and in particular to a phishing email dynamic defense decision-making method and system based on reinforcement learning. Background Art

[0002] With the rapid development of the Internet and the widespread use of email, phishing emails have become one of the most serious cybersecurity threats. Phishing attacks account for 36% of cybersecurity incidents, ranking first, posing a huge threat to personal privacy and the security of critical information infrastructure.

[0003] Traditional phishing email defense mostly uses feature engineering and blacklist and whitelist methods, which manually extract the significant features of phishing emails and build detection rules, and intercept them in combination with malicious domain names, IP address blacklists, etc. There are problems such as untimely rule updates and insufficient generalization capabilities, which make it difficult to cope with the ever-changing phishing emails. In recent years, machine learning methods have been widely used in the field of phishing email detection, but most existing defense methods regard phishing email detection as a static classification problem, using pre-trained static models, lacking dynamic update and adaptive adjustment capabilities, and are difficult to effectively deal with the ever-evolving phishing attack methods;

[0004] Therefore, a solution is urgently needed to solve the problems existing in the prior art. Summary of the invention

[0005] The embodiments of the present invention provide a method and system for dynamic defense decision-making against phishing emails based on reinforcement learning, which can at least solve some of the problems existing in the prior art.

[0006] A first aspect of an embodiment of the present invention provides a method for dynamic defense decision-making against phishing emails based on reinforcement learning, comprising:

[0007] Obtain multi-source heterogeneous historical phishing email data, extract features from the historical phishing email data through natural language processing technology and knowledge graph technology, generate multi-dimensional features, and determine the semantic structure and speech pattern in the multi-dimensional features through machine learning algorithms to construct a high-quality phishing email feature data set, build a phishing email dynamic defense decision model and train it based on a reinforcement learning algorithm, dynamically adjust sample utilization and feature weights in combination with an adaptive multi-scale exploration strategy and a priority experience playback mechanism, adjust defense decisions in combination with risk aversion mechanisms and cost constraints, and obtain a trained phishing email dynamic defense decision model;

[0008] The trained phishing email dynamic defense decision model is encapsulated into a standardized interface and integrated with a pre-set email security protection system. Newly received emails are obtained in real time through the email sending and receiving system, and initial email features corresponding to the newly received emails are extracted. An initial judgment is made through the phishing email dynamic defense decision model. Based on the initial judgment result, deep feature extraction is performed through heuristic analysis and behavioral sandbox, and secondary identification is performed in combination with context information to generate an accurate recognition result. If the accurate recognition result indicates that the current email is a phishing email, a phishing email attack portrait is generated, and an active defense strategy is generated according to the initial email features.

[0009] Based on the active defense strategy, multi-level progressive defense action instructions are output, and the defense action instructions are sent to the defense execution node through a standard interface to perform phishing email defense. During the defense process, user feedback and behavior data are obtained in real time. Based on the user feedback and behavior data, the high-quality phishing email feature data set is expanded through traffic mirroring and honeypot trapping. The global phishing email attack situation and defense effect are evaluated in combination with the security event correlation analysis algorithm. Based on the global phishing email attack situation and the defense effect, combined with the analysis feedback from domain experts, the action space and state space of the phishing email dynamic defense decision model are adaptively updated through an online learning algorithm, and the optimization is repeated until the model converges.

[0010] In an optional embodiment,

[0011] Obtain multi-source heterogeneous historical phishing email data, extract features from the historical phishing email data through natural language processing technology and knowledge graph technology, generate multi-dimensional features, and determine the semantic structure and speech pattern in the multi-dimensional features through machine learning algorithms to construct a high-quality phishing email feature data set, build a phishing email dynamic defense decision model and train it based on a reinforcement learning algorithm, dynamically adjust sample utilization and feature weights in combination with an adaptive multi-scale exploration strategy and a priority experience playback mechanism, and adjust defense decisions in combination with risk aversion mechanisms and cost constraints. The trained phishing email dynamic defense decision model includes:

[0012] Acquire multi-source heterogeneous historical phishing email data from public phishing email datasets and email service provider databases, perform deduplication, denoising and format standardization on the historical phishing email data, remove redundant emails and perform data encoding on the retained emails, parse the retained emails into a structured data format in combination with a pre-set email parsing library, extract email content based on the structured data format, perform sensitive information desensitization on the email content, and obtain standard email content;

[0013] Based on the standard email content, the lexical features, grammatical features and semantic features corresponding to the standard email content are extracted respectively by natural language processing technology to obtain language features comprehensively, entities, attributes and relationships are extracted from the standard email content by exponential graph technology, semantic triples are generated and linked and fused with existing general knowledge graphs and industry knowledge graphs, a phishing email semantic knowledge graph is constructed and graph features are extracted, the deep semantic relationships corresponding to the graph features are identified by semantic reasoning technology, and based on the deep semantic relationships, the graph features and the language features are fused to obtain the multi-dimensional features;

[0014] Based on the multi-dimensional features, the standard email content is decomposed into multiple topics through a topic modeling algorithm, each topic is assigned and the main semantic topic corresponding to the current email is determined based on the topic assignment result, the main semantic topics are combined to obtain the semantic structure, the complex sentence patterns and imperative statements corresponding to the standard email content are identified through dependency grammar analysis technology, and the speech pattern is extracted in combination with a convolutional neural network, representative features from the multi-dimensional features are selected through a feature importance analysis method, semantic features are extracted in combination with the semantic structure and the speech pattern, and the high-quality phishing email feature dataset is constructed based on the extracted semantic features;

[0015] The multi-dimensional features are used as state representations to generate a state space, define multi-level progressive defense actions, construct a defense strategy space based on the defense actions, construct a reward function based on the effectiveness and response speed of the defense actions, and construct a phishing email dynamic defense decision model based on the state space, the defense strategy space and the reward function. The phishing email dynamic defense decision model is trained through a reinforcement learning algorithm. During the strategy search process, the exploration amplitude is dynamically adjusted through an adaptive multi-scale exploration mechanism, the search depth and breadth are adjusted according to uncertainty and benefit changes, the empirical data of different transfer states are prioritized according to value, the sampling probability is adjusted according to the sorting result, the defense actions are labeled according to a pre-set risk threshold, high-risk defense actions are determined and punished in combination with a risk aversion mechanism, the defense actions and defense decisions are adjusted in combination with cost constraints, the defense strategy space is updated, and an optimized phishing email dynamic defense decision model is obtained.

[0016] In an optional embodiment,

[0017] The defensive actions and defensive decisions are adjusted in combination with the cost constraint as shown in the following formula:

[0018]

[0019] Among them, θ represents the parameters of the policy network, represents the gradient of the objective function J(θ) with respect to the parameter θ, is the expected value, indicating that under the current strategy π θ The weighted average of all possible future state-action sequences under the condition of T represents the total number of time steps, π θ (a t |s t ) means that in a given state s t Next, strategy π θ Select action a t The probability of Q(s t , a t ) is the state-action value function, which means that in state s t Take action a t After that, the expected value of the cumulative reward that may be obtained in the future, λ represents the cost weight coefficient, C(s t , a t ) is in state s t Take action a t The defense cost represents the resource consumption required to perform the defense behavior. represents the gradient operator on the policy network parameters θ.

[0020] In an optional embodiment,

[0021] The trained phishing email dynamic defense decision model is encapsulated as a standardized interface and integrated with a pre-set email security protection system. Newly received emails are obtained in real time through an email sending and receiving system, and initial email features corresponding to the newly received emails are extracted. An initial judgment is made through the phishing email dynamic defense decision model. Based on the initial judgment result, deep feature extraction is performed through heuristic analysis and behavioral sandbox, and secondary identification is performed in combination with context information to generate an accurate recognition result. If the accurate recognition result indicates that the current email is a phishing email, a phishing email attack portrait is generated. The active defense strategy generated according to the initial email features includes:

[0022] Encapsulate the trained phishing email dynamic defense decision model into a standardized interface, deploy the standardized interface into a pre-set email security protection system, and integrate it with the existing email processing process;

[0023] Acquire newly received emails in real time through the email sending and receiving system in the integrated email security protection system, perform multi-angle feature extraction on each newly received email, including text feature extraction, image feature extraction and attachment feature extraction, convert the extracted features into standard feature vectors, and combine them to obtain the initial email features;

[0024] Based on the initial email feature, the initial email feature is sent to the phishing email dynamic defense decision model in combination with traffic mirroring, and a nonlinear transformation is performed in a pre-set feedforward neural network to generate a threat level and confidence. Based on the threat level, it is determined whether the current email is a suspected phishing email, and the email marked as a suspected phishing email is sent to the re-determination system in the email security protection system. The re-determination system detects suspicious features in the suspected phishing email based on a heuristic analysis algorithm in combination with pre-set rules and feature libraries, and adds the attachments and links of the suspected phishing email to the behavioral sandbox for execution in combination with a behavioral sandbox analysis algorithm, monitors the running behavior and extracts dynamic features, obtains the sender's historical reputation and recipient feedback information corresponding to the suspected phishing email as context information, and fuses the context information with the suspicious features and the dynamic features to generate a second email feature. Based on the second email feature, a secondary identification is performed through the phishing email dynamic defense decision model to generate an accurate recognition result;

[0025] If the accurate identification result indicates that the current email is a phishing email, the attack target, disguise technique, bait content and malicious payload of the current email are read as key attack features, and a structured phishing email attack portrait is generated based on the key attack features. Based on the phishing email attack portrait and the initial email features, a pre-defined defense strategy template is matched to generate the active defense strategy.

[0026] In an optional embodiment,

[0027] The confidence corresponding to the threat level output by the feedforward neural network is shown in the following formula:

[0028]

[0029] Among them, p k Indicates the confidence that the current sample belongs to the kth threat level, is the weight of the k-th neuron in the L-th layer, representing the connection weight from the i-th neuron in the L-1-th layer to the k-th neuron in the L-th layer, represents the activation value of the i-th neuron in the L-1th layer, represents the bias term of the kth neuron in the Lth layer, K represents the total number of threat levels, is the weight of the jth neuron in the Lth layer, representing the connection weight from the i-th neuron in the L-1th layer to the j-th neuron in the Lth layer, Represents the bias term of the j-th neuron in the L-th layer.

[0030] In an optional embodiment,

[0031] Based on the active defense strategy, a multi-level progressive defense action instruction is output, and the defense action instruction is sent to the defense execution node through a standard interface to perform phishing email defense. In the defense process, user feedback and behavior data are obtained in real time. Based on the user feedback and behavior data, the high-quality phishing email feature data set is expanded through traffic mirroring and honeypot trapping. The global phishing email attack situation and defense effect are evaluated in combination with a security event correlation analysis algorithm. Based on the global phishing email attack situation and the defense effect, combined with the analysis feedback opinions issued by domain experts, the action space and state space of the phishing email dynamic defense decision model are adaptively updated through an online learning algorithm. Repeated optimization until the model converges includes:

[0032] Based on the priority and confidence of the active defense strategy, the defense level corresponding to each strategy is determined. In each defense level, the type and defense parameters corresponding to the active defense strategy are mapped to the corresponding defense action primitives. The defense action primitives are combined to obtain the multi-level progressive defense action instructions. The defense action instructions are sent to the defense execution node through a pre-set standard interface, wherein the defense action instructions include an early warning layer, a blocking layer and a tracing layer. The early warning layer warns of phishing risks through alarms, prompts and notifications. The blocking layer isolates the actual harm corresponding to phishing emails through isolation, interception and restriction actions. The tracing layer identifies the source of phishing emails through tracking and evidence collection.

[0033] When the defense execution node executes the defense action instruction, the feedback information and behavior data of the current user are obtained in real time. Based on the feedback information and behavior data, a traffic mirroring port is set next to the mail gateway through a traffic mirroring algorithm, the mail traffic is copied in real time, and parsed and extracted according to the mail protocol type to obtain a structured mail object, the structured mail object is parsed and added to the high-quality phishing mail feature data set, a honeypot mail account group is set up through a honeypot trapping algorithm and the account information is made public, the honeypot suspicious mails received by each honeypot mail account are regularly collected and the content is extracted, the honeypot suspicious mails are manually reviewed and annotated, and the honeypot suspicious mails are added to the high-quality phishing mail feature data set according to the annotated results;

[0034] Based on the updated high-quality phishing email feature data set and in combination with the security incident correlation analysis algorithm, the phishing email related data is correlated and analyzed with the pre-divided security incident data to identify phishing attack activities and phishing email attack chains, and the global phishing email attack situation is determined in combination with the attack scale, attack frequency and attack success rate. Based on the global phishing email attack situation, the defense effect of the current defense action instruction is determined in combination with the execution result and effect data corresponding to the current defense action instruction;

[0035] Based on the global phishing email attack situation and the defense effect, analysis feedback opinions given by domain experts are obtained, and structured feedback data is generated in combination with domain knowledge and experience. Through an incremental learning algorithm, the state space, action space and reward function of the phishing email dynamic defense decision model are locally updated based on the structured feedback data and the updated high-quality phishing email feature data set, and the update is repeated until the preset maximum number of updates is reached or the phishing email dynamic defense decision model converges.

[0036] In an optional embodiment,

[0037] Identify phishing attack activities and phishing email attack chains, and determine the global phishing email attack situation by combining the attack scale, attack frequency, and attack success rate as shown in the following formula:

[0038]

[0039] Where I represents the global phishing email attack situation value, w S Indicates the weight coefficient of the attack scale, N a represents the total number of phishing attacks identified within the time window, w F represents the weight coefficient of attack frequency, TI represents the length of the time window, and w R Represents the weight coefficient of attack success power, N S Indicates the number of successful phishing emails, N t represents the total number of phishing emails, w C Represents the attack chain complexity weight coefficient, N c Indicates the total number of identified attack chains, L e Indicates the length of the e-th attack chain, and e indicates the index of the attack chain.

[0040] A second aspect of an embodiment of the present invention provides a phishing email dynamic defense decision system based on reinforcement learning, including:

[0041] The first unit is used to obtain multi-source heterogeneous historical phishing email data, extract features from the historical phishing email data through natural language processing technology and knowledge graph technology, generate multi-dimensional features, and determine the semantic structure and speech pattern in the multi-dimensional features through machine learning algorithms to construct a high-quality phishing email feature data set, build a phishing email dynamic defense decision model and train it based on a reinforcement learning algorithm, dynamically adjust sample utilization and feature weights in combination with an adaptive multi-scale exploration strategy and a priority experience playback mechanism, and adjust defense decisions in combination with risk aversion mechanisms and cost constraints to obtain a trained phishing email dynamic defense decision model;

[0042] The second unit is used to encapsulate the trained phishing email dynamic defense decision model into a standardized interface and integrate it with a pre-set email security protection system, obtain newly received emails in real time through the email sending and receiving system, extract the initial email features corresponding to the newly received emails, make an initial judgment through the phishing email dynamic defense decision model, and based on the initial judgment result, perform deep feature extraction through heuristic analysis and behavioral sandbox, perform secondary identification in combination with context information, and generate an accurate recognition result. If the accurate recognition result indicates that the current email is a phishing email, a phishing email attack portrait is generated, and an active defense strategy is generated according to the initial email features;

[0043] The third unit is used to output multi-level progressive defense action instructions based on the active defense strategy, send the defense action instructions to the defense execution node through the standard interface and perform phishing email defense, obtain user feedback and behavior data in real time during the defense process, expand the high-quality phishing email feature data set through traffic mirroring and honeypot trapping based on the user feedback and behavior data, evaluate the global phishing email attack situation and defense effect in combination with the security event correlation analysis algorithm, based on the global phishing email attack situation and the defense effect, combined with the analysis feedback opinions issued by domain experts, adaptively update the action space and state space of the phishing email dynamic defense decision model through an online learning algorithm, and repeat the optimization until the model converges.

[0044] According to a third aspect of the embodiments of the present invention,

[0045] An electronic device is provided, comprising:

[0046] processor;

[0047] a memory for storing processor-executable instructions;

[0048] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0049] A fourth aspect of the embodiments of the present invention is:

[0050] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0051] In the present invention, the machine learning algorithm is used to automatically optimize the feature combination, which overcomes the limitations of manually designed features, effectively reduces the false alarm rate while improving the detection rate, introduces reinforcement learning theory, and explores and utilizes mechanisms such as balance and priority experience playback, so that the defense decision model can continuously optimize itself in the phishing attack and defense confrontation, learn the changing rules of the attacker's strategy, and dynamically adjust the defense strategy, so as to maintain a stable defense effect in a complex and changeable security environment. Through multi-level progressive active defense measures, a closed loop is formed from threat intelligence perception, real-time behavior analysis, attack situation assessment and other aspects, which blocks the attack at the first time while actively responding to unknown Threat profiling is performed to proactively hunt for potential risks, which minimizes the living space of phishing attacks. Online learning paradigm is used to continuously acquire actual operation data through user feedback, traffic mirroring, honeypot trapping, and other methods, and to dynamically expand the training data set and update the defense decision model in real time, so that the defense system can evolve autonomously in response to environmental changes, and continuously improve the timeliness and pertinence of defense. In summary, the present invention has made breakthroughs in accurate identification, proactive defense, situational awareness, and management assistance, which has greatly improved the intelligent level of phishing email prevention, and is of great significance for curbing the increasingly rampant phishing attacks and maintaining the security of cyberspace. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 It is a flowchart of a phishing email dynamic defense decision method based on reinforcement learning according to an embodiment of the present invention;

[0053] Figure 2 Schematic diagram of the structure of a dynamic defense decision system for phishing emails based on reinforcement learning according to an embodiment of the present invention. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0055] The technical solution of the present invention is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0056] Figure 1 FIG. 1 is a flow chart of a method for dynamic defense decision-making for phishing emails based on reinforcement learning according to an embodiment of the present invention. Figure 1 As shown, the method includes:

[0057] S1. Obtain multi-source heterogeneous historical phishing email data, extract features from the historical phishing email data through natural language processing technology and knowledge graph technology, generate multi-dimensional features, and determine the semantic structure and speech pattern in the multi-dimensional features through machine learning algorithms to construct a high-quality phishing email feature dataset, build a phishing email dynamic defense decision model and train it based on a reinforcement learning algorithm, dynamically adjust sample utilization and feature weights in combination with an adaptive multi-scale exploration strategy and a priority experience playback mechanism, and adjust defense decisions in combination with risk aversion mechanisms and cost constraints to obtain a trained phishing email dynamic defense decision model;

[0058] The natural language processing technology is a branch of artificial intelligence that aims to enable computers to understand, interpret and generate human language. The knowledge graph is a technology that represents entities and their relationships through nodes and edges, and aims to organize and represent structured information of knowledge. The multidimensional features refer to multiple features or attributes that describe data objects in data analysis and machine learning. The machine learning algorithm is a type of algorithm that recognizes patterns and makes predictions by learning from data. The semantic structure refers to the organization of semantic levels and relationships in a language. The speech pattern refers to a specific speaking style or technique used in language expression, which is usually used to achieve specific communication purposes. High-quality phishing email features The data set refers to a collection of phishing email samples that have been carefully selected and labeled, containing rich feature information. The phishing email dynamic defense decision model is a decision-making model based on real-time threat intelligence and dynamic risk assessment, which aims to take timely and effective defense measures against phishing email attacks. The adaptive multi-scale exploration strategy is a strategy for exploration and optimization in multi-scale space, which is often used in reinforcement learning and complex system optimization. The priority experience replay mechanism is a technology used for reinforcement learning, which accelerates the learning process by assigning different priorities to different experience samples. The risk aversion mechanism refers to the tendency to choose options with lower risks in the decision-making process to reduce possible losses.

[0059] In an optional embodiment,

[0060] Obtain multi-source heterogeneous historical phishing email data, extract features from the historical phishing email data through natural language processing technology and knowledge graph technology, generate multi-dimensional features, and determine the semantic structure and speech pattern in the multi-dimensional features through machine learning algorithms to construct a high-quality phishing email feature data set, build a phishing email dynamic defense decision model and train it based on a reinforcement learning algorithm, dynamically adjust sample utilization and feature weights in combination with an adaptive multi-scale exploration strategy and a priority experience playback mechanism, and adjust defense decisions in combination with risk aversion mechanisms and cost constraints. The trained phishing email dynamic defense decision model includes:

[0061] Acquire multi-source heterogeneous historical phishing email data from public phishing email datasets and email service provider databases, perform deduplication, denoising and format standardization on the historical phishing email data, remove redundant emails and perform data encoding on the retained emails, parse the retained emails into a structured data format in combination with a pre-set email parsing library, extract email content based on the structured data format, perform sensitive information desensitization on the email content, and obtain standard email content;

[0062] Based on the standard email content, the lexical features, grammatical features and semantic features corresponding to the standard email content are extracted respectively by natural language processing technology to obtain language features comprehensively, entities, attributes and relationships are extracted from the standard email content by exponential graph technology, semantic triples are generated and linked and fused with existing general knowledge graphs and industry knowledge graphs, a phishing email semantic knowledge graph is constructed and graph features are extracted, the deep semantic relationships corresponding to the graph features are identified by semantic reasoning technology, and based on the deep semantic relationships, the graph features and the language features are fused to obtain the multi-dimensional features;

[0063] Based on the multi-dimensional features, the standard email content is decomposed into multiple topics through a topic modeling algorithm, each topic is assigned and the main semantic topic corresponding to the current email is determined based on the topic assignment result, the main semantic topics are combined to obtain the semantic structure, the complex sentence patterns and imperative statements corresponding to the standard email content are identified through dependency grammar analysis technology, and the speech pattern is extracted in combination with a convolutional neural network, representative features from the multi-dimensional features are selected through a feature importance analysis method, semantic features are extracted in combination with the semantic structure and the speech pattern, and the high-quality phishing email feature dataset is constructed based on the extracted semantic features;

[0064] The multi-dimensional features are used as state representations to generate a state space, define multi-level progressive defense actions, construct a defense strategy space based on the defense actions, construct a reward function based on the effectiveness and response speed of the defense actions, and construct a phishing email dynamic defense decision model based on the state space, the defense strategy space and the reward function. The phishing email dynamic defense decision model is trained through a reinforcement learning algorithm. During the strategy search process, the exploration amplitude is dynamically adjusted through an adaptive multi-scale exploration mechanism, the search depth and breadth are adjusted according to uncertainty and benefit changes, the empirical data of different transfer states are prioritized according to value, the sampling probability is adjusted according to the sorting result, the defense actions are labeled according to a pre-set risk threshold, high-risk defense actions are determined and punished in combination with a risk aversion mechanism, the defense actions and defense decisions are adjusted in combination with cost constraints, the defense strategy space is updated, and an optimized phishing email dynamic defense decision model is obtained.

[0065] The data encoding is the process of converting raw data into a format that can be processed by a computer. The desensitization is a technology that processes sensitive data to protect privacy. The purpose is to use data for analysis or modeling without leaking personal privacy. The lexical features are features related to vocabulary in the text, which are mainly used to describe the basic composition of the text. The grammatical features refer to the features of the structure and grammatical components of sentences in the text. The semantic features focus on the meaning and contextual information of the text. The industry knowledge graph is a knowledge graph built for a specific industry, which is used to represent entities and their relationships within the industry. The link fusion is the process of integrating the same or related entities from different data sources. The main semantic theme is the core concept or theme expressed in the text. The dependency grammar analysis technology is used to analyze the dependency relationship between words in a sentence. The sampling probability refers to the probability of selecting a sample from a data set.

[0066] Obtain historical phishing email data from public phishing email datasets (such as PhishTank) and email service providers' (such as Gmail, Outlook) databases, deduplicate the obtained email data, remove emails with exactly the same content, perform denoising, clean up irrelevant characters and garbled characters in the emails, standardize the format, unify the encoding format of emails, remove redundant emails, and only retain emails with rich content and complete format. Encode the retained emails and assign unique numbers. Use the email parsing library to parse the emails into a structured data format, extract key fields such as sender, recipient, subject, and body, desensitize the extracted email content to sensitive information, such as hiding the real name, phone number, address, etc., to obtain standardized email content. For example, obtain 1,000 phishing emails from PhishTank, retain 800 emails after removing duplicate and noise data, and then remove emails with incomplete formats, and finally obtain 750 standard phishing email data;

[0067] Use natural language processing technologies, such as part-of-speech tagging and syntactic analysis, to extract lexical features (such as keyword frequency), grammatical features (such as sentence structure) and semantic features (such as sentiment tendency) of email content; use index graph technologies, such as named entity recognition and relationship extraction, to extract entities (such as organizations, names), attributes (such as positions, contact information) and relationships from email content, generate semantic triples, link and fuse the extracted semantic triples with existing general knowledge graphs and industry knowledge graphs, build a semantic knowledge graph in the field of phishing emails, extract graph features from the constructed knowledge graph, such as the importance of entities and the rarity of relationships, use semantic reasoning technology to explore deep semantic relationships between graph features and between graph features and language features, such as logical relationships such as cause and effect and implication, fuse graph features and language features, remove redundant and noisy features, and obtain high-quality multi-dimensional feature representations;

[0068] Use the topic model algorithm to divide the email content into different topics, assign the topics and determine the main semantic topics of the email based on the topic assignment results, and combine them to form a semantic structure representation. Use dependency syntax analysis technology to identify complex sentences and imperative statements. Use convolutional neural networks and word vector sequences as input to extract speech patterns and characterize the writing routines of phishing emails. Use feature importance analysis methods such as information gain and chi-square test to select representative features with high discrimination from multi-dimensional features. Combine semantic structure, speech pattern, and representative features to comprehensively extract the semantic features of phishing emails and construct a high-quality feature data set. For example, a phishing email with the theme of "Get a Free iPhone" contains a complex sentence pattern of "Just forward it to 10 friends to get a free phone", which reflects the "first-mover" speech routine. Comprehensively analyze the semantic theme, sentence characteristics, and writing routines to extract key semantic features such as "fake gifts" and "induced sharing";

[0069] Based on multi-dimensional features, the state space of phishing emails is fitted to characterize their content attributes, propagation characteristics, degree of harm, etc., and multi-level and progressive defense actions are designed. According to the effectiveness and response speed of each defense action, such as interception rate, misjudgment rate, reaction time, etc., a quantitative reward function is constructed. The state space, strategy space, and reward function are input into the reinforcement learning model. Through continuous trial and error, the optimal defense strategy is learned. In the learning process, an adaptive multi-scale exploration mechanism is used to dynamically adjust the exploration scope according to uncertainty, balance local utilization and global exploration, and the collected transfer state data is evaluated according to its value in improving the quality of the strategy. The model determines the risk level of a phishing email, and initially decides to “intercept it directly.” However, considering that it will affect normal commercial emails, the model changes the decision to “attach a warning label” and “restrict forwarding.” This not only reminds users of the risks, but also minimizes interference with normal communications, thus forming an overall defense strategy that takes into account both security and experience.

[0070] In this embodiment, the characteristics of phishing emails are characterized from multiple angles such as content, semantics, and language, which can comprehensively identify various types of hidden phishing techniques, greatly improve the detection recall rate of phishing emails, reduce the risk of missed judgment, and screen high-confidence discriminant features through methods such as feature importance analysis, thereby achieving accurate identification of phishing email risks and reducing the false positive rate. Risk thresholds and cost constraints are introduced, and a balance is made between defense effect and resource consumption to form a strategy that takes into account both security and economy. In summary, this embodiment improves the detection, identification and active defense capabilities of phishing emails from multiple dimensions, and can maximize the protection of users' information security and financial property security.

[0071] In an optional embodiment,

[0072] The defensive actions and defensive decisions are adjusted in combination with the cost constraint as shown in the following formula:

[0073]

[0074] Among them, θ represents the parameters of the policy network, represents the gradient of the objective function J(θ) with respect to the parameter θ, is the expected value, indicating that under the current strategy π θ The weighted average of all possible future state-action sequences under the condition of T represents the total number of time steps, π θ (a t |s t ) means that in a given state s t Next, strategy π θ Select action a t The probability of Q(s t , a t ) is the state-action value function, which means that in state s t Take action a t After that, the expected value of the cumulative reward that may be obtained in the future, λ represents the cost weight coefficient, C(s t , a t ) is in state s t Take action a t The defense cost represents the resource consumption required to perform the defense behavior. represents the gradient operator on the policy network parameters θ.

[0075] In this embodiment, after cost constraint optimization, the system can achieve the same or even better defense effect with fewer resources. When the effects are similar, defense means that occupy less resources are given priority, thereby maximizing the benefits of defense investment and improving the economy of system operation. By introducing explicit cost items, the optimized strategy can more clearly reflect the security benefits and resource overhead of each action, which helps security managers to formulate practical defense plans based on the strategy and quantify the defense effect. After the introduction of cost constraints, when optimizing long-term benefits, defense resources will be used more cautiously, and sufficient resources will be reserved to deal with high-risk events that may occur in the future, which helps to improve the overall defense effectiveness of the system.

[0076] S2. Encapsulate the trained phishing email dynamic defense decision model into a standardized interface and integrate it with the pre-set email security protection system, obtain newly received emails in real time through the email sending and receiving system, extract the initial email features corresponding to the newly received emails, make an initial judgment through the phishing email dynamic defense decision model, and based on the initial judgment result, perform deep feature extraction through heuristic analysis and behavioral sandbox, perform secondary identification in combination with context information, and generate an accurate recognition result. If the accurate recognition result indicates that the current email is a phishing email, generate a phishing email attack portrait, and generate an active defense strategy based on the initial email features;

[0077] The standardized interface refers to a unified interface specification defined in software or system design to ensure smooth interaction and compatibility between different systems or modules. The email security protection system refers to a set of comprehensive protection measures against email security threats (such as spam, phishing emails, malicious attachments, etc.). The heuristic analysis is a method based on experience and intuition to discover potential problems or threats. The contextual information refers to the background information of events, behaviors or data in a specific situation or environment. The phishing email attack portrait refers to a detailed description and analysis of phishing email attacks, including the attacker's behavior patterns, common characteristics of emails, types of victims and common attack methods. The active defense strategy refers to taking positive measures to actively defend and respond to potential security threats in information security.

[0078] In an optional embodiment,

[0079] The trained phishing email dynamic defense decision model is encapsulated as a standardized interface and integrated with a pre-set email security protection system. Newly received emails are obtained in real time through an email sending and receiving system, and initial email features corresponding to the newly received emails are extracted. An initial judgment is made through the phishing email dynamic defense decision model. Based on the initial judgment result, deep feature extraction is performed through heuristic analysis and behavioral sandbox, and secondary identification is performed in combination with context information to generate an accurate recognition result. If the accurate recognition result indicates that the current email is a phishing email, a phishing email attack portrait is generated. The active defense strategy generated according to the initial email features includes:

[0080] Encapsulate the trained phishing email dynamic defense decision model into a standardized interface, deploy the standardized interface into a pre-set email security protection system, and integrate it with the existing email processing process;

[0081] Acquire newly received emails in real time through the email sending and receiving system in the integrated email security protection system, perform multi-angle feature extraction on each newly received email, including text feature extraction, image feature extraction and attachment feature extraction, convert the extracted features into standard feature vectors, and combine them to obtain the initial email features;

[0082] Based on the initial email feature, the initial email feature is sent to the phishing email dynamic defense decision model in combination with traffic mirroring, and a nonlinear transformation is performed in a pre-set feedforward neural network to generate a threat level and confidence. Based on the threat level, it is determined whether the current email is a suspected phishing email, and the email marked as a suspected phishing email is sent to the re-determination system in the email security protection system. The re-determination system detects suspicious features in the suspected phishing email based on a heuristic analysis algorithm in combination with pre-set rules and feature libraries, and adds the attachments and links of the suspected phishing email to the behavioral sandbox for execution in combination with a behavioral sandbox analysis algorithm, monitors the running behavior and extracts dynamic features, obtains the sender's historical reputation and recipient feedback information corresponding to the suspected phishing email as context information, and fuses the context information with the suspicious features and the dynamic features to generate a second email feature. Based on the second email feature, a secondary identification is performed through the phishing email dynamic defense decision model to generate an accurate recognition result;

[0083] If the accurate identification result indicates that the current email is a phishing email, the attack target, disguise technique, bait content and malicious payload of the current email are read as key attack features, and a structured phishing email attack portrait is generated based on the key attack features. Based on the phishing email attack portrait and the initial email features, a pre-defined defense strategy template is matched to generate the active defense strategy.

[0084] The multi-angle feature extraction refers to analyzing and processing data from different perspectives or dimensions to extract more feature information. The feedforward neural network is an artificial neural network structure, in which information is transmitted from the input layer to the output layer through one or more hidden layers, and there is no cyclic connection. The threat level refers to an assessment of the severity or degree of harm of a certain security threat. The defense strategy template refers to a standardized framework or template for formulating and implementing information security protection strategies. The malicious payload refers to malicious code or data contained in a network attack, which can cause damage to the target system. Malicious payloads are usually spread through email attachments, download links or malicious websites.

[0085] Standardize and encapsulate the trained dynamic phishing email defense decision model, provide a unified input and output interface, and deploy the model to the email security protection system after encapsulation, so as to seamlessly integrate it with the existing email processing process (such as email receiving, filtering, delivery, etc.);

[0086] After receiving a new email, the email sending and receiving system immediately performs multi-angle analysis on the email, extracts comprehensive features, extracts text content such as email subject, body, attachment name, etc., performs word segmentation, word frequency statistics, word frequency-reverse file frequency calculation, etc., generates text feature vectors, analyzes image attachments in the email body, extracts image features such as color histograms and scale-invariant feature transformation feature points, analyzes meta information such as the type, size, and name of the email attachments, extracts content features such as the file header and hash value of the attachments, standardizes various features according to a predefined format, and combines them to form an initial email feature vector. For example, when a new email is received, extracts text features such as the subject "your account verification" and body keywords "verification", "account", "click", etc., extracts logo images in the body, generates color histograms, takes the file type, size, and message digest of the attachments, and finally combines them to generate a 200-dimensional initial feature vector;

[0087] Based on the traffic mirroring technology, without affecting the normal email processing, the initial feature vector is sent to the phishing email dynamic defense decision model. The model uses a pre-trained feedforward neural network to calculate the output vector and extract high-order semantic features through nonlinear transformation of several hidden layers. The output vector includes the threat level (such as low risk, medium risk, high risk) and confidence. It is determined whether it is a suspected phishing email according to the threat level. For example, the combined 200-dimensional initial feature vector is input into a three-layer fully connected neural network and processed by the activation function to generate a 2-dimensional output vector, indicating that the threat level of the email is 0.8 (high risk) for phishing emails, and the confidence is 0.9, which exceeds the suspected phishing threshold, and the email is marked as a suspected phishing email;

[0088] Suspected phishing emails are sent to the re-determination system for further analysis. Heuristic analysis is used to match the sender, subject, body, etc. of the email according to complex rules such as blacklists and abnormal link detection published by anti-spam organizations to find suspicious features. Executable attachments and links in the email are extracted and sent to the behavioral sandbox for simulated execution. The key calls and data flows during the sample execution process are recorded to form behavioral logs and feature reports. At the same time, the email sender's past reputation data, such as historical sending volume, user complaint rate, etc., as well as the recipient's feedback on the email, such as whether to report it as spam, are obtained to form the above The following environmental information summarizes the results of heuristic analysis, sandbox analysis, and environmental analysis, extracts key suspicious features, and fuses them with the initial features to form a second email feature vector. For example, a suspected phishing email sender "Service 1 Company" matches the domain name of "Service 1 Company" in the phishing blacklist. After the email attachment is executed by the sandbox, it attempts to connect to an overseas Internet Protocol address and download malicious code. Combined with the sender's reputation record of sending 10,000 emails in the past month, 2,000 of which were reported as spam by users, the key features of the second judgment are summarized and fused with the initial features to generate a new 250-dimensional feature vector;

[0089] Using the generated fusion features, the phishing email dynamic defense decision model is called again for judgment. The model generates more accurate threat judgment results based on global information and outputs the final phishing email identification label. For the identified phishing emails, key information such as attack targets (such as stealing users' online banking account passwords), disguise techniques (such as forging official bank emails), bait content (such as free gifts), and malicious payloads (such as malicious attachments) are extracted. Based on the predefined data structure, the attack elements are normalized and organized to form a structured phishing attack portrait. The phishing attack portrait is spliced ​​with the initial email features corresponding to the phishing email to form a complete phishing event feature. The spliced ​​event features are used to match the predefined defense strategy templates. Each defense strategy template is customized for a certain type of phishing attack, and the content includes threat intelligence acquisition, attack source tracing, email blocking, deception countermeasures, security reminders and other linkage disposal measures. Through template matching, the defense strategy that best fits the current attack features is selected to form a customized active defense plan.

[0090] In this embodiment, the dynamic defense decision model for phishing emails trained by a deep learning algorithm can automatically extract key features from massive email samples and make intelligent judgments, so as to identify complex and changeable phishing emails, significantly improve the accuracy of defense, and reduce the false negative rate. It uses a variety of analysis methods such as heuristic analysis, sandbox execution, and environmental analysis to cross-verify suspected phishing emails from multiple angles, and uses external information such as threat intelligence to enrich the judgment clues, so as to minimize the false alarm rate, avoid normal emails from being intercepted by mistake, and improve the availability and user experience of the defense system. By extracting email features in real time during the email processing process and using the encapsulated and deployed local intelligent model for rapid judgment, the entire detection process can be completed within milliseconds, realizing real-time detection and blocking of phishing emails, minimizing the attack window period, and reducing security risks. In summary, this embodiment shares the threat intelligence of the entire network through intelligent means, dynamically responds to unknown attacks, forms a new active defense concept, and provides a set of effective solutions for dealing with the increasingly rampant phishing email attacks.

[0091] In an optional embodiment,

[0092] The confidence corresponding to the threat level output by the feedforward neural network is shown in the following formula:

[0093]

[0094] Among them, p k Indicates the confidence that the current sample belongs to the kth threat level, is the weight of the k-th neuron in the L-th layer, representing the connection weight from the i-th neuron in the L-1-th layer to the k-th neuron in the L-th layer, represents the activation value of the i-th neuron in the L-1th layer, represents the bias term of the kth neuron in the Lth layer, K represents the total number of threat levels, is the weight of the jth neuron in the Lth layer, representing the connection weight from the i-th neuron in the L-1th layer to the j-th neuron in the Lth layer, Represents the bias term of the j-th neuron in the L-th layer.

[0095] In this embodiment, through the combination of nonlinear functions such as weighted summation and exponential normalization, the neural network can automatically learn the correspondence between phishing email features and threat levels, overcoming the limitation that traditional methods are difficult to characterize complex and changeable phishing patterns. The output in the form of probability is more fine-grained and provides rich confidence information. The judgment strategy can be flexibly adjusted according to different confidence thresholds to balance the accuracy and false alarm rate. End-to-end learning reduces the cost of manual feature engineering and improves the adaptability of the model. In summary, this embodiment makes full use of the advantages of deep learning technology in feature representation and classification discrimination, and lays a solid model foundation for efficiently and accurately identifying the threat level of phishing emails.

[0096] S3. Based on the active defense strategy, multi-level progressive defense action instructions are output, and the defense action instructions are sent to the defense execution node through the standard interface to perform phishing email defense. During the defense process, user feedback and behavior data are obtained in real time. Based on the user feedback and behavior data, the high-quality phishing email feature data set is expanded through traffic mirroring and honeypot trapping. The global phishing email attack situation and defense effect are evaluated in combination with the security event correlation analysis algorithm. Based on the global phishing email attack situation and the defense effect, combined with the analysis feedback from domain experts, the action space and state space of the phishing email dynamic defense decision model are adaptively updated through an online learning algorithm, and the optimization is repeated until the model converges.

[0097] The multi-level progressive defense action instructions are a defense strategy implemented in stages, with different defense measures in each stage to gradually improve the security of the system. The defense execution node refers to the component or system responsible for the specific implementation and execution of the defense strategy in the information security system. The honeypot trapping is a network security technology used to attract attackers and record their behavior in order to study and analyze their attack methods. The security event correlation analysis algorithm is used to correlate and analyze multiple security events or log data to discover potential attack patterns or abnormal behaviors. The global phishing email attack situation refers to a comprehensive analysis and evaluation of phishing email attack activities worldwide. The analysis feedback refers to the evaluation and improvement suggestions for security events or data analysis results. The online learning algorithm is a type of algorithm that can gradually update and improve the model when data streams continue to arrive.

[0098] In an optional embodiment,

[0099] Based on the active defense strategy, a multi-level progressive defense action instruction is output, and the defense action instruction is sent to the defense execution node through a standard interface to perform phishing email defense. In the defense process, user feedback and behavior data are obtained in real time. Based on the user feedback and behavior data, the high-quality phishing email feature data set is expanded through traffic mirroring and honeypot trapping. The global phishing email attack situation and defense effect are evaluated in combination with a security event correlation analysis algorithm. Based on the global phishing email attack situation and the defense effect, combined with the analysis feedback opinions issued by domain experts, the action space and state space of the phishing email dynamic defense decision model are adaptively updated through an online learning algorithm. Repeated optimization until the model converges includes:

[0100] Based on the priority and confidence of the active defense strategy, the defense level corresponding to each strategy is determined. In each defense level, the type and defense parameters corresponding to the active defense strategy are mapped to the corresponding defense action primitives. The defense action primitives are combined to obtain the multi-level progressive defense action instructions. The defense action instructions are sent to the defense execution node through a pre-set standard interface, wherein the defense action instructions include an early warning layer, a blocking layer and a tracing layer. The early warning layer warns of phishing risks through alarms, prompts and notifications. The blocking layer isolates the actual harm corresponding to phishing emails through isolation, interception and restriction actions. The tracing layer identifies the source of phishing emails through tracking and evidence collection.

[0101] When the defense execution node executes the defense action instruction, the feedback information and behavior data of the current user are obtained in real time. Based on the feedback information and behavior data, a traffic mirroring port is set next to the mail gateway through a traffic mirroring algorithm, the mail traffic is copied in real time, and parsed and extracted according to the mail protocol type to obtain a structured mail object, the structured mail object is parsed and added to the high-quality phishing mail feature data set, a honeypot mail account group is set up through a honeypot trapping algorithm and the account information is made public, the honeypot suspicious mails received by each honeypot mail account are regularly collected and the content is extracted, the honeypot suspicious mails are manually reviewed and annotated, and the honeypot suspicious mails are added to the high-quality phishing mail feature data set according to the annotated results;

[0102] Based on the updated high-quality phishing email feature data set and in combination with the security incident correlation analysis algorithm, the phishing email related data is correlated and analyzed with the pre-divided security incident data to identify phishing attack activities and phishing email attack chains, and the global phishing email attack situation is determined in combination with the attack scale, attack frequency and attack success rate. Based on the global phishing email attack situation, the defense effect of the current defense action instruction is determined in combination with the execution result and effect data corresponding to the current defense action instruction;

[0103] Based on the global phishing email attack situation and the defense effect, analysis feedback opinions given by domain experts are obtained, and structured feedback data is generated in combination with domain knowledge and experience. Through an incremental learning algorithm, the state space, action space and reward function of the phishing email dynamic defense decision model are locally updated based on the structured feedback data and the updated high-quality phishing email feature data set, and the update is repeated until the preset maximum number of updates is reached or the phishing email dynamic defense decision model converges.

[0104] The defense hierarchy refers to the multi-level protection of the system in the information security system according to different protection depths and strategies. The defense action primitive refers to the basic operation or action unit when building a defense strategy. The early warning layer refers to the layer in the defense system used to discover and report potential threats in advance. The blocking layer is a layer in the defense system, which is responsible for real-time interception and prevention of identified threats. The tracing layer refers to understanding the starting point and process of the attack by analyzing and tracing the source of the activity after the security incident occurs. The mirror traffic port refers to the port set up in the network, which is used to copy network traffic for monitoring and analysis. The honeypot email account group refers to a group of specially set up fake email accounts to attract and capture attackers. The honeypot suspicious email refers to emails sent to the honeypot email account. These emails are designed to have potential malicious features, such as phishing links or malicious attachments. The phishing email attack chain refers to the entire process from the initial phishing email being sent to the final attack target being successfully attacked.

[0105] According to the active defense strategy, the priority (such as urgency) and confidence (such as credibility) of each strategy are comprehensively considered to divide the strategy into different defense levels. In each defense level, the defense type (such as email blocking, attack source tracing) and corresponding defense parameters (such as blocking rules, tracing methods) of the strategy are further mapped to predefined defense action primitives (such as isolating emails, analyzing email headers). By combining multiple defense action primitives, a complete hierarchical progressive defense action instruction is formed. Through the pre-packaged standard interface, the defense action instruction is sent to the distributed defense execution nodes, such as email gateways, security network elements, etc. While each node executes specific defense actions, the user feedback information (such as user reports and complaints) and related behavior data (such as user clicks and email reply records) generated during the execution process are obtained in real time;

[0106] In order to obtain high-quality phishing email sample data, a traffic mirror port is deployed in the mail gateway bypass, and the email sending and receiving traffic on the gateway is copied to this port in real time. The mirror traffic is parsed according to common mail protocols such as Simple Mail Transfer Protocol and Post Office Protocol version 3, and structured email objects are extracted, including sender, recipient, title, body, attachments and other key information. After detailed content analysis of each part of the email, it is added to the high-quality phishing email feature dataset. Using honeypot trapping technology, a batch of honeypot mailbox accounts are pre-registered to simulate the email sending and receiving behavior of normal users, and the honeypot mailbox addresses are intentionally leaked on the Internet to induce phishing attackers to send phishing emails to the honeypot. Suspicious emails received by each honeypot account are regularly collected and the email content is extracted. The suspicious emails are manually reviewed by security analysts to identify and mark the real phishing emails and add them to the high-quality phishing email feature dataset.

[0107] With the continuous enrichment of high-quality phishing email feature data sets, we conduct correlation analysis based on various security data. We use correlation analysis algorithms such as multi-source fusion to correlate key data such as the sender's smart protocol address, attachment hash, domain name system information, and multi-dimensional security event data such as access logs, domain name registration information, and botnet intelligence to discover more hidden phishing attack clues and attack chains. Based on the correlation analysis results, we evaluate the overall situation of current phishing attacks, such as attack scale, attack frequency, and attack success rate, to provide a reference for judging the defense effect. We submit the correlation analysis results and the initial evaluation of the defense effect to network security experts inside and outside the company for analysis. Based on their own professional knowledge and experience, the expert group judges the rationality of the analysis results, evaluates the pertinence of the defense measures, puts forward subsequent improvement suggestions, and formalizes the expert opinions into structured feedback data (such as defense effectiveness score, new defense measures, etc.). Using machine learning algorithms such as incremental learning and parameter fine-tuning, the dynamic phishing email defense decision model is locally optimized based on feedback data and updated high-quality phishing email feature data sets, such as adding new defense actions, adjusting reward function weights, etc., so that the model can adapt to the latest attack and defense situation, and update repeatedly until the model performance is stable or the preset maximum number of iterations is reached.

[0108] In this embodiment, defense instructions are automatically issued through a standard interface, which greatly improves the speed and accuracy of defense response and reduces security operation costs. By combining alarms, isolation, and evidence collection, the penetration and harm of phishing attacks are comprehensively reduced, and the security of key businesses is protected to the maximum extent. High-quality phishing email samples are continuously obtained through technical means such as traffic mirroring and honeypot trapping, and the detection model is dynamically updated using an incremental learning algorithm, so that the model can quickly adapt to new attack modes, thereby improving the detection rate while reducing the false alarm rate. Security event correlation analysis technology is used to correlate and sort out phishing email-related data with multi-source heterogeneous data such as login logs and botnets, deeply explore the masterminds and the overall picture of phishing attacks, and assist in threat hunting. In summary, this embodiment greatly improves the intelligence level and defense effect of phishing email defense, provides a powerful grip for coping with the current complex and changeable network security forms, and is of great significance for improving the overall network security protection capabilities of organizations and ensuring the safe and stable operation of key businesses.

[0109] In an optional embodiment,

[0110] Identify phishing attack activities and phishing email attack chains, and determine the global phishing email attack situation by combining the attack scale, attack frequency, and attack success rate as shown in the following formula:

[0111]

[0112] Where I represents the global phishing email attack situation value, w S Indicates the weight coefficient of the attack scale, N a represents the total number of phishing attacks identified within the time window, w F represents the weight coefficient of attack frequency, TI represents the length of the time window, and w R Represents the weight coefficient of attack success power, N S Indicates the number of successful phishing emails, N t represents the total number of phishing emails, w C Represents the attack chain complexity weight coefficient, N c Indicates the total number of identified attack chains, L e Indicates the length of the e-th attack chain, and e indicates the index of the attack chain.

[0113] In this embodiment, the phishing attack situation is quantitatively evaluated from four dimensions, namely, attack scale, attack frequency, attack success rate, and attack chain complexity, covering all aspects of phishing attacks, and can comprehensively reflect the current phishing email security threat level, and intuitively present network security risks to security managers. The weighted sum form is adopted, and each weight coefficient intuitively reflects the degree of influence of different factors on the situation. The higher the attack scale, frequency, success rate, and attack chain complexity, the larger the weight coefficient and the higher the situation value. By adjusting the weight coefficient, the focus of the situation assessment can be flexibly presented, which is convenient for security managers to interpret the causes of the situation value. By tracking the numerical changes of the situation value, the defense effect can be judged, which is convenient for vertical comparison of the severity of phishing attack situations in different time periods. In summary, this embodiment realizes the visualization presentation of the macro network security situation, and provides a strong data basis and scientific indicators for objectively judging the effectiveness of phishing attack defense and dynamically adjusting defense strategies. It is of great significance to improving the security management level of the organization and coping with the increasingly severe phishing attack situation.

[0114] Figure 2 FIG. 1 is a schematic diagram of the structure of a dynamic phishing email defense decision system based on reinforcement learning according to an embodiment of the present invention. Figure 2 As shown, the system comprises:

[0115] The first unit is used to obtain multi-source heterogeneous historical phishing email data, extract features from the historical phishing email data through natural language processing technology and knowledge graph technology, generate multi-dimensional features, and determine the semantic structure and speech pattern in the multi-dimensional features through machine learning algorithms to construct a high-quality phishing email feature data set, build a phishing email dynamic defense decision model and train it based on a reinforcement learning algorithm, dynamically adjust sample utilization and feature weights in combination with an adaptive multi-scale exploration strategy and a priority experience playback mechanism, and adjust defense decisions in combination with risk aversion mechanisms and cost constraints to obtain a trained phishing email dynamic defense decision model;

[0116] The second unit is used to encapsulate the trained phishing email dynamic defense decision model into a standardized interface and integrate it with a pre-set email security protection system, obtain newly received emails in real time through the email sending and receiving system, extract the initial email features corresponding to the newly received emails, make an initial judgment through the phishing email dynamic defense decision model, and based on the initial judgment result, perform deep feature extraction through heuristic analysis and behavioral sandbox, perform secondary identification in combination with context information, and generate an accurate recognition result. If the accurate recognition result indicates that the current email is a phishing email, a phishing email attack portrait is generated, and an active defense strategy is generated according to the initial email features;

[0117] The third unit is used to output multi-level progressive defense action instructions based on the active defense strategy, send the defense action instructions to the defense execution node through the standard interface and perform phishing email defense, obtain user feedback and behavior data in real time during the defense process, expand the high-quality phishing email feature data set through traffic mirroring and honeypot trapping based on the user feedback and behavior data, evaluate the global phishing email attack situation and defense effect in combination with the security event correlation analysis algorithm, based on the global phishing email attack situation and the defense effect, combined with the analysis feedback opinions issued by domain experts, adaptively update the action space and state space of the phishing email dynamic defense decision model through an online learning algorithm, and repeat the optimization until the model converges.

[0118] According to a third aspect of the embodiments of the present invention,

[0119] An electronic device is provided, comprising:

[0120] processor;

[0121] a memory for storing processor-executable instructions;

[0122] The processor is configured to call the instructions stored in the memory to execute the aforementioned method.

[0123] A fourth aspect of the embodiments of the present invention is:

[0124] A computer-readable storage medium is provided, on which computer program instructions are stored. When the computer program instructions are executed by a processor, the aforementioned method is implemented.

[0125] The present invention may be a method, an apparatus, a system and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for executing various aspects of the present invention.

[0126] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A dynamic defense decision method for phishing emails based on reinforcement learning, characterized in that: include: Obtain multi-source heterogeneous historical phishing email data, extract features from the historical phishing email data through natural language processing technology and knowledge graph technology, generate multi-dimensional features, and determine the semantic structure and speech pattern in the multi-dimensional features through machine learning algorithms to construct a high-quality phishing email feature data set, build a phishing email dynamic defense decision model and train it based on a reinforcement learning algorithm, dynamically adjust sample utilization and feature weights in combination with an adaptive multi-scale exploration strategy and a priority experience playback mechanism, adjust defense decisions in combination with risk aversion mechanisms and cost constraints, and obtain a trained phishing email dynamic defense decision model; The trained phishing email dynamic defense decision model is encapsulated into a standardized interface and integrated with a pre-set email security protection system. Newly received emails are obtained in real time through the email sending and receiving system, and initial email features corresponding to the newly received emails are extracted. An initial judgment is made through the phishing email dynamic defense decision model. Based on the initial judgment result, deep feature extraction is performed through heuristic analysis and behavioral sandbox, and secondary identification is performed in combination with context information to generate an accurate recognition result. If the accurate recognition result indicates that the current email is a phishing email, a phishing email attack portrait is generated, and an active defense strategy is generated according to the initial email features. Based on the active defense strategy, multi-level progressive defense action instructions are output, and the defense action instructions are sent to the defense execution node through a standard interface to perform phishing email defense. During the defense process, user feedback and behavior data are obtained in real time. Based on the user feedback and behavior data, the high-quality phishing email feature data set is expanded through traffic mirroring and honeypot trapping. The global phishing email attack situation and defense effect are evaluated in combination with the security event correlation analysis algorithm. Based on the global phishing email attack situation and the defense effect, combined with the analysis feedback from domain experts, the action space and state space of the phishing email dynamic defense decision model are adaptively updated through an online learning algorithm, and the optimization is repeated until the model converges.

2. The method according to claim 1, characterized in that Obtain multi-source heterogeneous historical phishing email data, extract features from the historical phishing email data through natural language processing technology and knowledge graph technology, generate multi-dimensional features, and determine the semantic structure and speech pattern in the multi-dimensional features through machine learning algorithms to construct a high-quality phishing email feature data set, build a phishing email dynamic defense decision model and train it based on a reinforcement learning algorithm, dynamically adjust sample utilization and feature weights in combination with an adaptive multi-scale exploration strategy and a priority experience playback mechanism, and adjust defense decisions in combination with risk aversion mechanisms and cost constraints. The trained phishing email dynamic defense decision model includes: Acquire multi-source heterogeneous historical phishing email data from public phishing email datasets and email service provider databases, perform deduplication, denoising and format standardization on the historical phishing email data, remove redundant emails and perform data encoding on the retained emails, parse the retained emails into a structured data format in combination with a pre-set email parsing library, extract email content based on the structured data format, perform sensitive information desensitization on the email content, and obtain standard email content; Based on the standard email content, the lexical features, grammatical features and semantic features corresponding to the standard email content are extracted respectively by natural language processing technology to obtain language features comprehensively, entities, attributes and relationships are extracted from the standard email content by exponential graph technology, semantic triples are generated and linked and fused with existing general knowledge graphs and industry knowledge graphs, a phishing email semantic knowledge graph is constructed and graph features are extracted, the deep semantic relationships corresponding to the graph features are identified by semantic reasoning technology, and based on the deep semantic relationships, the graph features and the language features are fused to obtain the multi-dimensional features; Based on the multi-dimensional features, the standard email content is decomposed into multiple topics through a topic modeling algorithm, each topic is assigned and the main semantic topic corresponding to the current email is determined based on the topic assignment result, the main semantic topics are combined to obtain the semantic structure, the complex sentence patterns and imperative statements corresponding to the standard email content are identified through dependency grammar analysis technology, and the speech pattern is extracted in combination with a convolutional neural network, representative features from the multi-dimensional features are selected through a feature importance analysis method, semantic features are extracted in combination with the semantic structure and the speech pattern, and the high-quality phishing email feature dataset is constructed based on the extracted semantic features; The multi-dimensional features are used as state representations to generate a state space, define multi-level progressive defense actions, construct a defense strategy space based on the defense actions, construct a reward function based on the effectiveness and response speed of the defense actions, and construct a phishing email dynamic defense decision model based on the state space, the defense strategy space and the reward function. The phishing email dynamic defense decision model is trained through a reinforcement learning algorithm. During the strategy search process, the exploration amplitude is dynamically adjusted through an adaptive multi-scale exploration mechanism, the search depth and breadth are adjusted according to uncertainty and benefit changes, the empirical data of different transfer states are prioritized according to value, the sampling probability is adjusted according to the sorting result, the defense actions are labeled according to a pre-set risk threshold, high-risk defense actions are determined and punished in combination with a risk aversion mechanism, the defense actions and defense decisions are adjusted in combination with cost constraints, the defense strategy space is updated, and an optimized phishing email dynamic defense decision model is obtained.

3. The method according to claim 2, characterized in that The defensive actions and defensive decisions are adjusted in combination with the cost constraint as shown in the following formula: Among them, θ represents the parameters of the policy network, represents the gradient of the objective function J(θ) with respect to the parameter θ, is the expected value, indicating that under the current strategy π θ The weighted average of all possible future state-action sequences under the condition of T represents the total number of time steps, π θ (a t |s t ) means that in a given state s t Next, strategy π θ Select action a t The probability of Q(s t , a t ) is the state-action value function, which means that in state s t Take action a t After that, the expected value of the cumulative reward that may be obtained in the future, λ represents the cost weight coefficient, C(s t , a t ) is in state s t Take action a t The defense cost represents the resource consumption required to perform the defense behavior, and ▽θ represents the gradient operator of the policy network parameter θ.

4. The method according to claim 1, characterized in that The trained phishing email dynamic defense decision model is encapsulated as a standardized interface and integrated with a pre-set email security protection system. Newly received emails are obtained in real time through an email sending and receiving system, and initial email features corresponding to the newly received emails are extracted. An initial judgment is made through the phishing email dynamic defense decision model. Based on the initial judgment result, deep feature extraction is performed through heuristic analysis and behavioral sandbox, and secondary identification is performed in combination with context information to generate an accurate recognition result. If the accurate recognition result indicates that the current email is a phishing email, a phishing email attack portrait is generated. The active defense strategy generated according to the initial email features includes: Encapsulate the trained phishing email dynamic defense decision model into a standardized interface, deploy the standardized interface into a pre-set email security protection system, and integrate it with the existing email processing process; Acquire newly received emails in real time through the email sending and receiving system in the integrated email security protection system, perform multi-angle feature extraction on each newly received email, including text feature extraction, image feature extraction and attachment feature extraction, convert the extracted features into standard feature vectors, and combine them to obtain the initial email features; Based on the initial email feature, the initial email feature is sent to the phishing email dynamic defense decision model in combination with traffic mirroring, and a nonlinear transformation is performed in a pre-set feedforward neural network to generate a threat level and confidence. Based on the threat level, it is determined whether the current email is a suspected phishing email, and the email marked as a suspected phishing email is sent to the re-determination system in the email security protection system. The re-determination system detects suspicious features in the suspected phishing email based on a heuristic analysis algorithm in combination with pre-set rules and feature libraries, and adds the attachments and links of the suspected phishing email to the behavioral sandbox for execution in combination with a behavioral sandbox analysis algorithm, monitors the running behavior and extracts dynamic features, obtains the sender's historical reputation and recipient feedback information corresponding to the suspected phishing email as context information, and fuses the context information with the suspicious features and the dynamic features to generate a second email feature. Based on the second email feature, a secondary identification is performed through the phishing email dynamic defense decision model to generate an accurate recognition result; If the accurate identification result indicates that the current email is a phishing email, the attack target, disguise technique, bait content and malicious payload of the current email are read as key attack features, and a structured phishing email attack portrait is generated based on the key attack features. Based on the phishing email attack portrait and the initial email features, a pre-defined defense strategy template is matched to generate the active defense strategy.

5. The method according to claim 4, characterized in that The confidence corresponding to the threat level output by the feedforward neural network is shown in the following formula: Among them, p k Indicates the confidence that the current sample belongs to the kth threat level, is the weight of the k-th neuron in the L-th layer, representing the connection weight from the i-th neuron in the L-1-th layer to the k-th neuron in the L-th layer, represents the activation value of the i-th neuron in the L-1th layer, represents the bias term of the kth neuron in the Lth layer, K represents the total number of threat levels, is the weight of the jth neuron in the Lth layer, representing the connection weight from the i-th neuron in the L-1th layer to the j-th neuron in the Lth layer, Represents the bias term of the j-th neuron in the L-th layer.

6. The method according to claim 1, characterized in that Based on the active defense strategy, a multi-level progressive defense action instruction is output, and the defense action instruction is sent to the defense execution node through a standard interface to perform phishing email defense. In the defense process, user feedback and behavior data are obtained in real time. Based on the user feedback and behavior data, the high-quality phishing email feature data set is expanded through traffic mirroring and honeypot trapping. The global phishing email attack situation and defense effect are evaluated in combination with a security event correlation analysis algorithm. Based on the global phishing email attack situation and the defense effect, combined with the analysis feedback opinions issued by domain experts, the action space and state space of the phishing email dynamic defense decision model are adaptively updated through an online learning algorithm. Repeated optimization until the model converges includes: Based on the priority and confidence of the active defense strategy, the defense level corresponding to each strategy is determined. In each defense level, the type and defense parameters corresponding to the active defense strategy are mapped to the corresponding defense action primitives. The defense action primitives are combined to obtain the multi-level progressive defense action instructions. The defense action instructions are sent to the defense execution node through a pre-set standard interface, wherein the defense action instructions include an early warning layer, a blocking layer and a tracing layer. The early warning layer warns of phishing risks through alarms, prompts and notifications. The blocking layer isolates the actual harm corresponding to phishing emails through isolation, interception and restriction actions. The tracing layer identifies the source of phishing emails through tracking and evidence collection. When the defense execution node executes the defense action instruction, the feedback information and behavior data of the current user are obtained in real time. Based on the feedback information and behavior data, a traffic mirroring port is set next to the mail gateway through a traffic mirroring algorithm, the mail traffic is copied in real time, and parsed and extracted according to the mail protocol type to obtain a structured mail object, the structured mail object is parsed and added to the high-quality phishing mail feature data set, a honeypot mail account group is set up through a honeypot trapping algorithm and the account information is made public, the honeypot suspicious mails received by each honeypot mail account are regularly collected and the content is extracted, the honeypot suspicious mails are manually reviewed and annotated, and the honeypot suspicious mails are added to the high-quality phishing mail feature data set according to the annotated results; Based on the updated high-quality phishing email feature data set and in combination with the security incident correlation analysis algorithm, the phishing email related data is correlated and analyzed with the pre-divided security incident data to identify phishing attack activities and phishing email attack chains, and the global phishing email attack situation is determined in combination with the attack scale, attack frequency and attack success rate. Based on the global phishing email attack situation, the defense effect of the current defense action instruction is determined in combination with the execution result and effect data corresponding to the current defense action instruction; Based on the global phishing email attack situation and the defense effect, analysis feedback opinions given by domain experts are obtained, and structured feedback data is generated in combination with domain knowledge and experience. Through an incremental learning algorithm, the state space, action space and reward function of the phishing email dynamic defense decision model are locally updated based on the structured feedback data and the updated high-quality phishing email feature data set, and the update is repeated until the preset maximum number of updates is reached or the phishing email dynamic defense decision model converges.

7. The method according to claim 6, characterized in that Identify phishing attack activities and phishing email attack chains, and determine the global phishing email attack situation by combining the attack scale, attack frequency, and attack success rate as shown in the following formula: Where I represents the global phishing email attack situation value, w S Indicates the weight coefficient of the attack scale, N a represents the total number of phishing attacks identified within the time window, w F represents the weight coefficient of attack frequency, TI represents the length of the time window, and w R Represents the weight coefficient of attack success power, N S Indicates the number of successful phishing emails, N t Indicates the total number of phishing emails, w C Represents the attack chain complexity weight coefficient, N c Indicates the total number of identified attack chains, L e Indicates the length of the e-th attack chain, and e indicates the index of the attack chain.

8. A dynamic defense decision system for phishing emails based on reinforcement learning, used to implement the method described in any one of claims 1 to 7, characterized in that: include: The first unit is used to obtain multi-source heterogeneous historical phishing email data, extract features from the historical phishing email data through natural language processing technology and knowledge graph technology, generate multi-dimensional features, and determine the semantic structure and speech pattern in the multi-dimensional features through machine learning algorithms to construct a high-quality phishing email feature data set, build a phishing email dynamic defense decision model and train it based on a reinforcement learning algorithm, dynamically adjust sample utilization and feature weights in combination with an adaptive multi-scale exploration strategy and a priority experience playback mechanism, and adjust defense decisions in combination with risk aversion mechanisms and cost constraints to obtain a trained phishing email dynamic defense decision model; The second unit is used to encapsulate the trained phishing email dynamic defense decision model into a standardized interface and integrate it with a pre-set email security protection system, obtain newly received emails in real time through the email sending and receiving system, extract the initial email features corresponding to the newly received emails, make an initial judgment through the phishing email dynamic defense decision model, and based on the initial judgment result, perform deep feature extraction through heuristic analysis and behavioral sandbox, perform secondary identification in combination with context information, and generate an accurate recognition result. If the accurate recognition result indicates that the current email is a phishing email, a phishing email attack portrait is generated, and an active defense strategy is generated according to the initial email features; The third unit is used to output multi-level progressive defense action instructions based on the active defense strategy, send the defense action instructions to the defense execution node through the standard interface and perform phishing email defense, obtain user feedback and behavior data in real time during the defense process, expand the high-quality phishing email feature data set through traffic mirroring and honeypot trapping based on the user feedback and behavior data, evaluate the global phishing email attack situation and defense effect in combination with the security event correlation analysis algorithm, based on the global phishing email attack situation and the defense effect, combined with the analysis feedback opinions issued by domain experts, adaptively update the action space and state space of the phishing email dynamic defense decision model through an online learning algorithm, and repeat the optimization until the model converges.

9. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method described in any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon, characterized in that: When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images