Social worker phishing mail detection system and method based on threat intelligence

Abstract

The invention provides a social worker phishing mail detection system based on threat intelligence, and the system comprises a client which specifically comprises a mail gateway which captures incoming or outgoing mail flow, extracts a dynamic sample, submits the dynamic sample to a sandbox, and generates a static threat IOC feature index; the sandbox is used for executing operations contained in the dynamic samples and generating dynamic threat sample IOB behavior indexes; the client side is used for fusing the indexes to form overall threat intelligence information and reporting the overall threat intelligence information to the server side; the server specifically comprises a threat intelligence safety operation and monitoring analysis module used for processing the whole threat intelligence information, determining real threat intelligence and storing the real threat intelligence in a threat intelligence warehouse; and the mail security joint defense early warning module is used for regularly pulling the data of the threat intelligence warehouse and sharing the pulled data to the client for threat joint defense early warning. By adopting the scheme, the fishing threat of social workers can be effectively dealt with, and the overall safety protection level is improved.

Description

Technical Field

[0001] The present application relates to the field of network security technology, and in particular to a social engineering phishing email detection system and method based on threat intelligence. Background Art

[0002] With the advancement of digital transformation, network security and data security are becoming increasingly important, and malicious phishing emails pose a huge threat to network security.

[0003] The commonly used phishing email detection method is a web email security protection method based on intermediate proxy detection. Figure 1 This is a logical view of webmail security protection based on intermediate proxy detection, such as Figure 1 As shown, the method processes the web page and email interaction of intranet users as follows:

[0004] (1) If an intranet user initiates a plaintext email request to the external network, it needs to go through the intermediate proxy detection module;

[0005] (2) After determining that it is a plaintext connection, the intermediate proxy detection module acts as a proxy server to establish a plaintext connection with the intranet user;

[0006] (3) At the same time, the intermediate proxy detection module uses SSL encryption as a client to establish an SSL ciphertext connection to the external network mail server and initiate a request;

[0007] (4) The external web page and mail server return the response to the intermediate proxy detection module through SSL ciphertext, and the intermediate proxy detection module sends the plaintext content to the internal network user after SSL decryption;

[0008] (5) If an intranet user initiates SSL-encrypted email access to the external network, it needs to pass through the intermediate proxy detection module;

[0009] (6) The intermediate detection proxy module will periodically initiate an SSL request connection to the micro SSL certificate service;

[0010] (7) During the SSL negotiation process, the micro SSL certificate service and the intermediate proxy detection module will send a certificate to the intermediate proxy detection module;

[0011] (8) The intermediate proxy detection module checks whether the above certificate has been replaced. If it has been replaced, it proves that an SSL man-in-the-middle attack has occurred, and the current email access request is blocked;

[0012] (9) If the certificate has not been replaced, the intranet user is allowed to access emails, and the email server sends a securely encrypted response to the intranet user.

[0013] The disadvantages of the prior art are as follows:

[0014] (1) Defense limitations: Intermediary agents typically use rules and pattern matching to detect threats, which may result in missed or false positives. They may not be able to respond to emerging, unknown threats unless rules and signatures are continuously updated.

[0015] (2) Single point of failure: The intermediate proxy is a single point in the network. If the proxy fails or is attacked, the entire email flow will be affected. This may cause email delays or interruptions.

[0016] (3) Not applicable to end-to-end encrypted emails: For end-to-end encrypted emails, intermediate agents are usually unable to view the email content and therefore cannot effectively inspect and filter it.

[0017] (4) Limitations on advanced threats: Certain advanced threats and attacks may bypass intermediary inspections as attackers continue to evolve and find new ways to evade detection. Summary of the invention

[0018] The present application aims to solve one of the technical problems in the related art at least to some extent.

[0019] To this end, the first purpose of this application is to propose a social engineering phishing email detection system based on threat intelligence.

[0020] The second purpose of this application is to propose a method for detecting social engineering phishing emails based on threat intelligence.

[0021] To achieve the above-mentioned purpose, the first embodiment of the present application proposes a social engineering phishing email detection system based on threat intelligence, including a client and a server, wherein the client includes:

[0022] Mail gateway, used to capture incoming or outgoing mail traffic, extract dynamic samples from the captured mail traffic and submit them to the sandbox, and generate static threat IOC feature indicators;

[0023] Sandbox, used to perform the operations contained in the dynamic sample and generate IOB behavior indicators of the dynamic threat sample;

[0024] The client is used to integrate static threat IOC characteristic indicators and dynamic threat sample IOB behavior indicators to form overall threat intelligence information, and report the overall threat intelligence information to the server;

[0025] The server includes:

[0026] Threat intelligence security operation and monitoring analysis module, which is used to process the overall threat intelligence information, determine the real threat intelligence, and store it in the threat intelligence warehouse;

[0027] The email security joint defense warning module is used to periodically pull data from the threat intelligence warehouse and share the pulled data with the client for threat joint defense warning.

[0028] The social engineering phishing email detection system based on threat intelligence in the embodiment of the present application can achieve the following by deploying a mail gateway and a multi-scenario intelligent sandbox locally, relying on a "cloud" threat intelligence sharing platform for expert operation, coordinating with important industry units, and carrying out real-time linkage sharing of threat intelligence such as malicious sample features, malicious URLs, and abnormal behaviors:

[0029] (1) Effectively defend against social engineering phishing attacks, cut off hackers' access to the intranet attack springboard through phishing attacks, and comprehensively protect the security of the email system and data.

[0030] (2) It fills the current data gap in the field of domestic defense against social engineering phishing intelligence, and builds a complete data operation system for intelligence production, operation, and consumption.

[0031] (3) Effectively solve the problem of uneven email security awareness among corporate personnel during actual combat, realize automated response to email phishing and malicious program attacks through timely threat warning, and use intelligence operation mechanism to achieve the protection effect of "one-point monitoring, full network warning".

[0032] Optionally, in one embodiment of the present application, the client is specifically used to:

[0033] The overall threat intelligence information is processed according to preset standards to obtain the data to be reported, and access and data reporting are defined according to the prescribed coding and format.

[0034] Optionally, in one embodiment of the present application, the threat intelligence security operation and monitoring analysis module accesses at least one data source, and the threat intelligence security operation and monitoring analysis module is further used to collect threat intelligence data, wherein collecting threat intelligence data includes:

[0035] Receive data transmitted from the access data source through an encrypted network transmission protocol, where the data includes standardized alarm log information and threat intelligence information reported by the client;

[0036] Classify the received data and create corresponding collection adapters for each type of data. Complete distributed log data collection by unified scheduling and policy control of the collection adapters.

[0037] Process the overall threat intelligence information to determine the real threat intelligence, including online real-time analysis, offline analysis and manual assisted analysis.

[0038] Optionally, in one embodiment of the present application, the online real-time analysis includes:

[0039] Access the distributed message queue Kafka in real time through Flume's distributed Agent nodes to obtain real-time data;

[0040] Extract and transform real-time data through ETL and load it into the HDFS distributed file system;

[0041] Synchronize real-time data through the rule model library to optimize rules and models;

[0042] Based on Flink, real-time processing of data in Kafka / HDFS is used to generate real threat intelligence data through rule matching, model comparison, scenario and cluster analysis.

[0043] The threat intelligence security operation and monitoring analysis module is also used for:

[0044] The application layer service framework is used to schedule real threat intelligence data and store it in the threat intelligence warehouse.

[0045] Optionally, in one embodiment of the present application, the offline analysis includes:

[0046] Preprocess the historical data to obtain preprocessed historical data, and store the preprocessed historical data in the data warehouse;

[0047] Integrate information from different types into data warehouses and link them to analytical platforms;

[0048] Use data mining and analysis technology to analyze the data integrated in the data warehouse on the analysis platform to obtain analysis results;

[0049] The detection model is trained based on known threat intelligence data, and the pre-processed historical data is tested using the detection model to obtain the detection results.

[0050] Based on the analysis and detection results, identify potential threat information and obtain offline analysis results;

[0051] Manually judge and analyze the offline analysis results to form a security incident entry event model warehouse.

[0052] Optionally, in one embodiment of the present application, the manual assisted analysis includes:

[0053] Normalize and standardize the data to obtain standard data;

[0054] Use automated tools and techniques to screen and filter standard data to identify potential threat information;

[0055] Through manual judgment of real threat information.

[0056] Optionally, in one embodiment of the present application, the threat intelligence security operation and monitoring analysis module is further used to:

[0057] The email text content is vectorized using a large-scale pre-trained language model, and the model is trained using unsupervised learning. During training, the text semantic patterns are learned from the vectors to achieve incremental optimization of the model.

[0058] Use Spark SQL to access at least one data source to obtain distributed log data;

[0059] Execute SQL queries through Spark SQL to associate corresponding data in each data source to form threat data records;

[0060] Based on threat data records, detection rules for detecting security incidents are formed according to different dimensions and different threat description information;

[0061] Establish associations with existing attack organizations through attack organization characteristics, form an attack organization fingerprint library, and generate attack organization identification strategies;

[0062] Add the source intelligence data with credibility and activity greater than the threshold and the intelligence reassembled by the intelligence analysis model to the enthusiasm database in real time;

[0063] Mark the effective time for accurate and effective intelligence data, lower the credibility of false intelligence data and optimize the analysis model that generates this intelligence;

[0064] Dynamically adjust the time validity of intelligence detection rules according to the triggering probability of different types of intelligence in the actual detection scene environment;

[0065] The email security joint defense warning module is also used to send policies, rules, and models to the client for threat joint defense warning.

[0066] Optionally, in one embodiment of the present application, data from the threat intelligence warehouse is periodically pulled, and the pulled data is shared with the client for threat joint defense warning, including:

[0067] Verify the client's access rights using authentication and authorization token verification to identify authorized clients;

[0068] Extract corresponding threat intelligence data from the threat intelligence warehouse based on the subscription status of the authorized client;

[0069] The extracted threat intelligence data is classified, merged, encrypted and compressed to obtain processed threat intelligence data;

[0070] The processed threat intelligence data is shared to the corresponding authorized client through an encrypted network transmission protocol.

[0071] To achieve the above-mentioned purpose, the second embodiment of the present invention proposes a social engineering phishing email detection method based on threat intelligence, including:

[0072] The emails are detected through the social engineering phishing email detection system, and the detected emails are divided into threat emails and non-threat emails. Among them, non-threat emails include warning emails and normal emails;

[0073] The social engineering phishing email detection system adds warning information to the detected threat emails and / or warning emails, and delivers the threat emails and / or non-threat emails to the email server, and sends them to the user's inbox through the email server.

[0074] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0076] Figure 1 It is the logical view of webmail security protection based on intermediate proxy detection;

[0077] Figure 2 A schematic diagram of the structure of a social engineering phishing email detection system based on threat intelligence provided in Example 1 of the present application;

[0078] Figure 3 A schematic diagram of the structure of a social engineering phishing email detection system based on threat intelligence provided in Example 2 of the present application;

[0079] Figure 4 A flowchart of a method for detecting social engineering phishing emails based on threat intelligence provided in Example 3 of the present application;

[0080] Figure 5 A flowchart of a method for detecting social engineering phishing emails based on threat intelligence provided in Example 4 of the present application;

[0081] Figure 6 A flowchart of a social engineering phishing email detection method based on threat intelligence provided in the fifth embodiment of the present application;

[0082] Figure 7 A flowchart of a method for detecting social engineering phishing emails based on threat intelligence provided in Example 6 of the present application;

[0083] Figure 8A structural diagram of a social engineering phishing email detection device based on threat intelligence provided in Example 7 of the present application. DETAILED DESCRIPTION

[0084] Embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0085] The following describes a social engineering phishing email detection system and device based on threat intelligence according to an embodiment of the present application with reference to the accompanying drawings.

[0086] Figure 2 A structural diagram of a social engineering phishing email detection system based on threat intelligence provided in Example 1 of the present application.

[0087] like Figure 2 As shown in the figure, the threat intelligence-based social engineering phishing email detection system includes:

[0088] The client and the server, wherein the client includes:

[0089] Mail gateway, used to capture incoming or outgoing mail traffic, extract dynamic samples from the captured mail traffic and submit them to the sandbox, and generate static threat IOC feature indicators;

[0090] Sandbox, used to perform the operations contained in the dynamic sample and generate IOB behavior indicators of the dynamic threat sample;

[0091] The client is used to integrate static threat IOC characteristic indicators and dynamic threat sample IOB behavior indicators to form overall threat intelligence information, and report the overall threat intelligence information to the server;

[0092] The server includes:

[0093] Threat intelligence security operation and monitoring analysis module, which is used to process the overall threat intelligence information, determine the real threat intelligence, and store it in the threat intelligence warehouse;

[0094] The email security joint defense warning module is used to periodically pull data from the threat intelligence warehouse and share the pulled data with the client for threat joint defense warning.

[0095] The social engineering phishing email detection system based on threat intelligence in the embodiment of the present application can achieve the following by deploying a mail gateway and a multi-scenario intelligent sandbox locally, relying on a "cloud" threat intelligence sharing platform for expert operation, coordinating with important industry units, and carrying out real-time linkage sharing of threat intelligence such as malicious sample features, malicious URLs, and abnormal behaviors:

[0096] (1) Effectively defend against social engineering phishing attacks, cut off hackers' access to the intranet attack springboard through phishing attacks, and comprehensively protect the security of the email system and data.

[0097] (2) It fills the current data gap in the field of domestic defense against social engineering phishing intelligence, and builds a complete data operation system for intelligence production, operation, and consumption.

[0098] (3) Effectively solve the problem of uneven email security awareness among corporate personnel during actual combat, realize automated response to email phishing and malicious program attacks through timely threat warning, and use intelligence operation mechanism to achieve the protection effect of "one-point monitoring, full network warning".

[0099] See also Figure 3 , Figure 3 A schematic diagram of the structure of a social engineering phishing email detection system based on threat intelligence provided in Embodiment 2 of the present application is shown. The method is executed by an electronic device.

[0100] The system consists of three parts: client, threat intelligence security operation and monitoring analysis module and email security joint defense warning module: 1) The server includes threat intelligence security operation and monitoring analysis module and email security joint defense warning module, which are deployed on the application system and service side, installed and run on the server, responsible for data reception, threat intelligence analysis and processing, threat intelligence sharing, etc.; 2) The client includes email gateway and sandbox, which are deployed on the data collection side, installed and run on the server, responsible for uploading collected data, receiving and executing server instructions, etc. The overall plan is as follows Figure 3 As shown, the following steps are included:

[0101] (1) The email gateway first captures incoming or outgoing email traffic, including the email body, attachments, links, and email metadata such as sender, recipient, and subject;

[0102] (2) The extracted samples are submitted to a sandbox, an isolated environment for safely executing potentially malicious code or accessing malicious links;

[0103] (3) The email gateway and sandbox generate static threat intelligence IOC feature indicators and dynamic threat sample IOB behavior indicators respectively, and correlate and integrate them to form the overall threat intelligence information of the client for reporting;

[0104] (4) After receiving threat intelligence information reported by the client, the threat intelligence security operation and monitoring analysis module performs real-time filtering, system and manual analysis and verification to ensure the accuracy of the threat intelligence and store it in the threat intelligence warehouse;

[0105] (5) The email security joint defense warning module periodically pulls incremental data from the threat intelligence warehouse and shares it with the client for threat joint defense warning;

[0106] (6) While sharing threat intelligence, the intelligence is also used as training data for large models to enhance the capabilities of the client AI detection engine.

[0107] Specifically, the implementation process of the system includes:

[0108] a) Threat intelligence collection process

[0109] 1) The client encapsulates all the field information appearing in the alarm information into a complete set based on the unified data reporting metadata field standard, and refers to the naming, meaning, expression, and assignment of the unified fields, and then defines access and data reporting according to the unified coding and format.

[0110] 2) A single alarm log message is reconstructed in a JSON format according to the unified field standard to form a unified log format file. The generated log data information is encoded in Base64 encoding format during the reporting process to prevent reading and writing problems caused by special characters during data exchange and transmission.

[0111] 3) After each access data source completes the above two types of data standards, the regularized alarm data will be transmitted to the big data system through an encrypted network transmission protocol.

[0112] 4) According to the unified data collection standard, you can receive all kinds of data information from different systems, different platforms, and different manufacturers. Classify according to device functions, data types, log attributes, etc., create corresponding collection adapters for each type of data, and then use the unified data collection center to uniformly schedule and control the policies of each classification adapter to complete distributed log data collection.

[0113] b) Threat intelligence analysis process

[0114] Data analysis includes three steps: online real-time analysis, offline analysis, and manual assisted analysis:

[0115] (1) Real-time mining and analysis:

[0116] 1) Real-time data is connected to the distributed message queue Kafka in real time by Flume's distributed Agent nodes;

[0117] 2) Extract, transform, and load into the HDFS distributed file system through ETL;

[0118] 3) The rule model library synchronously consumes data in the message queue to optimize rules and models, and sends the optimization results to the real-time mining and analysis engine;

[0119] 4) Based on Flink, real-time processing of data in Kafka / HDFS is carried out to generate high-quality intelligence data through rule matching, model comparison, scenario and cluster analysis, etc.

[0120] 5) Use the application layer service architecture ClickHouse, Impala, and Azkaban to schedule data, and then provide it to the client through the email security joint defense warning module.

[0121] (2) Offline mining analysis:

[0122] 1) Clean and preprocess historical data to remove duplication, noise and unnecessary information to ensure data quality for subsequent analysis;

[0123] 2) Integrate information from different types into the same data warehouse and analytical platform for comprehensive analysis;

[0124] 3) Use data mining and analysis techniques to analyze the integrated data to identify potential threat signs and patterns, including detection of abnormal behavior activities, correlation analysis between various types of data, time series analysis of events, feature extraction, etc.;

[0125] 4) Use machine learning or deep learning methods to train models based on known threat intelligence data to help identify potential threats, and evaluate the accuracy, recall, precision, etc. of the trained models to determine their applicability and effectiveness;

[0126] 5) Based on the system analysis and model detection results, identify potential threat intelligence, attacks or abnormal activities, and discover new attack patterns, malware samples, attacker behaviors, etc.;

[0127] 6) Manually analyze the offline analysis results to form a security event entry event model warehouse, and provide security events and threat intelligence to the client through the email security joint defense early warning module.

[0128] (3) Human-assisted analysis:

[0129] 1) Normalize and standardize the intermediate data for comparison and analysis;

[0130] 2) Screen and filter data to quickly identify potential threat indicators and patterns by using automated tools and techniques;

[0131] 3) Conduct in-depth research to assess the authenticity and impact of the threat and identify potential risks associated with the organization;

[0132] 4) Communicate the results to relevant parties and develop response plans to mitigate threats and improve security.

[0133] c) Threat Intelligence Operation Process

[0134] (1) Using a self-developed large-scale pre-trained language model, we perform vector encoding on the email text content and gain the ability to understand the semantic representation of the text through unsupervised learning;

[0135] (2) The semantic vector retains the semantic attribute information of the text and can perform continuous unsupervised training to learn the text semantic patterns from the vector and achieve incremental optimization of the model.

[0136] (3) Use Spark SQL to access multiple data sources, including the original email extended information log stored in ClickHouse, various network security resource libraries stored in MySQL (such as vulnerability libraries, whitelist libraries, domain name libraries, etc.), and STIX2 format threat intelligence stored in Elasticsearch;

[0137] (4) When new data comes in, Spark SQL executes complex SQL queries to associate the above multiple data sources. These association operations include but are not limited to: JOIN operations, conditional filtering, field extraction, etc. In this way, a complete data record containing original logs, network security resource library information, and threat intelligence can be formed;

[0138] (5) Security operations personnel view and analyze various information such as logs, vulnerabilities, and threat intelligence on a unified interface. When page data does not meet analysis requirements, they can modify or add Spark Sql to meet the fields and analysis dimensions required by the new scenario in a timely manner.

[0139] (6) Threat information is divided into three dimensions: object, method, and event. Eight components, including observable data, attack indicators, security events, attack activities, threat subjects, attack targets, attack methods, and countermeasures, are used to describe threat information, forming detection rules that are used to detect "security events" again;

[0140] (7) Establish associations with existing attack organizations through attack organization characteristics. When the intelligence released detects that the network is attacked again, the attack organization of this attack is determined by comparing the attack traces with the attack organization fingerprint database;

[0141] (8) Add high-credibility, active source intelligence data and intelligence reassembled by intelligence analysis models to the intelligence database in real time;

[0142] (9) When the client updates and uses this intelligence to detect a threat, the new threat data discovered by the intelligence is transmitted back to the platform, and the security operations staff will re-evaluate the event data;

[0143] (10) Marking the effective time of accurate and effective intelligence data to improve the credibility of intelligence, lowering the credibility of false intelligence data and optimizing the analytical model that generates this intelligence;

[0144] (11) Finally, the time effectiveness of intelligence detection is dynamically adjusted based on the triggering probability of different types of intelligence in the actual detection scene environment.

[0145] d) Threat intelligence sharing process

[0146] (1) First, use authentication and authorization tokens to verify access rights to ensure that only authorized organizations can access and download shared threat intelligence;

[0147] (2) Extract corresponding threat intelligence data from the threat intelligence warehouse based on different organizational subscriptions;

[0148] (3) Classify, merge, encrypt and compress threat intelligence data;

[0149] (4) Share threat intelligence to the client via encrypted network transmission protocol.

[0150] This system is based on the HTTP protocol API interface, and other application layer protocols such as WebSocket can be used instead to implement data exchange; the SparkSql framework is used to support the association of multiple data sources, and the Apache Flink framework can be used for data fusion, both of which support real-time and batch processing; the middle-layer data uses Clickhouse's memory data table, which can be replaced by MySQL's memory storage engine (MEMORY).

[0151] See also Figure 4 , Figure 4 A flow chart of a method for detecting social engineering phishing emails based on threat intelligence provided in Example 3 of the present application is shown.

[0152] like Figure 4 As shown, this embodiment discloses a serial deployment (interception mode), which is suitable for implementation scenarios where there is no mail gateway, the security compliance requirements need to be met, and there is a strong demand for monitoring and interception of new phishing emails:

[0153] Step 1: Change the original email delivery path by modifying the upstream MTA and deliver the original email to the social engineering phishing detection and analysis system.

[0154] Step 2: The social engineering phishing detection and analysis system will perform analysis. Threat / phishing emails will be isolated / intercepted, safe emails will be released, and alert emails will modify the original email information and add warning information.

[0155] Step 3: The social engineering phishing detection and analysis system delivers the email to the email server.

[0156] See also Figure 5 , Figure 5 A flow chart of a method for detecting social engineering phishing emails based on threat intelligence provided in Embodiment 4 of the present application is shown.

[0157] like Figure 5 As shown, this embodiment discloses a serial deployment (monitoring mode), which is suitable for customers who are concerned about interception errors in serial deployment to avoid affecting email services, and hope to observe the serial capability and false alarm problems first, solve the encrypted traffic email restoration, and account login for monitoring:

[0158] Step 1: By modifying the upstream MTA, the original mail delivery path is changed and the original mail is delivered to the social engineering phishing detection and analysis system;

[0159] Step 2: Phishing emails and warning emails that have been analyzed and identified by the social engineering phishing detection and analysis system will automatically embed warning information in the original emails;

[0160] Step 3: The social engineering phishing detection and analysis system delivers the email to the email server.

[0161] See also Figure 6 , Figure 6 A flow chart of a method for detecting social engineering phishing emails based on threat intelligence provided in Example 5 of the present application is shown.

[0162] like Figure 6 As shown, this embodiment discloses a blind carbon copy (BCC) deployment mode, which is suitable for scenarios where customers are concerned about new phishing email attacks from outside to inside, and the client-side email gateway supports direct configuration of the BCC blind copy function.

[0163] Step 1: Configure the upstream MTA to deliver email copies to the social engineering phishing detection and analysis system through the blind carbon copy (BCC) mode;

[0164] Step 2: The social engineering phishing detection and analysis system analyzes the emails. It does not intercept or isolate the identified phishing emails or warning emails, but instead notifies the target email customers by resending warning emails.

[0165] See also Figure 7 , Figure 7 A flow chart of a method for detecting social engineering phishing emails based on threat intelligence provided in Example 6 of the present application is shown.

[0166] like Figure 7As shown, this embodiment discloses a bypass mirroring (SPAN) deployment mode, which is suitable for scenarios where customers require that the original mail service flow not be changed for easy deployment.

[0167] Step 1: Configure the upstream TAP or network switch to mirror and forward SMTP email traffic to the social engineering phishing detection and analysis system;

[0168] Step 2: The social engineering phishing detection and analysis system analyzes the email content, determines the nature of the email, and then generates a warning email and delivers it to the user's inbox;

[0169] To use this mode, ensure that the upstream TAP or network switch supports SMTP mail traffic mirroring.

[0170] In order to implement the above embodiment, the present application also proposes a social engineering phishing email detection method based on threat intelligence.

[0171] Figure 8 A structural diagram of a social engineering phishing email detection device based on threat intelligence provided in Example 7 of the present application.

[0172] like Figure 8 As shown, the threat intelligence-based social engineering phishing email detection method includes the following steps:

[0173] Step 801, detecting emails through a social engineering phishing email detection system, and classifying the detected emails into threat emails and non-threat emails, wherein the non-threat emails include warning emails and normal emails;

[0174] Step 802: Add warning information to the detected threat emails and / or warning emails through the social engineering phishing email detection system, and deliver the threat emails and / or non-threat emails to the email server, and send them to the user's inbox through the email server.

[0175] It should be noted that the aforementioned explanation of the embodiment of the social engineering phishing email detection method based on threat intelligence is also applicable to the social engineering phishing email detection device based on threat intelligence of this embodiment, and will not be repeated here.

[0176] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example" or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0177] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0178] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, fragment or portion of code comprising one or more executable instructions for implementing the steps of a custom logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0179] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

[0180] It should be understood that the various parts of the present application can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0181] A person skilled in the art may understand that all or part of the steps in the method for implementing the above-mentioned embodiment may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiment.

[0182] In addition, each functional unit in each embodiment of the present application may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into one module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium.

[0183] The storage medium mentioned above may be a read-only memory, a magnetic disk or an optical disk, etc. Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limiting the present application. A person of ordinary skill in the art may change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. A social engineering phishing email detection system based on threat intelligence, It is characterized in that It includes a client and a server, wherein the client includes: Mail gateway, used to capture incoming or outgoing mail traffic, extract dynamic samples from the captured mail traffic and submit them to the sandbox, and generate static threat IOC feature indicators; Sandbox, used to perform the operations contained in the dynamic sample and generate IOB behavior indicators of the dynamic threat sample; The client is used to integrate the static threat IOC characteristic indicators and the dynamic threat sample IOB behavior indicators to form overall threat intelligence information, and report the overall threat intelligence information to the server; The server includes: Threat intelligence security operation and monitoring analysis module, which is used to process the overall threat intelligence information, determine the real threat intelligence, and store it in the threat intelligence warehouse; The email security joint defense warning module is used to periodically pull data from the threat intelligence warehouse and share the pulled data with the client for threat joint defense warning.

2. The system according to claim 1, It is characterized in that The client is specifically used for: The overall threat intelligence information is processed according to preset standards to obtain the data to be reported, and access and data reporting are defined according to the prescribed coding and format.

3. The system according to claim 1, It is characterized in that The threat intelligence security operation and monitoring analysis module is connected to at least one data source, and the threat intelligence security operation and monitoring analysis module is also used to collect threat intelligence data, wherein the collection of threat intelligence data includes: Receive data transmitted from an access data source via an encrypted network transmission protocol, wherein the data includes standardized alarm log information and threat intelligence information reported by a client; Classify the received data and create corresponding collection adapters for each type of data. Complete distributed log data collection by unified scheduling and policy control of the collection adapters. The overall threat intelligence information is processed to determine the real threat intelligence, including online real-time analysis, offline analysis and manual assisted analysis.

4. The system according to claim 3, It is characterized in that The online real-time analysis includes: Access the distributed message queue Kafka in real time through Flume's distributed Agent nodes to obtain real-time data; The real-time data is extracted and converted through ETL and then loaded into the HDFS distributed file system; Synchronizing the real-time data through a rule model library to optimize rules and models; Based on Flink, real-time processing of data in Kafka / HDFS is used to generate real threat intelligence data through rule matching, model comparison, scenario and cluster analysis. The threat intelligence security operation and monitoring analysis module is also used to: The application layer service framework is used to schedule real threat intelligence data and store it in the threat intelligence warehouse.

5. The system according to claim 3, It is characterized in that The offline analysis includes: Preprocess the historical data to obtain preprocessed historical data, and store the preprocessed historical data in the data warehouse; Integrate information from different types into the data warehouse and link it to the analysis platform; Use data mining and analysis technology to analyze the data integrated in the data warehouse on the analysis platform to obtain analysis results; The detection model is trained based on known threat intelligence data, and the pre-processed historical data is tested using the detection model to obtain the detection results. Based on the analysis and detection results, identify potential threat information and obtain offline analysis results; Manually judge and analyze the offline analysis results to form a security incident entry event model warehouse.

6. The system according to claim 3, It is characterized in that The manual assisted analysis includes: Normalize and standardize the data to obtain standard data; Use automated tools and techniques to screen and filter standard data to identify potential threat information; Through manual judgment of real threat information.

7. The system of claim 1, It is characterized in that The threat intelligence security operation and monitoring analysis module is also used to: The email text content is vectorized using a large-scale pre-trained language model, and the model is trained using unsupervised learning. During training, the text semantic patterns are learned from the vectors to achieve incremental optimization of the model. Use Spark SQL to access at least one data source to obtain distributed log data; Execute SQL queries through Spark SQL to associate corresponding data in each data source to form threat data records; Based on threat data records, detection rules for detecting security incidents are formed according to different dimensions and different threat description information; Establish associations with existing attack organizations through attack organization characteristics, form an attack organization fingerprint library, and generate attack organization identification strategies; Add the source intelligence data with credibility and activity greater than the threshold and the intelligence reassembled by the intelligence analysis model to the enthusiasm database in real time; Mark the effective time for accurate and effective intelligence data, lower the credibility of false intelligence data and optimize the analysis model that generates this intelligence; Dynamically adjust the time validity of intelligence detection rules according to the triggering probability of different types of intelligence in the actual detection scene environment; The email security joint defense warning module is also used to send strategies, rules, and models to the client for threat joint defense warning.

8. The system of claim 1, It is characterized in that The method of periodically pulling data from the threat intelligence warehouse and sharing the pulled data with the client for threat joint defense warning includes: Verify the client's access rights using authentication and authorization token verification to identify authorized clients; Extract corresponding threat intelligence data from the threat intelligence warehouse based on the subscription status of the authorized client; The extracted threat intelligence data is classified, merged, encrypted and compressed to obtain processed threat intelligence data; The processed threat intelligence data is shared to the corresponding authorized client through an encrypted network transmission protocol.

9. A social engineering phishing email detection method based on threat intelligence, It is characterized in that include: Detecting emails by using the social engineering phishing email detection system as described in claims 1-8, and dividing the detected emails into threatening emails and non-threatening emails, wherein the non-threatening emails include warning emails and normal emails; The social engineering phishing email detection system adds warning information to the detected threatening emails and / or warning emails, and delivers the threatening emails and / or non-threatening emails to the email server, and sends them to the user's inbox through the email server.

Images