Indicator of compromise extraction method and apparatus, medium, and electronic device

Abstract

The present disclosure relates to an indicator of compromise (IOC) extraction method and apparatus, a medium, and an electronic device. The method comprises: using a machine learning-based classification model and at least one regular expression-based extraction model to respectively extract IOCs of data of IOCs to be extracted; and on the basis of a first extraction result of the classification model and a second extraction result of each extraction model, generating a target IOC of the data. In this way, a machine learning technique and a regular expression technique can be combined to form a multi-level and multi-angle IOC extraction system. On one hand, the extraction system can accurately extract known IOCs on the basis of known compromise features and patterns; on the other hand, the system can effectively identify and extract novel and unknown IOCs without depending on the known patterns, and in this way, can effectively extract various types of IOCs, and can also be adapted to the dynamic change and increasing complexity of cyber compromises, thereby enhancing the comprehensiveness and forward-looking capability of network security protection.

Description

Threat indicator extraction method, device, medium and electronic equipment

[0001] This application claims priority to Chinese Patent Application No. 202410195571.9 filed on February 21, 2024, and the contents of the above-mentioned Chinese patent application disclosure are hereby incorporated by reference in their entirety as a part of this application. Technical Field

[0002] The present disclosure relates to a threat indicator extraction method, device, medium and electronic equipment. Background Art

[0003] With the development and widespread use of network technology, network security issues are becoming increasingly prominent. Cyber ​​threats such as viruses, malware, and phishing attacks are becoming increasingly common, posing a serious threat to the network security of individuals and businesses. To combat these threats, various network security tools and technologies have emerged. For example, full network traffic analysis can be used to detect and prevent potential network attacks, safeguarding network security.

[0004] However, traditional network security tools and technologies primarily rely on known threat signatures and patterns for detection, often failing to effectively address new, unknown threats. Furthermore, traditional network traffic analysis tools typically only provide information at the network level, such as IP addresses and ports, but are unable to delve deeper into the application layer to extract valuable information from network traffic payloads, such as indicators of compromise (IOCs). Summary of the Invention

[0005] This summary is provided to briefly introduce concepts that will be described in detail in the detailed description below. This summary is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0006] In a first aspect, the present disclosure provides a threat indicator extraction method, comprising:

[0007] Obtain data on threat indicators to be extracted;

[0008] extracting threat indicators from the data using a machine learning-based classification model and at least one regular expression-based extraction model;

[0009] A target threat indicator of the data is generated according to the first extraction result of the classification model and the second extraction result of each of the extraction models.

[0010] In a second aspect, the present disclosure provides a threat indicator extraction device, comprising:

[0011] A first acquisition module is used to acquire data of threat indicators to be extracted;

[0012] an extraction module, configured to extract threat indicators from the data using a machine learning-based classification model and at least one regular expression-based extraction model;

[0013] A generating module is used to generate a target threat indicator of the data according to the first extraction result of the classification model and the second extraction result of each of the extraction models.

[0014] In a third aspect, the present disclosure provides a computer-readable medium having a computer program stored thereon, which, when executed by a processing device, performs the steps of the threat indicator extraction method provided in the first aspect of the present disclosure.

[0015] In a fourth aspect, the present disclosure provides an electronic device, comprising:

[0016] a storage device having a computer program stored thereon;

[0017] A processing device is used to execute the computer program in the storage device to implement the steps of the threat indicator extraction method provided in the first aspect of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The above and other features, advantages and aspects of the various embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that the originals and elements are not necessarily drawn to scale. In the drawings:

[0019] Fig. 1 is a flowchart showing a method for extracting threat indicators according to an exemplary embodiment.

[0020] Fig. 2 is a schematic diagram showing a threat indicator extraction method according to an exemplary embodiment.

[0021] Fig. 3 is a schematic diagram showing an attack scenario according to an exemplary embodiment.

[0022] Fig. 4 is a schematic diagram showing a data preprocessing according to an exemplary embodiment.

[0023] FIG5 is a schematic diagram showing an expert model extracting IOCs of threat indicator data to be extracted according to an exemplary embodiment.

[0024] FIG6 is a schematic diagram showing an IOC workflow for classifying and extracting data of threat indicators to be extracted according to an exemplary embodiment.

[0025] FIG7 is a schematic diagram showing a process of generating target threat indicators of data to be extracted based on a first extraction result of a classification model and a second extraction result of each extraction model, according to an exemplary embodiment.

[0026] Fig. 8 is a schematic diagram of an alarm data processing system according to an exemplary embodiment.

[0027] Fig. 9 is a block diagram showing a threat indicator extraction device according to an exemplary embodiment.

[0028] Fig. 10 is a schematic structural diagram of an electronic device according to an exemplary embodiment. DETAILED DESCRIPTION

[0029] To address the inability to extract IOCs at the application level, as mentioned in the background, regular expressions based on known attack types are currently commonly used to extract IOCs from network traffic. This approach typically relies on the experience and knowledge of security experts, writing precise regular expressions to match and extract specific patterns in network traffic, such as malicious URLs, suspicious IP addresses and ports, and malicious instructions. In the field of network security, IOCs are key data points used to identify and detect malicious activity. These indicators can include file hash values, malicious IP addresses, registry entries, and more.

[0030] This regular expression-based approach is effective for known, pattern-based threats, accurately extracting IOCs. However, this approach has some significant limitations. First, it may not be effective for detecting new, unknown threats, as their signatures are not covered by existing regular expressions. Second, writing and maintaining regular expressions requires significant manpower and time, limiting its scalability and ability to respond to dynamically changing and increasingly complex cyber threats.

[0031] Furthermore, since regular expressions can typically only match specific text patterns, this approach may not be effective for extracting complex, non-textual network traffic, such as binary data, encrypted data, etc. Therefore, although regular expression-based methods work well in some scenarios, there are still many challenges and problems to be solved.

[0032] Therefore, we need a new technical solution that can not only effectively extract IOCs from various types of network traffic, but also adapt to the dynamic changes and increasing complexity of network threats. In view of this, the present disclosure provides a threat indicator extraction method, device, medium, and electronic device.

[0033] The following describes embodiments of the present disclosure in more detail with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.

[0034] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0035] As used herein, the term "including" and its variations are open-ended, i.e., "including but not limited to." The term "based on" means "based, at least in part, on." The term "one embodiment" means "at least one embodiment," the term "another embodiment" means "at least one additional embodiment," and the term "some embodiments" means "at least some embodiments." Definitions of other terms are provided in the following description.

[0036] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0037] It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, they should be understood as "one or more".

[0038] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0039] It is understandable that before using the technical solutions disclosed in the various embodiments of this disclosure, the type, scope of use, usage scenarios, etc. of the personal information involved in this disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0040] For example, in response to a user's active request, a prompt message is sent to the user to clearly inform the user that the operation requested will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the electronic device, application, server, storage medium, or other software or hardware that performs the operations of the disclosed technical solution based on the prompt message.

[0041] As an optional but non-limiting implementation, in response to receiving a user's active request, the prompt information may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form. Furthermore, the pop-up window may also contain a selection control for the user to select "agree" or "disagree" to provide personal information to the electronic device.

[0042] It is understandable that the above notification and user authorization process are merely illustrative and do not limit the implementation of the present disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of the present disclosure.

[0043] At the same time, it is understood that the data involved in the technical solution of the present disclosure (including but not limited to the data itself, the acquisition or use of the data) should comply with the requirements of relevant laws, regulations and relevant provisions.

[0044] Fig. 1 is a flow chart showing a method for extracting threat indicators according to an exemplary embodiment. As shown in Fig. 1 , the method may include S101 to S103.

[0045] In S101 , data of threat indicators to be extracted are obtained.

[0046] In the present disclosure, the data from which threat indicators are to be extracted can be network traffic data or network text data. The network text data can include various types of text content on the internet, such as intelligence forums, online social content, and attack reports. Thus, the threat indicator extraction method provided by the present disclosure can not only extract IOCs from network traffic data, but can also effectively extract IOCs from various types of online text, providing a more comprehensive and in-depth network threat analysis beyond traffic load analysis.

[0047] In S102 , threat indicators of data to be extracted are respectively extracted using a classification model based on machine learning and at least one extraction model based on regular expressions.

[0048] In this disclosure, a regular expression uses a single string to describe and match a series of strings that conform to a certain IOC rule. If the regular expression successfully matches the data to be matched, it means that the data to be matched contains IOC.

[0049] The above-mentioned classification model based on machine learning can be a language model such as Bidirectional Encoder Representation from Transformers (BERT), or a deep learning model such as a convolutional neural network or a recurrent neural network.

[0050] In S103 , a target threat indicator of the data of threat indicators to be extracted is generated according to the first extraction result of the classification model and the second extraction result of each extraction model.

[0051] In the present disclosure, the number of extraction models can be one or more. The expert model and each extraction model concurrently execute the IOC extraction operation through a coroutine.

[0052] In one embodiment, there is one extraction model. In this case, a classification model based on machine learning can be used to extract IOCs from the data of threat indicators to be extracted to obtain a first extraction result. At the same time, the one extraction model based on regular expressions can be used to extract IOCs from the data of threat indicators to be extracted to obtain a second extraction result. Subsequently, a target threat indicator of the data of threat indicators to be extracted is generated based on the first extraction result and the second extraction result.

[0053] In another embodiment, there are multiple extraction models. In this case, a classification model based on machine learning can be used to extract IOCs from the data of threat indicators to be extracted to obtain a first extraction result. At the same time, multiple extraction models based on regular expressions can be used to extract IOCs from the data of threat indicators to be extracted to obtain multiple second extraction results. Thereafter, a target threat indicator of the data of threat indicators to be extracted is generated based on the first extraction result and the multiple second extraction results.

[0054] In the above technical solution, a machine learning-based classification model and at least one regular expression-based extraction model are first used to extract the IOCs of the threat indicator data to be extracted. Then, based on the first extraction result of the classification model and the second extraction result of each extraction model, target threat indicators are generated for the threat indicator data to be extracted. In this way, machine learning technology and regular expression technology can be combined to form a multi-level, multi-angle IOC extraction system. On the one hand, this extraction system can accurately extract indicators of known threats based on known threat characteristics and patterns, and on the other hand, it can effectively identify and extract indicators of new and unknown threats without relying on known patterns. In this way, it can not only effectively extract various types of IOCs, but also adapt to the dynamic changes and increasing complexity of network threats, thereby improving the comprehensiveness and foresight of network security protection. In addition, the machine learning-based classification model can automatically learn and extract IOCs, reducing the reliance on manually written and maintained regular expressions, thereby improving the efficiency and accuracy of IOC extraction while reducing manpower and time costs.

[0055] The at least one regular expression-based extraction model may include an expert model based on a regular expression of an attack type (Attack Type).

[0056] The attack type-based regular expressions are regular expressions summarized by security experts based on known attack types, that is, regular expressions summarized based on attack-related IOC rules. The expert model uses these regular expressions to extract attack threat indicators from the data to be extracted. Attack types may include one or more of the following: Structured Query Language (SQL) injection attacks, cross-site scripting (XSS) attacks, information leakage attacks, code injection attacks, file upload attacks, file inclusion attacks, weak password attacks, etc. This is only an illustrative description of the attack types and does not specifically limit them.

[0057] As shown in Figure 2, the expert model can use a summary of known attack scenarios to detect various types of threat indicators, including network addresses (IP), ports (Port), domain names (Domain), links (URL), commands (i.e., binary programs such as python, curl, wget, git, etc.), and attack type identifiers. It also provides an attack type identification function, giving additional attack scenario semantics to the data of threat indicators to be extracted, which is conducive to understanding attack data and supporting other upstream businesses.

[0058] As shown in Figure 3, known attack scenarios mainly include rebound shell (Reverse Shell, also known as reverse shell), file download, vulnerability verification, domain name query, encoding scenario, host command execution, sandbox escape and other attack scenarios. Among them, the adjacent content on the right side of each attack scenario in Figure 3 is the command for extracting the attack type under the corresponding attack scenario.

[0059] Specifically, as shown in Figure 3, for rebound Shell attack scenarios, tools and script commands such as icmp and dnscat2 are usually used to extract attack types. Programming language commands such as python, perl, java, ruby, php, golang, and lua are also used to extract attack types. System commands such as bash, whois, wget, curl, socat, nc / netcat, exec, telnet, powershell, and xterm are used to extract attack instructions. For domain name query attack scenarios, commands such as nslookup, dig, and ping are usually used to extract attack types. For file download attack scenarios, commands such as wget, curl, and Commands such as git, ftp, tftp, and scp are used to extract attack types. For vulnerability verification scenarios, commands such as rmi, ldap, jdbc, t3, and spring are usually used to extract attack types. For encoding scenarios, commands such as base64, urlencode, hex, and unicode are usually used to extract attack types. For host command execution attack scenarios, commands such as whoami, id, cat, ifconfig, ls, uname, w, pwd, iwconfig, cd, hostname, ping, lsof, and env are usually used to extract attack types. For sandbox escape attack scenarios, the execSync command is usually used to extract attack types.

[0060] In one embodiment, the number of the extraction model is one, and the one extraction model is an expert model based on a regular expression of the attack type.

[0061] In another embodiment, there are multiple extraction models, and the multiple extraction models include: an expert model based on regular expressions of attack types, and one or more general models based on general regular expressions.

[0062] In the present disclosure, a universal regular expression is an expression defined based on universal IOC rules, wherein the universal IOC rules are determined based on data features of corresponding IOCs, and the universal model is used to extract universal threat indicators.

[0063] The multiple extraction models may include an expert model based on regular expressions for attack types and a general model based on general regular expressions; alternatively, the multiple extraction models may include an expert model based on regular expressions for attack types and multiple general models based on general regular expressions. When there are multiple general models, the regular expressions employed by each general model differ. Different general models focus on different threat indicator types and are independent of the fixed patterns or attack scenarios of the threat indicator data to be extracted. They can proactively extract all IOCs of the corresponding threat indicator type from the threat indicator data to be extracted for unknown threats.

[0064] When multiple generic models are used, users can specify which generic model or models to use to extract IOCs from the threat indicator data. Different generic models may use different regular expressions to process the same threat indicator type, representing extraction strategies with varying degrees of rigor or standards. This can lead to different extraction results, enabling different generic models to provide detection capabilities for different threat indicator types.

[0065] For example, as shown in Figure 2, multiple extraction models include an expert model and three general models, namely general model A, general model B and general model C. Among them, general model A and general model B can both be used to detect three types of threat indicators, namely IP, Domain and URL, and model C is used to detect two types of threat indicators, namely IP and URL.

[0066] In the above implementation, the expert model relies on the experience and knowledge of experts, focusing on extracting threat indicators for known or fixed-pattern attack scenarios, achieving rapid detection efficiency and ensuring basic detection results. At the same time, to address the problem that regular expressions can generally only match specific text patterns, an additional extractor for general threat indicators (i.e., a general model) is provided to provide a certain degree of flexibility in detection capabilities. Even when expert experience is not accumulated, it can provide basic detection results for specific threat indicator types (e.g., IP addresses, domain names, URLs, etc.).

[0067] In addition, the above-mentioned expert model is constructed based on regular expressions of multiple attack types. The expert model can include multiple extraction plug-ins, wherein each extraction plug-in corresponds to a regular expression of at least one attack type, and the regular expressions of attack types corresponding to different extraction plug-ins do not overlap. In this way, IOC extraction can be performed by calling the extraction plug-in without affecting the normal execution of the existing execution process. Moreover, the extraction plug-in is configurable, which improves the flexibility of the expert model. At this time, the above-mentioned expert model can extract the IOC of the data of the threat indicator to be extracted through the following steps (11) to (15).

[0068] Step (11): Preprocess the data of threat indicators to be extracted to obtain text data.

[0069] In this disclosure, the data of threat indicators to be extracted may be normal and easy-to-read text, or it may be encoded network traffic data. Attackers usually use obfuscation technology to make the data of threat indicators to be extracted bypass basic detection. In addition, the data of threat indicators to be extracted is unpredictable and has a variety of different text forms, which brings more difficulties to the subsequent IOC extraction. Therefore, this disclosure sets up a data preprocessing link (as shown in Figure 2) to facilitate subsequent analysis and extraction.

[0070] As shown in Figure 4, the preprocessing process for extracting threat indicators can include the following three steps:

[0071] ① Data cleaning: This mainly involves removing invalid, redundant, and irrelevant data from the threat indicator data to improve data quality. For example, this involves removing redundant data packets, filling missing values, and processing outliers.

[0072] ② Data conversion: This primarily involves converting cleaned data into a format suitable for analysis. This includes converting binary or hexadecimal data into text, or converting unstructured data into structured data. It may also include data standardization, such as converting all data to the same scale or converting numerical data into text, to facilitate subsequent analysis and comparison.

[0073] ③ Data decoding: This involves using a decoder to decode the encoded data back to its original format for subsequent analysis. For example, data encoded with base64 or URL encoding can be decoded into regular text data that can be processed by the classification model.

[0074] Step (12): Perform regular matching on the text data according to the regular expression of the attack type.

[0075] Step (13): If there is a matching regular expression in the text data, for each matching regular expression, a target extraction plug-in corresponding to the matching regular expression is determined from multiple extraction plug-ins.

[0076] Step (14): Utilize the target extraction plug-in to extract the threat indicators in the matching regular expression.

[0077] Step (15): Merge the threat indicators in each matching regular expression to obtain the second extraction result of the expert model.

[0078] In the present disclosure, as shown in FIG5 , the regular expressions for multiple attack types used to construct the expert model can exist in the form of a regular expression pattern configuration set. As shown in FIG5 , after obtaining text data through data preprocessing, the text data can be compared with a regular expression whitelist to filter out the content in the regular expression whitelist from the text data; then, the filtered text data is subjected to regular matching with the regular expressions of each attack type in the regular expression pattern configuration set; if a matching regular expression exists in the filtered text data, then for each matching regular expression (i.e., each successfully matched regular matching result), a target extraction plug-in corresponding to the matching regular expression is determined from multiple extraction plug-ins (i.e., a regular plug-in corresponding to the matching regular expression is determined); then, the regular configuration (i.e., the configuration of the matching regular expression) and the corresponding extraction plug-in (i.e., the target extraction plug-in) are loaded, and the threat indicators in the matching regular expression are extracted using the target extraction plug-in; finally, the threat indicators in each matching regular expression are merged to remove duplicates, obtaining a second extraction result of the expert model, and the result is returned.

[0079] In addition, as shown in Figures 2 and 5, the expert model also supports adding new regular expressions to the regular expression whitelist, or adding the configuration of the new regular expression to the regular expression pattern configuration set to check suspicious data, thereby improving the scalability of the expert model to adapt to the dynamic changes and increasing complexity of network threats. Specifically, the above-mentioned threat indicator extraction method can also include the following steps:

[0080] In response to receiving a new regular expression for the expert model, determining an extraction plug-in that matches the new regular expression from a plurality of extraction plug-ins;

[0081] Establish a correspondence between matching extraction plug-ins and newly added regular expressions.

[0082] In addition, the above classification model can extract threat indicators from the data of threat indicators to be extracted through the following steps (21) to (25).

[0083] Step (21): pre-process the data of threat indicators to be extracted to obtain text data.

[0084] Step (22): Segment the text data to obtain a segmentation sequence.

[0085] In the present disclosure, a heuristic text tokenizer (i.e., the Tokenizer shown in FIG6 ) can be used to tokenize text data to obtain a token sequence, wherein tokens in the token sequence that contain sub-words of a preset type carry first identification information, and the length of the sub-word is shorter than the length of the token to which it belongs. For example, tokens such as URL and command contain sub-words such as IP and Port.

[0086] In order to adapt to the special form of network traffic load that is different from regular text, additional word segmentation symbols with special meanings (for example, backslash, hyphen, etc.) can be added to the symbol table of the text word segmenter, so that the granularity of word segmentation can match the symbols and language semantics of the network traffic load and the terminal command line, so as to improve the accuracy of word segmentation and thereby improve the accuracy of the first extraction result.

[0087] Step (23): vectorize the word segmentation sequence to obtain a text vector, input the text vector into the classification model, and obtain the classification results and threat indicator types of each word in the word segmentation sequence.

[0088] After obtaining a word sequence through word segmentation by a text word segmenter, the word sequence can be vectorized to obtain a text vector, and then the text vector is input into a classification model (for example, the BERT model shown in Figure 6). The classification model may include a linear classification layer and multiple encoders connected in sequence, and the linear classification layer is connected to the last encoder in the multiple encoders connected in sequence. The encoder is used to capture the complex relationship between the words in the word sequence and generate a high-dimensional vector representation of the text vector; the linear classification layer is used to generate the classification results and threat indicator types of each word in the word sequence based on the high-dimensional vector output by the last encoder. The classification result is used to characterize whether the corresponding word belongs to a threat indicator.

[0089] Step (24): for each segmented word in the segmented word sequence that carries the first identification information, extract the threat indicator in the subword in the segmented word to obtain a third extraction result.

[0090] In the present disclosure, the subwords in each segmentation may be input into a classification model to extract threat indicators from each subword as a third extraction result.

[0091] Step (25): Determine the segmented words belonging to threat indicators and their threat indicator types in the segmented word sequence, as well as the third extraction result as the first extraction result of the classification model.

[0092] After the classification model extracts IOCs and their threat indicator types from the threat indicator data to be extracted, and extracts IOCs from the subwords of each segmented word that carries the first identification information in the segmented word sequence, these are combined to obtain a first extraction result of the classification model. In this way, the classification model can not only extract IOCs and their threat indicator types from each segmented word in the threat indicator data to be extracted, but also extract IOCs from subwords of preset types within the segmented words, thereby improving the accuracy and comprehensiveness of IOC extraction.

[0093] In addition, after obtaining the first extraction result of the classification model, a labeling module (such as the ner module shown in Figure 6) can be used to automatically assign relevant labels to each threat indicator in the first extraction result based on the characteristics of the threat indicator in the first extraction result (such as type, source, target, etc.). These labels can help security experts and subsequent processes understand and process these threat indicators faster and more accurately.

[0094] In addition, the above classification model can be trained through steps (1) and (2):

[0095] Step (1): Get a text sample.

[0096] In the present disclosure, real alarm traffic from network full-flow analysis and network application firewalls can be used as the main source of training data, and some text content containing threat indicators can be added to enrich the training data; then, the training data is cleaned and labeled using security expert annotation and heuristic regular automatic annotation methods to obtain text samples.

[0097] A text sample may include multiple sample words and a label for each sample word. The label includes the classification result and threat indicator type used to characterize the sample word. If the sample word implies preset content, the text sample also includes the preset content, as well as the classification result and threat indicator type of the preset content. The classification result is used to indicate whether the corresponding sample word or preset content is a threat indicator. The preset content does not exist in the sample word but is implied by the sample word. To ensure the accuracy and comprehensiveness of IOC extraction by the classification model, the preset content, as well as the classification result and threat indicator type of the preset content, is added to the text sample.

[0098] For example, sample words such as URLs contain implicit port numbers. For example, a URL is an HTTP protocol, which has a default port number, such as 80. This port number does not exist in the HTTP protocol. However, to ensure the accuracy and comprehensiveness of the IOC extraction by the classification model, the default port number is added to the text sample, along with the classification results and threat indicator type for the default port number.

[0099] Step (2): Based on the text samples, fine-tune the pre-trained model based on machine learning to obtain a classification model.

[0100] During fine-tuning, the cross-entropy loss function can be used to measure the difference between the model's prediction results and the true labels, the Adam optimizer can be used to optimize the model's parameters, and the learning rate decay strategy can be used to adjust the learning rate.

[0101] The following describes in detail the specific implementation method for generating target threat indicators from the data to be extracted based on the first extraction result of the classification model and the second extraction result of each extraction model in S103. Specifically, this can be achieved through the following steps (31) and (32):

[0102] Step (31): Aggregate the first extraction result of the classification model and the second extraction result of each extraction model.

[0103] Step (32): Perform format verification on the extracted results obtained after the aggregation process to filter out the extraction results that do not comply with the format and obtain the target threat indicators.

[0104] As shown in Figures 2 and 7, after extracting a threat indicator set (i.e., the first extraction result) from the data of threat indicators to be extracted through the classification model, and extracting threat indicator results (i.e., the second extraction result) through the expert model and the general model respectively, the three threat indicator sets are aggregated to remove duplicate threat indicators (i.e., aggregation and deduplication); then, based on the threat indicator type, the entity legitimacy of each IOC in the extraction result obtained after the aggregation process is verified and filtered to filter out IOCs with non-compliant formats, thereby obtaining the final result (i.e., the target threat indicator).

[0105] In addition, the above method may further include the following steps:

[0106] If it is determined based on the target threat indicator that threat intelligence exists in the data of the threat indicator to be extracted, threat alarm information is generated.

[0107] After obtaining the target threat indicator of the data of the threat indicator to be extracted, the target threat indicator is matched with the IOC whitelist. If the target threat indicator contains a threat indicator outside the IOC whitelist, it is determined that threat intelligence exists in the data of the threat indicator to be extracted. At this time, threat alarm information can be generated to warn users to protect themselves from threats in a timely manner.

[0108] The present disclosure also provides an alarm data processing system, as shown in Figure 8, which includes a traffic automatic analysis module and a data association analysis module, wherein the traffic automatic analysis module includes a data preprocessing module and a network traffic data intelligent analysis module. The data preprocessing module is used to filter out some common false alarm data or alarm data of unrelated business scenarios from the alarm data to be processed. In this case, the data of the threat indicators to be extracted is the alarm data. The network traffic data intelligent analysis module mainly distinguishes the alarm data based on knowledge scenarios and natural language technology, automatically identifies IOC information in common high-risk scenarios and provides a unified interface service for structuring the extracted threat indicators to obtain structured threat indicators (i.e., the formatted alarm data in Figure 8) for use by the data association analysis module. The network traffic data intelligent analysis module is the most important and basic link to improve the accuracy of alarm analysis and operational efficiency. After identifying the IOC, the identified IOC can be matched with the IOC whitelist, and the IOCs outside the IOC whitelist among the identified IOCs are determined as threat intelligence.

[0109] As shown in FIG8 , a plurality of alarm data to be processed can be obtained from a Transmit Load Balancing (TLB) alarm log message queue and a Network Traffic Analysis (NTA) alarm log message queue.

[0110] The data correlation analysis module is used to correlate each structured threat indicator with Domain Name System (DNS) data, session data, intelligence data, and Host-based IDS (HIDS) data (hereinafter referred to as host data) to correlate relevant historical data, thereby obtaining multiple pieces of suspected risk information. Based on these multiple pieces of suspected risk information, alarm analysis is performed to identify real risk information, which is then provided to the Security Information and Event Management (SIEM) / Security Orchestration Automation and Response (SOAR) operations platform to generate security work orders based on the real risk information.

[0111] FIG9 is a block diagram of a threat indicator extraction device according to an exemplary embodiment. As shown in FIG9 , the device 200 includes:

[0112] A first acquisition module 201 is used to acquire data of threat indicators to be extracted;

[0113] An extraction module 202, configured to extract threat indicators from the data using a machine learning-based classification model and at least one regular expression-based extraction model;

[0114] The generating module 203 is configured to generate a target threat indicator for the data according to the first extraction result of the classification model and the second extraction result of each of the extraction models.

[0115] In the above technical solution, a machine learning-based classification model and at least one regular expression-based extraction model are first used to extract the IOCs of the threat indicator data to be extracted. Then, based on the first extraction result of the classification model and the second extraction result of each extraction model, target threat indicators are generated for the threat indicator data to be extracted. In this way, machine learning technology and regular expression technology can be combined to form a multi-level, multi-angle IOC extraction system. On the one hand, this extraction system can accurately extract indicators of known threats based on known threat characteristics and patterns, and on the other hand, it can effectively identify and extract indicators of new and unknown threats without relying on known patterns. In this way, it can not only effectively extract various types of IOCs, but also adapt to the dynamic changes and increasing complexity of network threats, thereby improving the comprehensiveness and foresight of network security protection. In addition, the machine learning-based classification model can automatically learn and extract IOCs, reducing the reliance on manually written and maintained regular expressions, thereby improving the efficiency and accuracy of IOC extraction while reducing manpower and time costs.

[0116] Optionally, the at least one regular expression-based extraction model comprises an expert model based on regular expressions of attack types.

[0117] Optionally, there are multiple extraction models, and the multiple extraction models further include one or more general models based on general regular expressions. When there are multiple general models, the regular expressions used by the multiple general models are different.

[0118] Optionally, the expert model includes a plurality of extraction plug-ins, each of which corresponds to a regular expression of at least one of the attack types;

[0119] The extraction module 202 includes:

[0120] A preprocessing submodule, used for performing data preprocessing on the data to obtain text data;

[0121] A regular matching submodule, configured to perform regular matching on the text data according to the regular expression of the attack type;

[0122] a first determination submodule configured to, if a matching regular expression exists in the text data, determine, for each matching regular expression, a target extraction plug-in corresponding to the matching regular expression from the plurality of extraction plug-ins; and a first extraction submodule configured to extract threat indicators from the matching regular expression using the target extraction plug-in;

[0123] The merging submodule is used to merge the threat indicators in each of the matched regular expressions to obtain the second extraction result of the expert model.

[0124] Optionally, the apparatus 200 further includes:

[0125] a determination module configured to, in response to receiving a new regular expression for the expert model, determine an extraction plug-in from the plurality of extraction plug-ins that matches the new regular expression;

[0126] An establishing module is used to establish a corresponding relationship between the matching extraction plug-in and the newly added regular expression.

[0127] Optionally, the extraction module includes:

[0128] A preprocessing submodule, used for performing data preprocessing on the data to obtain text data;

[0129] A word segmentation submodule, configured to segment the text data to obtain a word segmentation sequence, wherein a word segmentation sequence containing a subword of a preset type carries first identification information, and a length of the subword is smaller than a length of the word segmentation to which it belongs;

[0130] A vectorization submodule, configured to perform vectorization processing on the segmented word sequence to obtain a text vector, input the text vector into the classification model, and obtain a classification result and threat indicator type for each segmented word in the segmented word sequence, wherein the classification result is used to indicate whether the corresponding segmented word belongs to a threat indicator;

[0131] a second extraction submodule, configured to extract, for each segmented word in the segmented word sequence carrying the first identification information, threat indicators from the subwords in the segmented word sequence to obtain a third extraction result;

[0132] The second determining submodule is configured to determine the segmented words belonging to threat indicators and their threat indicator types in the segmented word sequence, and the third extraction result as the first extraction result of the classification model.

[0133] Optionally, the classification model is obtained by training a model training device, wherein the model training device includes:

[0134] a second acquisition module, configured to acquire a text sample, wherein the text sample includes a plurality of sample words and a label for each sample word, wherein the label includes a classification result and a threat indicator type for characterizing the sample word; and when the sample word implies preset content, the text sample also includes the preset content, as well as a classification result and a threat indicator type for the preset content, wherein the classification result is used to characterize whether the corresponding sample word or preset content is a threat indicator;

[0135] The fine-tuning module is used to fine-tune the pre-training model based on machine learning according to the text sample to obtain the classification model.

[0136] Optionally, the classification model is a deep learning model or a language model.

[0137] Optionally, the generating module 203 includes:

[0138] an aggregation processing submodule, configured to perform aggregation processing on the first extraction result of the classification model and the second extraction result of each of the extraction models;

[0139] The format check submodule is used to perform format check on the extraction results obtained after the aggregation processing to filter out extraction results that do not comply with the format and obtain the target threat indicators.

[0140] Optionally, the data is network traffic data or network text data.

[0141] Optionally, the apparatus 200 further includes:

[0142] The alarm module is configured to generate threat alarm information if it is determined that threat intelligence exists in the data of the threat indicator to be extracted based on the target threat indicator.

[0143] The present disclosure also provides a computer-readable medium having a computer program stored thereon, which, when executed by a processing device, implements the steps of the above-mentioned threat indicator extraction method provided by the present disclosure.

[0144] Reference is now made to FIG10 , which illustrates a schematic diagram of the structure of an electronic device (e.g., a terminal device or server) 600 suitable for implementing embodiments of the present disclosure. The terminal device in the embodiments of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. The electronic device illustrated in FIG10 is merely an example and should not limit the functionality and scope of use of the embodiments of the present disclosure.

[0145] As shown in Figure 10, the electronic device 600 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 into a random access memory (RAM) 603. Various programs and data required for the operation of the electronic device 600 are also stored in the RAM 603. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0146] Typically, the following devices may be connected to the I / O interface 605: an input device 606 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 608 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 609. The communication device 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. Although FIG10 illustrates the electronic device 600 with various devices, it should be understood that not all of the devices shown are required to be implemented or present. More or fewer devices may alternatively be implemented or present.

[0147] According to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 609, or installed from the storage device 608, or installed from the ROM 602. When the computer program is executed by the processing device 601, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.

[0148] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or component. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0149] In some embodiments, the client and server can communicate using any currently known or later developed network protocol, such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or later developed network.

[0150] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0151] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device is caused to: obtain data of threat indicators to be extracted; use a classification model based on machine learning and at least one extraction model based on regular expressions to respectively extract threat indicators from the data of threat indicators to be extracted; and generate target threat indicators for the data based on a first extraction result of the classification model and a second extraction result of each of the extraction models.

[0152] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including, but not limited to, object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0153] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0154] The modules described in the embodiments of the present disclosure may be implemented in software or hardware. In some cases, the name of a module does not necessarily limit the module itself. For example, the first acquisition module may also be described as a "module for acquiring data on threat indicators to be extracted."

[0155] The functions described above in the present disclosure may be performed at least in part by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0156] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0157] According to one or more embodiments of the present disclosure, Example 1 provides a threat indicator extraction method, including:

[0158] Obtain data on threat indicators to be extracted;

[0159] Extracting threat indicators from the data of threat indicators to be extracted using a classification model based on machine learning and at least one extraction model based on regular expressions;

[0160] A target threat indicator of the data is generated according to the first extraction result of the classification model and the second extraction result of each of the extraction models.

[0161] According to one or more embodiments of the present disclosure, Example 2 provides the method of Example 1, wherein the at least one regular expression-based extraction model includes an expert model based on regular expressions of attack types.

[0162] According to one or more embodiments of the present disclosure, Example 3 provides the method of Example 2, wherein there are multiple extraction models, and the multiple extraction models also include one or more general models based on general regular expressions, wherein, when there are multiple general models, the regular expressions used between the multiple general models are different.

[0163] According to one or more embodiments of the present disclosure, Example 4 provides the method of Example 2 or Example 3,

[0164] The expert model includes a plurality of extraction plug-ins, each of which corresponds to a regular expression of at least one of the attack types;

[0165] The expert model extracts threat indicators from the data of threat indicators to be extracted in the following manner:

[0166] Performing data preprocessing on the data to obtain text data;

[0167] Performing regular matching on the text data according to the regular expression of the attack type;

[0168] If a matching regular expression exists in the text data, for each matching regular expression, determining a target extraction plug-in corresponding to the matching regular expression from the plurality of extraction plug-ins; and extracting threat indicators from the matching regular expression using the target extraction plug-in;

[0169] The threat indicators in each of the matched regular expressions are merged to obtain the second extraction result of the expert model.

[0170] According to one or more embodiments of the present disclosure, Example 5 provides the method of Example 4, further comprising:

[0171] In response to receiving a new regular expression for the expert model, determining an extraction plug-in that matches the new regular expression from the plurality of extraction plug-ins;

[0172] A correspondence between the matching extraction plug-in and the newly added regular expression is established.

[0173] According to one or more embodiments of the present disclosure, Example 6 provides the method of Example 1, wherein the classification model extracts threat indicators from the data in the following manner:

[0174] Performing data preprocessing on the data to obtain text data;

[0175] Segmenting the text data to obtain a segmentation sequence, wherein a segmentation containing a subword of a preset type in the segmentation sequence carries first identification information, and a length of the subword is smaller than a length of the segmentation to which it belongs;

[0176] Performing vectorization processing on the word segmentation sequence to obtain a text vector, inputting the text vector into the classification model to obtain a classification result and threat indicator type for each word in the word segmentation sequence, wherein the classification result is used to indicate whether the corresponding word is a threat indicator;

[0177] For each segmented word in the segmented word sequence that carries the first identification information, extract the threat indicator in the subword in the segmented word to obtain a third extraction result;

[0178] The segmented words belonging to threat indicators and their threat indicator types in the segmented word sequence, as well as the third extraction result, are determined as the first extraction result of the classification model.

[0179] According to one or more embodiments of the present disclosure, Example 7 provides the method of Example 1, wherein the classification model is trained in the following manner:

[0180] Obtaining a text sample, wherein the text sample includes a plurality of sample words and a label for each of the sample words, wherein the label includes a classification result and a threat indicator type for characterizing the sample word; and when the sample word implies preset content, the text sample also includes the preset content, as well as a classification result and a threat indicator type for the preset content, wherein the classification result is used to characterize whether the corresponding sample word or preset content is a threat indicator;

[0181] According to the text samples, the pre-trained model based on machine learning is fine-tuned to obtain the classification model.

[0182] According to one or more embodiments of the present disclosure, Example 8 provides the method of Example 1, wherein the classification model is a deep learning model or a language model.

[0183] According to one or more embodiments of the present disclosure, Example 9 provides the method of Example 1, wherein generating a target threat indicator for the data based on the first extraction result of the classification model and the second extraction result of each of the extraction models includes:

[0184] performing aggregation processing on the first extraction result of the classification model and the second extraction result of each of the extraction models;

[0185] The extracted results obtained after the aggregation processing are format checked to filter out extraction results with non-compliant formats and obtain the target threat indicators.

[0186] According to one or more embodiments of the present disclosure, Example 10 provides the method described in any one of Examples 1-3 and 5-9, wherein the data is network traffic data or network text data.

[0187] According to one or more embodiments of the present disclosure, Example 11 provides the method of any one of Examples 1-3 and 5-9, further comprising:

[0188] If it is determined according to the target threat indicator that threat intelligence exists in the data, threat warning information is generated.

[0189] According to one or more embodiments of the present disclosure, Example 12 provides a threat indicator extraction device, including:

[0190] A first acquisition module is used to acquire data of threat indicators to be extracted;

[0191] an extraction module, configured to extract threat indicators from the data using a machine learning-based classification model and at least one regular expression-based extraction model;

[0192] A generating module is used to generate a target threat indicator of the data according to the first extraction result of the classification model and the second extraction result of each of the extraction models.

[0193] According to one or more embodiments of the present disclosure, Example 13 provides a computer-readable medium having a computer program stored thereon, which implements the steps of the method described in any one of Examples 1-11 when executed by a processing device.

[0194] According to one or more embodiments of the present disclosure, Example 14 provides an electronic device, including:

[0195] a storage device having a computer program stored thereon;

[0196] A processing device is used to execute the computer program in the storage device to implement the steps of the method described in any one of Examples 1-11.

[0197] The above description is merely an example of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned disclosed concepts. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this disclosure.

[0198] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details have been included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0199] Although the present disclosure has been described using language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims. Regarding the apparatus in the above-described embodiments, the specific manner in which each module performs operations has been described in detail in the embodiments related to the method and will not be elaborated upon here.

Claims

1. A threat indicator extraction method, comprising: Obtain data on threat indicators to be extracted; extracting threat indicators from the data using a machine learning-based classification model and at least one regular expression-based extraction model; A target threat indicator of the data is generated according to the first extraction result of the classification model and the second extraction result of each of the extraction models.

2. The method according to claim 1, wherein The at least one regular expression based extraction model comprises an expert model based on regular expressions of attack types.

3. The method according to claim 2, wherein: There are multiple extraction models, and the multiple extraction models further include one or more general models based on general regular expressions; when there are multiple general models, the regular expressions adopted by the multiple general models are different.

4. The method according to claim 2 or 3, wherein: The expert model includes a plurality of extraction plug-ins, each of which corresponds to a regular expression of at least one of the attack types; The expert model extracts threat indicators from the data in the following ways: Performing data preprocessing on the data to obtain text data; Performing regular matching on the text data according to the regular expression of the attack type; If a matching regular expression exists in the text data, for each matching regular expression, determining a target extraction plug-in corresponding to the matching regular expression from the plurality of extraction plug-ins; and extracting threat indicators from the matching regular expression using the target extraction plug-in; The threat indicators in each of the matched regular expressions are merged to obtain the second extraction result of the expert model.

5. The method according to claim 4, further comprising: In response to receiving a new regular expression for the expert model, determining an extraction plug-in that matches the new regular expression from the plurality of extraction plug-ins; A correspondence between the matching extraction plug-in and the newly added regular expression is established.

6. The method according to any one of claims 1 to 5, wherein: The classification model extracts threat indicators from the data in the following manner: Performing data preprocessing on the data to obtain text data; Segmenting the text data to obtain a segmentation sequence, wherein a segmentation containing a subword of a preset type in the segmentation sequence carries first identification information, and a length of the subword is smaller than a length of the segmentation to which it belongs; Performing vectorization processing on the word segmentation sequence to obtain a text vector, inputting the text vector into the classification model to obtain a classification result and threat indicator type for each word in the word segmentation sequence, wherein the classification result is used to indicate whether the corresponding word is a threat indicator; For each segmented word in the segmented word sequence that carries the first identification information, extracting the threat indicator in the subword in the segmented word to obtain a third extraction result; The segmented words belonging to threat indicators and their threat indicator types in the segmented word sequence, as well as the third extraction result, are determined as the first extraction result of the classification model.

7. The method according to any one of claims 1 to 6, wherein: The classification model is trained in the following way: Obtaining a text sample, wherein the text sample includes a plurality of sample words and a label for each of the sample words, wherein the label includes a classification result and a threat indicator type for characterizing the sample word; and when the sample word implies preset content, the text sample also includes the preset content, as well as a classification result and a threat indicator type for the preset content, wherein the classification result is used to characterize whether the corresponding sample word or preset content is a threat indicator; According to the text samples, the pre-trained model based on machine learning is fine-tuned to obtain the classification model.

8. The method according to any one of claims 1 to 7, wherein: The classification model is a deep learning model or a language model.

9. The method according to any one of claims 1 to 8, wherein: Generating a target threat indicator of the data according to the first extraction result of the classification model and the second extraction result of each of the extraction models includes: performing aggregation processing on the first extraction result of the classification model and the second extraction result of each of the extraction models; The extracted results obtained after the aggregation processing are format checked to filter out extraction results with non-compliant formats and obtain the target threat indicators.

10. The method according to any one of claims 1 to 9, wherein The data is network traffic data or network text data.

11. The method according to any one of claims 1 to 10, further comprising: If it is determined according to the target threat indicator that threat intelligence exists in the data, threat warning information is generated.

12. A threat indicator extraction device comprising: A first acquisition module is configured to acquire data of threat indicators to be extracted; an extraction module configured to respectively extract threat indicators from the data using a machine learning-based classification model and at least one regular expression-based extraction model; The generating module is configured to generate a target threat indicator of the data according to the first extraction result of the classification model and the second extraction result of each of the extraction models.

13. A computer-readable medium storing a computer program, wherein: When the computer program is executed by a processing device, the threat indicator extraction method according to any one of claims 1 to 11 is implemented.

14. An electronic device comprising: a storage device storing a computer program; A processing device is configured to execute the computer program in the storage device to implement the threat indicator extraction method according to any one of claims 1 to 11.

Images