A method and device for acquiring threat intelligence based on intrinsic security

Abstract

The present application discloses a method and device for acquiring threat intelligence based on intrinsic security, including: acquiring intelligence data from multiple intelligence sources. Based on the threat intelligence template corresponding to the intelligence source, capturing threat intelligence from the intelligence data of multiple intelligence sources to obtain threat intelligence to be processed. Preprocessing the threat intelligence to be processed to obtain processed threat intelligence. Conflict detection is performed on the processed threat intelligence to obtain at least two conflicting threat intelligence. Then, conflict resolution is performed on at least two conflicting threat intelligence to obtain conflict-resolved threat intelligence. The present application provides a method for extracting and fusing threat intelligence from multiple intelligence sources. Based on the threat intelligence template corresponding to the intelligence source, threat intelligence is extracted from the intelligence data of multiple intelligence sources, and then threat intelligence data is preprocessed and conflict resolved, so that the threat intelligence finally obtained by fusion is more standardized and the quality of the threat intelligence obtained is higher.

Description

Technical Field

[0001] The present application relates to the field of Internet security technology, and in particular to a method and device for acquiring threat intelligence based on intrinsic security. Background Art

[0002] Threat intelligence is evidence-based knowledge, including context, mechanism, indication, meaning and executable suggestions, which is used to provide information support for enterprises or institutions to respond to or deal with threats. For example, domain names that generate security threats can be used as threat intelligence. Threat intelligence vividly expresses the characteristics of data-driven security under the endogenous system. Endogenous security must be intelligence-first.

[0003] Intelligence repositories store and apply threat intelligence, and are a common source of threat intelligence, which can be called intelligence sources. More and more companies are adding threat intelligence from intelligence repositories to their security protection networks.

[0004] In actual applications, enterprises will obtain multiple different intelligence sources and use threat intelligence platforms to obtain threat intelligence from multiple intelligence sources. However, the content of multiple intelligence sources currently obtained is uneven, and how to fuse data from multiple intelligence sources is an urgent problem to be solved. Summary of the invention

[0005] In order to solve the above technical problems, the present application provides a method and device for acquiring threat intelligence based on intrinsic security, which is used to automatically perform data fusion processing on multiple intelligence sources to obtain threat intelligence with better quality.

[0006] In order to achieve the above purpose, the technical solutions provided by the embodiments of the present application are as follows:

[0007] The present application provides a method for acquiring threat intelligence based on intrinsic security, the method comprising:

[0008] Obtain intelligence data from multiple intelligence sources;

[0009] Based on the threat intelligence template corresponding to the intelligence source, the threat intelligence is captured from the intelligence data of multiple intelligence sources to obtain the threat intelligence to be processed;

[0010] Performing data preprocessing on the threat intelligence to be processed to obtain processed threat intelligence;

[0011] Performing conflict detection on the processed threat intelligence to obtain at least two conflicting threat intelligence;

[0012] Conflict resolution is performed on the at least two conflicting threat intelligences to obtain threat intelligence after the conflict resolution.

[0013] Optionally, the performing conflict resolution on the at least two conflicting threat intelligences to obtain the threat intelligence after the conflict resolution includes:

[0014] Determine the object intelligence sources to which the at least two conflicting threat intelligences respectively belong;

[0015] Based on the at least two conflicting threat intelligences and at least two of the object intelligence sources, obtaining an attribute importance value of a target intelligence source; the target intelligence source is any one of the object intelligence sources;

[0016] Based on the attribute importance value of the target intelligence source, the object intelligence source is prioritized to determine the object intelligence source within a preset priority ranking range;

[0017] The target threat intelligence among the at least two conflicting threat intelligences is retained to obtain the threat intelligence after the conflict is resolved; the target threat intelligence is the threat intelligence corresponding to the object intelligence source within the preset priority sorting range.

[0018] Optionally, acquiring the attribute importance value of the target intelligence source based on the at least two conflicting threat intelligences and the at least two object intelligence sources includes:

[0019] Calculate the ratio of the amount of intelligence from the target intelligence source to the total amount of intelligence from at least two of the target intelligence sources during the target time period;

[0020] Obtaining a first quantity of at least two conflicting threat intelligences;

[0021] determining a second number of base data tags and a third number of enriched data tags in the target intelligence source;

[0022] determining the confidence level of the target intelligence source;

[0023] Based on the proportionality coefficient, the first number, the second number, the third number, and the confidence level, the attribute importance value of the target intelligence source is calculated.

[0024] Optionally, determining the confidence of the target intelligence source includes:

[0025] When the target intelligence source is provided by a user, the confidence level set by the user is used to determine the confidence level of the target intelligence source;

[0026] Alternatively, performing a quality assessment on the target intelligence source to generate a quality assessment result, and generating a confidence level of the target intelligence source based on the quality assessment result;

[0027] Alternatively, the confidence level corresponding to the target intelligence source is determined according to the type of the target intelligence source.

[0028] Optionally, performing quality assessment on the intelligence source, generating a quality assessment result, and generating the confidence of the intelligence source based on the quality assessment result includes:

[0029] Determine the quality assessment result of the intelligence source according to whether the intelligence data provided by the intelligence source meets the preset conditions and whether there is an intelligence conflict in the threat intelligence in the intelligence source;

[0030] A confidence level of the intelligence source is generated based on the quality assessment result.

[0031] Optionally, performing conflict detection on the processed threat intelligence to obtain at least two conflicting threat intelligences includes:

[0032] Determining the processed threat intelligence as keywords, and constructing a keyword co-occurrence matrix;

[0033] Binarizing the keyword co-occurrence matrix to obtain a binary keyword co-occurrence matrix;

[0034] Establishing a keyword co-occurrence network according to the binary keyword co-occurrence matrix;

[0035] Extracting content data from the keyword co-occurrence network to obtain a content data set;

[0036] Conflict detection is performed on the content data in the content data set, and the threat intelligence to which at least two conflicting content data belong is determined as at least two conflicting threat intelligence.

[0037] Optionally, the data preprocessing includes at least one or more of matching whitelist, data cleaning, data deduplication, data tail removal, data verification, data format unification, data label marking and data normalization.

[0038] Optionally, the data label is marked as attribute information for marking the intelligence data; the attribute information includes at least one or more of the first discovery time, the last discovery time, the data type label, the region, the organization, the industry, the attack type, the threat level, the intelligence source and the family gang information.

[0039] The embodiment of the present application also provides a threat intelligence acquisition device based on intrinsic security, the device comprising:

[0040] An acquisition unit, used to acquire intelligence data from multiple intelligence sources;

[0041] A capture unit is used to capture threat intelligence from intelligence data of multiple intelligence sources based on a threat intelligence template corresponding to the intelligence source, and obtain threat intelligence to be processed;

[0042] A preprocessing unit, used to perform data preprocessing on the threat intelligence to be processed to obtain processed threat intelligence;

[0043] A detection unit, configured to perform conflict detection on the processed threat intelligence and obtain at least two conflicting threat intelligences;

[0044] The conflict processing unit is used to perform conflict processing on the at least two conflicting threat intelligences and obtain the threat intelligence after the conflict processing.

[0045] Optionally, the conflict handling unit includes:

[0046] A first determining subunit is used to determine the object intelligence sources to which the at least two conflicting threat intelligences respectively belong;

[0047] A first acquisition subunit is configured to acquire an attribute importance value of a target intelligence source based on the at least two conflicting threat intelligences and at least two of the object intelligence sources; the target intelligence source is any one of the object intelligence sources;

[0048] A sorting subunit, configured to prioritize the target intelligence sources based on the attribute importance values ​​of the target intelligence sources, and determine the target intelligence sources within a preset priority sorting range;

[0049] The second acquisition subunit is used to retain the target threat intelligence among the at least two conflicting threat intelligences and obtain the threat intelligence after the conflict is resolved; the target threat intelligence is the threat intelligence corresponding to the object intelligence source within the preset priority sorting range.

[0050] An embodiment of the present application also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, all or part of the steps of any method for acquiring threat intelligence based on intrinsic security are implemented.

[0051] An embodiment of the present application also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, all or part of the steps of the threat intelligence acquisition method based on intrinsic security as described in any one of the above are implemented.

[0052] Through the above technical solution, it can be seen that the present application has the following beneficial effects:

[0053] The embodiment of the present application provides a method and device for acquiring threat intelligence based on intrinsic security, including: acquiring intelligence data from multiple intelligence sources. Based on the threat intelligence template corresponding to the intelligence source, threat intelligence is captured from the intelligence data of multiple intelligence sources to obtain threat intelligence to be processed. Because the intelligence data in different intelligence sources have data problems such as inconsistent types and formats. It is necessary to pre-process the threat intelligence to be processed to obtain the processed threat intelligence. There may be conflicting threat intelligence in the processed threat intelligence, and it is necessary to perform conflict detection on the processed threat intelligence to obtain at least two conflicting threat intelligence. Then, conflict resolution is performed on at least two conflicting threat intelligence to obtain the conflict-processed threat intelligence. The embodiment of the present application provides a method for extracting and fusing threat intelligence from multiple intelligence sources. Based on the threat intelligence template corresponding to the intelligence source, after extracting threat intelligence from the intelligence data in multiple intelligence sources, threat intelligence data is pre-processed and conflict resolution is performed, so that the threat intelligence finally obtained by fusion is more standardized and the quality of the threat intelligence obtained is higher. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0055] Figure 1 A schematic diagram of an exemplary application scenario provided for an embodiment of the present application;

[0056] Figure 2 A flowchart of a method for acquiring threat intelligence based on intrinsic security provided in an embodiment of the present application;

[0057] Figure 3 A flowchart of a threat intelligence conflict handling process provided in an embodiment of the present application;

[0058] Figure 4 A schematic diagram of the structure of a threat intelligence acquisition device based on intrinsic security provided in an embodiment of the present application. DETAILED DESCRIPTION

[0059] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the embodiments of the present application are further described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0060] To facilitate understanding of the technical solution provided by this application, the background technology involved in this application will be described below.

[0061] Threat intelligence is evidence-based knowledge, including context, mechanism, indication, meaning and executable suggestions, which is used to provide information support for enterprises or institutions to respond to or deal with threats. For example, domain names that generate security threats can be used as threat intelligence. Threat intelligence aims to provide comprehensive, accurate, executable and decision-making knowledge and information for enterprises or institutions facing threats.

[0062] Endogenous security is the adaptive, autonomous and self-growing security capability that is constantly developed within the information system. It is a security construction system with user characteristics. Endogenous intelligence is the core of endogenous security. The intelligence database stores threat intelligence and is a common source of threat intelligence, known as an intelligence source. Based on this, more and more companies are adding threat intelligence from the intelligence database to their security protection network.

[0063] In actual applications, enterprises will obtain multiple different intelligence sources and use threat intelligence platforms to obtain threat intelligence from multiple intelligence sources. However, the content of multiple intelligence sources currently obtained is uneven, and how to fuse data from multiple intelligence sources is an urgent problem to be solved.

[0064] Based on this, the embodiment of the present application provides a method and device for acquiring threat intelligence based on intrinsic security. Figure 1 See the example scenario shown. Figure 1 , which is a schematic diagram of an exemplary application scenario provided by an embodiment of the present application. The method can be applied to a threat intelligence platform. Typically, a threat intelligence platform can aggregate and normalize threat data, then correlate the data and apply it into context.

[0065] In actual applications, the threat intelligence platform obtains intelligence data from multiple intelligence sources. Based on the threat intelligence template corresponding to the intelligence source, threat intelligence is captured from the intelligence data of multiple intelligence sources to obtain threat intelligence to be processed. Furthermore, data preprocessing is performed on the threat intelligence to be processed to obtain processed threat intelligence. Since conflicting threat intelligence may exist in the processed threat intelligence, the threat intelligence platform performs conflict detection on the processed threat intelligence to obtain at least two conflicting threat intelligence. Furthermore, the threat intelligence platform performs conflict resolution on at least two conflicting threat intelligence to obtain threat intelligence after conflict resolution.

[0066] Those skilled in the art will understand that Figure 1 The framework diagram shown is only an example in which the embodiments of the present application can be implemented. The scope of application of the embodiments of the present application is not limited by any aspect of the framework.

[0067] Based on the above description, the threat intelligence acquisition method based on intrinsic security provided by the present application will be described in detail below with reference to the accompanying drawings.

[0068] See also Figure 2 , which is a flowchart of a method for acquiring threat intelligence based on intrinsic security provided by an embodiment of the present application. The method can be executed by the above-mentioned threat intelligence platform, such as Figure 2 As shown, the method may include S201-S208:

[0069] S201: Obtain intelligence data from multiple intelligence sources.

[0070] The threat intelligence platform can obtain multiple intelligence sources. There are many types of intelligence sources, such as commercial intelligence sources, open source intelligence sources, agency intelligence sources, etc. Among them, the intelligence data in the open source intelligence source is data collected from public sources.

[0071] After acquiring multiple intelligence sources, the threat intelligence platform needs to read data from multiple intelligence sources to obtain intelligence data from multiple intelligence sources. It is understandable that the types of intelligence data are diverse, and the returned data format can be automatically determined based on the uniform resource locator URL of the intelligence source, such as html, csv, txt, zip, rar, json, etc.

[0072] S202: Based on the threat intelligence template corresponding to the intelligence source, threat intelligence is captured from intelligence data of multiple intelligence sources to obtain threat intelligence to be processed.

[0073] IOC, also known as threat intelligence, is the remote command and control server intelligence used by attackers to control victim hosts. For example, domain names, Internet Protocol IP addresses, URLs, etc. In actual applications, threat intelligence is pushed to different security devices for detection and real-time interception. It provides rich contextual information such as the hazard level of threat events, attack groups, and malicious families to help determine the priority of threat events and guide subsequent security response activities.

[0074] Therefore, after obtaining intelligence data from multiple intelligence sources, it is necessary to extract threat intelligence from the intelligence data, integrate the acquired threat intelligence, and store it on the threat intelligence platform to provide a rich source of threat intelligence to the outside world.

[0075] In an embodiment of the present application, threat intelligence templates corresponding to different intelligence sources are pre-generated. It should be noted that the threat intelligence template is dynamically set according to the type and data format of the intelligence source. Different intelligence sources correspond to different threat intelligence templates, and different threat intelligence templates are set with different fields. Fields are used to segment intelligence data. For example, for commercial intelligence sources, threat intelligence templates corresponding to commercial intelligence sources are pre-set, and for agency intelligence sources, threat intelligence templates corresponding to agency intelligence sources are pre-set. The fields set in the threat intelligence template corresponding to the commercial intelligence source and the threat intelligence template corresponding to the agency intelligence source may be different.

[0076] During specific implementation, according to the type and data format of the intelligence source, multiple fields corresponding to the intelligence source (which can be understood as a data segmentation form) and the data format corresponding to the fields are set in the threat intelligence template to form a field mapping. Among them, the fields set by the dynamic threat intelligence template include basic threat intelligence fields and enriched threat intelligence fields. The basic threat intelligence fields include fields such as the network interconnection protocol IP, the domain name DOMAIN, and the message digest algorithm version 5 MD5. The enriched threat intelligence fields can be dynamically set according to the type of intelligence source. It can be understood that the basic threat intelligence fields and the enriched threat intelligence fields are set according to the actual needs of the actual application scenarios, and they are not limited here. For example, for the intelligence data of the malsilo IP type of a certain intelligence source:

[0077] #Firstseen,Lastseen,ipv4:port,threat,tags

[0078] "2020-01-13","2020-01-30","99.252.27.6:80","emotet","peexe64,doc,assembly,docx,data,pedll,mz,pegui,zip,peexe32"

[0079] "2020-01-13","2020-01-30","72.189.57.105:80","emotet","peexe64,doc,assembly,docx,data,pedll,mz,pegui,zip,peexe32"

[0080] Based on this, the threat intelligence template is set as follows:

[0081] [',"','Firstseen','Lastseen','ipv4:port','threat','tags']

[0082] Among them, 'Firstseen', 'Lastseen', 'ipv4:port', 'threat', and 'tags' are the field names of different fields in the threat intelligence template, such as Firstseen is the first discovery time, Lastseen is the last discovery time, 'ipv4:port' is the IP address, threat is the attack type, and tags is the data type tag. The basic threat intelligence field is 'ipv4:port', and the rest are enriched threat intelligence fields.

[0083] It is understandable that in order to make the threat intelligence extraction more accurate, a more detailed threat intelligence template can be set. For example, if the business intelligence source is further divided into types, the threat intelligence template corresponding to the business intelligence source is further divided accordingly.

[0084] Therefore, in practical applications, after obtaining intelligence data from multiple intelligence sources, the corresponding relationship between intelligence sources and threat intelligence templates is used to obtain threat intelligence templates corresponding to multiple intelligence sources. Then, based on the threat intelligence template corresponding to the intelligence source, the intelligence data in the intelligence source and the fields set in the threat intelligence template are automatically matched to automatically segment the intelligence data in the intelligence source, complete the automatic capture of threat intelligence, and obtain the threat intelligence to be processed. In this way, the workload of parsing intelligence data from multiple intelligence sources can be reduced.

[0085] S203: Perform data preprocessing on the threat intelligence to be processed to obtain processed threat intelligence.

[0086] The quality of the threat intelligence to be processed obtained by the threat intelligence platform varies, and data preprocessing is required for the threat intelligence to be processed. The processed threat intelligence is more standardized.

[0087] In specific implementation, data preprocessing includes at least one or more of matching whitelist, data cleaning, data deduplication, data tail removal, data verification, data format unification, data label marking and data normalization.

[0088] Among them, the threat intelligence to be processed is matched to the whitelist. The whitelist corresponds to the blacklist. Usually, after the blacklist is enabled, users, IP addresses, IP packets, emails, viruses, etc. included in the blacklist cannot pass. After the whitelist is enabled, users, IP addresses, IP packets, emails, etc. in the whitelist can pass. The threat intelligence in the embodiment of the present application is intelligence data that threatens the security of the enterprise. Therefore, if the threat intelligence to be processed is in the whitelist, subsequent data processing will not be performed. If the threat intelligence to be processed is not in the whitelist, other data preprocessing methods will continue to be used for data processing.

[0089] It is understandable that before matching the whitelist, the whitelist must be set in advance, and the content in the whitelist can be set according to the actual application scenario, which is not limited in the embodiments of the present application.

[0090] In addition, data cleaning is to discover and correct identifiable errors in the threat intelligence data files to be processed, so as to improve the accuracy of the threat intelligence to be processed. Data deduplication is to discover and remove the same threat intelligence in the threat intelligence to be processed, so as to reduce the subsequent processing cost and improve the processing efficiency. Data tail removal is to discover and remove the messy data in the threat intelligence to be processed, such as some decimal data, so as to improve the consistency of the threat intelligence to be processed. Data verification is to discover and remove the empty data in the threat intelligence to be processed, so as to reduce the subsequent processing cost and improve the processing efficiency.

[0091] Since the data formats of threat intelligence in different intelligence sources are inconsistent, in the embodiment of the present application, the data format of the threat intelligence to be processed is unified. For example, the data format of the threat intelligence to be processed is unified by using standards such as national standards, STIX, TAXII, OpenIOC, or user-defined engineering standards, so as to improve the consistency of the threat intelligence to be processed.

[0092] Data labeling is to label the attribute information of intelligence data, wherein the attribute information includes at least one or more of the first discovery time, the last discovery time, the data type label, the region, the organization, the industry, the attack type, the threat level, the intelligence source and the family gang information.

[0093] After the threat intelligence to be processed is labeled with data tags, the threat intelligence to be processed corresponds to the labeled data tags, which are the attribute information of the tags. Among them, data tags are divided into basic data tags and rich data tags. For example, basic data tags are domain names, URLs, etc., and rich data tags are first discovery time, last discovery time, threat level, etc.

[0094] Normalize the threat intelligence to be processed to improve the consistency of the threat intelligence to be processed. In specific implementation, by specifying a series of constraints, the threat intelligence to be processed is processed according to the specified constraints and context completion / association is performed to achieve normalization. It is understandable that normalization includes context completion.

[0095] Among them, the standard field information contained in the threat intelligence to be processed extracted from different intelligence sources may be incomplete. For example, some intelligence sources only provide IP addresses without any other information. At this time, before the intelligence data is processed further, it is necessary to enrich the fields and associate the context of the intelligence data to improve the richness and high availability of the intelligence data, such as adding data tags to IP addresses and supplementing attribute information such as the attack group or discovery time.

[0096] S204: Perform conflict detection on the processed threat intelligence to obtain at least two conflicting threat intelligence.

[0097] There are conflicts in threat intelligence from different intelligence sources. For example, one threat intelligence determines that the attack type of a URL is a phishing email, while another threat intelligence determines that the attack type of the URL is a malicious Trojan. It is determined that the two threat intelligences are conflicting.

[0098] Therefore, after obtaining the processed threat intelligence, it is necessary to perform conflict detection on the processed threat intelligence, obtain at least two conflicting threat intelligence, and then perform conflict resolution on the conflicting threat intelligence.

[0099] In a possible implementation, the embodiment of the present application provides a specific implementation method of performing conflict detection on processed threat intelligence and obtaining at least two conflicting threat intelligence, including:

[0100] Determine the processed threat intelligence as a keyword and construct a keyword co-occurrence matrix. Binarize the keyword co-occurrence matrix to obtain a binary keyword co-occurrence matrix. Establish a keyword co-occurrence network based on the binary keyword co-occurrence matrix. Extract content data in the keyword co-occurrence network to obtain a content data set. Perform conflict detection on the content data in the content data set, and determine the threat intelligence to which at least two conflicting content data belong as at least two conflicting threat intelligence.

[0101] This conflict detection method is based on the keyword co-occurrence network. It uses association analysis to detect and judge the processed threat intelligence in real time. There is no need to describe and store the intelligence data in a structured manner, which reduces the amount of calculation. It overcomes the technical defect that the accuracy of content conflict detection is poor due to the inability to synchronize data updates with the public text intelligence of big data with high real-time performance. It realizes the detection of content conflicts in multi-source intelligence with big data characteristics.

[0102] S205: Conflict resolution is performed on at least two conflicting threat intelligences to obtain threat intelligences after conflict resolution.

[0103] Conflict resolution is performed on at least two conflicting threat intelligences, one of the threat intelligences is selected to be retained, and the other conflicting threat intelligences are discarded to obtain the threat intelligence after the conflict resolution.

[0104] In specific implementation, different priorities can be set for different intelligence sources. If the priority of the intelligence source to which the conflicting threat intelligence belongs is low, it will be discarded. Based on this, in a possible implementation method, the embodiment of the present application provides a specific implementation method for conflict handling of at least two conflicting threat intelligences and obtaining the threat intelligence after conflict handling. For details, please refer to S301-S304 below.

[0105] Based on the contents of S201-S205, it can be known that intelligence data from multiple intelligence sources is obtained. Based on the threat intelligence template corresponding to the intelligence source, threat intelligence is captured from the intelligence data of multiple intelligence sources to obtain threat intelligence to be processed. Because the intelligence data in different intelligence sources have data problems such as inconsistent types and formats. It is necessary to preprocess the threat intelligence to be processed to obtain the processed threat intelligence. There may be conflicting threat intelligence in the processed threat intelligence, and it is necessary to perform conflict detection on the processed threat intelligence to obtain at least two threat intelligences that are in conflict. Then, conflict resolution is performed on at least two conflicting threat intelligences to obtain the threat intelligence after conflict resolution. The embodiment of the present application provides a method for extracting and fusing threat intelligence from multiple intelligence sources. Based on the threat intelligence template corresponding to the intelligence source, after threat intelligence is extracted from the intelligence data in multiple intelligence sources, threat intelligence data is preprocessed and conflict resolution is performed, so that the threat intelligence finally obtained by fusion is more standardized and the quality of the threat intelligence obtained is higher.

[0106] In one possible implementation, combining Figure 3 The embodiment of the present application provides a specific implementation method of performing conflict resolution on at least two conflicting threat intelligence in S205 to obtain the threat intelligence after the conflict resolution. Figure 3 A flowchart of a threat intelligence conflict handling process provided by an embodiment of the present application. The specific implementation method includes:

[0107] S301: Determine the object intelligence sources to which at least two conflicting threat intelligence belong respectively.

[0108] The target intelligence source is the intelligence source to which the threat intelligence belongs, and the threat intelligence is the intelligence extracted from the intelligence source.

[0109] S302: Based on at least two conflicting threat intelligence sources and at least two object intelligence sources, obtain an attribute importance value of a target intelligence source; the target intelligence source is any one of the object intelligence sources.

[0110] In an embodiment of the present application, the attribute importance value of the target intelligence source can represent the priority of the intelligence source. The larger the attribute importance value, the higher the priority of the intelligence source.

[0111] In a possible implementation, the attribute importance value of the target intelligence source may be set according to the confidence of the object intelligence source.

[0112] In another possible implementation, the embodiment of the present application provides a specific implementation method for obtaining the attribute importance value of the target intelligence source based on at least two conflicting threat intelligence and at least two object intelligence sources, including:

[0113] A1: Calculate the ratio of the amount of intelligence from the target intelligence source to the total amount of intelligence from at least two target intelligence sources during the target time period.

[0114] As an optional example, the target time period is within 30 days or 90 days before the current day. The ratio coefficient of the amount of intelligence from the target intelligence source to the total amount of intelligence from at least two target intelligence sources is set to M.

[0115] in, Where m1 is the number of intelligence from the target intelligence source. n is the total number of intelligence sources from at least two target intelligence sources. i represents the i-th target intelligence source in the target intelligence source. k is the number of intelligence in each target intelligence source. k is the total amount of intelligence from n intelligence sources.

[0116] A2: The first number of at least two conflicting threat intelligences is obtained.

[0117] The first number is the number of conflicting threat intelligence. For example, if there are 4 conflicting threat intelligence, the first number is 4, namely threat intelligence A, threat intelligence B, threat intelligence C, and threat intelligence D.

[0118] A3: Determine a second number of base data labels and a third number of enriched data labels in the target intelligence source.

[0119] Set the second number of basic data labels in the target intelligence source to 0n, and the third number of enriched data labels to Pn.

[0120] A4: Determine the confidence level of the target intelligence source. Typically, the confidence level is an integer value, specified in the range of 1-5.

[0121] Confidence reflects the credibility of the intelligence source and its reference value. The confidence level of the target intelligence source is determined to be D.

[0122] A5: Based on the proportionality coefficient, the first quantity, the second quantity, the third quantity and the confidence level, the attribute importance value of the target intelligence source is calculated.

[0123] As an optional example, the attribute importance value W of the target intelligence source is obtained by the following formula:

[0124] W=(M*N*((2+0n)+4(1+Pn)))+D

[0125] S303: Prioritize the target intelligence sources based on the attribute importance values ​​of the target intelligence sources, and determine the target intelligence sources within a preset priority range.

[0126] The higher the attribute importance value of the target intelligence source, the more frequently the target intelligence source is used, and the higher the priority of the target intelligence source. Based on the attribute importance value, each object intelligence source is prioritized to determine the object intelligence source within the preset priority sorting range. It is understandable that the preset priority sorting range can be determined according to the actual needs of the actual application scenario, and the embodiments of the present application do not limit this. For example, when there are only two threat intelligence conflicts, the preset priority sorting range can be selected as the highest priority.

[0127] In some possible implementations, the present application provides some specific implementations for determining the confidence of a target intelligence source, including:

[0128] When the target intelligence source is provided by the user, the confidence level set by the user is used to determine the confidence level of the target intelligence source;

[0129] Alternatively, a quality assessment is performed on the target intelligence source, a quality assessment result is generated, and a confidence level of the target intelligence source is generated based on the quality assessment result;

[0130] Alternatively, the confidence level corresponding to the target intelligence source is determined according to the type of the target intelligence source.

[0131] In a possible implementation, the embodiment of the present application provides a specific implementation method of performing quality assessment on an intelligence source, generating a quality assessment result, and generating the confidence of the intelligence source based on the quality assessment result, including:

[0132] Determine the quality assessment result of the intelligence source based on whether the intelligence data provided by the intelligence source meets the preset conditions and whether there is intelligence conflict in the threat intelligence in the intelligence source;

[0133] Generate a confidence level for the intelligence source based on the quality assessment results.

[0134] Among them, the preset condition is that the information completeness of the intelligence data meets the preset range. For example, taking the query of domain name as an example, the domain name is obtained by querying the whois information. Among them, whois information is a transmission protocol used to query the IP and owner of the domain name. That is, whois is a database used to query whether the domain name has been registered, as well as detailed information of the registered domain name. When the whois information of intelligence source A includes registrant information, creation time, etc. The whois information of intelligence source B is empty. After comparison, it can be determined that the information completeness of intelligence source A is higher than that of intelligence source B.

[0135] It is understandable that the preset range can be set according to the needs of the actual application scenario and is not limited here.

[0136] In the embodiment of the present application, the higher the quality assessment result, the higher the confidence of the intelligence source.

[0137] S304: retain the target threat intelligence among the at least two conflicting threat intelligences, and obtain the threat intelligence after the conflict is resolved; the target threat intelligence is the threat intelligence corresponding to the object intelligence source within the preset priority sorting range.

[0138] According to the priority order of the object intelligence source, the threat intelligence corresponding to the object intelligence source within the preset priority sorting range is determined, that is, the target threat intelligence. The target threat intelligence among at least two conflicting threat intelligences is retained, and after conflict resolution is performed on all conflicting threat intelligences, the threat intelligence after conflict resolution is obtained.

[0139] In addition, the confidence of the intelligence source can be adjusted according to the attribute importance value of the intelligence source.

[0140] Based on the contents of S301-S304, it can be seen that by determining the priority of intelligence sources by means of attribute importance values, conflicting threat intelligence can be handled. Based on the set priority of intelligence sources, enterprises can give up purchasing intelligence sources with low priority, reduce the cost of purchasing intelligence sources, and help enterprises purchase and use higher quality intelligence sources.

[0141] In order to facilitate understanding and explanation of the contents of S301-S304, the embodiment of the present application provides an application scenario example of performing conflict resolution on at least two conflicting threat intelligences and obtaining threat intelligences after conflict resolution, as follows:

[0142] Assume that there is a conflict between the threat intelligence in intelligence sources A and B.

[0143] There are 1000 pieces of intelligence data in intelligence source A. The confidence of intelligence source A is 4. The format of the intelligence data is as follows:

[0144] #Firstseen,Lastseen,domain,threat,tags

[0145] Among them, there are 5 data tags for intelligence source A. Firstseen is the first discovery time, Lastseen is the last discovery time, domain is the domain name, threat is the attack type, and tags is the data type tag. The number of basic data tags is 1, which is the domain name, and the rest are rich data tags, which are 4 in number.

[0146] For example, there are two threat intelligence in intelligence source A as follows:

[0147] "2020-09-16","2021-06-08","gudangalami.com","emotet","doc"

[0148] "2021-01-31","2021-06-10","moasocialcoop.com","emotet","doc".

[0149] Among them, "2020-09-16" and "2021-01-31" are the first discovery time, "2021-06-08" and "2021-06-10" are the last discovery time, "gudangalami.com" and "moasocialcoop.com" are domain names, "emotet" is a computer malware program, and "doc" is a data type.

[0150] Intelligence source B has 1,000 pieces of intelligence data. The confidence level of intelligence source B is 4. The format of the intelligence data is as follows:

[0151] #id,dateadded,url,url_status,threat,tags,urlhaus_link,reporter

[0152] Among them, there are 8 data tags in total for intelligence source A. id is the identification number, dateadded is the discovery time, url is the uniform resource locator, url_status is the status of the uniform resource locator, threat is the attack type, tags is the data type tag, urlhaus_link is the specific link address of the uniform resource locator, and reporter is the malware URL.

[0153] For example, the threat intelligence from intelligence source B is as follows:

[0154] "1432743","2021-07-07 06:49:14","http: / / 115.51.105.45:33488 / Mozi.a","online","malware_download","elf,Mozi","https: / / urlhaus.abuse.ch / url / 1432743 / ","lrz_urlhaus".

[0155] Among them, "1432743" is the identification number, "2021-07-07 06:49:14" is the discovery time, the Uniform Resource Locator is "http: / / 115.51.105.45:33488 / Mozi.a", the status of the Uniform Resource Locator is "online", "malware_download" is the attack type, "elf,Mozi" is the data type, the specific link address of the Uniform Resource Locator is "https: / / urlhaus.abuse.ch / url / 1432743 / ", and the malware URL is "lrz_urlhaus".

[0156] Then, the calculation formula for the attribute importance value of intelligence source A is:

[0157] W=(1000 / 2000*2*((2+4)+4(1+1)))+4=16

[0158] The calculation formula for the attribute importance value of intelligence source B is:

[0159] W=(1000 / 2000*2*((2+6)+4(1+2))+4)=24

[0160] It can be concluded that the W value of intelligence source B is high, so the conflicting threat intelligence corresponding to intelligence source B is selected as the intelligence output, and the conflicting threat intelligence corresponding to intelligence source A is discarded.

[0161] Based on the method for acquiring threat intelligence based on intrinsic security provided in the above method embodiment, the embodiment of the present application also provides a device for acquiring threat intelligence based on intrinsic security. The device for acquiring threat intelligence based on intrinsic security will be described below in conjunction with the accompanying drawings.

[0162] See also Figure 4 , which is a schematic diagram of the structure of a threat intelligence acquisition device based on intrinsic security provided in an embodiment of the present application. For technical details of the threat intelligence acquisition device based on intrinsic security provided in the device embodiment, please refer to the above method embodiment. Figure 4 As shown, the threat intelligence acquisition device based on intrinsic security includes:

[0163] An acquisition unit 401 is used to acquire intelligence data from multiple intelligence sources;

[0164] A capture unit 402 is used to capture threat intelligence from intelligence data of multiple intelligence sources based on a threat intelligence template corresponding to the intelligence source, and obtain threat intelligence to be processed;

[0165] The preprocessing unit 403 is used to perform data preprocessing on the threat intelligence to be processed to obtain the processed threat intelligence;

[0166] A detection unit 404 is used to perform conflict detection on the processed threat intelligence to obtain at least two conflicting threat intelligences;

[0167] The conflict handling unit 405 is used to perform conflict handling on the at least two conflicting threat intelligences and obtain threat intelligence after the conflict handling.

[0168] In a possible implementation, the conflict handling unit 405 includes:

[0169] A first determining subunit is used to determine the object intelligence sources to which the at least two conflicting threat intelligences respectively belong;

[0170] A first acquisition subunit is configured to acquire an attribute importance value of a target intelligence source based on the at least two conflicting threat intelligences and at least two of the object intelligence sources; the target intelligence source is any one of the object intelligence sources;

[0171] A sorting subunit, configured to prioritize the target intelligence sources based on the attribute importance values ​​of the target intelligence sources, and determine the target intelligence sources within a preset priority sorting range;

[0172] The second acquisition subunit is used to retain the target threat intelligence among the at least two conflicting threat intelligences and obtain the threat intelligence after the conflict is resolved; the target threat intelligence is the threat intelligence corresponding to the object intelligence source within the preset priority sorting range.

[0173] In a possible implementation, the first acquiring subunit includes:

[0174] A first calculation subunit is used to calculate a ratio coefficient between the amount of intelligence from a target intelligence source and the total amount of intelligence from at least two of the target intelligence sources within a target time period;

[0175] A third acquisition subunit is used to acquire a first quantity of at least two conflicting threat intelligences;

[0176] A second determining subunit, configured to determine a second number of basic data tags and a third number of enriched data tags in the target intelligence source;

[0177] A third determination subunit, used to determine the confidence of the target intelligence source;

[0178] The second calculation subunit is used to calculate the attribute importance value of the target intelligence source based on the proportional coefficient, the first number, the second number, the third number and the confidence level.

[0179] In a possible implementation manner, the third determining subunit includes:

[0180] a fourth determining subunit, configured to determine the confidence level of the target intelligence source using the confidence level set by the user when the target intelligence source is provided by the user;

[0181] Alternatively, the first generating subunit is used to perform quality assessment on the target intelligence source, generate a quality assessment result, and generate the confidence of the target intelligence source based on the quality assessment result;

[0182] Alternatively, the fifth determination subunit is used to determine the confidence level corresponding to the target intelligence source according to the type of the target intelligence source.

[0183] In a possible implementation, the generating subunit includes:

[0184] A sixth determination subunit is used to determine the quality assessment result of the intelligence source according to whether the intelligence data provided by the intelligence source meets the preset conditions and whether there is an intelligence conflict in the threat intelligence in the intelligence source;

[0185] The second generating subunit is used to generate the confidence of the intelligence source based on the quality assessment result.

[0186] In a possible implementation, the detection device 404 includes:

[0187] A seventh determination subunit is used to determine that the processed threat intelligence is a keyword and construct a keyword co-occurrence matrix;

[0188] A binarization processing subunit, used for performing binarization processing on the keyword co-occurrence matrix to obtain a binarized keyword co-occurrence matrix;

[0189] Establishing a subunit, for establishing a keyword co-occurrence network according to the binary keyword co-occurrence matrix;

[0190] An extraction subunit, used to extract content data in the keyword co-occurrence network to obtain a content data set;

[0191] The conflict detection subunit is used to perform conflict detection on the content data in the content data set, and determine the threat intelligence to which at least two conflicting content data belong as at least two conflicting threat intelligence.

[0192] In a possible implementation, the data preprocessing includes at least one or more of matching whitelist, data cleaning, data deduplication, data tail removal, data verification, data format unification, data label marking and data normalization.

[0193] In one possible implementation, the data label is marked as attribute information for marking the intelligence data; the attribute information includes at least one or more of the first discovery time, the last discovery time, the data type label, the region, the organization, the industry, the attack type, the threat level, the intelligence source and the family gang information.

[0194] In addition, an embodiment of the present application also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein when the processor executes the program, all or part of the steps of the method for acquiring threat intelligence based on intrinsic security as described in any of the above embodiments are implemented.

[0195] In addition, an embodiment of the present application also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, all or part of the steps of the method for obtaining threat intelligence based on intrinsic security as described in any of the above embodiments are implemented.

[0196] The embodiments of the present application provide a threat intelligence acquisition device and equipment based on intrinsic security, including: obtaining intelligence data from multiple intelligence sources. Based on the threat intelligence template corresponding to the intelligence source, threat intelligence is captured from the intelligence data of multiple intelligence sources to obtain threat intelligence to be processed. Since the intelligence data in different intelligence sources have data problems such as inconsistent types and formats. It is necessary to perform data preprocessing on the threat intelligence to be processed to obtain the processed threat intelligence. There may be conflicting threat intelligence in the processed threat intelligence, and it is necessary to perform conflict detection on the processed threat intelligence to obtain at least two conflicting threat intelligence. Then, conflict resolution is performed on the at least two conflicting threat intelligence to obtain the threat intelligence after conflict resolution. Based on the threat intelligence template corresponding to the intelligence source, after threat intelligence is extracted from the intelligence data in multiple intelligence sources, threat intelligence data preprocessing and conflict resolution are performed, so that the threat intelligence finally obtained by fusion is more standardized and the quality of the threat intelligence obtained is higher.

[0197] It can be known from the description of the above implementation mode that those skilled in the art can clearly understand that all or part of the steps in the above-mentioned embodiment method can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product can be stored in a storage medium such as ROM / RAM, a disk, an optical disk, etc., including several instructions for enabling a computer device (which can be a personal computer, a server, or a network communication device such as a media gateway, etc.) to execute the methods described in the various embodiments of the present application or certain parts of the embodiments.

[0198] It should be noted that the various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the various embodiments can be referred to each other. For the method disclosed in the embodiment, since it corresponds to the system disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the system part description.

[0199] It should also be noted that, in this article, the term "includes", "comprising" or any other variation thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of more restrictions, an element defined by the sentence "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

[0200] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for acquiring threat intelligence based on intrinsic security, characterized in that: The method comprises: Obtain intelligence data from multiple intelligence sources; Based on the threat intelligence template corresponding to the intelligence source, the threat intelligence is captured from the intelligence data of multiple intelligence sources to obtain the threat intelligence to be processed; Performing data preprocessing on the threat intelligence to be processed to obtain processed threat intelligence; Performing conflict detection on the processed threat intelligence to obtain at least two conflicting threat intelligence; Determine the object intelligence sources to which the at least two conflicting threat intelligences respectively belong; Calculate the ratio coefficient between the amount of intelligence from the target intelligence source and the total amount of intelligence from at least two of the target intelligence sources within the target time period; the target intelligence source is any one of the target intelligence sources; Obtaining a first quantity of at least two conflicting threat intelligences; Determining a second number of basic data tags and a third number of enriched data tags in the target intelligence source; the basic data tags include a domain name and a uniform resource locator URL; the enriched data tags include: first discovery time, last discovery time, and threat level; determining the confidence level of the target intelligence source; Based on the proportional coefficient, the first number, the second number, the third number, and the confidence level, the attribute importance value of the target intelligence source is calculated, and the attribute importance value is used to characterize the priority of the intelligence source: The attribute importance value of the target intelligence source = {proportional coefficient × first quantity × [(2 + second quantity) + 4 (1 + third quantity)]} + confidence; The attribute importance value is used to represent the priority of the intelligence source; Based on the attribute importance value of the target intelligence source, the object intelligence source is prioritized to determine the object intelligence source within a preset priority ranking range; The target threat intelligence among the at least two conflicting threat intelligences is retained to obtain the threat intelligence after the conflict is resolved; the target threat intelligence is the threat intelligence corresponding to the object intelligence source within the preset priority sorting range.

2. The method according to claim 1, characterized in that Determining the confidence of the target intelligence source includes: When the target intelligence source is provided by a user, the confidence level set by the user is determined as the confidence level of the target intelligence source; Alternatively, performing a quality assessment on the target intelligence source to generate a quality assessment result, and generating a confidence level of the target intelligence source based on the quality assessment result; Alternatively, the confidence level corresponding to the target intelligence source is determined according to the type of the target intelligence source.

3. The method according to claim 2, characterized in that The step of performing quality assessment on the intelligence source, generating a quality assessment result, and generating the confidence of the intelligence source based on the quality assessment result includes: Determine the quality assessment result of the intelligence source according to whether the intelligence data provided by the intelligence source meets the preset conditions and whether there is an intelligence conflict in the threat intelligence in the intelligence source; A confidence level of the intelligence source is generated based on the quality assessment result.

4. The method according to any one of claims 1 to 3, characterized in that: The performing conflict detection on the processed threat intelligence to obtain at least two conflicting threat intelligences includes: Determining the processed threat intelligence as keywords, and constructing a keyword co-occurrence matrix; Binarizing the keyword co-occurrence matrix to obtain a binary keyword co-occurrence matrix; Establishing a keyword co-occurrence network according to the binary keyword co-occurrence matrix; Extracting content data from the keyword co-occurrence network to obtain a content data set; Conflict detection is performed on the content data in the content data set, and the threat intelligence to which at least two conflicting content data belong is determined as at least two conflicting threat intelligence.

5. The method according to any one of claims 1 to 3, characterized in that: The data preprocessing includes at least one or more of matching whitelist, data cleaning, data deduplication, data tail removal, data verification, data format unification, data label marking and data normalization.

6. The method according to claim 5, characterized in that The data label is used to mark attribute information of the intelligence data; the attribute information includes at least one or more of the first discovery time, the last discovery time, the data type label, the region, the organization, the industry, the attack type, the threat level, the intelligence source and the family gang information.

7. A threat intelligence acquisition device based on intrinsic security, characterized in that: The device comprises: An acquisition unit, used to acquire intelligence data from multiple intelligence sources; A capture unit is used to capture threat intelligence from intelligence data of multiple intelligence sources based on a threat intelligence template corresponding to the intelligence source, and obtain threat intelligence to be processed; A preprocessing unit, used to perform data preprocessing on the threat intelligence to be processed to obtain processed threat intelligence; A detection unit, configured to perform conflict detection on the processed threat intelligence and obtain at least two conflicting threat intelligences; A conflict processing unit, used to perform conflict processing on the at least two conflicting threat intelligences, and obtain the threat intelligence after the conflict processing; The conflict handling unit comprises: A first determining subunit is used to determine the object intelligence sources to which the at least two conflicting threat intelligences respectively belong; A first acquisition subunit is used to acquire an attribute importance value of a target intelligence source based on the at least two conflicting threat intelligences and at least two of the object intelligence sources; the target intelligence source is any one of the object intelligence sources; the attribute importance value is used to characterize the priority of the intelligence source; The first acquisition subunit includes: A first calculation subunit is used to calculate a ratio coefficient between the amount of intelligence from a target intelligence source and the total amount of intelligence from at least two of the target intelligence sources within a target time period; A third acquisition subunit is used to acquire a first quantity of at least two conflicting threat intelligences; A second determination subunit is used to determine a second number of basic data tags and a third number of enriched data tags in the target intelligence source; the basic data tags include a domain name and a uniform resource locator URL; the enriched data tags include: first discovery time, last discovery time, and threat level; A third determination subunit, used to determine the confidence of the target intelligence source; A second calculation subunit is used to calculate the attribute importance value of the target intelligence source based on the proportional coefficient, the first number, the second number, the third number and the confidence level: The attribute importance value of the target intelligence source = {proportional coefficient × first quantity × [(2 + second quantity) + 4 (1 + third quantity)]} + confidence; A sorting subunit, configured to prioritize the target intelligence sources based on the attribute importance values ​​of the target intelligence sources, and determine the target intelligence sources within a preset priority sorting range; The second acquisition subunit is used to retain the target threat intelligence among the at least two conflicting threat intelligences and obtain the threat intelligence after the conflict is resolved; the target threat intelligence is the threat intelligence corresponding to the object intelligence source within the preset priority sorting range.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, all or part of the steps of the method according to any one of claims 1 to 6 are implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, all or part of the steps of the method according to any one of claims 1 to 6 are implemented.

Images