Threat intelligence oriented entity identification method and system

Abstract

The invention relates to a threat intelligence oriented entity identification method and system. The method comprises the following steps: 1) performing coarse word segmentation on a threat information text serving as a training corpus; 2) constructing a threat information entity common word dictionary library and a rule library, and performing dictionary matching and rule matching on a coarse word segmentation result; 3) marking an entity label for each word based on a matching result to form a training set; 4) constructing a feature template, establishing an indication word bank to perfect the screening form of the feature template, generating context features for the training set by using the feature template, screening, and inputting the screened features into a machine learning modelto carry out parameter iterative training; and 5) performing coarse word segmentation, dictionary matching and rule matching on the threat information text to be identified, and performing entity identification by using the trained machine learning model. According to the threat information entity extraction method, the threat information entity extraction is completed by adopting a means of combining a rule, a dictionary and a model, so that the entity identification precision of the threat information is remarkably improved.

Description

Technical field [0001] The present invention proposes a threat intelligence-oriented entity identification method and system. It quotes linguistic standards in the threat intelligence field, covering natural language processing rule extraction, dictionary extraction and machine learning methods. A total of 28 related entities can be extracted, belonging to The intersection of computer science and network security. Background technique [0002] At present, the number of netizens in my country has reached 772 million. At the same time, my country is constantly suffering from serious cyber attacks, and the outbreak of large-scale security incidents has sharply endangered the security situation of cyberspace. In order to adapt to the rapid evolution of cyber threats, cyber security analysts in various countries are actively collecting statistics on cyber security indicators (Indicators of Compromise, IOC) (e.g. malicious information) from public sources of threat intelligence (such a...

AI Technical Summary

Problems solved by technology

However, although various entity extraction methods vary widely in technical implementation, their extraction effects often have a strong dependence on specific resources (artificial vocabulary or artificial word segmentation data), resulting in that although existing entity extraction methods are in various open evaluations Excellent performance, but the performance in the field of network threat intelligence where corpus resources are scarce is still not satisfactory, that is, the current technology cannot meet the high standards (mainly precision and recall) input expected by IOC, especially for threat intelligence field, the F1 value of entity extraction is only 0%-30% through experiments, so there is still a lot of room for entity extraction research in professional fields

In foreign countries, named entity recognition technology is also in the golden age of development, but Chinese sentences have particularity and complexity, unlike English words separated by spaces to complete word segmentation directly, and there are no uppercase and lowercase marks and word form transformation features, so Chinese Threat intelligence entity identification can only refer to and not directly refer to foreign threat intelligence entity identification tools

[0006] To sum up, the current artificial entity extraction for threat intelligence still requires experienced analysts to spend a lot of energy to complete, which cannot meet the needs. Although there have been some preliminary applications for automated analysis, most of them are very basic trends. For There is often a strong dependence on specific resources. At present, there is no mature entity extraction technology in the field of threat intelligence in China, which is one of the obstacles to making emergency judgments on domestic network security threats.

Images