Embedded expression device, embedded expression creation method and embedded expression creation program
The embedded expression creation device addresses the issue of low-quality representations by segmenting and comparing cyber threat information using type determination and document division, enhancing expressiveness and analysis efficiency.
Patent Information
- Application Number
- JP2024045356
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2024-03-21
- Publication Date
- 2025-10-03
AI Technical Summary
Existing methods for embedding cyber threat-related information using Transformer-based LLMs, such as BERT, often include unnecessary information, leading to low-quality embedded representations due to the tokenization of input strings from social networking sites or bulletin boards.
An embedded expression creation device that includes a type determination unit to assign second tags to sentences based on the distribution of first tags, a document division unit to reconstruct documents into partial documents with shared tags, and an embedding processing unit to obtain embedded representations using a language model, while evaluating similarity between documents.
The method enhances the expressiveness of cyber threat-related information by accurately segmenting and comparing embedded representations, improving information analysis efficiency and contributing to sustainable development goals.
Smart Images
Figure 2025145266000001_ABST
Abstract
Description
[Technical Field]
[0001] The present invention relates to techniques for collecting and analyzing cyber threat information. [Background technology]
[0002] Conventionally, cyber threat-related information has been published in vulnerability databases such as the National Vulnerability Database (NVD, see Non-Patent Document 1, for example), or has also been disseminated through blogs of security vendors, social networking sites, bulletin boards, and the like. When collecting and analyzing information about a specific cyber threat, it is necessary to aggregate this scattered unstructured data and convert it into a format that can be understood by humans or computers. Commonly used methods include converting cyber threat-related information into a knowledge graph and analyzing the data (see, for example, Non-Patent Document 2), or using LLMs (Large Language Models) to embed or fine-tune the data (see, for example, Non-Patent Document 3). [Prior art documents] [Non-patent literature]
[0003] [Non-Patent Document 1] NATIONAL VULNERABILITY DATABASE (NVD), [online], September 11, 2023, NIST, [Retrieved March 12, 2024], Internet<https: / / nvd.nist.gov / general> [Non-patent document 2] Aritran Piplai et al., Creating Cybersecurity Knowledge Graphs From Malware After Action Reports, IEEE Access, vol. 8, 211691-211703, 2020. [Non-patent document 3] Jian Liu et al., TriCTI: an actionable cyber threat intelligence discovery system via trigger-enhanced neural network, Cybersecurity, vol. 5(1), 1-8, 2022. [Non-patent document 4] Md Tanvirul Alam et al., CyNER: A Python Library for Cybersecurity Named Entity Recognition, arXiv:2204.05754, 2022. [Non-Patent Document 5] Xuren Wang et al., DNRTI: A Large-Scale Dataset for Named Entity Recognition in Threat Intelligence, 2020 IEEE 19th International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom). [Non-patent document 6] Nils Reimers et al., Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks, arXiv:1908.10084, 2019. Summary of the Invention [Problem to be solved by the invention]
[0004] When using a Transformer-based LLM, for example in BERT, the input string is tokenized, and the encoded data is used as input to the Transformer, ultimately obtaining embedded representations corresponding to each token and the input string. However, in general, if information from social networking sites or bulletin boards is used as input strings, unnecessary information may be included, resulting in low-quality embedded representations.
[0005] The present invention aims to provide a method for embedding cyber threat-related information with high expressive power. [Means for solving the problem]
[0006] The embedded expression creation device of the present invention includes a type determination unit that determines a second tag for each sentence included in a document based on the distribution of first tags that indicate the information type assigned to the words that make up the sentence; a document division unit that concatenates multiple sentences that share the second tag and reconstructs the document into multiple partial documents to which the second tags are assigned; and an embedding processing unit that obtains embedded expressions for each of the partial documents using a predetermined language model.
[0007] The type determination unit may determine the second tag by majority vote of the first tags.
[0008] The type determination unit may determine the second tag for the sentence using a classifier trained using training data.
[0009] The teacher data may be data in which an information type that exceeds a predetermined rate in the distribution of the first tags is set as the second tag.
[0010] The document division unit may connect consecutive sentences in the document according to a predetermined rule regarding the syntax of the sentences, even if the second tags are different.
[0011] The embedded representation creation device may include a similarity evaluation unit that evaluates the similarity between the documents by comparing the embedded representations for each of the partial documents.
[0012] When the amount of information of the partial document is below a threshold, the similarity evaluation unit may exclude the embedded representation of the partial document from comparison targets.
[0013] If the amount of information in only one of the partial documents between the documents is below a threshold, the document division unit may combine the partial document with the other partial document while maintaining the order of sentences within the document, and generate a new partial document to which a new tag that combines the second tag is assigned.
[0014] The embedded representation creation method of the present invention is performed by a computer, and includes the following steps: a type determination step of determining a second tag for each sentence included in a document based on the distribution of first tags indicating the information type assigned to the words that make up the sentence; a document division step of concatenating multiple sentences that share the second tag and reconstructing the document into multiple partial documents to which the second tags are assigned; and an embedding processing step of obtaining embedded representations for each partial document using a predetermined language model.
[0015] An embedded expression creation program according to the present invention is for causing a computer to function as the embedded expression creation device. [Effects of the Invention]
[0016] According to the present invention, an embedded representation of cyber threat-related information with high expressive power can be obtained. [Brief explanation of the drawings]
[0017] [Figure 1] 1 is a block diagram showing a functional configuration of an embedded expression creation device according to an embodiment. [Figure 2] 10 is a flowchart showing the processing steps of an embedded expression creation method according to an embodiment. [Figure 3]FIG. 1 is a schematic diagram showing a flow up to the creation of an embedded expression in an embodiment. DETAILED DESCRIPTION OF THE INVENTION
[0018] An example of an embodiment of the present invention will be described below. When targeting a certain cyber threat, multiple pieces of cyber threat-related information (mainly documents) can be obtained by using keywords such as CVE (Common Vulnerabilities and Exposures) identification numbers. In the field of cybersecurity, there are particularly important types of information, such as organizations involved in cyberattacks, attack methods, and countermeasures against attacks. Standards such as STIX (Structured Threat Information eXpression) organize the information necessary to counter cyberthreats, and technology has been proposed to assign tags that comply with the standards to words (see, for example, Non-Patent Documents 4 and 5).
[0019] In this embodiment, it is assumed that tags indicating the type of information can be added to at least some of the words in a document regarding cyber threat-related information collected from various information sources. The embedded expression creation device creates embedded expressions for each information type in the document based on the tags, and further associates the documents with each other and presents them.
[0020] FIG. 1 is a block diagram showing the functional configuration of an embedded expression creation device 1 according to this embodiment. The embedded expression creation device 1 is an information processing device that includes a control unit 10, a storage unit 20, and various input / output interfaces.
[0021] The control unit 10 is a part that controls the entire embedded expression creation device 1, and realizes each function of this embodiment by appropriately reading and executing various programs stored in the storage unit 20. The control unit 10 may be a CPU. The storage unit 20 is a storage area for storing various programs for causing the hardware group to function as the embedded expression creation device 1, various data, and the like, and may be a ROM, RAM, flash memory, hard disk drive (HDD), or the like.
[0022] The control unit 10 executes software (embedded expression creation program) stored in the storage unit 20 to function as a type determination unit 11, a document division unit 12, an embedding processing unit 13, and a similarity evaluation unit .
[0023] The type determination unit 11 determines a second tag for each sentence included in the collected documents based on the distribution of first tags indicating information types that have been assigned to the words that make up the sentence by existing methods. At this time, the type determination unit 11 may count the number of first tags in the sentence for each information type, and determine the second tag to be assigned to the sentence by majority vote.
[0024] Furthermore, in order to further improve accuracy, the type determination unit 11 may determine a second tag for a sentence using a classifier trained using training data. In general, it is rare for a sentence to consist of only words with the same tag, and the first tags assigned to words by mechanical methods are not necessarily accurate. Therefore, in order to build a highly accurate classifier, training data in which reliable second tags are assigned to sentences is required. For this reason, it is possible to collect documents related to cyber threats in advance and generate training data from these documents, for example, as follows.
[0025] The second tags may be assigned manually by an expert, or may be assigned automatically to sentences using the first tags assigned to words. For example, among the primary tags of words contained in each sentence in a document, the primary tag that appears most frequently and that appears more frequently than the sum of the other primary tags is designated as the secondary tag of the sentence, and sentences that do not meet the conditions are excluded from the training data. Specifically, suppose a sentence s contains a, b, and c instances of words with primary tags A, B, and C, respectively. In this case, if a > b + c (majority), the secondary tag A is assigned to sentence s. If there are no instances that exceed the majority, the secondary tag is not assigned. This allows for the generation of a certain number of highly accurate sentence and secondary tag pairs. Note that the conditions for adopting sentences as training data can be set as appropriate, such as exceeding a certain percentage (e.g., 2 / 3 or 70%) instead of the majority.
[0026] By constructing a classifier using sentences to which secondary tags have been assigned in this way as correct answer data, secondary tags can be assigned to each sentence of newly collected documents. For example, a classifier can be constructed by fine-tuning using the method of sentence BERT (see Non-Patent Document 6).
[0027] The document dividing unit 12 connects multiple sentences that share a second tag, and reconstructs the document into multiple partial documents to which the second tag is assigned. At this time, in order to maintain the context, the document dividing unit 12 may connect consecutive sentences in the document into partial documents, even if the second tags are different, in accordance with predetermined rules regarding sentence syntax. For example, a demonstrative pronoun such as "it" refers to a noun in a previous sentence, so the sentence is considered to be closely related to the previous sentence. Therefore, by linking the previous and subsequent sentences according to specific words or syntax, related sentences can be grouped into a single partial document.
[0028] The embedding processor 13 obtains an embedded expression based on a predetermined language model for each partial document. For example, when using BERT, each segmented document is represented by a 768-dimensional vector. Therefore, if the number of classifications (number of second tags) is m, the document is represented in 768 × m dimensions.
[0029] The similarity evaluation unit 14 compares documents by information type, that is, compares the embedded expressions of each partial document, thereby evaluating the similarity between the documents. Here, when documents X and Y are information of the same level (for example, both are vulnerability information expressed by CVE-ID), they are expressed in the same dimension (for example, 768 × m dimensions). In this case, X and Y can be directly compared, but the similarity evaluation unit 14 may exclude the embedded expression of any partial document from the comparison targets if the amount of information of the partial document among the m types of information types (second tags) is below a threshold. The amount of information may be, for example, entropy. Alternatively, the number of sentences or the number of words may be used as a simpler measure of the amount of information.
[0030] On the other hand, if X and Y are not the same piece of information, for example, if X is vulnerability information expressed by a CVE-ID and Y is a report on an attack method, among the m subdivided partial documents representing Y, many will have an amount of information below the threshold, but some partial documents related to attacks will have a large amount of information. For example, in Non-Patent Document 5, the tags "way" and "purp" can be considered tags related to attacks. On the other hand, there are cases where the partial documents assigned with "way" and "purp" do not necessarily contain sufficient information for X. For example, if the amount of information in the way for X is below a threshold, only the purp can be used in the similarity comparison, and the information in Y cannot be utilized.
[0031] Therefore, when there is such a bias in the information, the second tags may be aggregated. That is, when the amount of information in only one partial document between documents is below the threshold, the document division unit 12 combines this partial document with the other partial documents while maintaining the order of the sentences in the original document, and generates a new partial document to which a new tag that combines the second tags is assigned. For example, if a partial document of the way tag contains sentences w1 and w2, and a partial document of the purp tag contains sentences p1 and p2, if the order of appearance of the sentences in the original document before subdivision is w1, p1, p2, w2, this positional relationship is preserved, and w1, p1, p2, w2 are used as partial documents of the way+purp tag. This makes it possible to compare embedded expressions without losing information from document Y, even if document X has few partial documents of a way.
[0032] FIG. 2 is a flowchart showing the processing steps of the embedded representation creation method according to this embodiment. Here, we assume that multiple documents have been collected as cyber threat-related information, and that a classifier for tagging sentences has already been constructed.
[0033] In step S1, the control unit 10 assigns tags (first tags) indicating the type of information to words contained in document X, which is cyber threat-related information, using existing methods such as Non-Patent Documents 4 and 5. If a document that has already been tagged is input, this step is omitted.
[0034] In step S2, the type determination unit 11 determines which information type each sentence in document X belongs to based on the distribution of tags assigned to words in step S1, and assigns a second tag to each sentence.
[0035] In step S3, the document dividing unit 12 merges the sentences classified by the second tags and subdivides them into a plurality of partial documents for each information type.
[0036] In step S4, the embedding processor 13 obtains an embedded representation for each fragmented partial document through LLM.
[0037] In step S5, the similarity evaluation unit 14 compares the embedded expressions between the documents for each information type, that is, for each partial document, and evaluates the similarity.
[0038] The embedded representation creation device 1 evaluates the similarity between, for example, X and Y, which are seemingly unrelated, based on the similarity evaluation results, discovers a connection between X and Y, and adjusts the representation method according to the information analysis task.
[0039] FIG. 3 is a schematic diagram showing the flow up to the creation of an embedded expression in this embodiment. Here, a document with a CVE-ID of 20xx-xxxx is input to the embedded representation creation device 1, and this document is broken down into individual sentences (1). At least some of the words contained in the sentences are given tags (first tags) A, B, ... indicating the type of information.
[0040] Next, a fine-tuned classifier based on word tags classifies each sentence into one information type and assigns the corresponding tag (second tag) A, B, ... (2). Sentences that could not be classified with low accuracy are excluded.
[0041] Next, multiple sentences with the same tag are merged and reconstructed into subdocuments, the number of which is the number of tags (m) (3). For each of these partial documents, we use LLM such as BERT to generate an embedding v A ,v B , ... are created, and these are combined to create an embedding representation of the original document, vCVE-20xx-xxxx (4)
[0042] According to this embodiment, the embedded representation creation device 1 can obtain expressions for each information type by breaking down cyber threat-related information into domain-specific information types and passing them through the LLM. The embedded representation creation device 1 can improve the expressiveness of cyber threat-related information by combining and presenting these. In addition, the embedded expression creation device 1 can express cyber threat-related information with greater expressiveness by evaluating the similarity of embedded expressions for each document depending on the task for which the embedded expressions are used, and then combining and presenting them.
[0043] This will, for example, improve the expressiveness of embedded representations of cyber threat-related information and make information analysis more efficient, which will contribute to Goal 9 of the United Nations-led Sustainable Development Goals (SDGs): "Build resilient infrastructure, promote sustainable industrialization and foster innovation."
[0044] Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments. Furthermore, the effects described in the above-described embodiments are merely a list of the most preferable effects resulting from the present invention, and the effects of the present invention are not limited to those described in the embodiments.
[0045] The embedded expression creation method by the embedded expression creation device 1 is realized by software. When realized by software, the programs constituting this software are installed in an information processing device (computer). These programs may be recorded on removable media such as CD-ROMs and distributed to users, or may be distributed by being downloaded to the user's computer via a network. Furthermore, these programs may be provided to the user's computer as a web service via a network without being downloaded. [Explanation of symbols]
[0046] 1. Embedded Representation Creation Device 10 Control Unit 11 Type determination section 12 Document division 13 Embedding processing section 14 Similarity Evaluation Section 20 Memory section
Claims
1. a type determination unit that determines a second tag for each sentence included in a document based on a distribution of first tags indicating information types assigned to words that constitute the sentence; a document division unit that connects multiple sentences that share the second tag and reconstructs the document into multiple partial documents to which the second tag is assigned; and an embedding processing unit that acquires an embedded expression based on a predetermined language model for each of the partial documents.
2. The embedded expression creation device according to claim 1 , wherein the type determination unit determines the second tag by majority vote of the first tags.
3. The embedded expression creation device according to claim 1 , wherein the type determination unit determines the second tag for the sentence using a classifier trained using training data.
4. The embedded expression creation device according to claim 3 , wherein the training data is data in which information types that exceed a predetermined rate in the distribution of the first tags are set as the second tags.
5. 5. The embedded expression creation device according to claim 1, wherein the document division unit connects consecutive sentences in the document according to predetermined rules regarding the syntax of the sentences, even if the second tags are different.
6. 5. The embedded expression creation device according to claim 1, further comprising a similarity evaluation unit that evaluates the similarity between the documents by comparing the embedded expressions for each of the partial documents.
7. The embedded expression creation device according to claim 6 , wherein the similarity evaluation unit excludes the embedded expression of the partial document from comparison targets when the amount of information of the partial document is below a threshold.
8. The embedded expression creation device described in claim 6, wherein when the information amount of only one of the partial documents between the documents is below a threshold, the document division unit combines the partial document with the other partial documents while maintaining the order of sentences within the document, and generates a new partial document to which a new tag that combines the second tag is assigned.
9. a type determination step of determining a second tag for each sentence included in the document based on a distribution of first tags indicating information types assigned to words constituting the sentence; a document division step of concatenating a plurality of sentences having the second tag in common and reconstructing the document into a plurality of partial documents to which the second tag is assigned; an embedding processing step of acquiring an embedded representation using a predetermined language model for each of the partial documents, the method being executed by a computer.
10. An embedded expression creation program for causing a computer to function as the embedded expression creation device according to any one of claims 1 to 8.