Information complementing device and information complementing method

Abstract

A data collection unit (11) collects information related to an object and sets the information as collected data (101). A data normalization unit (12) adds, as an input type, the type of data to the collected data (101) collected by the data collection unit (11), and thereby generates training data (102). A training unit (13) trains a machine learning model (103), which is a large-scale language model, on the basis of the training data (102) generated by the data normalization unit (12). By using the trained machine learning model (103) and a portion of visualization data that includes configuration information, state information, and evaluation information of the object and information indicating relationships of the object with other related objects, an inference unit (14) infers the information in the visualization data other than said portion. A correction unit (15) corrects the information in the visualization data inferred by the inference unit (14), on the basis of the collected data (101).

Description

Information supplementation device and information supplementation method

[0001] The present invention relates to an information supplementation device and an information supplementation method.

[0002] For security measures for supply chains, there are technologies that perform security inspections such as configuration inspections and vulnerability inspections using data that visualizes the configuration, status, and evaluation of devices, systems, services, etc. (hereinafter referred to as "visualized data") For example, as an example of a technology for sharing configuration information, SBOM (Software Bill of Materials) has been proposed as a data format and process for listing and documenting the components and dependencies of software products.

[0003] More specifically, an SBOM is a software "bill of materials," a data format and process for listing and documenting a software product's components and dependencies. SBOM provides transparency into software products and is used for purposes such as traceability, security, legal compliance, license management, or vulnerability management.

[0004] “The Minimum Elements For a Software Bill of Materials [SBOM]”, [online], July 12, 2021, The United States Department of Commerce, [Retrieved October 24, 2023], Internet <URL: https: / / www.ntia.doc.gov / files / ntia / publications / sbom_minimum_elements_report.pdf>

[0005] Here, the visualization data of devices provided by providers to users often contains confidential information about the provider, such as proprietary innovations. Therefore, it is difficult for providers to provide all of the visualization data due to concerns about leaks of confidential information. It is also possible that the information provided by providers may contain unintentional errors. Thus, even if an SBOM or similar document is provided, the information necessary for security testing may be insufficient or may contain errors, making it difficult to conduct a highly accurate test.

[0006] The present invention has been made in view of the above, and has an object to improve the accuracy of testing.

[0007] To solve the above-mentioned problems and achieve the object, a data collection unit collects information related to an object and generates collected data. A data normalization unit generates training data by adding a data type as an input type to the collected data collected by the data collection unit. A training unit trains a large-scale language model based on the training data generated by the data normalization unit. An inference unit infers information other than the part of the visualization data using the trained large-scale language model and a part of the visualization data including configuration information, status information, evaluation information, and information indicating relationships with other related objects of the object. A correction unit corrects the visualization data inferred by the inference unit based on the collected data.

[0008] According to the present invention, the accuracy of the test can be improved.

[0009] FIG. 1 is a block diagram showing an example of an information complementation device. FIG. 2 is a diagram showing an example of collected data. FIG. 3 is a diagram showing an example of learning data which is sentence-type data. FIG. 4 is a diagram showing an example of learning data which is word list-type data. FIG. 5 is a diagram showing an example of a procedure for creating learning data when GPT2 is used. FIG. 6 is a diagram showing an example of a correction process. FIG. 7 is a flowchart of a complementation process performed by the information complementation device according to the first embodiment. FIG. 8 is a diagram for explaining an evaluation procedure. FIG. 9 is a diagram showing an example of an evaluation result. FIG. 10 is a diagram showing an example of a computer that executes an information complementation program.

[0010] Hereinafter, an embodiment of an information supplementation device and an information supplementation method disclosed in the present application will be described in detail with reference to the accompanying drawings. Note that the information supplementation device and the information supplementation method disclosed in the present application are not limited to the following embodiment.

[0011] First Embodiment (Configuration of Information Complement Device) Fig. 1 is a block diagram showing an example of an information complement device 1. The information complement device 1 has two operation phases: a learning phase and an inference phase.

[0012] In the learning phase, the information supplementation device 1 acquires object-related information related to each element, such as a product, system, or service, that constitutes the supply chain. The object-related information can be any information related to each element, and may be part of configuration information, status information, evaluation information, and information indicating the relationship between elements, i.e., part of visualization data. The information supplementation device 1 then generates a large-scale language model that infers visualization data from the acquired object information.

[0013] In the inference phase, the information complementing device 1 uses a trained large-scale language model to infer missing information from a portion of the visualized data obtained for various inference targets constituting a supply chain. In particular, the information complementing device 1 infers information required for vulnerability testing, such as a Common Platform Enumeration (CPE) or a Persistent Uniform Resource Locator (PURL).

[0014] Here, configuration information refers to information that identifies the products, systems, services, etc. that make up the supply chain, as well as information that describes their contents and specifications. Products refer to various devices included in the supply chain and the components that realize those devices, including both hardware and software components. Specifications are information that define the basic functions that each product, system, etc., represented by the configuration information, has and provides to the outside world. For example, specifications define the physical or electrical input / output of an object, such as physical actions, displays, or the transmission and reception of data and signals.

[0015] The configuration information also includes information about hardware components, such as component model numbers and serial numbers. The configuration information also includes information about software components, such as package names, file names, and associated attribute information. Software component information is information obtained, for example, from data in an OS (Operating System) package management system or an SBOM.

[0016] Status information is information that is generated as a result of operating or manipulating products, systems, services, etc. that make up a supply chain. For example, logs and resource usage of products, systems, services, etc. are examples of status information.

[0017] The evaluation information is information calculated based on predetermined indicators for products, systems, services, etc. that make up the supply chain. For example, the evaluation information includes inspection results and security risk information for products, systems, services, etc.

[0018] Information indicating the relationships between elements is information that indicates the connection relationships, topology, and inclusion relationships between the products, systems, or services that make up the supply chain. Information indicating the relationships between elements indicates, for example, the relationship between a "finished product" element and its constituent "parts," or the relationship between the parts of a specific finished product.

[0019] Visualization data is a general term for configuration information, status information, evaluation information, and information indicating relationships between elements associated with a device, system, service, or the like.

[0020] As shown in FIG. 1, the information supplementing device 1 is connected to an external system 2 that holds information about products, systems, services, etc., and a user terminal 3 via a network.

[0021] The external system 2 is not limited to a single system, but is a general term for various information sources, such as the Internet, that exist outside the information supplementation device 1. The external system 2 includes OSINT (Open Source Intelligence), public websites where information about objects is published by object providers, and databases where vulnerability information is collected. OSINT refers to the process and method of collecting and analyzing publicly available information. Generally, OSINT is collected from the Internet and public information sources and used for various purposes, such as security, business, journalism, and research.

[0022] The external system 2 includes, for example, Hugging Face, which is a platform that provides machine learning models and datasets, and has a machine learning model 21. The machine learning model 21 is a pre-trained model of a large-scale language model, and for example, a Generative Pretrained Transformer (GPT) (registered trademark) or a Text-to-Text Transfer Transformer (T5) is used.

[0023] The external system 2 also includes an NVD (NATIONAL VULNERABILITY DATABASE) 22. The NVD 22 holds information on CVEs (Common Vulnerabilities and Exposures) 221 and CPEs 222. The CVEs 221 include information on CVE descriptions 2211. CPEs are a standard format for uniquely identifying information such as computer systems and application software, and are primarily used in security vulnerability information and vulnerability assessment tools. The CPEs 222 include information on CPE names 2221 and CPE titles 2222. The external system 2 may also include license information, vulnerability information, information included in visualization data, and the like.

[0024] The user terminal 3 is a computer of a user who uses the information supplementation device 1. The user also includes an administrator of the information supplementation device 1. The user can transmit data to the information supplementation device 1 using the user terminal 3.

[0025] For example, in the learning phase, the user uses the user terminal 3 to send visualization data to be used for machine learning to the information supplementation device 1. In this case, the visualization data does not need to be all in one, and only a portion of the visualization data may be sent. Also, in the inference phase, the user uses the user terminal 3 to send inference data, such as a portion of the visualization data, to the information supplementation device 1, causing inference to be performed on the visualization data for various inference targets that make up the supply chain.

[0026] In this embodiment, the information supplementation device 1 performs learning using data on products, systems, and services included in a supply chain and performs inference on any of the products, systems, and services, but is not limited to this. Alternatively, the information supplementation device 1 may perform learning and inference on a single device.

[0027] 1, the information supplementation device 1 includes a data collection unit 11, a data normalization unit 12, a learning unit 13, an inference unit 14, and a correction unit 15. The configuration of the information supplementation device 1 according to this embodiment will be described in detail below with reference to FIG.

[0028] The data collection unit 11 acquires object-related information, which is data related to various elements that make up the supply chain and is used for pre-learning and fine-tuning of a large-scale language model, as collected data 101. For example, the data collection unit 11 acquires license information, vulnerability information, visualization data, etc. as object-related information.

[0029] Fig. 2 is a diagram showing an example of collected data. In addition, the data collection unit 11 can acquire object-related information as shown in Fig. 2 and use it as collected data 101 to be used for learning data 102. The data in Fig. 2 are easier to obtain as they move up the page.

[0030] The data collection unit 11 can crawl and acquire information stored in the NVDs 22, such as the CVEs 221 and CPEs 222, and public information, such as information on laws and regulations and industry standards. The data collection unit 11 then extracts object-related information related to various elements that make up the supply chain from the acquired public information, and sets the extracted information as collected data 101.

[0031] Public information collection is performed, for example, in the following manner: A user of the information complement device 1 obtains visualization data from the provider of each element that makes up the supply chain, in which the names of software present in the device and the like are described as part of the visualization data. Then, based on the software names obtained by the user, the data collection unit 11 collects the names of the libraries necessary for the software's operation by crawling open source, such as information on the Web, from the perspective of dependencies between the libraries. This allows the data collection unit 11 to use the data for learning the relationships between the software and libraries included in each element that makes up the supply chain.

[0032] For example, if the website crawling and keyword extraction is performed using Python (registered trademark), the data collection unit 11 can use "Requests," "Beautiful Soup," "Scrapy," etc. to extract websites containing software names and automatically extract character strings associated with the word "library" from the extracted websites, thereby collecting library names. Additionally, the data collection unit 11 can automatically extract related libraries by using a database that provides library dependencies for OSS (Open Source Software), such as the "Open Source Insights Project" provided by Google (registered trademark).

[0033] The data collection unit 11 can also acquire visualization data including configuration information such as an SBOM from a manufacturer or a service provider. The visualization data acquired from the provider may consist of only configuration information such as an SBOM, or may include information such as status information and evaluation information. The data collection unit 11 can use the acquired visualization data as collected data 101.

[0034] The data collection unit 11 can also acquire specifications, manuals, etc. as electronic data. Then, the data collection unit 11 extracts object-related information related to various elements that make up the supply chain from the acquired electronic data such as specifications, manuals, etc. using natural language processing, and sets the extracted information as collected data 101.

[0035] The data collection unit 11 can also acquire source code from an external site or a manufacturer. The data collection unit 11 then extracts code snippets, function hashes, and the like from the acquired source code to generate the collected data 101.

[0036] The data collection unit 11 can also receive input of external specifications such as an appearance image obtained from an actual physical part, and use the input as collected data 101. The data collection unit 11 can also acquire a communication log obtained by operating an actual product, and use the communication log as collected data 101.

[0037] Communication logs are collected, for example, as follows: Users of each element in the supply chain operate the corresponding device or system, and the data collection unit 11 captures the communication log as operation information. Furthermore, if the operation log can be viewed directly or remotely using a tool such as rsyslog, the data collection unit 11 can also utilize these. From the collected communication logs, the data collection unit 11 acquires, as features related to the status information of the visualization data, features such as protocol numbers, port numbers, and communication content included in the IP (Internet Protocol) header and IP payload. Based on the acquired protocol names and port numbers, the data collection unit 11 can then collect software names by crawling open source resources, such as information on the Web, from the perspective of the software names that use these protocols by default and the dependencies between elements. This allows the data collection unit 11 to use the acquired data to learn the relationships between applications running in the same environment.

[0038] For example, if the characteristics of the target device include the HTTP protocol and port 80, the data collection unit 11 can determine that a web server application is installed from an open source. Furthermore, the data collection unit 11 can crawl the web server application as input and extract possible application names such as "Apache," "Nginx," and "IIS." Furthermore, if the HTTP header can be confirmed, the data collection unit 11 can confirm the type and version of the web server application from the Server tag. For example, the data collection unit 11 can infer the presence of a web servlet container application such as tomcat / Jetty / Jboss if port 8080 is used, RemoteSH if port 514 is used, and ssh if port 22 is used.

[0039] The data collection unit 11 can also acquire binaries from an actual product, disassemble the acquired binaries, extract operation specifications, and use the acquired specifications as collected data 101. The data collection unit 11 can also acquire operation logs accumulated in the product by operating the product, and use the acquired logs as collected data 101. The data collection unit 11 can also acquire detailed configuration information from a program that acquires configuration information that operates inside an actual physical component, and use the acquired logs as collected data 101.

[0040] Detailed configuration information is collected, for example, in the following manner: A user of the information complement device 1 obtains visualization data from the provider of each element that makes up the supply chain, in which the names of software present in each element and other information are described as part of the visualization data. The data collection unit 11 collects pre-created software and file dependencies from the software names obtained by the user, from the perspective of dependencies between files required by the software or owned by the software. This allows the data collection unit 11 to use the data for learning the relationships between software and files included in each element that makes up the supply chain.

[0041] The following methods exist for creating pre-created software and file dependencies. A user of the information complement device 1 prepares a separate device with a well-known OS or software installed and acquires information about the files that make up a specific OS or software. Additionally, the user of the information complement device 1 sorts out from the acquired files, for example, by directory, extension, property information, etc., files that remain constant in any environment (such as executable files) and files that change (such as configuration files or log files), for the same software, and acquires hash values ​​for the unchanged files. The user of the information complement device 1 prepares a separate device with a well-known OS or software installed and, for example, in the case of Linux (registered trademark), searches for dependent libraries using commands such as "ldd" or "readelf -d" and then creates a database of the dependent libraries along with the OS, software name, and version information. Alternatively, the user of the information complement device 1 may use various commands, such as the "wget ​​-S" command to check server applications, or the "dig version bind" command to check the version of BIND (Berkeley Internet Name Domain) used by the DNS (Domain Name System) server.

[0042] Here, in this embodiment, a case will be described in which the collected data 101 is mainly collected from the external system 2, but the data collection unit 11 may also acquire part of the visualized data of each element constituting the supply chain via the user terminal 3. As described above, the data collection unit 11 collects information related to the target object and generates the collected data 101.

[0043] The data normalization unit 12 generates training data 102 by adding the type of data as an input type as preprocessing for improving inference accuracy to each of the collected data 101 collected by the data collection unit 11. An example of the preprocessing by the data normalization unit 12 will be described below.

[0044] For example, when collected data 101 is input into the machine learning model 103, there is a possibility that a weakly relevant answer will be output due to probability, which may result in a decrease in inference accuracy. For example, the word "Java (registered trademark)" can be used as a programming language for developing applications or as a platform built around a Java Virtual Machine. When an inference result for an application is desired, an inference result for the platform may be output, which may lead to an incorrect answer. Therefore, the data normalization unit 12 combines collected data 101 and the input type of the data in the collected data 101 as training data 102, thereby obtaining a highly relevant result according to the purpose of inference. Here, the data normalization unit 12 may use information on the source of the data as the input type.

[0045] For example, the data normalization unit 12 adds an input type indicating the type of data to the beginning of the training data 102 including the collected data 101, and causes the machine learning model 103 to simultaneously learn the collected data 101 and the input type. Here, the input type may list multiple data types, such as application description, application sbom, or application files.

[0046] Specifically, if the collected data 101 is a file list related to an application, the data normalization unit 12 sets the input type of the collected data 101 to "application files" and sets it as the training data 102. When training is performed in this manner, during inference, the machine learning model 103 receives input of inference data with the input type added, thereby being able to acquire a file list from the files information, and further being able to infer the package name, version, OS, architecture, etc. required for the CPE description based on the file configuration.

[0047] Furthermore, if the collected data 101 contains meaningless character strings, using the collected data 101 to train the machine learning model 103 may result in degradation of the model. More specifically, if the machine learning model 103 learns data in which the character strings themselves are meaningless, the model may not be able to correctly learn meaningful sentence connections and may generate random alphabetic strings during inference. Furthermore, if existing LLMs (Large Language Models) such as GPT are used as the machine learning model 103, the model may degrade if natural sentences are input as training data, such as vulnerability information or visualization data, which do not form sentences written in a prescribed format. Furthermore, the machine learning model 103 has a limit to the token size that can be used, and it may be difficult to directly input visualization data of tens of MB as training data.

[0048] Therefore, the data normalization unit 12 selects parts of the collected data 101 that are useful for inference, or adds data that is useful for inference, and converts the collected data 101 into a format that is useful for inference, thereby generating training data 102.

[0049] For example, the data normalization unit 12 removes from the collected data 101 items that are written in an alphabetical list format from the learning target and sets the data as the learning data 102. Specifically, when the collected data 101 is visualized data, the data normalization unit 12 removes an ID (identifier) ​​number, a software hash value, a certificate signature, and the like from the collected data 101 and sets the data as the learning data 102.

[0050] Additionally, the data normalization unit 12 links related sentences and the like based on the collected data 101 to generate sentence-type learning data 102. Specifically, when inferring a CPE name 201 and the CVE 221 indicates a vulnerability present in the CPE 222 including the CPE name 201, the data normalization unit 12 links the correct CPE name 201 with the input type and the CVE description 2211 of the CVE 221 to generate learning data 102.

[0051] 3 is a diagram showing an example of learning data that is text-type data. For example, a case where T5 used as a pre-learning model is used will be described, and learning data 121 for inferring a CPE name 201 will be described. Here, a CVE explanation sentence 2211 is used as the text-type data. The data normalization unit 12 normalizes the data to match T5.

[0052] The data normalization unit 12 acquires the CVE description 2211 held by the CVE 221, which indicates a vulnerability of the CPE 222 including the CPE name 201. Here, because the CVE description 2211 is long, the data normalization unit 12 limits it to 1,000 characters and generates the CVE information 203. Then, the data normalization unit 12 adds an input type (Type) to the beginning of the CVE information 203 as the data type of the CVE information 203. Furthermore, here, the data normalization unit 12 adds the manufacturer (Vendor), product information (Product), and version (Version) to the beginning of the CVE information 203. That is, the data normalization unit 12 adds attribute information 202 including the input type to the CVE information 203. The data normalization unit 12 then associates the CVE information 203 to which the attribute information 202 has been added with the CPE name 201, which is the correct answer to the inference, to generate the training data 121 shown in Figure 3. The data normalization unit 12 can also generate training data 122 by adding the data type to the CVE description 2211 and deleting predetermined information (here, cpe:2.3:) from the CPE name 201, and using this as the correct answer to the inference. In this way, by removing the same character string that is included in most of the correct answers as the predetermined information, it is possible to prevent meaningless relationships from being learned and improve learning accuracy.

[0053] Here, since one vulnerability may be included in multiple pieces of software, and one piece of software may have multiple vulnerabilities, one CVE 221 may correspond to multiple CPEs 222, and one CPE 222 may correspond to multiple CVEs 221. Therefore, the data normalization unit 12 generates learning data 102 having a CVE explanation 2211 for each combination of a CPE 222 and a corresponding CVE 221.

[0054] 4 is a diagram showing an example of learning data that is word-list data. As another example, a case will be described in which the data normalization unit 12 generates learning data for inferring a CPE name 211 using word-list data. Here, the data normalization unit 12 also performs normalization in accordance with T5. Here, the data normalization unit 12 extracts and arranges words from the CPE title 2222 corresponding to the CPE name 211, and treats them as word-list data.

[0055] The data normalization unit 12 acquires the CPE title 2222 of the CPE 222 having the CPE name 211. The data normalization unit 12 then adds attribute information 212 indicating the input type (Type) to the beginning of the CPE title 2222 as the data type of the CPE title 2222. The data normalization unit 12 then associates the CPE title 2222 to which the attribute information 212 has been added with the CPE name 211 that is the correct answer to the inference for that CPE title 2222, thereby generating the training data 123 shown in FIG. 4. The data normalization unit 12 can also generate training data 124, 125, etc. by adding the attribute information 212 to the CPE title 2222 and deleting predetermined information (here, cpe:2.3:) from the CPE name 211, as in the case of sentence-type data, as the correct answer to the inference.

[0056] Also, GPT2 may be used as a pre-training model. FIG. 5 is a diagram showing an example of a procedure for creating training data when GPT2 is used. For example, the special tokens used to recognize context in GPT2 generated by a tokenizer include those shown in sentence 131. Each special token in sentence 131 is indicated here by a symbol written after a colon. For example, "bos_token" is <s>It is expressed as:

[0057] For example, the data normalization unit 12 can use the format 132 as the training data 102 when using GPT2. Therefore, the data normalization unit 12 generates training data 133 according to the format 132 using the CPE name 2221 and attribute information. Here, data 135 is data obtained from the collected data 101, such as the CVE explanation 2211. Data 134 is the input type of data 135, such as application. And data 136 is the CPE name 2221, which is the correct answer for the inference on data 135.

[0058] Here, if the training data 102 is generated using input data containing multiple words, the training will be performed in a manner that strongly responds to the order in which the words appear. Therefore, the data normalization unit 12 can also generate multiple training data 102 by shuffling the order of the words in the input data.

[0059] As described above, the data normalization unit 12 processes the collected data 101 collected by the data collection unit 11 by adding the type of data as an input type, and generates learning data so that learning can be performed effectively.

[0060] Furthermore, in this embodiment, the data normalization unit 12 may perform the following preprocessing when normalizing the data. First, since inconsistent writing has a negative impact on word similarity, the data normalization unit 12 converts all writing to lowercase.

[0061] Furthermore, the inclusion of characters unrelated to the text will adversely affect the comprehension of the text's relationships, so the data normalization unit 12 removes escape symbols such as "¥".

[0062] In addition, to eliminate ambiguity in the sentence structure, the data normalization unit 12 writes an end symbol such as "." at the end of the correct answer data and the data body to clearly indicate the end of the sentence. By performing each of the above preprocessing steps, the data normalization unit 12 generates the training data 102.

[0063] Furthermore, the pre-training models are not limited to T5 and GPT2, and other pre-training models such as GPT4, PaLM2 (Pathways Language Model 2), and Llama2 (Large Language Model Meta AI 2) may also be used. The data normalization unit 12 normalizes the data in a format suitable for fine tuning using the pre-training model to be actually used, and generates training data 102.

[0064] Furthermore, the data normalization unit 12 receives input of inference data in the inference phase. The inference data is visualization data from which some information is missing. In this embodiment, the data normalization unit 12 acquires visualization data from the user terminal 3 from which some information to be inferred is missing. The data normalization unit 12 then adds an input type to the inference data in accordance with the format of the training data 102 used during training. The data normalization unit 12 then outputs the inference data to which the input type has been added to the inference unit 14. In this way, the data normalization unit 12 generates input data to which the input type has been added from the inference data, based on the format of the training data 102 used to train the machine learning model 103.

[0065] Continuing the explanation, returning to Figure 1, the learning unit 13 acquires a machine learning model 21, which is a pre-training model of a large-scale language model. Then, the learning unit 13 performs fine tuning, which is supervised learning, on the machine learning model 21 using training data 102 to generate a machine learning model 103. By using the pre-training model acquired from the external system 2, the learning unit 13 can easily generate the machine learning model 103 at low cost.

[0066] In this way, the learning unit 13 causes a large-scale language model to learn based on the training data 102 generated by the data normalization unit 12, and generates a machine learning model 103, which is a large-scale language model (LLM). For example, the learning unit 13 causes the large-scale language model to learn associations between words in the data included in the collected data 101, associations between words in the data and input types, and associations between correct data and words in the data included in the collected data 101 and input types.

[0067] During inference, the inference unit 14 receives input of inference data to which attribute information has been added from the data normalization unit 12. Next, the inference unit 14 inputs the inference data to the trained machine learning model 103. Then, the inference unit 14 acquires the inference result output from the machine learning model 103. For example, using the training data of FIG. 3 as an example, the inference unit 14 uses AI to determine the relevance between the sentence in the input inference data and the CVE information 203 in the training data to which attribute information 202 has been added, and outputs the CPE name 2221 corresponding to the CVE information 203 to which the related attribute information 202 has been added. The inference unit 14 can also perform inference multiple times using different inference data and combine the results to improve accuracy.

[0068] For example, consider a case where the inference unit 14 performs one inference using configuration information of an element to be inferred, etc. In this case, it is further assumed that a list of files related to the element to be inferred can be acquired based on its relationship with other elements.

[0069] The inference unit 14 uses the input type and data of the acquired file list as inference data to acquire information on the OS and architecture related to the element to be inferred. As a result, even if a CPE name 2221 capable of identifying the CVE 221 cannot be output as an inference result in a single inference, the inference unit 14 can narrow down the CPE name 2221 to be output by comparing it with the OS and architecture inferred from the file list. Therefore, the inference unit 14 can improve the accuracy of inference by performing inference on the element to be inferred multiple times while changing the inference data.

[0070] The inference unit 14 performs the above inference for visualized data items other than the CPE name using the machine learning model 103, and acquires visualized data related to the inference target item. Then, the inference unit 14 outputs the acquired visualized data to the correction unit 15.

[0071] In this way, the inference unit 14 infers information other than the part of the visualized data, using a part of the visualized data including configuration information, state information, evaluation information, and information indicating relationships with other related objects of the object, and the machine learning model 103, which is a trained large-scale language model. Furthermore, the inference unit 14 inputs the input data for inference generated by the data normalization unit 12 to the trained machine learning model 103 to obtain an inference result.

[0072] The correction unit 15 receives an input of visualized data that is an inference output by the inference unit 14. Next, the correction unit 15 compares the acquired inference output with the collected data 101. If similar data exists, the correction unit 15 corrects the visualized data of the inference output using the similar data included in the collected data 101. In this way, the correction unit 15 corrects the visualized data inferred by the inference unit 14 using the collected data 101. The correction unit 15 then outputs the corrected visualized data 30 as an inference result. In this way, the correction unit 15 can correct typos and other errors in the visualized data inferred by the inference unit 14 and output the correct answer, thereby enabling accurate testing.

[0073] 6 is a diagram illustrating an example of the correction process. For example, the correction unit 15 acquires a CPE name 141 included in visualized data, which is the inference output. Next, the correction unit 15 compares the CPE name 141 with each of the CPE names 2221 included in the collected data 101. For example, for each CPE name 2221, the correction unit 15 identifies one element that mismatches the inferred CPE name, and then performs a partial match search to determine whether the elements other than the mismatched element match. If there are no mismatched search results other than the one mismatched element, the correction unit 15 determines that similar data exists and corrects the CPE name, which is the inference result, so that it matches the CPE name 2221 included in the collected data 101. For example, the CPE name 142 differs from the CPE name 141 in character 143. Therefore, since the CPE name 141 and the CPE name 142 differ in one element portion including the character 143, the correcting unit 15 corrects the character 143 portion of the CPE name 141 to match the CPE name 142.

[0074] Furthermore, when creating the learning data 122 in FIG. 3 or the learning data 124 and 125 in FIG. 4, information (here, cpe:2.3:) that is deleted from the correct CPE name in order to improve the accuracy of inference is also corrected by comparing it with the collected data 101 in the correction process, and the finally output visualization data is output with the deleted information added.

[0075] (Complementation Process) Fig. 7 is a flowchart of the complementation process performed by the information complement device according to the first embodiment. Here, the complementation process includes a learning process and an inference process. Next, the flow of the complementation process performed by the information complement device 1 according to this embodiment will be described with reference to Fig. 7.

[0076] The data collection unit 11 collects object-related information including the CVE description 2211, the CPE name 2221, and the CPE title 2222 from the NVD 22 of the external system 2, etc., and generates the collected data 101 (step S1).

[0077] Next, the data normalization unit 12 performs preprocessing such as adding an input type to the collected data 101, selecting used parts, or linking related information to the collected data 101, thereby normalizing the collected data 101 and generating learning data 102 (step S2).

[0078] Next, the learning unit 13 acquires the machine learning model 21 as a pre-learning model. Then, the learning unit 13 causes the machine learning model 21 to learn using the training data 102, thereby generating the machine learning model 103 (step S3). This completes the learning phase.

[0079] Next, the process moves to the inference phase. The data normalization unit 12 acquires the inference data input by the user from the user terminal 3. The data normalization unit 12 then modifies the inference data by adding an input type to match the format of the training data 102 (step S4).

[0080] Next, the inference unit 14 obtains the inference data with the input type added from the data normalization unit 12, inputs it into the trained machine learning model 103, and performs inference by obtaining the visualized data output from the machine learning model 103 (step S5).

[0081] Next, the correction unit 15 compares the visualized data that is the inference output from the inference unit 14 with the collected data 101, and if similar data exists, corrects the visualized data that is the inference output using the similar data (step S6). Thereafter, the correction unit 15 outputs the corrected visualized data 30 as the inference result.

[0082] (Effects of Information Processing Device and Configuration Information Generation Process) As described above, the information supplementation device 1 according to this embodiment collects object-related information about each element, such as a product, a system, or a service, in the learning phase. Next, the information supplementation device 1 trains a pre-learning model using training data 102, which is collected data 101 that has been preprocessed to improve inference accuracy, to generate a machine learning model 103. In the inference phase, the information supplementation device 1 modifies the acquired inference data to match the format of the training data 102, and then performs inference using the trained machine learning model 103, correcting the inference output based on the collected data 101 to obtain an inference result.

[0083] This allows for incomplete visualization data to be supplemented by inference, improving the comprehensiveness of the visualization data. Therefore, even when collectable information is incomplete, license inspections, vulnerability inspections, and the like can be performed appropriately, improving the accuracy of the inspections.

[0084] Furthermore, even if the obtainable visualization data does not output information correctly, the accuracy of the visualization data can be confirmed by comparing it with visualization data output by separate inference, and the visualization data can be replaced with more accurate information, thereby obtaining more appropriate visualization data. This makes it possible to further improve the accuracy of license inspections, vulnerability inspections, etc., and to improve the accuracy of inspections.

[0085] The following describes the evaluation of visualization data complemented by the information complement device 1 according to this embodiment. Here, a case where a CPE is generated from the visualization data as the complemented data will be described. The evaluation method confirmed that the items required for vulnerability testing were correctly output for the CPE generated by the information complement device 1.

[0086] FIG. 8 is a diagram illustrating the evaluation procedure. The CPE includes, for example, information shown in the format 301 illustrated in FIG. 8 . When text is output using a typical large-scale language model (LLM), it is often possible to determine that the meaning is the same even if the text appears differently. In contrast, the information supplementation device 1 according to this embodiment makes it difficult to determine vulnerabilities unless the four items of type, vendor name, product name, and version are correctly output. In other words, even if the same information is output, if the order is incorrect, it is considered an incorrect answer. Therefore, here, a generated CPE is defined as correct if all four items of type, vendor name, product name, and version are correct. For example, as shown in Table 302, when a certain CPE is generated, if the type, vendor name, and product name are correct but the version is incorrect, it is considered an incorrect answer.

[0087] Here, fine tuning was performed on the T5 pre-trained model and the GPT2 pre-trained model obtained from Hugging Face, with the number of epochs set to 3. For fine tuning, 1.5 million pieces of training data 102, which are sentence-type data as shown in Figure 3, and 1.11 million pieces of training data 102, which are word list-type data as shown in Figure 4, were used.

[0088] Inference was performed using 1,000 pieces of data. The 1,000 pieces of data used for inference were not used for learning, and the same data was used for T5 and GPT2. FIG. 9 is a diagram showing an example of the evaluation results. In FIG. 9, perfect match represents a case where the inference output by the inference unit 14 is determined to be correct, and match after correction represents a case where the inference is determined to be correct after correction by the correction unit 15.

[0089] As shown in Figure 9, when GPT2 is used, the number of correct answers in inference improves as the number of data items used for fine tuning increases, both for exact matches and total correct answers. When T5 is used, both the exact matches and total correct answers peak when the number of data items used for fine tuning is around 20,000, and then fluctuates, showing signs of improvement in accuracy at 500,000 or more items. Regardless of which model is used, the maximum accuracy rate exceeds 80%.

[0090] In this way, the visualized data complemented by the information complement device 1 has sufficient accuracy, and by using this visualized data, it is possible to improve the accuracy of license inspections, vulnerability inspections, etc.

[0091] (Second Embodiment) Next, a second embodiment will be described. The information supplementation device 1 according to this embodiment is also represented by the block diagram of FIG. 1. The information supplementation device 1 according to this embodiment differs from the first embodiment in that a pre-training model is created by the device itself using training data 102. Details of the information supplementation device 1 according to this embodiment will be described below. In the following description, explanations of the operations of the same components as those in the first embodiment will be omitted.

[0092] In the information supplementation device 1 according to this embodiment, in creating a pre-learning model, learning is performed on language structures and patterns, rather than learning correct answer data as in fine tuning. Therefore, the data normalization unit 12 selects data that is likely to be used for inference, and further normalizes the data into a format that is easily recognizable as text data.

[0093] Specifically, the data normalization unit 12 removes insignificant parts of each piece of collected data 101 collected by the data collection unit 11 to generate training data. For example, the data normalization unit 12 removes extension regions in SBOM. More specifically, the data normalization unit 12 removes the ExternalRef part of SPDX (Software Package Data Exchange) and the properties part of Cyclone DX.

[0094] Furthermore, the data normalization unit 12 removes items that are written in a list of letters of the alphabet from the training data, specifically, the data normalization unit 12 removes ID numbers, software hash values, certificate signatures, and the like.

[0095] Furthermore, the data normalization unit 12 removes delimiters such as "{", "}", and "", and escape symbols such as "¥".

[0096] Then, the data normalization unit 12 connects the character strings of the main parts of each data with "." By performing each of the above pre-processing steps, the data normalization unit 12 generates training data 102 that has been normalized for pre-training.

[0097] The learning unit 13 has a sufficient amount of learning data 102 for generating a pre-trained model, and is able to use machine resources for generating the pre-trained model.

[0098] The learning unit 13 performs pre-learning, which is unsupervised learning, using the training data 102 normalized for pre-learning, to generate a pre-learning model. Then, the learning unit 13 performs fine tuning on the generated pre-learning model using the training data 102 normalized for fine tuning, to generate a machine learning model 103.

[0099] In this way, the learning unit 13 generates a pre-training model by performing unsupervised learning based on the training data 102 normalized for pre-training, or acquires a pre-training model from an external source. Then, the learning unit 13 generates a machine learning model 103 by performing supervised learning on the pre-training model based on the training data 102 normalized for fine tuning.

[0100] As described above, the information supplementation device 1 according to this embodiment creates a pre-training model by itself using the training data 102, and then uses the pre-training model to generate the machine learning model 103. This enables the information supplementation device 1 to use a pre-training model that is customized specifically for analyzing target data, and more appropriately generates the machine learning model 103 for inferring visualized data.

[0101] In this way, when the information supplementation device 1 can collect a sufficient amount of training data 102 and has machine resources available for generating a pre-trained model, it can perform pre-training and use a pre-trained model customized specifically for analyzing target data. On the other hand, when it is difficult to prepare a sufficient amount of training data 102 or machine resources for generating a pre-trained model, it is preferable to obtain a pre-trained model from an external source such as an external system 2.

[0102] (System Configuration, etc.) Furthermore, the components of each device shown in the drawings are conceptual functional units and do not necessarily have to be physically configured as shown. In other words, the specific form of distribution and integration of each device is not limited to that shown, and all or part of the devices can be functionally or physically distributed or integrated in any unit depending on various loads, usage conditions, etc. Furthermore, all or any part of the processing functions performed by each device can be realized by a CPU (Central Processing Unit) and a program analyzed and executed by the CPU, or can be realized as hardware using wired logic.

[0103] Furthermore, among the processes described in this embodiment, all or part of the processes described as being performed automatically can be performed manually, or all or part of the processes described as being performed manually can be performed automatically using a known method.In addition, the information including the processing procedures, control procedures, specific names, various data and parameters shown in the above documents and drawings can be changed as desired unless otherwise specified.

[0104] (Program) In one embodiment, the information supplementation device 1 can be implemented by installing an information processing program that executes the above information processing as package software or online software on a desired computer. For example, the information supplementation device 1 can be made to function by having the information processing device execute the above information processing program. The information supplementation device 1 referred to here includes desktop and notebook personal computers. In addition, the information supplementation device 1 also includes mobile communication terminals such as smartphones, mobile phones, and PHS (Personal Handy-phone Systems), as well as slate terminals such as PDAs (Personal Digital Assistants).

[0105] The information supplementation device 1 can also be implemented as a server device that provides services related to the above-mentioned information processing to a client terminal device used by a user. For example, the information supplementation device 1 is implemented as a server device that receives input of information obtained from an external source such as an external system 2 and provides services to supplement components. In this case, the server device may be implemented as a web server or as a cloud that provides services related to the above-mentioned information processing by outsourcing.

[0106] 10 is a diagram showing an example of a computer that executes an information complementation program. The computer 1000 includes, for example, a memory 1010 and a CPU 1020. The computer 1000 also includes a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These components are connected by a bus 1080.

[0107] The memory 1010 includes a read-only memory (ROM) 1011 and a random access memory (RAM) 1012. The ROM 1011 stores a boot program such as a basic input output system (BIOS). The hard disk drive interface 1030 is connected to a hard disk drive 1090. The disk drive interface 1040 is connected to a disk drive 1100. A removable storage medium such as a magnetic disk or optical disk is inserted into the disk drive 1100. The serial port interface 1050 is connected to a mouse 1110 and a keyboard 1120, for example. The video adapter 1060 is connected to a display 1130, for example.

[0108] The hard disk drive 1090 stores, for example, an OS 1091, an application program 1092, a program module 1093, and program data 1094. That is, an information processing program that defines each process of the information supplementation device 1, which has the same functions as the information supplementation device 1, is implemented as a program module 1093 in which computer-executable code is written. The program module 1093 is stored, for example, in the hard disk drive 1090. For example, the program module 1093 for executing processes similar to those of the functional configuration of the information supplementation device 1 is stored in the hard disk drive 1090. Note that the hard disk drive 1090 may be replaced with an SSD (Solid State Drive).

[0109] Furthermore, setting data used in the processing of the above-described embodiment is stored as program data 1094, for example, in the memory 1010 or the hard disk drive 1090. Then, the CPU 1020 reads the program module 1093 or the program data 1094 stored in the memory 1010 or the hard disk drive 1090 into the RAM 1012 as necessary, and executes the processing of the above-described embodiment.

[0110] The program module 1093 and program data 1094 may not necessarily be stored in the hard disk drive 1090, but may also be stored in a removable storage medium and read by the CPU 1020 via the disk drive 1100 or the like. Alternatively, the program module 1093 and program data 1094 may be stored in another computer connected via a network (such as a local area network (LAN) or a wide area network (WAN)). The program module 1093 and program data 1094 may then be read by the CPU 1020 from the other computer via the network interface 1070.

[0111] REFERENCE SIGNS LIST 1 Information supplement device 2 External system 3 User terminal 11 Data collection unit 12 Data normalization unit 13 Learning unit 14 Inference unit 15 Correction unit 21 Machine learning model 22 NVD 30 Visualization data 101 Collected data 102 Learning data 103 Machine learning model 221 CVE 222 CPE 2211 CVE description 2221 CPE name 2222 CPE title< / s>

Claims

1. A data collection unit that collects information related to an object and uses it as collected data; a data normalization unit that adds the data type as an input type to the collected data collected by the data collection unit to generate learning data; a learning unit that trains a large language model based on the learning data generated by the data normalization unit; an inference unit that infers information other than the part of the visualization data using a part of the visualization data including the configuration information, status information, evaluation information of the object, and information indicating the relationship with other related objects, and the trained large language model; and a correction unit that corrects the information of the visualization data inferred by the inference unit based on the collected data. An information complementation device characterized by comprising these components.

2. The learning unit generates a pre-trained model, which is a large language model, by performing unsupervised learning based on the learning data or obtains the pre-trained model from the outside, and performs supervised learning based on the learning data on the pre-trained model to generate the trained large language model. The information complementation device according to claim 1, characterized by this.

3. The data normalization unit generates input data with the input type added to a part of the visualization data based on the format of the learning data used for training the large language model, and the inference unit obtains an inference result by inputting the input data generated by the data normalization unit into the trained large language model. The information complementation device according to claim 1, characterized by this.

4. A computer executes a data collection process of collecting information related to an object and using it as collected data; a data normalization process of adding the data type as an input type to the collected data collected in the data collection process to generate learning data; a learning process of training a large language model based on the learning data generated in the data normalization process; an inference process of inferring information other than the part of the visualization data using a part of the visualization data including the configuration information, status information, evaluation information of the object, and information indicating the relationship with other related objects, and the trained large language model; and a correction process of correcting the information of the visualization data inferred in the inference process based on the collected data. An information complementation method characterized by this.