Search device, search method, and search program

Abstract

To provide a search device, search method, and search program that can appropriately retrieve information desired by a user from information, including variations in spelling, recorded within a repository. [Solution] The search device 1 includes a keyword reception unit 11 that receives search keywords for each item in a repository where information on multiple items is linked to each other, a keyword extension unit 12 that receives the specification of variations in notation for the search keyword and extends the search keyword, a similarity evaluation unit 13 that calculates an evaluation value by aggregating the similarity between the string stored in the item corresponding to the search keyword in the repository and each of the extended search keywords, and a result output unit 14 that ranks records where information on multiple items is linked based on a score obtained by summing the evaluation values ​​for each item and outputs them as search results.

Description

[Technical Field]

[0001] The present invention relates to an apparatus, method, and program for retrieving information stored in a repository. [Background technology]

[0002] Traditionally, repositories (databases) have been used to share information obtained from numerous information providers among multiple users. For example, information on product vulnerabilities in the supply chain, from procurement of components to sales, consists of the product's component information (Bill of Materials, BOM) and vulnerability information associated with each component. Non-patent document 1 proposes a method for aggregating information using a knowledge graph. By describing information in a specific format and linking multiple pieces of information to construct a knowledge graph, it becomes possible to obtain an overview of the supply chain, which would be difficult to do by simply checking individual pieces of information. Furthermore, it becomes possible to obtain related information linked to specific products or companies. [Prior art documents] [Patent Documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2015-203961 [Patent Document 2] Japanese Patent Publication No. 2017-201478 [Patent Document 3] Japanese Patent Publication No. 2020-91607 [Patent Document 4] Japanese Patent Publication No. 2024-75335 [Non-patent literature]

[0004] [Non-Patent Document 1] W. Zhang et al., "The construction of a domain knowledge graph and its application in supply chain risk analysis," IEEE International Conference on E-Business Engineering (ICEBE), 2019. [Non-Patent Document 2] T. Zhang et al., "BERTScore: Evaluating Text Generation with BERT," Proc. International Conference on Learning Representations, 2020. [Non-Patent Document 3] Y. Wu et al., "Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation," 2016, <https: / / arxiv.org / abs / 1609.08144v2>. [Non-Patent Document 4] J. Devlin et al., "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding," Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171-4186, 2019. [Non-Patent Document 5] GV Cormack et al., "Reciprocal rank fusion outperforms condorcet and individual rank learning methods," Proc. Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval, pp. 758-759, 2009. [Overview of the project] [Problems that the invention aims to solve]

[0005] Conventional technologies have proposed supply chain information management methods, but they required standardization of information format and notation. Generally, BOMs and related information may be provided by each supplier of the components that make up a product, resulting in inconsistent granularity and format, making it difficult to search and utilize this information.

[0006] For example, product information may list the manufacturer's name using different spellings, such as "ABC Corporation" or "A-B-C," for the same company. In particular, when different character sets are used, such as "ABC" and "A-B-C," it is difficult to recognize them as the same company using simple methods. Furthermore, if "ABC Corporation" has a group company with a name like "ABC Solutions Corporation," it is difficult to clearly distinguish between the two.

[0007] Document search technologies include, for example, Patent Documents 1 to 4. These technologies enable the search for documents related to information entered by the user. However, when information is described in multiple fields, as in general databases, and there are variations in the notation of the described information, it has been difficult for the user to obtain the desired search results.

[0008] An object of the present invention is to provide a search device, a search method, and a search program that can appropriately search for information desired by a user from information including notation variations recorded in a repository.

Means for Solving the Problems

[0009] A search device according to the present invention includes a keyword reception unit that receives a search keyword for each item with respect to a repository in which information of a plurality of items is associated with each other, a keyword expansion unit that receives a designation of a notation variation for the search keyword and expands the search keyword, a similarity evaluation unit that calculates an evaluation value obtained by aggregating the similarity between a character string stored in an item corresponding to the search keyword in the repository and each of the expanded search keywords, and a result output unit that ranks records in which information of the plurality of items is associated with each other according to a score obtained by summing the evaluation values for each item and outputs the result as a search result.

[0010] The keyword expansion unit may generate a notation variation of the search keyword by a generative AI.

[0011] The similarity evaluation unit may aggregate each of the search keyword and the character string into a vector and calculate the evaluation value based on the cosine similarity.

[0012] The similarity evaluation unit may convert each of the search keyword and the character string into a series of vectors and calculate the evaluation value by a function that evaluates the semantic similarity in natural language processing.

[0013] The similarity evaluation unit may calculate the maximum value of the similarity for each of the expanded search keywords as the evaluation value.

[0014] The result output unit may calculate the score by multiplying the evaluation value by a coefficient corresponding to the importance for each item and then summing them.

[0015] The search method according to the present invention is such that a computer receives, at a keyword reception unit, a search keyword for each item with respect to a repository in which information of a plurality of items is associated with each other, receives, at a keyword expansion unit, a designation of notation variation for the search keyword, expands the search keyword, calculates, at a similarity evaluation unit, an evaluation value obtained by aggregating the degrees of similarity between the character string stored in the item corresponding to the search keyword in the repository and each of the expanded search keywords, and ranks, at a result output unit, the records in which the information of the plurality of items is associated with each other by a score obtained by summing the evaluation values for each item, and outputs the result as a search result.

[0016] The search program according to the present invention is for causing a computer to function as the search device.

Advantages of the Invention

[0017] According to the present invention, the information desired by the user can be appropriately searched from the information including notation variations recorded in the repository.

Brief Description of the Drawings

[0018] [Figure 1] It is a diagram illustrating the configuration of a repository system for managing vulnerability information in an embodiment. [Figure 2] It is a block diagram showing the functional configuration of a search device in an embodiment. [Figure 3] It is a diagram illustrating a search screen presented by a search device in an embodiment. [Figure 4] It is a flowchart showing the flow of a search process by a search device in an embodiment.

Modes for Carrying Out the Invention

[0019] Hereinafter, an example of an embodiment of the present invention will be described. The search method of this embodiment evaluates the similarity with search keywords and searches for relevant information that matches the search conditions in repositories that include variations in notation and where the granularity and format of the information are not standardized. Here, we focus on hierarchically related sets of information, that is, sets of information that can be referenced as a starting point for related information. As an example, we will take vulnerability information in supply chains.

[0020] Figure 1 is a diagram illustrating the configuration of a repository system 100 for managing vulnerability information in this embodiment. The information handled in a repository can be broadly categorized into three types: product information, company information, and vulnerability information.

[0021] The information input unit 110 is equipped with input functions corresponding to each type of information, acquires information from the corresponding external system, and records it in the corresponding table of the information storage unit 120. The information storage unit 120 organizes tables for each type of information and stores each piece of information. Here, the information contained in each table is linked through identifiers such as IDs.

[0022] The information formatting unit 130 organizes, searches, and aggregates the information stored in each table of the information storage unit 120 and outputs it to an external system. In this embodiment, the information formatting unit 130 is configured to include the search device 1.

[0023] Product information is information that includes information about the products that make up the hardware, and for example, it includes a value that uniquely identifies the product (product information ID), the product name, the product model number, the manufacturer's company information (company information ID), the product's use or application (product use), and a list of products used in the product (parts list).

[0024] Corporate information includes information about the company that provides the hardware, and for example, it includes a value that uniquely identifies the company (corporate information ID), the company name, the company's nationality, and information about its subsidiaries, affiliates, business partners, etc. (related company list).

[0025] Vulnerability information includes known vulnerabilities related to hardware, and may include, for example, a value that uniquely identifies the vulnerability information (vulnerability information ID), a value indicating the degree of threat of the vulnerability (vulnerability threat level), the vulnerability registration date, the vulnerability update date, the conditions under which the vulnerability occurs (vulnerability conditions), the impact when the vulnerability is attacked (vulnerability impact), the source of the vulnerability information (vulnerability source), and a list of affected products (product list). Vulnerability information may also include information such as flags indicating that suspicions arose during inspection.

[0026] For example, product information such as "notebook PC" is associated with product information such as "CPU" and "SSD" as part of its component list, and further, the product information for "CPU" and "SSD" is associated with corresponding vulnerability information. When aggregating vulnerability information related to "notebook PCs," the vulnerability information associated with product information included in component lists such as "CPU" and "SSD" will be aggregated. In this case, if the "CPU" and "SSD" each have vulnerability information described in different ways, such as "causing malfunction at high temperatures," the search device 1 will simultaneously search for and output information of the same nature.

[0027] Figure 2 is a block diagram showing the functional configuration of the search device 1 in this embodiment. The search device 1 is an information processing device (computer) equipped with a control unit 10 and a storage unit 20, as well as various input / output interfaces.

[0028] The control unit 10 controls the entire search device 1 and realizes the functions of each functional unit described later by appropriately reading and executing various programs stored in the memory unit 20. The control unit 10 is not particularly limited, but may be a CPU.

[0029] The memory unit 20 is a storage area for various programs and data that enable the hardware group to function as a search device 1, and is not particularly limited to ROM, RAM, flash memory, hard disk drive (HDD), solid state drive (SSD), etc.

[0030] The control unit 10 includes a keyword reception unit 11, a keyword expansion unit 12, a similarity evaluation unit 13, and a result output unit 14.

[0031] The keyword reception unit 11 accepts search keywords for at least one item for each item in a repository where information from multiple items is linked to each other.

[0032] The keyword extension unit 12 accepts further requests for variations in spelling of the search keywords received by the keyword reception unit 11, and extends the search keywords. In this case, the keyword extension unit 12 may accept strings with variations in spelling from the user, or it may automatically generate variations in the spelling of search keywords using a generation AI.

[0033] Figure 3 shows an example of an input screen presented to the user by the search device 1 in this embodiment. For each item in the repository, such as company name, product name, and conditions under which vulnerabilities occur, users can specify keywords (search values) along with a coefficient (importance) to determine the search priority.

[0034] Furthermore, variations in spelling may be specified for each search item on the screen accessed via the settings button. In this case, the variations in spelling may be those specified by the user, or they may be those automatically generated using, for example, a generation AI. For example, by using a prompt such as, "Please output a list of 10 possible variations in the spelling of the original search keyword 'XXX'," multiple variations can be automatically generated.

[0035] Furthermore, when providing this function to users, an API (Application Programming Interface) may be used instead of a search screen. In this case as well, users can specify settings to the API that are equivalent to those used on the search screen.

[0036] The similarity evaluation unit 13 calculates an evaluation value that aggregates the similarity between the strings stored in the items corresponding to the search keywords in the repository and each of the extended search keywords. For example, the similarity evaluation unit 13 may aggregate the search keywords and the registry strings into vectors and calculate an evaluation value using cosine similarity. Alternatively, the similarity evaluation unit 13 may convert the search keywords and the registry strings into a series of vectors and calculate an evaluation value using a predetermined function that evaluates semantic similarity in natural language processing.

[0037] Here, the similarity evaluation unit 13 may calculate the maximum similarity value as the evaluation value as a method for aggregating the similarity for each of the extended search keywords, but is not limited to this, and any aggregation function such as the average may be used.

[0038] The results output unit 14 ranks records linked to information from multiple items based on a similarity score obtained by summing the evaluation values ​​for each item, and outputs them as search results. In this case, the result output unit 14 may calculate a similarity score by multiplying the evaluation value by a coefficient corresponding to the importance of each item and then summing them up.

[0039] Specifically, the similarity score between the search keywords and the information within the repository is calculated, for example, as follows:

[0040] First, let I be the set of search items (table column names), and let K be the set of keywords. Then, let k be the keyword corresponding to a certain search item i∈I. i This is denoted as (∈K). Keyword k iRegarding the pattern of notation variation, let there be n ki cases. The patterns of notation variation include those obtained by mechanically converting full-width and half-width alphabets, those extracted from dictionary data, those specified by the search user, those automatically generated using generative AI, etc. Also, when not considering notation variation, n = 1. For k i regarding the j-th (j ∈ [n ki ), let the string be k i,j .

[0041] Also, in the repository, when the information of each table is concatenated so as to include all search items, the information for one line is called a record. Let the set of records be R, and represent one line of it as r. Also, let the string corresponding to the search item i recorded in the record r of the repository be r i .

[0042] Let the function for evaluating the similarity between two strings be E. The function E returns a value between 0 and 1, and the higher the value, the more similar the meanings of the two are. Let the coefficient (importance) for determining the priority of the search corresponding to the search item i be ε i . This ε i can be input by the user from the input screen of FIG. 3 provided by the search system of the repository, or the initial value may be given by the search system. At this time, the similarity score S(r) is determined as follows.

Equation

[0043] As an implementation of the function E, for example, the following can be mentioned. ·Indicators such as BM25, ROUGE for evaluating the similarity between two given strings. ·Vectorize the two given strings using an Embedding model respectively, and the vector sequence Vki,j and V ri For this purpose, we use the cosine similarity of the vectors v1 and v2, which are obtained by aggregating (for example, averaging) them. The two given strings are vectorized using the Embedding model, and the sequence of vectors V ki,j and V ri For this purpose, a function such as the BERT Score (see, for example, Non-Patent Document 1) is used to evaluate semantic similarity in natural language processing. • Use a value (e.g., the mean) that combines the calculation results of multiple methods, including the above, for evaluating similarity.

[0044] The Embedding model takes a string as input and outputs a sequence of vectors corresponding to that string. When the Embedding model processes at the token level, the sequence of vectors represents the sequence of vectors corresponding to each token; when the Embedding model processes the entire string, the sequence consists of only one vector. Examples of Embedding models include the WordPiece model (see, for example, Non-Patent Document 2), which vectorizes tokens (sequences of characters, the smallest units that make up a sentence), and the BERT model (see, for example, Non-Patent Document 3), which vectorizes sequences of tokens (i.e., entire strings).

[0045] The result output unit 14 calculates a similarity score S(r) for all records r∈R for which the search keyword is specified, and obtains a list of search results by sorting the obtained similarity scores S(r) in descending order. The result output unit 14 then assigns ranks to the obtained list from the top of the list and outputs them as search results.

[0046] Furthermore, the result output unit 14 may re-evaluate the rankings by combining them with search results from conventional methods such as exact keyword matching or partial keyword matching. For example, evaluation methods can be appropriately adopted, such as narrowing the results to only those records found using other methods, or sorting the results in descending order of the score obtained by summing the reciprocals of the ranks from each method, using the records and their ranks from the search results obtained using other methods (see, for example, Non-Patent Document 4).

[0047] Figure 4 is a flowchart showing the flow of the search process by the search device 1 in this embodiment.

[0048] In step S1, the keyword reception unit 11 receives search keywords for each item in the information stored in the repository.

[0049] In step S2, the keyword extension unit 12 accepts the specification of variations in spelling for the search keyword and extends the search keyword.

[0050] In step S3, the similarity evaluation unit 13 calculates an evaluation value that aggregates the similarity between the strings stored in the items corresponding to the search keywords in the repository and each of the extended search keywords.

[0051] In step S4, the result output unit 14 ranks records linked to information from multiple items based on a similarity score obtained by summing the evaluation values ​​for each item, and outputs them as search results.

[0052] According to this embodiment, the search device 1 expands the search keywords to include variations in spelling, calculates an evaluation value that aggregates the similarity to each record in the repository, and outputs records ranked by a similarity score obtained by summing the evaluation values ​​for each item. Therefore, the search device 1 can appropriately retrieve information desired by the user, such as specific products and similar products, related to the specified search keyword, from the information recorded in the repository, including variations in spelling.

[0053] Furthermore, by automatically generating variations in spelling using a generation AI, the search device 1 can efficiently expand search keywords without necessarily requiring user input, and can appropriately search for the information desired by the user.

[0054] Furthermore, by aggregating strings into vectors, the search device 1 can evaluate the similarity between search keywords and records in the repository using metrics such as cosine similarity. Alternatively, the search device 1 can convert strings into a sequence of vectors and use a function that evaluates semantic similarity in natural language processing. This allows the search device 1 to appropriately automate the information retrieval process.

[0055] When the search device 1 aggregates the similarity for each of the expanded search keywords, it uses the maximum value as the evaluation value, thereby appropriately extracting information that may be related to the search keywords. Furthermore, the search device 1 calculates a similarity score by multiplying the evaluation value by a coefficient corresponding to the importance of each item and then summing them up, thereby prioritizing the search for information desired by the user. In addition, the user can adjust the coefficient while reviewing the search results, for example, to obtain the desired search results.

[0056] Furthermore, this embodiment allows users to appropriately search for desired information from a large amount of accumulated data, thereby contributing to Goal 9 of the United Nations-led Sustainable Development Goals (SDGs), "Build resilient infrastructure, promote sustainable industrialization and foster innovation."

[0057] Although embodiments of the present invention have been described above, the present invention is not limited to the embodiments described above. Furthermore, the effects described in the embodiments described above are merely a list of the most preferred effects resulting from the present invention, and the effects of the present invention are not limited to those described in the embodiments.

[0058] The search method by search device 1 is implemented by software. When implemented by software, the programs constituting this software are installed on an information processing device (computer). These programs may be distributed to users by being recorded on removable media such as a CD-ROM, or by being downloaded to the user's computer via a network. Furthermore, these programs may be provided to the user's computer as a web service via a network without being downloaded. [Explanation of Symbols]

[0059] 1. Search device 10 Control Unit 11 Keyword Reception Department 12 Keyword Extension Section 13 Similarity Evaluation Unit 14. Result Output Section 20 Memory section

Claims

1. A keyword reception unit that receives search keywords for each item in a repository where information on multiple items is linked to each other, A keyword extension unit accepts the specification of variations in spelling for the aforementioned search keyword and extends the aforementioned search keyword, A similarity evaluation unit calculates an evaluation value that aggregates the similarity between the strings stored in the items corresponding to the search keywords in the repository and each of the extended search keywords. A search device comprising: a result output unit that ranks records linked to information on multiple items based on a score obtained by summing the evaluation values ​​for each of the items, and outputs them as search results.

2. The search device according to claim 1, wherein the keyword expansion unit generates variations in the notation of the search keyword using generating AI.

3. The search device according to claim 1 or 2, wherein the similarity evaluation unit aggregates the search keyword and the string into vectors and calculates the evaluation value using cosine similarity.

4. The search device according to claim 1 or 2, wherein the similarity evaluation unit converts each of the search keyword and the string into a sequence of vectors and calculates the evaluation value using a function that evaluates semantic similarity in natural language processing.

5. The search device according to claim 1 or 2, wherein the similarity evaluation unit calculates the maximum value of the similarity for each of the expanded search keywords as the evaluation value.

6. The search device according to claim 1 or claim 2, wherein the result output unit multiplies the evaluation value by a coefficient corresponding to the importance of each item, and then sums them up to calculate the score.

7. Computers The keyword reception unit receives search keywords for each item in a repository where information on multiple items is linked to each other. The keyword extension unit accepts the specification of variations in spelling for the search keyword, and extends the search keyword. The similarity evaluation unit calculates an evaluation value by aggregating the similarity between the strings stored in the items corresponding to the search keywords in the repository and each of the extended search keywords. A search method in which, in the results output unit, records linked to information on multiple items are ranked by a score obtained by summing the evaluation values ​​for each item, and output as search results.

8. A search program for causing a computer to function as a search device according to claim 1 or claim 2.

Images