Similarity determination device, similarity determination system, and similarity determination method

Abstract

To provide similarity determination means capable of comparing sentence similarity at high speed and with high accuracy.SOLUTION: Provided is a similarity determination device, including: a keyword management database that stores a set of first domain keywords corresponding to a first domain; a domain graph database that stores a first domain graph generated based on the set of first domain keywords for a first sentence corresponding to the first domain; a graph creation part that generates a second domain graph for a second sentence corresponding to the first domain on the basis of the first set of domain keywords; and a similarity determination part that generates a comparison result indicating similarity between the first domain graph and the second domain graph by comparing the first domain graph with the second domain graph.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a similarity determination device, a similarity determination system, and a similarity determination method. [Background technology]

[0002] In recent years, cyber attacks have been evolving year by year due to the discovery of new vulnerabilities and the emergence of new attack methods. Under these circumstances, issues such as whether information systems have defensive capabilities against cyber attacks and the extent of damage that may occur when a cyber attack occurs have become important concerns for society and organizations.

[0003] For this reason, for example, companies and other organizations are required to obtain various reports on newly discovered vulnerabilities and cyber attacks (hereinafter referred to as cyber events) in order to understand the risks to their business, and to take appropriate measures based on these reports.

[0004] In order to take appropriate measures against cyber events that may pose a risk, it is desirable to determine whether a reported cyber event has already been dealt with or is currently being dealt with. If the reported cyber event is similar (or identical) to a cyber event for which measures have already been developed, there is no need to develop measures again, which can save time and resources required for developing measures.

[0005] One way to determine whether a cyber event has already been addressed or is currently being addressed is to determine whether there are other reports that are highly similar to the report reporting the vulnerability or cyber attack.

[0006] Conventionally, there have been several proposals for comparing sentences and determining the degree of similarity. For example, Rettinger et al. (Non-Patent Document 1) states that "Evaluating document relevance is at the core of many applications, such as document retrieval and recommendation. Most similarity approaches operate on word distribution-based document representations. These approaches are computationally fast, but suffer from problems when documents are in different languages, vocabulary, or types, and ignore the rich relational knowledge available in knowledge graphs. On the other hand, graph-based document models can leverage valuable knowledge about relationships between entities, but graph manipulation is resource-intensive, making similarity evaluation infeasible for many applications. This paper presents an efficient semantic similarity approach that exploits explicit hierarchical and cross-sectional relationships. In our experiments, we show that (i) our approach's similarity measure correlates significantly more highly with human perception of document similarity than comparable measures, (ii) this holds true even for short documents with few annotations, and (iii) document similarity can be computed more efficiently compared to other graph traversal-based approaches." [Prior art documents] [Non-patent literature]

[0007] [Patent Document 1] Paul C., Rettinger A., ​​Mogadala A., Knoblock CA, Szekely P. (2016) Efficient Graph-Based Document Similarity. In: Sack H., Blomqvist E., d'Aquin M., Ghidini C., Ponzetto S., Lange C. (eds) The Semantic Web. Latest Advances and New Domains. ESWC 2016. Lecture Notes in Computer Science, vol 9678. Springer, Cham. https: / / doi.org / 10.1007 / 978-3-319-34129-3_21 Summary of the Invention [Problem to be solved by the invention]

[0008] The above-mentioned non-patent document 1 describes a means for determining the similarity of sentences by utilizing hierarchical and cross-sectional relationships in sentences and using a graph traversal means.

[0009] However, in the method described in Non-Patent Document 1, the entire sentence is represented as a graph, so the longer the sentence, the larger the graph becomes, and there is a problem that more computing resources and time are required to store and process the graph. Therefore, when the method described in Non-Patent Document 1 is applied to, for example, a report on a cyber event, the time it takes for the comparison judgment results to be output may be delayed depending on the size of the graph, which may delay the development of countermeasures against the cyber event.

[0010] Therefore, an object of the present disclosure is to provide a similarity determination means capable of performing high-speed and highly accurate comparison of sentence similarity by reducing the scale of the graph representation corresponding to the sentence while maintaining relationship information between elements in the sentence. [Means for solving the problem]

[0011] In order to solve the above problems, a representative similarity determination device of the present invention includes a keyword management database that stores a set of first domain keywords corresponding to a first domain, a domain graph database that stores a first domain graph generated for a first sentence corresponding to the first domain based on the set of first domain keywords, a graph generation unit that generates a second domain graph for a second sentence corresponding to the first domain based on the set of first domain keywords, and a similarity determination unit that compares the first domain graph with the second domain graph to generate a comparison result indicating the similarity between the first domain graph and the second domain graph. Effect of the Invention

[0012] According to the present disclosure, it is possible to provide a similarity determination means capable of performing high-speed and highly accurate text similarity comparison by reducing the scale of the graph representation corresponding to the text while maintaining relationship information between elements in the text. Other objects, configurations and effects will become apparent from the following description of the preferred embodiment of the invention. [Brief description of the drawings]

[0013] [Figure 1] FIG. 1 illustrates a computer system for implementing an embodiment of the present disclosure. [Diagram 2] FIG. 2 is a diagram illustrating an example of a configuration of a similarity determination system according to an embodiment of the present disclosure. [Diagram 3] FIG. 3 is a diagram showing the flow of a pre-graph generation process and the flow of a graph comparison process in the similarity determination process according to an embodiment of the present disclosure. [Figure 4] FIG. 4 is a diagram illustrating details of the flow of a pre-graph generation process according to an embodiment of the present disclosure. [Diagram 5] FIG. 5 is a diagram illustrating an example of a domain graph and a normalized domain graph according to an embodiment of the present disclosure. [Figure 6] FIG. 6 is a diagram showing details of the flow of the graph comparison process according to the embodiment of the present disclosure. [Figure 7] FIG. 7 is a diagram illustrating a flow of an update process for updating domain keywords and domain graphs according to an embodiment of the disclosure. [Figure 8] FIG. 8 is a diagram showing an example of a keyword management screen according to an embodiment of the present disclosure. [Figure 9] FIG. 9 is a diagram illustrating an example of a text management screen according to an embodiment of the present disclosure. [Figure 10] FIG. 10 is a diagram illustrating an example of a flow of a multi-domain management process according to an embodiment of the present disclosure. [Figure 11] FIG. 11 is a diagram illustrating a specific example of similarity determination for two domain graphs according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0014] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. Note that the present invention is not limited to the embodiment. In addition, in the description of the drawings, the same parts are denoted by the same reference numerals. In addition, terms such as "first," "second," and "third" may be used to describe various elements or components in this disclosure, but it will be understood that these elements or components should not be limited by these terms. These terms are used only to distinguish one element or component from another element or component. Thus, a first element or component discussed below can also be referred to as a second element or component without departing from the teachings of the inventive concept.

[0015] First, referring to Fig. 1, a computer system 100 for implementing an embodiment of the present disclosure will be described. The mechanisms and devices of the various embodiments disclosed herein may be applied to any suitable computing system. The main components of the computer system 100 include one or more processors 102, memory 104, a terminal interface 112, a storage interface 113, an I / O (input / output) device interface 114, and a network interface 115. These components may be interconnected via a memory bus 106, an I / O bus 108, a bus interface unit 109, and an I / O bus interface unit 110.

[0016] Computer system 100 may include one or more general purpose programmable central processing units (CPUs) 102A and 102B, collectively referred to as processors 102. In some embodiments, computer system 100 may include multiple processors, while in other embodiments, computer system 100 may be a single CPU system. Each processor 102 executes instructions stored in memory 104 and may include an on-board cache.

[0017] In one embodiment, memory 104 may include random access semiconductor memory, storage devices, or storage media (either volatile or non-volatile) for storing data and programs. Memory 104 may store all or part of the programs, modules, and data structures that implement the functions described herein. For example, memory 104 may store a similarity determination application 150. In one embodiment, similarity determination application 150 may include instructions or descriptions to execute on processor 102 the functions described below.

[0018] In some embodiments, the similarity determination application 150 may be implemented in hardware via semiconductor devices, chips, logic gates, circuits, circuit cards, and / or other physical hardware devices instead of or in addition to a processor-based system. In some embodiments, the similarity determination application 150 may include data other than instructions or descriptions. In some embodiments, a camera, sensor, or other data input device (not shown) may be provided to communicate directly with the bus interface unit 109, the processor 102, or other hardware of the computer system 100.

[0019] Computer system 100 may include a bus interface unit 109 that provides communication between processor 102, memory 104, display system 124, and I / O bus interface unit 110. I / O bus interface unit 110 may couple to an I / O bus 108 for transferring data to and from various I / O units. I / O bus interface unit 110 may communicate via I / O bus 108 with multiple I / O interface units 112, 113, 114, and 115, also known as I / O processors (IOPs) or I / O adapters (IOAs).

[0020] Display system 124 may include a display controller, a display memory, or both. The display controller may provide video, audio, or both data to display device 126. Computer system 100 may also include one or more sensors or other devices configured to collect data and provide the data to processor 102.

[0021] For example, computer system 100 may include biometric sensors to collect heart rate data, stress level data, etc., environmental sensors to collect humidity data, temperature data, pressure data, etc., and motion sensors to collect acceleration data, movement data, etc. Other types of sensors may also be used. Display system 124 may be connected to a display device 126, such as a standalone display screen, a television, a tablet, or a handheld device.

[0022] The I / O interface unit provides the ability to communicate with various storage or I / O devices. For example, the terminal interface unit 112 may be attached to user I / O devices 116, such as user output devices, such as a video display, a television with speakers, and user input devices, such as a keyboard, a mouse, a keypad, a touchpad, a trackball, buttons, a light pen, or other pointing device. A user may use a user interface to enter input data or instructions to the user I / O devices 116 and the computer system 100, and receive output data from the computer system 100, by manipulating the user input devices. The user interface may be displayed on a display, played through speakers, or printed via a printer, for example, via the user I / O devices 116.

[0023] Storage interface 113 allows attachment of one or more disk drives or direct access storage device 117 (usually a magnetic disk drive storage device, but may be an array of disk drives or other storage devices configured to appear as a single disk drive). In some embodiments, storage device 117 may be implemented as any secondary storage device. Contents of memory 104 may be stored in storage device 117 and retrieved from storage device 117 as needed. I / O device interface 114 may provide an interface to other I / O devices such as printers, fax machines, etc. Network interface 115 may provide a communications path to allow computer system 100 and other devices to communicate with each other. This communications path may be, for example, network 130.

[0024] In one embodiment, computer system 100 may be a device that receives requests from other computer systems (clients) without a direct user interface, such as a multi-user mainframe computer system, a single-user system, or a server computer, etc. In other embodiments, computer system 100 may be a desktop computer, a portable computer, a laptop, a tablet computer, a pocket computer, a telephone, a smartphone, or any other suitable electronic device.

[0025] Next, a similarity determination system according to an embodiment of the present disclosure will be described with reference to FIG.

[0026] 2 is a diagram illustrating an example of a configuration of a similarity determination system 200 according to an embodiment of the present disclosure. The similarity determination system 200 generates a comparison result indicating the similarity between a plurality of sentences, 2, the similarity determination system 200 includes a similarity determination device 210, a communication network 250, and a user terminal 260. The similarity determination device 210 and the user terminal 260 may be connected to each other via the communication network 250.

[0027] The similarity determination device 210 is a device for determining the similarity of sentences, and as shown in FIG. In one embodiment, the similarity determination device 210 may be implemented by the computer system 100 shown in FIG.

[0028] The memory 220 is for implementing the function of the similarity determination means according to the embodiment of the present disclosure. It may be a memory for storing a similarity determination application 150. This similarity determination application 150 may include processing instructions for implementing the functions of software modules such as a graph generating unit 221, a similarity determining unit 222, and an updating unit 223, as shown in FIG.

[0029] The graph generation unit 221 is a functional unit for generating a domain graph corresponding to a target sentence. In the present disclosure, the "domain graph" may be a directed acyclic graph in which unique domain keywords in the target sentence are represented as nodes and domain keywords are represented as edges. In addition, in the present disclosure, the "domain" refers to a specific field or topic, and may include any topic, such as "famous physicists," "automobiles," and "DoS attacks." By comparing domain graphs generated for different sentences, the similarity between the sentences can be determined. The details of the functions of the graph generating unit 221 will be described later, and therefore will not be described here.

[0030] The similarity determination unit 222 is a functional unit for determining the similarity between a plurality of sentences using the domain graph generated by the graph generation unit 221, and outputting a comparison result indicating the determined similarity. The function of the similarity determination unit 222 will be described in detail later, and therefore will not be described here.

[0031] The update unit 223 is a functional unit for adding new domain keywords and deleting existing domain keywords to the domain keywords stored in the keyword management DB 231 described later, and for updating the domain graphs stored in the domain graph DB 232 based on changes to the domain keywords. The details of the function of the update unit 223 will be described later, and therefore will not be described here.

[0032] The storage unit 230 is a storage area that accommodates a database (hereinafter, "DB") for storing various information according to an embodiment of the present disclosure, and may include a keyword management DB 231 and a domain graph DB 232 as shown in FIG.

[0033] The keyword management DB 231 is a database for storing a set of domain keywords according to an embodiment of the present disclosure. In the present disclosure, a "domain keyword" is a word that is particularly important in a particular domain. These domain keywords may be selected by a user (e.g., a user of a user terminal 260, described later) who requests a comparison result indicating the similarity between sentences. As an example, in the domain of "particle physics," "Higgs particle" may be selected as a domain keyword. In an embodiment, the keyword management DB 231 may store a set of domain keywords corresponding to various different domains. As described later, these domain keywords are used in generating a domain graph according to an embodiment of the present disclosure.

[0034] The domain graph DB 232 is a database for storing domain graphs generated by the graph generating unit 221. As described later, a domain graph (e.g., a first domain graph) stored in the domain graph DB 232 is used as a comparison target for a newly generated domain graph (e.g., a second domain graph).

[0035] The processor 244 is a processing unit for executing processing instructions that define the functions of each functional unit of the similarity determination application 150 stored by the memory 220 .

[0036] The input / output unit 246 is a functional unit for receiving information input to the similarity determination device 210 and outputting information such as a comparison result generated by the similarity determination device 210. In an embodiment, the input / output unit 246 may include, for example, a keyboard, a mouse, a display for displaying a GUI (Graphical User Interface), and the like.

[0037] Communications network 250 may include, for example, a local area network (LAN), a wide area network (WAN), a satellite network, a cable network, a WiFi network, or any combination thereof.

[0038] The user terminal 260 is a terminal device that can be used by a user of the similarity determination device 210. By using the user terminal 260, the user can request a comparison result indicating the similarity between sentences, select a domain keyword, and confirm the comparison result, for example, by using a GUI provided by the input / output unit 246. As an example, the user terminal 260 may include, for example, a smartphone, a smart watch, a tablet, a personal computer, etc., and is not particularly limited. For convenience of explanation, FIG. 2 illustrates an example of a configuration including one user terminal 260, but the number of user terminals 260 is not particularly limited.

[0039] According to the similarity determination device 210 described above, it is possible to provide a similarity determination means capable of high-speed and high-precision comparison of sentence similarity by reducing the scale of the graph representation corresponding to the sentence while maintaining relationship information between elements in the sentence.

[0040] Next, an overall flow of a similarity determination process according to an embodiment of the present disclosure will be described with reference to FIG.

[0041] As described above, one aspect of the embodiment of the present disclosure relates to a process for determining the similarity between a plurality of sentences. Moreover, this similarity determination process mainly includes a pre-graph generation process 400 and a graph comparison process 600. Fig. 3 is a diagram showing the flow of the pre-graph generation process 400 and the flow of the graph comparison process 600 in the similarity determination process according to the embodiment of the present disclosure.

[0042] The pre-graph generation process 400 is a process for generating in advance a domain graph (for example, a normalized first domain graph) that serves as a reference when comparing sentences based on an existing sentence (first sentence), and storing the domain graph in the domain graph DB 232. More specifically, as shown in Fig. 3, in the pre-graph generation process 400, the graph generation unit 221 generates a normalized first domain graph 415 that shows the first sentence 412 in a graph format using the first sentence 412 that corresponds to the first domain and a first domain keyword that is stored in the keyword management DB 231 and corresponds to the first domain, and stores the normalized first domain graph in the domain graph DB 232.

[0043] The graph comparison process 600 is a process for generating a new domain graph (e.g., a normalized second domain graph) corresponding to a target sentence (second sentence) and comparing it with an existing domain graph (e.g., the normalized first domain graph generated by the pre-graph generation process 400 shown in FIG. 4) to generate a comparison result indicating the similarity between the normalized first domain graph and the normalized second domain graph.

[0044] 3, in the graph comparison process 600, the graph generation unit 221 generates a normalized second domain graph 425 showing the second sentence 422 in a graph format, using the second sentence 422 corresponding to the first domain and a first domain keyword stored in the keyword management DB 231 and corresponding to the first domain. After that, the similarity determination unit 222 compares the normalized first domain graph 415 stored in the domain graph DB 232 with the newly generated normalized second domain graph 425, thereby generating a comparison result indicating the similarity between the normalized first domain graph and the normalized second domain graph.

[0045] Note that, although the general flows of the pre-graph generation process 400 and the graph comparison process 600 included in the similarity determination process in the embodiment of the present disclosure have been described above, the details of these processes will be described later, and therefore will not be described here.

[0046] Next, details of the advance graph generation process according to the embodiment of the present disclosure will be described with reference to FIG.

[0047] 4 is a diagram showing details of the flow of the pre-graph generation process 400 according to an embodiment of the present disclosure. As described above, the pre-graph generation process 400 is a process for generating a normalized first domain graph in advance and storing it in the domain graph DB 232, and is mainly performed by the graph generation unit 221 of the similarity determination device 210.

[0048] First, in step S410, the graph generation unit 221 extracts a set of domain keywords corresponding to the domain of the first sentence 412 from the keyword management DB 231, and identifies a subset of the domain keywords contained in the first sentence 412 from the extracted set of domain keywords.

[0049] As an example, if the first text 412 relates to the domain of "physics", the graph generating unit 221 extracts a set of domain keywords corresponding to "physics" from the keyword management DB 231. If this set of domain keywords includes "atom", "gravity", "neutron", "electric charge", "electron" and "uncertainty principle", the graph generating unit 221 searches for and identifies these domain keywords in the first text 412. For example, if only "atom", "neutron", and "electron" appear in the first text 412 out of the set of domain keywords, the graph generating unit 221 identifies "atom", "neutron", and "electron" as a subset of the domain keywords.

[0050] Next, in step S420, the graph generator 221 generates nodes based on the subset of domain keywords identified in step S410. The node here is a data structure for representing the subset of domain keywords in a domain graph, which will be described later. In one embodiment, the graph generator 221 may create a node for each unique domain keyword of the subset of domain keywords identified in the first sentence 412. As an example, when "atom", "neutron", and "electron" are identified as the subset of domain keywords, the graph generator 221 may generate a set of nodes including nodes corresponding to "atom", "neutron", and "electron", respectively.

[0051] Next, in step S430, the graph generation unit 221 calculates a node score for each node generated in step S420. In this disclosure, a "node score" is a quantitative measure that indicates the relative importance in a sentence of a domain keyword corresponding to a specific node; the higher the importance, the higher the node score. In one embodiment, the graph generation unit 221 may calculate the node score of a specific node based on the number of times that a domain keyword corresponding to the node appears in the first sentence 412 (the number of occurrences). In this case, the graph generation unit 221 calculates the node score N of each node using the following formula 1: i (N1, N2, ... N T ) may be calculated.

number

[0052] Next, in step S440, the graph generation unit 221 generates edges indicating the relationships between the sets of nodes generated in step S420. The edges here are data structures for expressing the relationships between the nodes in a domain graph. In one embodiment, the graph generation unit 221 may create a knowledge graph corresponding to the first sentence in advance, and generate edges indicating the relationships between the sets of nodes based on the knowledge graph. In general, a knowledge graph is a data structure that systematically links various knowledge and represents them in a graph structure. Here, the means for generating the knowledge graph is not particularly limited, and any existing means such as natural language processing or neural networks may be used.

[0053] Next, in step S450, the graph generation unit 221 calculates an edge weight for each edge generated in step S440. In this disclosure, the "edge weight" is a measure that quantitatively indicates the relevance of domain keywords corresponding to two nodes connected by an edge in a domain graph, and the higher the relevance of the domain keywords, the higher the edge weight. In one embodiment, the graph generation unit 221 may calculate the edge weight based on a knowledge graph that is created in advance based on the first sentence. In this case, the graph generation unit 221 may calculate the edge weight based on the shortest distance (number of connections) in the knowledge graph between domain keywords corresponding to two nodes. In one embodiment, the graph generation unit 221 calculates the edge weight E of the edge between node i and node j using the following formula 2: ij may be calculated.

number

[0054] Next, in step S460, the graph generation unit 221 generates a first domain graph using the set of nodes generated in step S420, the node scores calculated in step S430, the set of edges generated in step S440, and the edge weights calculated in step S450. This first domain graph is a directed acyclic graph in which a subset of domain keywords identified in the first document 412 is represented as nodes, and the domain keywords are represented as edges. Each node in this first domain graph is associated with a node score indicating the importance of the domain keyword corresponding to the node, and each edge is associated with an edge weight indicating the degree of relevance between the nodes connected by the edge.

[0055] Next, in step S470, the graph generation unit 221 generates a normalized first domain graph by normalizing the first domain graph 415 generated in step S460. More specifically, the graph generation unit 221 may generate the normalized first domain graph by normalizing the node score of each node and the edge score of each edge in the normalized first domain graph. Here, the graph generation unit 221 generates the normalized node score NS i is calculated based on the following formula 3, and the normalized edge weight NE ij may be calculated based on Equation 4 below.

number

number

[0056] Next, in step S480, the graph generating unit 221 stores the normalized first domain graph generated in step S470 in the domain graph DB 232 in association with the first sentence 412 and the subset of domain keywords identified in step S410.

[0057] By performing the above-described pre-graph generation process 400 on various sentences, domain graphs corresponding to many different sentences can be prepared in advance and stored in the domain graph DB 232. As described below, the normalized first domain graph generated by the pre-graph generation process 400 is used to compare with a normalized second domain graph generated based on a second sentence in a graph comparison process 600 described below. Therefore, by performing the pre-graph generation process 400 on various sentences and preparing many domain graphs for comparison, a wider range of similarity determinations becomes possible, and it becomes easier to identify a first sentence that has a high similarity to a second sentence.

[0058] Next, a domain graph and a normalized domain graph according to an embodiment of the present disclosure will be described with reference to FIG.

[0059] FIG. 5 is a diagram illustrating an example of a domain graph 510 and a normalized domain graph 520 according to an embodiment of the present disclosure.

[0060] As described above, the domain graph in the present disclosure may be a directed acyclic graph in which unique domain keywords in a target sentence are represented as nodes and connections between domain keywords are represented as edges.

[0061] As an example, as shown in FIG. 5, a domain graph 510 may include a number of nodes, such as node "i", node "i+1", node "i+x", etc. Additionally, each node in the domain graph is associated with a node score indicating the relative importance of that node, and each edge is associated with an edge weight indicating the relevance of the nodes connected by that edge. For example, node "i" is associated with a node score "Ni", and the edge connecting node "i" and node "i+1" is associated with an edge weight "E i,i+1 " is associated with an edge weight of ".

[0062] Moreover, by normalizing each node score and edge weight in the domain graph 510, a normalized domain graph 520 can be generated. As shown in FIG. 5, each node in the normalized domain graph 520 is associated with a normalized node score, and each edge is associated with a normalized edge weight. For example, node "i" is associated with a normalized node score "NSi", and the edge connecting node "i" and node "i+1" is associated with a normalized node score "NSi", and the edge connecting node "i" and node "i+1" is associated with a normalized edge weight "NSi". i,i+1 " is associated with an edge weight of ".

[0063] In this way, by representing a sentence as a domain graph, it becomes possible to determine similarity with higher accuracy. As described above, each node in the domain graph is associated with a node score indicating the importance of the domain keyword corresponding to the node in the sentence, and each edge in the domain graph is associated with an edge weight indicating the relevance of the domain keyword corresponding to the node connected by the edge. This makes it possible to reduce the size of the domain graph while maintaining semantic information related to particularly important keywords in the sentence, even if the entire sentence is not represented as a domain graph. Furthermore, by generating a normalized domain graph in which each node score and edge weight in the domain graph is normalized, it is possible to prevent a decrease in similarity judgment due to differences in the size of the domain graphs when comparing domain graphs.

[0064] Next, details of the graph comparison process according to the embodiment of the present disclosure will be described with reference to FIG.

[0065] 6 is a diagram showing details of the flow of the graph comparison process 600 according to the embodiment of the present disclosure. As described above, the graph comparison process 600 according to the embodiment of the present disclosure is a process for generating a new domain graph (e.g., a normalized second domain graph) corresponding to a target sentence (second sentence) and comparing it with an existing domain graph (e.g., a normalized first domain graph generated by the pre-graph generation process 400 shown in FIG. 4) to generate a comparison result 685 indicating the similarity between the normalized first domain graph and the normalized second domain graph. The graph comparison process 600 is mainly performed by the graph generation unit 221 and the similarity determination unit 222. In the graph comparison process 600 described below, the process of generating a normalized second domain graph corresponding to the second sentence (steps S610 to S670) is substantially similar to the pre-graph generation process 400 shown in FIG. 4, so repeated explanations will be omitted.

[0066] First, in step S605, the input / output unit 246 receives an input including the second text 422 and information indicating the domain name of the second text 422 from the user terminal 260. The second text 422 may be the same as the first text 412 shown in FIG. 4, or may be a different text. In an embodiment, the second text 422 may be a text for which a similarity determination to an existing text is desired. The input / output unit 246 transfers the received second text 422 to the graph generation unit 221, and transfers information indicating the domain name of the second text 422 to the keyword management DB 231 and the domain graph DB 232.

[0067] Next, in step S610, the graph generation unit 221 extracts a set of domain keywords corresponding to the domain of the second sentence 422 from the keyword management DB 231, and identifies a subset of domain keywords (a subset of the first domain keywords) contained in the second sentence 422 from the extracted set of domain keywords.

[0068] Next, in step S620, the graph generating unit 221 generates nodes (a first node set) based on the subset of the domain keywords identified in step S610.

[0069] Next, in step S630, graph generation unit 221 calculates a node score for each node generated in step S620. Here, in order to calculate the node score, the above-mentioned Equation 1 may be used.

[0070] Next, in step S640, the graph generating unit 221 generates edges that indicate relationships between the sets of nodes generated in step S620.

[0071] Next, in step S650, the graph generating unit 221 calculates an edge weight for each edge generated in step S640. Here, in order to calculate the edge weight, the knowledge graph generated based on the second document 422 and the above-mentioned formula 2 may be used.

[0072] Next, in step S660, graph generation unit 221 generates a second domain graph using the set of nodes generated in step S620, the node scores calculated in step S630, the set of edges generated in step S640, and the edge weights calculated in step S650.

[0073] Next, in step S670, the graph generation unit 221 generates a normalized second domain graph by normalizing the second domain graph generated in step S660. Here, in order to normalize the second domain graph and generate the normalized second domain graph, the graph generation unit 221 may use the above-mentioned Equation 3 and Equation 4.

[0074] Next, in step S680, the similarity determination unit 222 obtains the normalized first domain graph generated by the pre-graph generation process 400 shown in FIG. 4 from the domain graph DB 232, and then determines the similarity between the normalized first domain graph and the normalized second domain graph generated in step S670.

[0075] Here, first, the similarity determination unit 222 calculates a common node score (CNS) indicating the importance of nodes common to the normalized first domain graph and the normalized second domain graph based on the normalized first domain graph and the normalized second domain graph using the following formula 5.

number

[0076] Next, based on the normalized first domain graph and the normalized second domain graph, the similarity determination unit 222 calculates a relationship score for common nodes (Rs), which indicates the degree of relevance of edges between nodes common to the normalized first domain graph and the normalized second domain graph, using the following formula 6.

number

[0077] Next, the similarity determination unit 222 uses the calculated CNS and Rs to calculate the similarity between the normalized first domain graph and the normalized second domain graph by the following formula 7. This similarity may be expressed as a percentage. Thus, a similarity of "100%" means that the normalized first domain graph and the normalized second domain graph (and the first and second sentences that are the source of the graphs) are identical.

number

[0078] Next, the similarity determination unit 222 generates a comparison result 685 indicating the similarity calculated here, and transmits it to the user terminal 260 .

[0079] According to the graph comparison process 600 described above, it is possible to provide a similarity determination means capable of high-speed and high-precision text similarity comparison by reducing the scale of the graph representation corresponding to the text while maintaining the relationship information between elements in the text.

[0080] Next, a process for updating domain keywords and domain graphs according to an embodiment of the present disclosure will be described with reference to FIG.

[0081] As described above, the keyword management DB 231 according to the embodiment of the present disclosure can store sets of domain keywords corresponding to various different domains. In addition, since the similarity determination according to the embodiment of the present disclosure is performed using a domain graph generated based on these domain keywords, in order to promote a highly accurate similarity determination, it may be desirable to update the set of domain keywords corresponding to a specific domain by adding a new domain keyword or deleting an existing domain keyword, for example.

[0082] Furthermore, when a set of specific domain keywords stored in the keyword management DB 231 is updated, it is desirable to update the domain graph that is generated based on the set of domain keywords and stored in the domain graph DB 232. Accordingly, one aspect of the present disclosure relates to updating a set of domain keywords stored in the keyword management DB 231 and a domain graph stored in the domain graph DB 232 based on an update request input by a user via the user terminal 260. Fig. 7 is a diagram showing a flow of an update process 700 for updating domain keywords and domain graphs according to an embodiment of the disclosure.

[0083] First, the input / output unit 246 receives an update request from the user terminal 260, which requests an update to a specific set of domain keywords (e.g., a first domain keyword set) stored in the domain graph DB 232. The update request here may be a user input requesting addition of a new domain keyword, deletion of an existing domain keyword, or modification of an existing domain keyword to the specific set of domain keywords. In an embodiment, the update request may be input via a user interface presented to the user by the input / output unit 246.

[0084] Next, the update unit 223 updates the set of specified domain keywords in the keyword management DB 231 based on the update request acquired from the user terminal 260. As an example, when the addition of a domain keyword with "photon" to the set of domain keywords corresponding to the domain with "physics" is specified by the update request, the update unit 223 may add the domain keyword with "photon" to the set of domain keywords corresponding to the domain with "physics" stored in the keyword management DB 231.

[0085] After that, the update unit 223 instructs the graph generation unit 221 to update, based on the updated set of domain keywords, the domain graphs corresponding to the domains whose sets of domain keywords have been updated among the domain graphs stored in the domain graph DB 232. Here, based on the updated set of domain keywords, the graph generation unit 221 may add new nodes and edges to the domain graph, delete existing nodes and edges, and recalculate the node scores of each node and the edge scores of each edge.

[0086] As described above, according to the update process 700 of the embodiment of the present disclosure, it is possible to update the set of domain keywords stored in the keyword management DB 231 and to update the domain graph corresponding to the set of domain keywords. This makes it possible to reflect changes in domain keywords due to progress in a specific domain and to set domain keywords suited to the purpose and values ​​of a user, thereby improving the accuracy of similarity determination.

[0087] Next, a keyword management screen according to an embodiment of the present disclosure will be described with reference to FIG.

[0088] 8 is a diagram showing an example of a keyword management screen 800 according to an embodiment of the present disclosure. The keyword management screen 800 allows a user to register or update domain keywords stored in the keyword management DB 231 of the similarity determination device 210. In an embodiment, the keyword management screen 800 may be a user interface screen generated by, for example, the input / output unit 246 and presented on the user terminal 260.

[0089] As shown in FIG. 8, a keyword management screen 800 may include a new domain registration window 810 and a domain keyword update window 820.

[0090] In the new domain registration window 810, the user can register a new domain in the keyword management section DB 231. For example, the user can enter the domain name "architecture" in the input area 811 of the new domain registration window 810 and press the confirm button 812 to register the new domain name "architecture" in the keyword management section DB 231.

[0091] In the domain keyword update window 820, the user can add a new domain keyword, delete an existing domain keyword, or change an existing domain keyword for a registered domain (e.g., a domain registered in the new domain registration window 810). For example, the user can select a specific domain name in the domain selection window 821, and then input the domain keyword to be added, deleted, or changed for the domain in the domain keyword input window 822. Thereafter, by pressing the confirm button 823, the user can select the action (add, delete, or change) to be performed on the domain keyword entered in the domain keyword input window 822. Furthermore, the user can input an update request and perform the update process shown in FIG. 7 by pressing the domain graph update 824.

[0092] On the keyword management screen 800 described above, the user can easily manage the domain keywords stored in the keyword management DB 231.

[0093] Next, a text management screen according to an embodiment of the present disclosure will be described with reference to FIG.

[0094] 9 is a diagram illustrating an example of a text management screen 900 according to an embodiment of the present disclosure. This text management screen 900 allows a user to search for other texts similar to a specific text, and to compare two texts (determine similarity). In an embodiment, this text management screen 900 may be a user interface screen generated by, for example, the input / output unit 246 and presented on the user terminal 260.

[0095] As shown in FIG. 9, the text management screen 900 may include a similar text search window 910 and a text comparison window 920.

[0096] In the similar sentence search window 910, the user can search for other sentences similar to a specific sentence. For example, the user can input the domain name "physics" in the domain selection window 911 of the similar sentence search window 910, input the sentence "laws of physics" in the file input window 912, and press the confirm button 913 to execute the graph comparison process 600 according to the embodiment of the present disclosure and search for sentences that are highly similar to "laws of physics" from the domain graph DB 232.

[0097] In the text comparison window 920, a user can compare two specific texts. For example, a user can input a domain name "architecture" in a domain selection window 921 of the text comparison window 920, input a text "ancient Roman architecture" in a first file input window 922, input a text "ancient Greek architecture" in a second file input window 923, and press a comparison button 924 to perform the graph comparison process 600 according to an embodiment of the present disclosure and determine the similarity between "ancient Roman architecture" and "ancient Greek architecture."

[0098] On the text management screen 900 described above, the user can easily determine the similarity between predetermined texts.

[0099] Next, a multi-domain management process according to an embodiment of the present disclosure will be described with reference to FIG.

[0100] Generally, there are sentences related to a plurality of different domains. Therefore, one aspect of the present disclosure relates to a multi-domain management process for determining a domain that has a high correspondence with a given sentence and generating a domain graph using domain keywords of the determined domain in order to provide a highly accurate similarity determination result for sentences related to a plurality of different domains. FIG. 10 is a diagram showing an example of the flow of a multi-domain management process 1000 according to an embodiment of the present disclosure. This multi-domain management process 1000 is mainly performed by the graph generation unit 221 of the similarity determination device 210.

[0101] First, in step S1010, the graph generating unit 221 accepts an input of a second sentence for which a similarity determination is desired, and then determines a domain that has a high correspondence with this second sentence. More specifically, the graph generating unit 221 calculates a domain score indicating the relevance of the second sentence to each set of domain keywords stored in the keyword management DB 231. In an embodiment, the graph generating unit 221 may calculate the domain score based on the proportion of domain keywords included in the second sentence among each set of domain keywords.

[0102] As an example, the keyword management DB 231 stores a set of first domain keywords corresponding to a first domain with “physicist” and a set of second domain keywords corresponding to a second domain with “chemist.” The graph generating unit 221 also accepts input of a second sentence with “famous scientists” related to chemists in various scientific fields, such as physicists, chemists, and biologists. In this case, the graph generation unit 221 calculates a first domain score indicating the relevance of the second sentence "famous scientist" to the first domain "physicist" and a second domain score indicating the relevance of the second sentence "famous scientist" to the second domain "chemist".

[0103] For example, if 70% of the domain keywords in a set of first domain keywords corresponding to the first domain with "physicist" are included in the second sentence with "famous scientist", the graph generation unit 221 may set the first domain score of the second sentence with "famous scientist" with respect to the first domain with "physicist" to "70%". In addition, if 30% of the domain keywords in the set of second domain keywords corresponding to the second domain with "chemist" are included in the second sentence with "famous scientist", the graph generation unit 221 may set the second domain score of the second sentence with "famous scientist" for the second domain with "chemist" to "30%". In this way, the graph generation unit 221 can calculate a domain score indicating the relevance of the second sentence to each set of domain keywords stored in the keyword management DB 231 based on the proportion of each set of domain keywords contained in the second sentence.

[0104] Next, in step S1020, the graph generating unit 221 acquires an appropriate set of domain keywords from the keyword management DB 231 based on the domain scores calculated in step S1010. In one embodiment, the graph generating unit 221 may acquire an appropriate set of domain keywords based on the relationship between the domain scores calculated for each domain and a predetermined domain score threshold. More specifically, the graph generation unit 221 may obtain a set of first domain keywords corresponding to the first domain from the keyword management DB 231 when the first domain score indicating the relevance of the second sentence to the first domain exceeds the second domain score indicating the relevance of the second sentence to the second domain and satisfies a predetermined domain score threshold (e.g., 50%). On the other hand, the graph generation unit 221 may obtain a set of second domain keywords corresponding to the second domain from the keyword management DB 231 when the second domain score indicating the relevance of the second sentence to the second domain exceeds the first domain score indicating the relevance of the second sentence to the first domain and satisfies a predetermined domain score threshold (e.g., 50%).

[0105] Next, in step S1030, the graph generating unit 221 generates a normalized second domain graph corresponding to the second sentence, using the domain keywords acquired in step S1020. The details of the process of generating the normalized second domain graph have been described with reference to FIG. 6, and therefore will not be described here.

[0106] According to the above-described multi-domain management process 1000, even if a given sentence is related to multiple domains, it is possible to determine appropriate domain keywords for generating a domain graph corresponding to the sentence, thereby improving the accuracy of similarity determination.

[0107] Next, a specific example of similarity determination according to an embodiment of the present disclosure will be described with reference to FIG.

[0108] Fig. 11 is a diagram showing a specific example of similarity determination for two domain graphs according to an embodiment of the present disclosure. Fig. 11 shows parameters such as node scores, normalized node scores, edge weights, and normalized edge weights for two domain graphs, domain graph A and domain graph B.

[0109] More specifically, a node in the domain graph A is f A Each node score is assigned a node score of {3, 2, 1, 4}. The normalized node score f is calculated by dividing each node score by the sum of the node scores of the domain graph (3+2+1+4=10). A You can get (NS)={0.3, 0.2, 0.1, 0.4}. Similarly, a node in domain graph B is f B Each node score is assigned a node score of {3, 2, 1, 4}. The normalized node score f is calculated by dividing each node score by the sum of the node scores of the domain graph (3+2+1+4=10). B You can get (NS)={0.3, 0.2, 0.1, 0.4}.

[0110] Figure 11 also shows the number of connections between nodes in domain graph A, f A (L) is shown in Table 1105, and the number of connections between each node in the domain graph B, f B (L) is shown in Table 1110. By taking the inverse of these connections, the edge weights E ij can be calculated. In addition, by using the above-mentioned formula 3, the normalized edge weight NE of the domain graph A is ij The normalized edge weights of domain graph A are shown in table 1115, and the normalized edge weights of domain graph B are shown in table 1120.

[0111] The normalized node score f of the domain graph A explained above A (NS) and the normalized node score f of the domain graph B B By substituting (NS) into Equation 5, we can calculate the CNS between domain graph A and domain graph B. In this case, the normalized node score f A (NS) and the normalized node score f of the domain graph B B Since (NS) is the same, Equation 5 becomes (1-0=1).

[0112] Moreover, by substituting the normalized edge weights of domain graph A and domain graph B described above into Equation 6, it is possible to calculate the RS between domain graph A and domain graph B. In this case, Equation 6 becomes (1-0.2163=0.7837).

[0113] Then, by substituting the calculated CNS and RS into Equation 7, it is possible to calculate the similarity between domain graph A and domain graph B. In this case, as a result of the calculation using Equation 7, the similarity between domain graph A and domain graph B is "89.185%".

[0114] In this way, by representing sentences as domain graphs and comparing these domain graphs, it is possible to determine the similarity between two sentences with high accuracy and speed.

[0115] The similarity determination means according to the embodiment of the present disclosure has been described above. As described above, one aspect of the present disclosure relates to representing sentences as domain graphs, which allows for more accurate similarity determination by expressing semantic relationships between words in sentences compared to a direct comparison of the texts of the sentences.

[0116] However, as mentioned above, in the conventional method of representing an entire sentence as a domain graph, every word contained in the sentence is represented as a node, so when the sentence is long, the number of nodes and edges becomes enormous, which causes a problem of slowing down processing such as comparison of the graph. Therefore, one aspect of the present disclosure relates to generating a domain graph that represents only specific keywords in a sentence as nodes, rather than the entire sentence. This makes it possible to reduce the size of the graph compared to conventional means of representing the entire chapter as a domain graph. In addition, by reducing the size of the graph, the time required for processing such as graph comparison can be shortened, enabling high-speed similarity determination.

[0117] However, if a domain graph is generated based only on specific keywords in a sentence, rather than on the entire sentence, there is a possibility of information loss compared to when the entire sentence is represented as a domain graph. Therefore, in one aspect of the present disclosure, each node in the domain graph is associated with a node score that indicates the importance of the domain keyword corresponding to that node, and each edge is associated with an edge weight that indicates the degree of relatedness between the nodes that the edge connects. This makes it possible to maintain semantic information about particularly important keywords in a sentence and reduce information loss without having to represent the entire sentence as a domain graph.

[0118] Furthermore, one aspect of the present disclosure relates to generating a normalized domain graph by normalizing each node score and edge weight in the domain graph. By normalizing the domain graph in this manner, it is possible to prevent a decrease in similarity determination caused by differences in the sizes of the domain graphs when comparing the domain graphs.

[0119] When the similarity determination means according to the embodiment of the present disclosure described above is applied to, for example, a report on a cyber event, it is possible to identify an existing report that is highly similar to a specific report. If there is an existing report whose similarity to the new report satisfies a predetermined similarity criterion, it is possible to determine that the cyber event reported in the report has already been dealt with or is being dealt with, and it is possible to save time and resources required to formulate countermeasures again. On the other hand, if there is no existing report whose similarity to the new report satisfies a predetermined similarity criterion, it is possible to determine that the cyber event reported in the report has not been dealt with and that new countermeasures need to be formulated. In this way, it is possible to determine with high accuracy whether a cyber event reported in a report has already been dealt with, is being dealt with, or has not been dealt with. In addition, since the domain graph used in the similarity determination according to the embodiment of the present disclosure is a domain graph based only on specific predetermined keywords, a comparison result indicating the similarity of reports can be generated quickly. This makes it possible to quickly formulate countermeasures against cyber events.

[0120] In this way, according to the present disclosure, it is possible to provide a similarity determination means capable of performing high-speed and high-precision sentence similarity comparison by reducing the scale of the graph representation corresponding to the sentence while maintaining relationship information between elements in the sentence.

[0121] Although the above describes an example in which the similarity determination means according to the embodiment of the present disclosure is implemented in an apparatus, a system, and a method, the present disclosure is not limited thereto, and may be implemented, for example, as a computer program. This computer program may be introduced into a computer system that implements the similarity determination means according to the embodiment of the present disclosure from a storage medium of an external device via a network and / or via a portable storage medium.

[0122] For example, one aspect of an embodiment of the present disclosure is a similarity determination computer program, in a computer system including a memory for storing processing instructions and a processor, the processing instructions stored in the memory include a step of acquiring a set of first domain keywords corresponding to a first domain, a step of generating a normalized first domain graph for a first sentence corresponding to the first domain, the normalized first domain graph being generated based on the set of first domain keywords, a step of identifying a subset of first domain keywords included in a second sentence corresponding to the first domain based on the set of first domain keywords, a step of determining the number of occurrences of the identified subset of first domain keywords in the second sentence, a step of generating a first node set including at least a first node and a second node based on the subset of first domain keywords, a step of calculating a node score indicating the importance of each node included in the first node set in the second sentence based on the determined number of occurrences, and a step of generating a knowledge graph indicating a semantic relationship between each node included in the first node set based on the second sentence; a step of determining a shortest distance in the knowledge graph between the first node and the second node; a step of calculating an edge weight indicating a degree of association between the first node and the second node based on the determined shortest distance; a step of generating an edge indicating a relationship between the first node and the second node and corresponding to the edge weight; a step of generating a second domain graph corresponding to the second sentence based on the first node set, the node scores, the edge weights, and the edges; a step of generating a normalized second domain graph by normalizing the node scores and the edge weights;

[0123] Although the embodiment of the present invention has been described above, the present invention is not limited to the above-described embodiment, and various modifications are possible without departing from the gist of the present invention. [Explanation of symbols]

[0124] 150 Similarity Judgment Application 200 Similarity Judgment System 210 Similarity determination device 220 Memory 221 Graph Generation Unit 222 Similarity determination unit 223 Update Department 230 Storage section 231 Keyword Management DB 232 Domain Graph DB 244 processors 246 Input / output section 250 Communication Network 260 User Terminals

Claims

1. A similarity determination device, comprising: A processor and a memory, The memory includes: a keyword management database for storing a set of first domain keywords corresponding to the first domain; a domain graph database that stores a first domain graph generated for a first sentence corresponding to the first domain based on the set of first domain keywords; a graph generator for generating a second domain graph for a second sentence corresponding to the first domain based on the set of first domain keywords; a similarity determination unit that generates a comparison result indicating a similarity between the first domain graph and the second domain graph by comparing the first domain graph and the second domain graph; The similarity determination device further comprises a processing instruction for causing the processor to function as a similarity determination device.

2. The graph generation unit Identifying a subset of first domain keywords contained in the second sentence based on the set of first domain keywords; determining a number of occurrences of the identified subset of first domain keywords in the second sentence; generating a first node set based on the subset of first domain keywords, the first node set including at least a first node and a second node; calculating a node score indicating the importance of each node included in the first node set in the second sentence based on the determined number of occurrences; 2. The similarity determination device according to claim 1, wherein the similarity determination device comprises:

3. The graph generation unit generating a knowledge graph indicating semantic relationships of each node included in the first node set based on the second sentence; determining a shortest distance in the knowledge graph between the first node and the second node; Calculating an edge weight indicating a degree of association between the first node and the second node based on the determined shortest distance; generating an edge indicating a relationship between the first node and the second node and associated with the edge weight; 3. The similarity determination device according to claim 2.

4. generating the second domain graph corresponding to the second sentence based on the first node set, the node scores, the edge weights, and the edges; generating a normalized second domain graph by normalizing the node scores and the edge weights; 4. The similarity determination device according to claim 3.

5. The keyword management database includes: in addition to the set of first domain keywords corresponding to the first domain, a set of second domain keywords corresponding to a second domain; The graph generation unit calculating a first domain score indicative of a relevance of the second sentence to the first domain; calculating a second domain score indicative of a relevance of the second sentence to the second domain; if the first domain score exceeds the second domain score and satisfies a predefined domain score threshold, generating the second domain graph based on the set of first domain keywords; if the second domain score exceeds the first domain score and satisfies a predefined domain score threshold, generating the second domain graph based on the set of second domain keywords; 2. The similarity determination device according to claim 1, wherein the similarity determination device comprises:

6. When an update request is received from a user requesting addition or deletion of a domain keyword to the first domain keyword set, an update unit configured to update the second domain graph based on addition or deletion of a domain keyword indicated in the update request; 2. The similarity determination device according to claim 1, wherein the similarity determination device comprises:

7. A similarity determination system in which a user terminal and a similarity determination device are connected via a communication network, The similarity determination device includes: A processor and a memory, The memory includes: a keyword management database for storing a set of first domain keywords corresponding to the first domain; a domain graph database that stores a first domain graph generated for a first sentence corresponding to the first domain based on the set of first domain keywords; an input / output unit that acquires a second document corresponding to the first domain from the user terminal; a graph generator for generating a second domain graph for the second sentence based on the set of first domain keywords; a similarity determination unit that generates a comparison result indicating a similarity between the first domain graph and the second domain graph by comparing the first domain graph and the second domain graph, and transmits the comparison result to the user terminal; The similarity determination system further comprises a processing instruction for causing the processor to function as a similarity determination system.

8. In a computer system including a memory for storing processing instructions and a processor, the processing instructions stored in the memory include obtaining a set of first domain keywords corresponding to the first domain; generating a normalized first domain graph for a first sentence corresponding to the first domain, the normalized first domain graph being generated based on the set of first domain keywords; identifying a subset of first domain keywords included in a second sentence corresponding to the first domain based on the set of first domain keywords; determining a number of occurrences of the identified subset of first domain keywords in the second sentence; generating a first node set based on the subset of first domain keywords, the first node set including at least a first node and a second node; calculating a node score indicating the importance of each node included in the first node set in the second text based on the determined number of occurrences; generating a knowledge graph based on the second sentence, the knowledge graph indicating a semantic relationship of each node included in the first node set; determining a shortest distance in the knowledge graph between the first node and the second node; calculating an edge weight indicating a degree of association between the first node and the second node based on the determined shortest distance; generating an edge indicating a relationship between the first node and the second node and associated with the edge weight; generating a second domain graph corresponding to the second sentence based on the first node set, the node scores, the edge weights, and the edges; generating a normalized second domain graph by normalizing the node scores and the edge weights; comparing the normalized first domain graph with the normalized second domain graph to generate and output a comparison result indicating a similarity between the normalized first domain graph and the normalized second domain graph; The similarity determination method is characterized by causing the processor to execute the above steps.

Images