Information processing device and method

Abstract

An information processing device according to one embodiment of the present invention comprises: an acquisition unit that acquires, from among a plurality of entities stored in a first database and classified into a group to be subjected to entity resolution from among a plurality of groups obtained by classifying a plurality of entities in the first database and a second database, a first adjacent entity representing an item having an adjacent relationship with an item represented by an entity to be subjected to entity resolution, and acquires, for each of the plurality of entities classified into the group to be subjected to entity resolution and stored in the second database, a second adjacent entity representing an item having an adjacent relationship with the item representing said entity; and a determination unit that determines the degree of similarity between the second adjacent entity and the first adjacent entity to thereby determine the second adjacent entity from among the second adjacent entities that is most similar to the first adjacent entity, and determines that the entity to be subjected to entity resolution and an entity that represents an item having an adjacent relationship with the item represented by the second adjacent entity, and that is classified into a group to be subjected to the entity resolution and is stored in the second database, are entities representing the same item.

Description

Information processing device and method

[0001] FIELD Embodiments of the present invention relate to an information processing apparatus and method.

[0002] There is a method for merging identical entities, which are entities representing the same item stored in two databases (DBs), by using adjacent entities representing items adjacent to the item represented by the identical entities (see, for example, Patent Document 1). In this method, for an entity stored in one DB, an identical entity that exists in the other DB and represents the same item as the item represented by the entity stored in the one DB is selected by the following (1), (2), and (3).

[0003] (1) Calculate string similarity, which is the degree of similarity between strings related to each entity stored in the DB to be searched, using edit distance and TF-IDF. (2) Narrow down the candidates for the entities to be searched based on the calculation result in (1) and the adjacency information indicating the adjacency relationships between the entities. (3) Output information indicating the correspondence of the name matching based on the candidates for the entities to be searched obtained by the narrowing down.

[0004] Japanese Patent Application Publication No. 2020-123210

[0005] In the calculation of the above-mentioned string similarity, since the edit distance generally assigns equal costs to addition, deletion, and substitution operations, if the importance of each cost differs, it is necessary to adjust the cost parameters. Also, since the above-mentioned TF-IDF is a method for evaluating the importance according to the frequency of occurrence of words in a sentence, it is not suitable for calculating the similarity between words when entities are input as words.

[0006] In addition, since the similarity calculation is performed for all entities stored in the DB to be searched, the amount of calculation can become very large when the number of entities to be searched is extremely large, such as in big data.

[0007] The present invention has been made in light of the above circumstances, and its purpose is to provide an information processing device and method that can appropriately identify information indicating the same item.

[0008] An information processing device according to one aspect of the present invention includes a selection unit that selects an entity to be matched from among a plurality of entities that are classified into a group to be matched and stored in the first database among a plurality of groups into which a plurality of entities stored in a first database and a plurality of entities stored in a second database are classified; an acquisition unit that acquires a first adjacent entity that is an entity representing an item that has an adjacent relationship with an item represented by the entity selected by the selection unit, and acquires a second adjacent entity that is an entity representing an item that has an adjacent relationship with an item represented by the entity for each of a plurality of entities that are classified into the group to be matched and stored in the second database; The system is equipped with a determination unit that determines the similarity between the second adjacent entity associated with each of the multiple entities classified into the group of entities to be matched and stored in the second database and the first adjacent entity acquired by the acquisition unit, thereby determining the second adjacent entity that is most similar to the first adjacent entity among the second adjacent entities associated with each of the multiple entities classified into the group of entities to be matched and stored in the second database, and determines that the entity that represents an item having an adjacent relationship with the item represented by the second adjacent entity and is classified into the group of entities to be matched and stored in the second database and the entity to be matched are entities that represent the same item.

[0009] An information processing method according to one aspect of the present invention is a method performed by an information processing device, the method including: selecting, by a selection unit of the information processing device, entities to be matched from among a plurality of entities classified into a group to be matched and stored in the first database, among a plurality of groups into which a plurality of entities stored in a first database and a plurality of entities stored in a second database are classified; acquiring, by an acquisition unit of the information processing device, first adjacent entities which are entities representing an item having an adjacent relationship with the item represented by the entity selected by the selection unit; and acquiring, for each of a plurality of entities classified into the group to be matched and stored in the second database, second adjacent entities which are entities representing an item having an adjacent relationship with the item represented by the entity. and determining, by a determination unit of the information processing device, the similarity between the second adjacent entity associated with each of the plurality of entities classified into the group of entities to be matched and stored in the second database, acquired by the acquisition unit, and the first adjacent entity acquired by the acquisition unit, thereby determining the second adjacent entity that is most similar to the first adjacent entity among the second adjacent entities associated with each of the plurality of entities classified into the group of entities to be matched and stored in the second database, and determining that the entity that represents an item having an adjacent relationship with the item represented by the second adjacent entity and that is classified into the group of entities to be matched and stored in the second database and the entity to be matched are entities that represent the same item.

[0010] According to the present invention, information indicating the same item can be appropriately identified.

[0011] FIG. 1 is a diagram showing an application example of an information processing device according to an embodiment of the present invention. FIG. 2 is a flowchart showing an example of a procedure for processing operations by an information processing device according to an embodiment of the present invention. FIG. 3 is a diagram explaining an example of grouping target entities stored in multiple DBs. FIG. 4 is a diagram explaining an example of selecting a group to be matched. FIG. 5 is a diagram explaining an example of acquiring adjacent entities. FIG. 6 is a diagram explaining an example of determining the similarity between adjacent entities and determining whether they are the same entity. FIG. 7 is a block diagram showing an example of the hardware configuration of an information processing device according to an embodiment of the present invention.

[0012] An embodiment of the present invention will be described below. Fig. 1 is a diagram showing an application example of an information processing device according to an embodiment of the present invention. As shown in Fig. 1, an information processing device 100 according to an embodiment of the present invention has target entity acquisition units 10a and 10b, a blocking unit 20, adjacent entity acquisition units 30a and 30b, and an identical entity acquisition unit 40. The processing performed by each unit will be described later.

[0013] The task of integrating data stored in multiple databases involves identifying the same entity stored across the databases and integrating the information associated with that entity. This integration makes it possible to view a wide variety of information stored in different databases side-by-side. Therefore, data stored in multiple databases is integrated for the purpose of multifaceted data analysis or reducing database management workload.

[0014] When the same entity is written differently in each database where it is stored, it is highly likely that it will be difficult to find the same entity. In this embodiment, linking entities that represent the same thing but are written differently is called name matching. In the business of accurately integrating items stored in multiple databases, it is necessary to perform name matching across multiple databases.

[0015] There are various characters and formats for the entities stored in the DB. In this embodiment, it is assumed that the DB is a DB in which relatively short entities at the word level are entered in each attribute.

[0016] Furthermore, in this embodiment, it is assumed that each entity entered in a certain attribute has an adjacent entity representing an item having an adjacent relationship with the item represented by that entity. For example, if the DB is a facility information DB that stores facility information of a communication network, the attribute is "building name," and the names of buildings adjacent to each other on the communication network are stored in "building name," the adjacent entities for each entity of "building name" refer to the entities of "building name" that are adjacent to each other on the communication network. In this embodiment, it is assumed that adjacent entities are stored in the DB.

[0017] In this embodiment, it is assumed that attribute X, which is an attribute containing the same entity, is known in multiple DBs. In this embodiment, entities stored in attribute X in each DB are matched. The entity of attribute X is referred to as the target entity.

[0018] Here, the number of DBs is arbitrary, but when the name matching of the target entities between two different DBs is performed by a brute force method, the name matching of the entities is completed in all DBs. Therefore, there is no loss of generality even when the name matching of the entities is performed between two DBs.

[0019] In addition, in this embodiment, it is assumed that the target entity has adjacent entities. The adjacent entities do not need to be stored in attribute A. For example, consider a case where the DB is a facility information DB that stores facility information of a communication network, attribute A is "building name," and the names of buildings adjacent to each other on the communication network are stored in "building name."

[0020] In this case, the adjacent entities for each "Building Name" entity refer to "Building Name" entities that are adjacent to each other on the communication network. The adjacent entities corresponding to each target entity may be different. In this embodiment, it is assumed that the adjacent entities are known to each target entity stored in one of the DBs.

[0021] In this embodiment, one target entity stored in "DB1" of two DBs, "DB1" and "DB2," is specified, and all entities stored in "DB2" are the search targets.

[0022] Then, from among these search targets, entities that are candidates for the specified target entity and that are linked as the same entity are selected using an appropriate blocking method.

[0023] Furthermore, among the neighboring entities of each entity selected as a candidate, an entity that is closest to the neighboring entity of the specified target entity is selected using LLMs (Large Language Models).

[0024] In this embodiment, the LLM is input with each of the neighboring entities selected as candidates, and a question is asked about the entity that is closest to the neighboring entity of the specified target entity. The entity corresponding to the neighboring entity included in the answer to this question is also determined to be the entity that is closest to the neighbor of the specified entity in terms of neighbor information. Therefore, it is determined that the things represented by these entities represent the same thing.

[0025] The above process is performed for all target entities stored in "DB1." As a result, it is possible to extract the same target entities between "DB1" and "DB2."

[0026] In this embodiment, the blocking method and LLM are appropriately used to improve the problems of conventional methods. First, the blocking method does not require parameter adjustment to select candidates for the same entity, and the blocking method is applicable regardless of whether the entity is a sentence or a word. The blocking method generally has the effect of reducing the number of similarity calculations between entities, thereby reducing the amount of calculation required for name matching.

[0027] In this embodiment, the LLM is applied after reducing the amount of calculation required for name identification in this way. Since the LLM, which inputs neighbor information, does not require fine tuning, it can be applied without parameter adjustment or pre-learning.

[0028] As described above, in this embodiment, it is possible to construct a name matching technology that is efficient, does not require parameter adjustment, and is applicable regardless of whether the entity is a sentence or a word.

[0029] Fig. 2 is a flowchart showing an example of a processing operation procedure by an information processing device according to an embodiment of the present invention. Fig. 3 is a diagram illustrating an example of grouping target entities stored in multiple DBs. In this embodiment, entities stored in "Database 1" and "Database 2" are matched. It is assumed that information on entities adjacent to each entity is known in both databases.

[0030] First, the target entity acquisition unit 10a of the information processing device 100 acquires a target entity (symbol a in FIG. 1 ) that is an entity stored in "Database 1." Furthermore, the target entity acquisition unit 10b acquires a target entity that is an entity stored in "Database 2" (S10). Next, the blocking unit 20 classifies an entity that combines the target entities acquired by the target entity acquisition unit 10a and the target entities acquired by the target entity acquisition unit 10b into N groups, from "Group 1" to "Group N," using a blocking technique (S20).

[0031] Specifically, as shown in Figure 3, each group from "Group 1" to "Group N" classifies the target entities acquired by the target entity acquisition unit 10a and the target entities acquired by the target entity acquisition unit 10b.

[0032] FIG. 4 is a diagram illustrating an example of selecting a group to be subjected to name matching. Next, the blocking unit 20 selects one group from the groups classified in S20 to be subjected to entity matching (S30). Here, the subsequent processing will be described for an example in which, as shown in FIG. 4, a total of six entities are assigned to "Group X," which is one of the groups from "Group 1" to "Group N," with three entities assigned from each database. An example will be described in which "Group X," which is one of the groups from "Group 1" to "Group N," with three entities assigned from each database, and "Group X" is selected in S30 as the group to be subjected to entity matching.

[0033] As shown in FIG. 4, the entities classified into “Group X” and originating from “Database 1” are “A”, “000000”, and “111111”, and the entities classified into “Group X” and originating from “Database 2” are “I”, “A′”, and “II”.

[0034] The adjacent entity acquisition unit 30a selects one entity, in this case entity "A," from among the multiple entities derived from "Database 1," which is the target entity classified within "Group X" selected in S30 and acquired by the target entity acquisition unit 10a in S10, as the entity to be matched (S40).

[0035] 5A and 5B are diagrams illustrating an example of acquiring adjacent entities. As shown in Fig. 5A, the adjacent entities of the entity "A" selected in S40 are known, and are the combination of entities "B" and "BB." The adjacent entity acquisition unit 30a acquires the adjacent entities.

[0036] 5B, among the entities derived from "Database 2" that are the target entities classified in S20 and selected in S30 into the same "Group X" and acquired by the target entity acquisition unit 10b in S10, the one-to-one adjacent entities of entities "I," "A'," and "II" are known and are the following combinations of entities: Adjacent entities of entity "I": entities "AAA" and "AAZ" Adjacent entities of entity "A'": entities "B'" and "BB'" Adjacent entities of entity "II": entities "SZD" and "RTG"

[0037] The adjacent entity acquisition unit 30b acquires these adjacent entities. In this way, adjacent entities of the entities classified into the selected "group X" as described above and originating from "database 1" and adjacent entities of the entities classified into the selected "group X" as described above and originating from "database 2" are acquired (S50).

[0038] 6 is a diagram illustrating an example of determining the similarity between adjacent entities and determining whether they are the same entity. Next, the same entity acquisition unit 40 uses LLM to determine the similarity between a combination of adjacent entities of entities originating from "Database 1" (acquired in S50 for the entities classified into the group selected in S30 and selected in S40) and a combination of adjacent entities of entities originating from "Database 2" (acquired in S50 for the entities classified into the group selected in S30). By performing this determination, the same entity acquisition unit 40 determines the combination of adjacent entities (symbol b in FIG. 1) that is most similar to the adjacent entities of the entity originating from "Database 1" from among multiple combinations of adjacent entities of entities originating from "Database 2" (S60).

[0039] Here, as shown in Figure 6, the identical entity acquisition unit 40 uses LLM to determine, from among the combinations of adjacent entities derived from "Database 2" acquired in S50, the combination of adjacent entities that is most similar to "Entity B" and "Entity BB," which is the combination of adjacent entities of "Entity A" derived from "Database 1" selected in S40 as the entity to be matched.

[0040] The same entity acquisition unit 40 inputs into the LLM the adjacent entities of "Entity A," i.e., entities "B" and "BB" (symbol a in Figure 6), as well as the adjacent entities of "Entity I," "Entity A'," and "Entity II," i.e., entities "AAA," "AAZ," "B'," "BB'," "SZD," and "RTG" (symbol b in Figure 6), and the question text.

[0041] This question is a question asking about the most similar adjacent entities as mentioned above, and is expressed, for example, as follows: (Question) "Please answer by selecting the entity combination that is most similar to the entity combination (B, BB) from (AAA, AAZ), (B', BB'), and (SZD, RTG)." Here, we will explain an example in which the adjacent entity combination indicated by the answer from the LLM in response to the above input is (B', BB'), as in the answer expressed as follows: (Answer) "It is (B', BB')."

[0042] As shown in Figure 6, the combination of adjacent entities indicated by this response (B', BB') is an adjacent entity of "Entity A'", so the same entity acquisition unit 40 determines that the entity to be matched, entity "A" classified into "Group X" and originating from "Database 1", and "Entity A'", which is classified into the same group as the group to which this entity is classified, i.e., "Group X" and originating from "Database 2", are the same entity (S70).

[0043] Furthermore, in this embodiment, the identical entity acquisition unit 40 can also determine whether there is an identical entity among the entities "Entity I", "Entity A'", and "Entity II", which are derived from "Database 2", for other entities classified in "Group X" and derived from "Database 1", namely, entities "000000" and "111111".

[0044] In this embodiment, the above-described process up to determining identical entities can be performed on all groups, i.e., on each group other than the above-mentioned "Group X" and "Group X" among "Group 1" to "Group N" shown in Fig. 3. In this way, the identical entity acquisition unit 40 obtains as output a combination of identical entities, which is the result obtained for each classified group.

[0045] 7 is a block diagram showing an example of the hardware configuration of an information processing device according to an embodiment of the present invention. In the example shown in FIG. 7, the information processing device 100 according to the embodiment is configured, for example, by a server computer or a personal computer, and has a hardware processor 111A such as a CPU (Central Processing Unit). A program memory 111B, a data memory 112, an input / output interface 113, and a communication interface 114 are connected to this hardware processor 111A via a bus 115. The same applies to the other devices shown in FIG. 1.

[0046] The communication interface 114 includes, for example, one or more wireless communication interface units, and enables transmission and reception of information to and from a communication network. As the wireless interface, for example, an interface that adopts a low-power wireless data communication standard such as a wireless LAN (Local Area Network) is used.

[0047] An input device 200 and an output device 300 attached to the information processing device 100 and used by a user or the like are connected to the input / output interface 113. The input / output interface 113 can take in operation data input by a user or the like through the input device 200 such as a keyboard, a touch panel, a touchpad, or the like, and can output and display output data to an output device 300 including a display device using a liquid crystal or an organic electroluminescence (EL) display, or the like. The input device 200 and the output device 300 may be devices built into the information processing device 100, or may be input devices and output devices of other information terminals that can communicate with the information processing device 100 via a network.

[0048] The program memory 111B is a non-transitory tangible storage medium that is a combination of a non-volatile memory that can be written to and read from at any time, such as a hard disk drive (HDD) or a solid state drive (SSD), and a non-volatile memory such as a read only memory (ROM), and can store programs necessary to execute various control processes, etc., according to one embodiment.

[0049] The data memory 112 is a tangible storage medium that is, for example, a combination of the above-mentioned nonvolatile memory and a volatile memory such as RAM (Random Access Memory), and can be used to store various data or information acquired and created during various processes.

[0050] An information processing apparatus 100 according to an embodiment of the present invention can be configured as an information processing apparatus having the units shown in FIG. 1 as software-based processing function units.

[0051] The information storage unit used as a work memory or the like in the information processing device 100 can be configured by using the data memory 112 shown in Fig. 7. However, these configured storage areas are not essential components within the information processing device 100, and may be areas provided in an external storage medium such as a USB (Universal Serial Bus) memory, or a storage device such as a database server located in the cloud.

[0052] The processing function units in each of the above units can be realized by reading and executing a program stored in the program memory 111B by the hardware processor 111A. Note that some or all of these processing function units may be realized in various other forms, including integrated circuits such as an application specific integrated circuit (ASIC) or a field-programmable gate array (FPGA).

[0053] The methods described in each embodiment can be stored as a program (software means) that can be executed by a computer on a recording medium such as a magnetic disk (floppy disk, hard disk, etc.), optical disk (CD-ROM, DVD, MO, etc.), or semiconductor memory (ROM, RAM, flash memory, etc.), and can also be distributed by transmitting it via a communication medium. The program stored on the medium also includes a configuration program that configures the software means (including not only execution programs but also tables or data structures) that the computer executes. The computer that realizes this device reads the program stored on the recording medium and, in some cases, configures the software means using the configuration program, and executes the above-mentioned processing by controlling the operation of this software means. The term "recording medium" as used herein is not limited to a storage medium for distribution, but also includes a storage medium such as a magnetic disk or semiconductor memory installed inside the computer or in a device connected via a network.

[0054] The present invention is not limited to the above-described embodiments, and various modifications can be made in the implementation stage without departing from the spirit of the invention. Furthermore, the embodiments may be implemented in appropriate combinations, in which case the combined effects can be obtained. Furthermore, the above-described embodiments include various inventions, and various inventions can be extracted by combining selected elements from the disclosed elements. For example, if the problem can be solved and the desired effect can be obtained even if some elements are deleted from all elements shown in the embodiments, the configuration from which these elements are deleted can be extracted as an invention.

[0055] 100: Information processing device 10a, 10b: Target entity acquisition unit 20: Blocking unit 30a, 30b: Adjacent entity acquisition unit 40: Same entity acquisition unit

Claims

1. A selection unit that selects an entity to be matched from among a plurality of entities that are classified into a group to be matched and stored in the first database among a plurality of groups into which a plurality of entities stored in a first database and a plurality of entities stored in a second database are classified; an acquisition unit that acquires a first adjacent entity that is an entity representing an item that has an adjacent relationship with an item represented by the entity selected by the selection unit, and acquires a second adjacent entity that is an entity representing an item that has an adjacent relationship with an item represented by the entity for each of a plurality of entities that are classified into the group to be matched and stored in the second database; an information processing device comprising: a determination unit that determines the similarity between the second adjacent entity associated with each of the multiple entities classified into the group of entities to be matched and stored in the second database, which is acquired by the acquisition unit, and the first adjacent entity acquired by the acquisition unit, thereby determining the second adjacent entity that is most similar to the first adjacent entity among the second adjacent entities associated with each of the multiple entities classified into the group of entities to be matched and stored in the second database, and determines that the entity that represents an item having an adjacent relationship with the item represented by the second adjacent entity and is classified into the group of entities to be matched and stored in the second database and the entity to be matched are entities that represent the same item.

2. An information processing device as described in claim 1, further comprising a classification unit that classifies multiple entities stored in the first database and multiple entities stored in the second database into multiple groups using a blocking technique, and the selection unit selects entities to be matched from among the multiple entities that are classified into groups that are the target of entity match-up among the groups classified by the classification unit and stored in the first database.

3. The information processing device of claim 1, wherein the determination unit determines the similarity between the second adjacent entity associated with each of the multiple entities classified into the group of entities to be matched and stored in the second database, acquired by the acquisition unit, and the first adjacent entity acquired by the acquisition unit using a large-scale language model, thereby determining the second adjacent entity most similar to the first adjacent entity among the second adjacent entities associated with each of the multiple entities classified into the group of entities to be matched and stored in the second database, and determining that the entity that represents an item having an adjacent relationship with the item represented by the second adjacent entity and that is classified into the group of entities to be matched and stored in the second database and the entity to be matched are entities representing the same item.

4. A method performed by an information processing device, comprising: a selection unit of the information processing device selecting an entity to be matched from among a plurality of entities classified into a group to be matched and stored in the first database, among a plurality of groups into which a plurality of entities stored in a first database and a plurality of entities stored in a second database are classified; an acquisition unit of the information processing device acquiring a first adjacent entity, which is an entity representing an item having an adjacent relationship with the item represented by the entity selected by the selection unit, and acquiring a second adjacent entity, which is an entity representing an item having an adjacent relationship with the item represented by the entity, for each of a plurality of entities classified into the group to be matched and stored in the second database; an information processing method comprising: a determination unit of the information processing device determining a similarity between the second adjacent entity associated with each of a plurality of entities classified into the group of entities to be matched and stored in the second database, which is acquired by the acquisition unit, and the first adjacent entity acquired by the acquisition unit, thereby determining the second adjacent entity that is most similar to the first adjacent entity among the second adjacent entities associated with each of a plurality of entities classified into the group of entities to be matched and stored in the second database, and determining that the entity that represents an item having an adjacent relationship with the item represented by the second adjacent entity and that is classified into the group of entities to be matched and stored in the second database and the entity to be matched are entities that represent the same item.

Images