Information processing device, information processing method, manufacturing method, and program

Abstract

In order to better predict the identity between record pairs, an information processing device (1) is provided with an acquisition means (11) that acquires a record pair, a similarity calculation means (12) that uses a plurality of similarity functions to calculate a plurality of similarities for the record pair, a prediction means (13) that references the record pair and the plurality of similarities and uses a significance determined in accordance with the record pair to perform an identity prediction for the record pair, and an output means (14) that outputs the prediction results from the prediction means (13).

Description

[Technical Field]

[0001] The present invention relates to a technique for predicting identity between record pairs. [Background technology]

[0002] A process is performed to identify and associate combinations of identical or similar records from records stored in different tables. This process is also called name matching. Name matching enables centralized management of tables and expansion of data. Techniques for performing name matching include machine learning or rule-based matching. For example, Patent Document 1 and Non-Patent Document 1 describe techniques for performing name matching using machine learning. In particular, the name matching processing device described in Patent Document 1 is composed of an information processing device, a storage unit, and an operation terminal. This name matching processing device calculates the similarity between record pairs using multiple similarity functions that calculate the similarity between record pairs, and learns the weights of the similarity through machine learning using training data. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-185244 [Non-patent literature]

[0004] [Non-Patent Document 1] Pradap Konda, et. al., Magellan: Toward Building Entity Matching Management Systems, Proceedings of the VLDB Endowment, 2016 Summary of the Invention [Problem to be solved by the invention]

[0005] There are various methods for determining the identity of record pairs. For example, the accuracy of determining identity for the record pair "ice" and "aisu" can be improved by changing the notation from katakana to hiragana. Also, the accuracy of determining identity for the record pair "potato chips" and "potachi" can be improved by extracting substrings. As such, the method suitable for determining identity for a record pair may differ depending on the record pair. The techniques described in Patent Document 1 and Non-Patent Document 1 have had the problem of being unable to properly determine identity for some record pairs.

[0006] One aspect of the present invention has been made in view of the above-mentioned problems, and one object of the present invention is to provide a technique that can more suitably predict the identity of a record pair. [Means for solving the problem]

[0007] An information processing device according to one aspect of the present invention includes an acquisition means for acquiring a record pair, a similarity calculation means for calculating a plurality of similarities for the record pair using a plurality of similarity functions, a prediction means for predicting identity of the record pair by referring to the record pair and the plurality of similarities and using an importance determined according to the record pair, and an output means for outputting a prediction result by the prediction means.

[0008] Furthermore, an information processing device according to one aspect of the present invention includes an acquisition means for acquiring training data including a plurality of pairs of record pairs and labels related to identity of the record pairs; and a parameter generation means for generating, by referring to the training data, at least one parameter of an importance calculation model used for calculating the importance by a prediction means that performs identity prediction of the record pair to be predicted using one or more parameters of a plurality of similarity functions for calculating a plurality of similarities for the record pair to be predicted, the record pair to be predicted, and the plurality of similarities.

[0009]

[0016] Also, an information processing method according to one aspect of the present invention includes acquiring a record pair; calculating a plurality of similarities for the record pair using a plurality of similarity functions; referring to the record pair and the plurality of similarities, performing an identity prediction for the record pair using an importance determined according to the record pair; and outputting a prediction result based on the identity prediction for the record pair.

[0010] Furthermore, an information processing method according to one aspect of the present invention includes acquiring training data including a plurality of pairs of record pairs and labels related to identity of the record pairs; and a prediction means that performs identity prediction for the record pair to be predicted using an importance determined according to the record pair to be predicted by referring to one or more parameters of each of a plurality of similarity functions for calculating a plurality of similarities for the record pair to be predicted, the record pair to be predicted, and the plurality of similarities, generates at least one parameter of an importance calculation model used to calculate the importance by referring to the training data.

[0011] Furthermore, a manufacturing method according to one aspect of the present invention includes acquiring training data including a plurality of pairs of record pairs and labels related to the identity of the record pairs; and generating, by referring to the training data, at least one of the importance calculation models used by a prediction means that calculates a plurality of similarities for a record pair to be predicted, and that predicts the identity of the record pair to be predicted using an importance determined according to the record pair to be predicted, by referring to the record pair to be predicted and the plurality of similarities.

[0012] Furthermore, a program according to one aspect of the present invention causes a computer to execute an acquisition process for acquiring a record pair; a similarity calculation process for calculating a plurality of similarities for the record pair using a plurality of similarity functions; a prediction process for predicting identity of the record pair by referring to the record pair and the plurality of similarities and using an importance determined according to the record pair; and an output process for outputting a prediction result obtained by the prediction process.

[0013] Furthermore, a program according to one aspect of the present invention causes a computer to execute an acquisition process of acquiring training data including a plurality of pairs of record pairs and labels related to identity of the record pairs; and a parameter generation process of generating, by referring to the training data, at least one parameter of an importance calculation model used for calculating the importance by a prediction means that performs identity prediction of the record pair to be predicted using one or more parameters of each of a plurality of similarity functions for calculating a plurality of similarities for the record pair to be predicted, the record pair to be predicted, and the plurality of similarities. [Effects of the Invention]

[0014] According to one aspect of the present invention, the identity of a record pair can be more appropriately predicted. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a block diagram showing a configuration of an information processing device according to a first exemplary embodiment. [Figure 2] 1 is a flowchart showing the flow of an information processing method according to the first exemplary embodiment. [Figure 3] 1 is a block diagram showing a configuration of an information processing device according to a first exemplary embodiment. [Figure 4] 1 is a flowchart showing the flow of an information processing method according to the first exemplary embodiment. [Figure 5] FIG. 10 is a block diagram showing the configuration of an information processing device according to a second exemplary embodiment. [Figure 6] 10A and 10B are diagrams illustrating specific examples of first data and second data according to the second exemplary embodiment. [Figure 7] 10 is a flowchart showing the flow of an information processing method according to the second exemplary embodiment. [Figure 8] FIG. 10 is a diagram showing a specific example of integrated data according to the second exemplary embodiment. [Figure 9] FIG. 10 is a block diagram showing a configuration of an information processing device according to a third exemplary embodiment. [Figure 10] FIG. 10 is a flow chart showing the flow of an information processing method according to the third exemplary embodiment. [Figure 11] FIG. 10 is a block diagram showing the configuration of an information processing device according to a fourth exemplary embodiment. [Figure 12] 10A and 10B are diagrams illustrating examples of screen displays according to the fourth exemplary embodiment. [Figure 13] FIG. 1 is a block diagram illustrating a configuration of a computer that functions as an information processing device according to each exemplary embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0016] Exemplary Embodiment 1 A first exemplary embodiment of the present invention will be described in detail with reference to the drawings. This exemplary embodiment is a basic form of the exemplary embodiments described below.

[0017] <Configuration of information processing device 1> The configuration of an information processing device 1 according to this exemplary embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the configuration of the information processing device 1. The information processing device 1 is a device that performs identity prediction for record pairs. The information processing device 1 includes an acquisition unit 11, a similarity calculation unit 12, a prediction unit 13, and an output unit 14.

[0018] (Acquisition part 11) The acquisition unit 11 acquires record pairs.

[0019] (Record vs. Record) A record pair is a set of multiple records. For example, a record is a row of a table, and includes one or more sets of attribute names and attribute values ​​corresponding to columns of the table. The number of records included in a record pair may be two, or may be three or more. For example, a record pair is a set of a record included in a first table and a record included in a second table. For example, the first table and the second table are tables that store customer information of a business, or tables that store product information. However, the first table and the second table are not limited to the above examples, and may be other tables. Furthermore, the first table and the second table may be the same or different.

[0020] (Similarity calculation unit 12) The similarity calculation unit 12 calculates a plurality of similarities using a plurality of similarity functions for the record pairs acquired by the acquisition unit 11. In other words, the similarity calculation unit 12 calculates a plurality of similarities using k (k is an integer of 2 or more) similarity functions φ i (1≦i≦k) is used to calculate k similarities for one record pair.

[0021] (Similarity function) Similarity function φ i is a function for calculating the similarity between records included in a record pair. In the following, the similarity function φ i is also called the "similarity calculation model." The similarity function φ i The input of is a record pair, and the similarity function φ i The output of the function is the similarity between the records in the record pair. i can be a target of learning by the information processing device 2 described later. i is generated by machine learning, the similarity function φ iThe machine learning technique is not limited, and for example, a decision tree-based, linear regression, or neural network technique may be used, or two or more of these techniques may be used. Examples of decision tree-based techniques include LightGBM (Light Gradient Boosting Machine), random forest, and XGBoost. Examples of linear regression techniques include Bayesian regression, support vector regression, Ridge regression, Lasso regression, and ElasticNet. Examples of neural networks include deep learning.

[0022] Similarity function φ i For example, the similarity function φ outputs a numerical value between 0 and 1 as the similarity. i For example, the Jaccard coefficient can be used as the similarity function φ. The Jaccard coefficient calculates |A∩B| / |A∪B| for a set A = {a1, a2, ...} and a set B = {b1, b2, ...}. i For example, the method described in Non-Patent Document 1 may be used as the similarity function φ i For example, the method described in the document "Yuliang Li, Jinfeng Li, Yoshihiko Suhara, AnHai Doan, Wang-Chiew Tan, Deep Entity Matching with Pre-Trained Language Models, Proceedings of the VLDB Endowment, 2016" (hereinafter referred to as "Non-Patent Document 2") may be used as the similarity function φ. i is not limited to the above example, and other methods may be used to calculate the similarity between record pairs.

[0023] (Prediction Section 13) The prediction unit 13 refers to the record pair and a plurality of similarities, and predicts the identity of the record pair using the importance determined according to the record pair.

[0024] (Importance) The importance is information determined according to the record pair. As an example, the importance is calculated with reference to the record pair. More specifically, as an example, the prediction unit 13 calculates the importance using an importance calculation model for calculating the importance. In this case, the input of the importance calculation model is the record pair. Also, the output of the importance calculation model is the importance. The importance calculation model may be a target for learning by the information processing device 2 described later. When the importance calculation model is generated by machine learning, the machine learning method of the importance calculation model is not limited, and as an example, a decision tree-based, linear regression, or neural network method may be used, or two or more of these methods may be used.

[0025] The importance calculation model is generated using a language model such as BERT (Bidirectional Encoder Representations from Transformers), fastText, word2vec, tf-idf, or BM25, for example. The importance calculation model may also include a language model. A specific example of the importance calculation process when using a language model will be described. For example, the prediction unit 13 converts record pairs into vectors using a language model, and then converts these vectors into vectors in another feature space. Furthermore, the prediction unit 13 inputs these vectors into a k-class classifier (such as a softmax function) to calculate k importances. Each of the k calculated importances is calculated using k similarity functions φ i corresponds to each of the following.

[0026] However, the method for calculating the importance is not limited to the above example, and the prediction unit 13 may calculate the importance using other methods. As an example, the prediction unit 13 may calculate the importance using rule-based processing. For example, the prediction unit 13 may calculate the importance by referring to a table that associates the importance with information about the record pair. Here, the information about the record pair may include, for example, feature amounts of the records included in the record pair, classification results of the records, or names of the records.

[0027] As an example, the prediction unit 13 predicts identity between record pairs using a linear sum of multiple similarities calculated by the similarity calculation unit 12, where each importance is used as a weighting coefficient. However, the method by which the prediction unit 13 predicts identity is not limited to the method using a linear sum, and the prediction unit 13 may predict identity between record pairs using other methods.

[0028] For example, the prediction unit 13 may perform identity prediction for a record pair by inputting the record pair and the similarity into a prediction model generated by machine learning. In this case, for example, the input to the prediction model includes a set of k similarities and the record pair. Furthermore, for example, the output of the prediction model includes a predicted result of identity. Furthermore, the prediction unit 13 calculates the parameters of the prediction model as importance. The machine learning method for the prediction model is not limited, and for example, a decision tree-based method, a linear regression method, or a neural network method may be used, or two or more of these methods may be used.

[0029] (Output unit 14) The output unit 14 outputs the prediction result by the prediction unit 13. The prediction result includes, for example, information indicating whether the records included in the record pair are identical or information indicating the similarity between the records included in the record pair.

[0030] The prediction result by the prediction unit 13 is used, for example, in table integration processing or information search processing. When integrating tables, by linking records predicted to be identical by the prediction unit 13, multiple tables can be integrated and data can be centrally managed. Furthermore, in information search, the prediction unit 13 may perform identity prediction for a record pair between a record used as a search key (for example, a record specified by a user) and any other record registered in a specified table. In this case, the information processing device 1 may output, as a search result, a record included in the record pair predicted to be identical by the prediction unit 13. This enables search processing in a table that is not linked with the record used as the search key.

[0031] <Effects of information processing device 1> As described above, the information processing device 1 according to this exemplary embodiment is configured to calculate multiple similarities for a record pair using multiple similarity functions, and to perform identity prediction for the record pair by referring to the record pair and the multiple similarities and using importance determined for the record pair. Here, because the importance is determined for each record pair, the result of identity prediction based on multiple similarities is not based on a uniform method, but reflects the importance of each record pair. Therefore, the information processing device 1 according to this exemplary embodiment can achieve the effect of more appropriately predicting identity for a record pair.

[0032] <Flow of information processing method S1> The flow of the information processing method S1 according to this exemplary embodiment will be described with reference to FIG. 2. FIG. 2 is a flow diagram showing the flow of the information processing method S1. In step S11, the acquisition unit 11 acquires a record pair. In step S12, the similarity calculation unit 12 calculates multiple similarities for the record pair using multiple similarity functions. In step S13, the prediction unit 13 refers to the record pair and the multiple similarities, and predicts identity of the record pair using importance determined according to the record pair. In step S14, the output unit 14 outputs the prediction result by the prediction unit 13.

[0033] <Effects of information processing method S1> As described above, the information processing method S1 according to this exemplary embodiment employs a configuration in which a plurality of similarities are calculated for a record pair using a plurality of similarity functions, and identity prediction for the record pair is performed by referring to the record pair and the plurality of similarities and using importance determined for the record pair. Therefore, the information processing method S1 according to this exemplary embodiment has the effect of more suitably predicting identity between the record pair.

[0034] <Configuration of information processing device 2> Next, the configuration of an information processing device 2 according to this exemplary embodiment will be described with reference to Fig. 3. Fig. 3 is a block diagram showing the configuration of the information processing device 2. The information processing device 2 is a device that generates parameters used to predict the identity of a record pair. The information processing device 2 includes an acquisition unit 21 and a parameter generation unit 22.

[0035] The acquiring unit 21 acquires training data including a plurality of pairs of record pairs and labels relating to the identity of the record pairs. As an example, the labels relating to the identity indicate whether the records included in the record pairs are identical or not.

[0036] The parameter generating unit 22 (i) generates a plurality of similarity functions φ for calculating a plurality of similarities for a record pair to be predicted. i and (ii) one or more parameters of an importance calculation model used by a prediction unit 13 that performs identity prediction of a record pair to be predicted using importance determined according to the record pair to be predicted by referring to the record pair to be predicted and a plurality of similarities, to calculate the importance, the prediction unit 13 generates at least one of the parameters of the importance calculation model used to calculate the importance by referring to the training data.

[0037] <Effects of information processing device 2> As described above, the information processing device 2 according to this exemplary embodiment acquires training data including a plurality of pairs of record pairs and labels related to the identity of the record pairs, and uses one or more parameters of each of a plurality of similarity functions for calculating a plurality of similarities for the record pair to be predicted, as well as the record pair to be predicted and the plurality of similarities, to perform identity prediction for the record pair to be predicted using an importance determined according to the record pair to be predicted. The prediction means generates at least one of one or more parameters of an importance calculation model used to calculate the importance by referring to the training data. Therefore, the information processing device 2 according to this exemplary embodiment has the effect of being able to generate parameters that can more appropriately predict the identity of the record pairs.

[0038] <Flow of information processing method S2> The flow of the information processing method S2 according to this exemplary embodiment will be described with reference to FIG. 4. FIG. 4 is a flow diagram showing the flow of the information processing method S2. In step S21, the acquisition unit 21 acquires training data including a plurality of pairs of record pairs and labels related to the identity of the record pairs. In step S22, the parameter generation unit 22 generates, with reference to the training data, at least one of the following parameters: (i) one or more parameters of each of a plurality of similarity functions for calculating a plurality of similarities for the record pair to be predicted; and (ii) one or more parameters of an importance calculation model used for calculating importance by a prediction means that performs identity prediction for the record pair to be predicted by referring to the record pair to be predicted and the plurality of similarities and using importance determined according to the record pair to be predicted.

[0039] <Effects of information processing method S2> As described above, the information processing method S2 according to this exemplary embodiment acquires training data including a plurality of pairs of record pairs and labels related to the identity of the record pairs, and uses one or more parameters of each of a plurality of similarity functions for calculating a plurality of similarities for the record pair to be predicted, as well as the record pair to be predicted and the plurality of similarities, to perform identity prediction for the record pair to be predicted using an importance determined according to the record pair to be predicted. The prediction means generates at least one of the one or more parameters of an importance calculation model used to calculate the importance by referring to the training data. Therefore, the information processing method S2 according to this exemplary embodiment has the effect of generating parameters that can more appropriately predict the identity of the record pairs.

[0040] <Manufacturing method> The information processing device 2 can also be specified as a device that executes a method for producing a trained model. Here, the method for producing a trained model includes acquiring training data including a plurality of pairs of record pairs and labels related to the identity of the record pairs, and generating at least one of a plurality of similarity calculation models and importance calculation models by referring to the training data.

[0041] Exemplary Embodiment 2 A second exemplary embodiment of the present invention will be described in detail with reference to the drawings. Note that components having the same functions as those described in the first exemplary embodiment are given the same reference numerals, and their description will not be repeated.

[0042] <Configuration of information processing device 1A> 5 is a block diagram showing the configuration of an information processing device 1A according to this exemplary embodiment. The information processing device 1A includes a control unit 10A, a storage unit 20A, a communication unit 30A, and an input / output unit 40A.

[0043] (Communication unit 30A) The communication unit 30A communicates with devices external to the information processing device 1A via a communication line. While the specific configuration of the communication line does not limit the present exemplary embodiment, examples of the communication line include a wireless LAN (Local Area Network), a wired LAN, a WAN (Wide Area Network), a public line network, a mobile data communication network, or a combination thereof. The communication unit 30A transmits data supplied from the control unit 10A to other devices, and supplies data received from other devices to the control unit 10A.

[0044] (Input / output section 40A) Input / output devices such as a keyboard, a mouse, a display, a printer, and a touch panel are connected to the input / output unit 40A. The input / output unit 40A receives various types of information input to the information processing device 1A from the connected input devices. Furthermore, the input / output unit 40A outputs various types of information to connected output devices under the control of the control unit 10A. An example of the input / output unit 40A is an interface such as a USB (Universal Serial Bus).

[0045] (Control unit 10A) As shown in FIG. 5, the control unit 10A includes an acquisition unit 11, a similarity calculation unit 12, a prediction unit 13, an output unit 14, and an integration unit 15A.

[0046] (Acquisition part 11) The acquiring unit 11 acquires first data x including a first record e included in the record pair, and second data x' including a second record e' included in the record pair. The first data x and the second data x' are, for example, tables including a plurality of records. As an example, the first record e∈x and the second record e'∈x' are expressed as follows: e=(a1:v1,a2:v2,…,a d :v d ) e´=(a1:v1,a2:v2,…,a d´ :v´ d´ )

[0047] where a l ∈A l (l=1, 2, …d), and a´ m ∈A´ m (m=1, 2, …d´), are attribute names, and A l and A' m is, for example, the string space. l ∈V l and v´ m ∈V´ m is the attribute value, and V l and V´ mis, for example, a character string space or a real number space. d is the number of attributes that record e has, and d' is the number of attributes that record e' has. In other words, the first record e and the second record e' each include multiple sets of attribute names and attribute values.

[0048] FIG. 6 shows tables T1 and T2, which are specific examples of first data x and second data x'. Tables T1 and T2 consist of rows and columns, with rows corresponding to records and columns corresponding to attributes. In other words, table T1 includes multiple first records e1, e2, .... Table T2 includes multiple second records e'1, e'2, ....

[0049] 6 is expressed as e2=(product name: potato chips, price: 198). In the first record e2, the attribute value of the attribute whose attribute name is "product name" is "potato chips", and the attribute value of the attribute whose attribute name is "price" is "198".

[0050] The attribute name and attribute value of table T1 may be the same as or different from the attribute name and attribute value of table T2. In the example of Fig. 6, the record pair (e, e') acquired by the acquisition unit 11 is a pair of any one of first records e1, e2, ... included in table T1 and any one of second records e'1, e'2, ... included in table T2.

[0051] (Similarity calculation unit 12) The similarity calculation unit 12 calculates k (k is an integer equal to or greater than 2) similarity functions φ i (1≦i≦k) for one record pair (e, e´), the k similarities s i The similarity calculation unit 12 calculates the k similarities s i The process of calculating the value of will be described in detail later.

[0052] (Prediction Section 13) The prediction unit 13 calculates a record pair (e, e') and a plurality of similarities s iand performs identity prediction for the record pair using the importance determined according to the record pair (e, e'). In this exemplary embodiment, the prediction unit 13 includes an importance calculation unit 131A that calculates the importance by referring to the record pair (e, e'). Details of the identity prediction process performed by the prediction unit 13 and the importance calculation process performed by the importance calculation unit 131A will be described later.

[0053] (Output unit 14) The output unit 14 outputs the prediction result by the prediction unit 13. For example, the prediction result includes information indicating whether the records included in the record pair are identical. The prediction result may also include information indicating the degree of similarity between the records included in the record pair. The output unit 14 may output the prediction result by writing it to the storage unit 20A or an external storage device, or may output it to an output device (such as a display device or a printer) connected to the input / output unit 40A. The output unit 14 may also output the prediction result by transmitting it to another device via the communication unit 30A.

[0054] (Integrated Section 15A) The integration unit 15A generates integrated data from the first data and the second data by referring to the prediction result output by the output unit 14. The integrated data generation process performed by the integration unit 15A will be described later in detail.

[0055] (Storage unit 20A) The storage unit 20A stores the first data x and the second data x' acquired by the acquisition unit 11, and also stores the prediction result PR of the prediction unit 13. The storage unit 20A also stores a plurality of similarity functions φ i , the importance calculation model g, and the parameter P are stored.

[0056] Similarity function {φ1,…,φ k} is a function for calculating the similarity using, for example, the Jaccard coefficient or the method described in Non-Patent Document 1 or Non-Patent Document 2, as shown in the above-described exemplary embodiment 1. The similarity function φ iis input by, for example, a user of the information processing device 1A. i For example, the similarity function φ outputs a numerical value between 0 and 1 for a record pair (e, e'). In this case, for example, the closer the output value is to 1, the higher the similarity, and the closer it is to 0, the lower the similarity. i is, as an example, a function with learnable parameters.

[0057] The importance calculation model g is a model used by the importance calculation unit 131A to calculate importance. As shown in the above-described exemplary embodiment 1, the importance calculation model g is generated using a language model such as BERT, fastText, word2vec, tf-idf, or BM25. The importance calculation model g may also include a language model.

[0058] The parameter P stored in the storage unit 20A is a function of k similarity functions φ i Each of the parameters θ i , and one or more parameters w included in the importance calculation model g.

[0059] <Flow of information processing method S1A> 7 is a flow diagram showing the flow of an information processing method S1A, which is an example of an information processing method executed by the information processing device 1A. Note that some steps may be executed in parallel or in a different order. Also, the contents that have already been explained will not be explained again.

[0060] (Step S101) In step S101, the acquisition unit 11 acquires first data and second data. As an example, the acquisition unit 11 acquires the first data and second data input by a user of the information processing device 1A or the like using an input device connected to the input / output unit 40A. Alternatively, the acquisition unit 11 may acquire the first data and second data by receiving the first data and second data from another device via the communication unit 30A. Alternatively, the acquisition unit 11 may acquire the first data and second data by reading the first data and second data from an externally connected storage device. The acquisition unit 11 stores the acquired first data and second data in the storage unit 20A.

[0061] (Step S102) In step S102, the acquisition unit 11 acquires the parameter P stored in the storage unit 20A.

[0062] (Step S103) In step S103, the obtaining unit 11 obtains a record pair (e, e') that is the prediction target.

[0063] (Step S104) In step S104, the similarity calculation unit 12 calculates k similarity functions φ i Using the above, k similarities s i Calculate the k similarity functions φ i Since each of the k similarities is different, the k similarities s i For example, in the case of a record pair of "ice" and "aisu", the similarity s calculated after changing the spelling is i indicates high similarity, while the similarity s calculated by extracting substrings i is a value that indicates low similarity. In the case of a record pair of "potato chips" and "potato chips", the similarity s calculated after changing the notation is i indicates low similarity, while the similarity s calculated by extracting substrings i is a value that indicates high similarity.

[0064] (Step S105) In step S105, the importance calculation unit 131A refers to the record pair (e, e') and calculates a plurality of similarities s i The importance of each of g i For example, the importance calculation unit 131A calculates the importance g using the importance calculation model g. i Calculate.

[0065] The importance calculation model g uses multiple similarities s i For each of the following, the importance g i The importance calculation model g is a model for calculating the following, for example:

number

[0066] The importance g calculated by the importance calculation unit 131A i A specific example of the calculation process will be described. First, the importance calculation unit 131A converts the character strings of each attribute value of the first record e and the second record e' into vectors using a language model. Specifically, for example, the importance calculation unit 131A converts a record pair (e = (product name: potato chips, price: 198), e' = (product name: potato chips, rating: 5)) into a character string such as "[CLS][COL] product name [VAL] potato chips [COL] price [VAL] 198 [SEP][COL] product name [VAL] potato chips [COL] rating [VAL] 5 [SEP]" using the function serialize(e, e') that converts the record pair (e, e') into a character string. Here, [CLS], [COL], [VAL], and [SEP] are symbols that indicate the start of a sentence, an attribute name, an attribute value, and a record separator, respectively.

[0067] Furthermore, the importance calculation unit 131A converts the generated character string into a vector using a language model (for example, BERT). Subsequently, the importance calculation unit 131A applies concatenation, summation, deep learning, etc. to the vector obtained by the language model to convert it into a new L-dimensional vector z.

[0068] Furthermore, the importance calculation unit 131A inputs the converted L-dimensional vector z to a k-class classifier to calculate k importance {g(e, e')} i The k-class classifier may be, for example, a linear classifier, deep learning, or other similar technology. For example, the k-class classifier may be the technology described in the document "Robert A. Jacobs, Michael Jordan, Geoffrey Hinton: Adaptive Mixtures of Local Experts, Neural Computation 3, 79-87 (1991)" or the technology described in the document "Noam Shazeer, Quoc Le, Geoffrey Hinton: Jeffrey Dean: OUTRAGEOUSLY LARGE NEURAL NETWORKS: THE SPARSELY-GATED MIXTURE-OF-EXPERTS LAYER, ICLR 2017."

[0069] For example, for i=1,...,k, the L-dimensional vector w i The importance {g(e,e´)} is the i-th dimension output of the linear softmax function. i teeth, exp(w i ^T·z) / (exp(w1^T·z)+exp(w2^T·z)+…+exp(w k ^T·z)) where the L-dimensional vector w i is an example of a learnable parameter w of the importance calculation model g. i ^T·z" is an L-dimensional vector w i and the dot product of an L-dimensional vector z.

[0070] (Step S106) In step S106, the prediction unit 13 calculates the similarity s calculated by the similarity calculation unit 12. i and the record pair (e, e'). The prediction unit 13 predicts the identity of the record pair (e, e'). For example, the prediction unit 13 predicts the identity of the record pair (e, e') by using k similarities s i The similarity between the records in the record pair (e, e') is calculated using the above formula, and if the calculated similarity is greater than a threshold q (e.g., q=0.5), it is predicted that the records e and e' are identical, and if the calculated probability is equal to or less than the threshold q, it is predicted that they are not identical.

[0071] The probability calculated by the prediction unit 13 is calculated by the k similarities s i The similarity s indicates the result of integrating and predicting the record pair (e, e') and the similarity s i The probability is calculated using a probability function h that takes k similarities s as input. i =φ i Using (e, e'), it is expressed as follows (Equation 1).

number

[0072] In the above (Equation 1), the importance {g(e, e')} i is the importance calculated by the importance calculation unit 131A, and the similarity s i =φ i (e, e´) is the similarity function φ i When using (Equation 1), in other words, the prediction unit 13 calculates a plurality of similarities s i The linear sum of each importance {g(e,e´)} i The identity prediction is performed using a linear sum with weighting coefficients.

[0073] In this exemplary embodiment, k similarities s calculated for each of a plurality of different record pairs (e, e') are iEven if the importance {g(e,e´)} i In other words, the prediction result by the prediction unit 13 includes the similarity s i In addition, the importance g determined by the record pair i In this way, the method by which the prediction unit 13 predicts identity can differ depending on the record pair.

[0074] (Step S107) In step S107, the output unit 14 outputs the prediction result of the prediction unit 13. As an example, the output unit 14 stores the prediction result in the storage unit 20A.

[0075] (Step S108) In step S108, the prediction unit 13 determines whether identity prediction has been performed for all record pairs (e, e') to be predicted. If the prediction process has been completed for all record pairs (e, e') to be predicted (step S108; YES), the prediction unit 13 proceeds to the process of step S109. On the other hand, if there are still record pairs (e, e') to be predicted (step S108; NO), the prediction unit 13 returns to the process of step S103 and performs identity prediction for the next record pair (e, e'). That is, the information processing device 1A executes the processes of steps S103 to S107 for all record pairs (e, e') to be predicted.

[0076] (Step S109) In step S109, the integrating unit 15A generates integrated data from the first data and the second data by referring to the prediction result output by the output unit 14. As an example, the integrated data includes a record obtained by integrating records included in the record pair predicted by the prediction unit 13 to be identical.

[0077] FIG. 8 is a diagram showing table T3, which is an example of integrated data. Table T3 includes multiple records f1, f2, .... Record f1 is a record obtained by integrating the first record e1 and the second record e'2 in FIG. 6. Record f2 is a record obtained by integrating the first record e2 and the second record e'3 in FIG. 6. Record f3 is a record obtained by integrating the first record e3 and the second record e'1 in FIG. 6.

[0078] <Example> Next, a specific example of this exemplary embodiment will be described. In this example, the similarity function {φ i Similarity functions φ1 to φ3 are used as}. Similarity function φ1 is a function that calculates the Jaccard coefficient of the product name of the record pair. Similarity function φ2 is a function that converts the product name of the record pair in hiragana to katakana and then calculates the Jaccard coefficient. Similarity function φ3 is a function that calculates the similarity using the method described in Non-Patent Document 2 mentioned above. Here, similarity function φ3 has a learnable parameter θ3.

[0079] (Step S101) In step S101 of FIG. 7, the acquisition unit 11 acquires test data D test ={((Product name: Soy sauce rice crackers, price: 268), (Product name: Soy sauce rice crackers, rating: 4)), ..., ((Product name: Mugwort dumplings, price: 190), (Product name: Mitarashi dango, rating: 3))}.

[0080] (Steps S102 to S104) The similarity calculation unit 12 calculates the similarity S=(s1, s2, s3). Here, the similarity calculation unit 12 reads the parameter θ3 from the storage unit 20A and calculates the similarity s3 using the read parameter θ3. Specifically, the similarity calculation unit 12 calculates the similarity S=(s1, s2, s3) using the test data D testCalculate the similarity S=(φ1(e,e´),φ2(e,e´),φ3(e,e´))^T=(0,1,0.7)^T of the record pair {e=(product name: soy sauce rice cracker, price: 268), e´=(product name: soy sauce rice cracker, rating: 4)}.

[0081] (Step S105) The prediction unit 13 uses a function serialize(e,e') that concatenates the attribute names and attribute values ​​of the record pair (e,e') to create a string "[CLS][COL] Product name [VAL] Shoyu Senbei [COL] Price [VAL] 268 [SEP] [COL] Product name [VAL] Shoyu Senbei [COL] Rating [VAL] 4 [SEP]" from the record pair (e,e'). The prediction unit 13 also obtains an L-dimensional vector v, which is a vector representation of this string, using BERT, a pre-trained language model. Furthermore, the prediction unit 13 uses a linear softmax function to calculate a similarity function φ for i=1, 2, 3. i The importance g, which is the weight of i of, g i =e^(w i ^T·v) / (e^(w1^T·v)+e^(w2^T·v)+e^(w3^T·v)) and obtain (g1, g2, g3) = (0.1, 0.6, 0.3), where w1, w2, and w3 are real vectors and are examples of learnable parameters w of the importance calculation model g.

[0082] (Step S106) In step S106, the prediction unit 13 adds importance to the similarity S calculated by the similarity calculation unit 12. i The sum of the multiplication is calculated as the probability. Since the similarity S = (0, 1, 0.7)^T and the importance (g1, g2, g3) = (0.1, 0.6, 0.3), h(e,e´)=0.1×0+0.6×1+0.3×0.7=0.81 Since the calculated value "0.81" is greater than the predetermined threshold value q=0.5, the prediction unit 13 predicts that the records e and e' included in the record pair (e, e') are the same.

[0083] (Step S107) In step S107, the output unit 14 outputs the identity prediction result for the record pair (e, e'). test This is applied to all record pairs in

[0084] <Effects of information processing device 1A> As described above, in the information processing device 1A according to this exemplary embodiment, the importance g i Calculate the importance g i Therefore, in addition to the effects of the information processing device 1 according to the first exemplary embodiment, the information processing device 1A according to the present exemplary embodiment has the advantage of predicting the identity of a record pair by using the importance g calculated using the record pair (e, e'). i This allows identity prediction to be performed taking into account the above, thereby achieving the effect of more appropriately predicting the identity of the record pair (e, e').

[0085] <Modification> In the above-described exemplary embodiment, the acquisition unit 11 further acquires auxiliary data u, and the prediction unit 13 obtains a record pair (e, e′) and a plurality of similarities s i and the auxiliary data u, and the importance g determined according to the record pair (e, e') and the auxiliary data u i may be used to perform identity prediction for the record pair (e, e').

[0086] The auxiliary data u includes, for example, information indicating the name of the record, the feature amount of the record, and / or the classification result of the record (sweets, person's name, etc.). Here, for example, the auxiliary data u may include information about the record obtained from external data such as Wikipedia (registered trademark). Also, for example, the auxiliary data u includes information about the record obtained from external data such as Wikipedia (registered trademark). iThe auxiliary data u may include the number of training data used in learning the parameter θ of the importance calculation model g and / or the parameter w of the importance calculation model g. However, the auxiliary data u is not limited to the above examples and may include other information. As an example, the auxiliary data u is a one-hot vector representing discrete information.

[0087] In this case, auxiliary data u is input to the importance calculation model g in addition to the record pair (e, e'). As an example, the auxiliary data u, which is a vector, is connected to the above-mentioned L-dimensional vector z, and the importance g is calculated using the connected vector and the parameter w. i is calculated.

[0088] In this modification, the prediction unit 13 calculates a record pair (e, e') and a plurality of similarities s i and the auxiliary data u, and the importance g determined according to the record pair (e, e') and the auxiliary data u i This allows the prediction unit 13 to improve the accuracy of predicting the identity of the record pair (e, e').

[0089] Exemplary Embodiment 3 A third exemplary embodiment of the present invention will be described in detail with reference to the drawings. Note that components having the same functions as those described in the first and second exemplary embodiments are denoted by the same reference numerals, and their description will not be repeated.

[0090] <Configuration of information processing device 1B> 9 is a block diagram showing the configuration of an information processing device 1B according to this exemplary embodiment. A control unit 10A of the information processing device 1B includes an acquisition unit 11, a similarity calculation unit 12, a prediction unit 13, an output unit 14, an integration unit 15A, and a learning unit 16B.

[0091] The acquisition unit 11 according to this exemplary embodiment acquires a record pair (e j ,e´ j ) and the record pair (e j ,e´ j ) identity label y jTraining data D containing multiple pairs of tr Further obtain the training data D tr is used to learn the parameters P mentioned above. The training data D tr As an example,

number

[0092] The learning unit 16B (i) calculates the similarity s i Multiple similarity functions φ used to calculate i Each of the parameters θ i and (ii) one or more parameters w included in an importance calculation model g used by the importance calculation unit 131A to calculate importance, by referring to the training data. The learning unit 16B is an example of the "parameter generating means" according to the present specification.

[0093] <Flow of information processing method S2B> 10 is a flow diagram showing the flow of an information processing method S2B, which is an example of an information processing method executed by information processing device 1B. Note that some steps may be executed in parallel or in a different order. Also, the contents already explained will not be explained again.

[0094] (Steps S201 and S202) In step S201, the acquisition unit 11 acquires training data D tr Obtain the training data D tris input by the user of the information processing device 1B, for example. In step S202, the acquisition unit 11 acquires a plurality of similarity functions φ i Obtain the similarity function φ i is input by the user of the information processing device 1B, for example.

[0095] (Step S203) In step S203, the learning unit 16B uses the training data D tr Using the parameter θ i and parameter w. Here, the parameter θ i is the similarity function φ i Furthermore, the parameter w is a set of parameters that the importance calculation model g has.

[0096] For example, the learning unit 16B calculates a parameter θ i and parameter w are optimized. This optimization can be done, for example, as follows:

number

number

[0097] In the objective function L, α is a non-negative hyperparameter. The hyperparameter α may be determined by a user of the information processing device 1B or may be determined by the training data D tr Alternatively, the identity may be determined automatically using a set of known record pairs. Ω is a regularization term for the parameters, and the L2 norm may be used. In the above formula, the parameter θ i Alternatively, we can fix and optimize only the parameter w.

[0098] The learning unit 16B uses the generated parameters w and θ i The parameter w and the parameter θ generated by the learning unit 16B are stored in the storage unit 20A. i is the similarity s calculated by the similarity calculation unit 12. i and / or the identity prediction process by the prediction unit 13.

[0099] <Example> Next, a specific example of this exemplary embodiment will be described. For example, for a first record e1=(product name: potato chips, price: 198) and a first record e2=(product name: ice cream, price: 148) in a table T1, and a second record e'1=(product name: potato chips, rating: 5) and a second record e'2=(product name: ice cream, rating: 4) in a table T2, the training data D tr of, D tr ={(e1,e´1,1),(e2,e´2,1),(e1,e´2,0),(e2,e´1,0)} Let's say.

[0100] Also, the similarity function {φ i Similarity functions φ1 to φ3 are used as the similarity functions φ1 to φ3. The similarity functions φ1 to φ3 are the same as the similarity functions φ1 to φ3 shown in the examples of the above-described exemplary embodiment 1. The similarity function φ3 has a learnable parameter θ3.

[0101] In step S201, the acquisition unit 11 acquires the training data D trIn step S203, the learning unit 16B obtains the training data D tr Record pair (e j ,e´ j ) so that the identity prediction is often correct, we use the parameter w of the importance calculation model g and the similarity function φ based on the cross-entropy error. i parameter θ i is optimized using stochastic gradient descent. The optimized parameters w and θ i is stored in the storage unit 20A.

[0102] <Effects of information processing device 1B> As described above, in the information processing device 1B according to this exemplary embodiment, the parameter w and the similarity function φ of the importance calculation model g are i The parameter θ i At least one of the parameters is set to the training data D tr Therefore, the information processing device 1B according to this exemplary embodiment has the effect of generating parameters that can more suitably predict the identity of a record pair, in addition to the effect achieved by the information processing device 1 according to the first exemplary embodiment.

[0103] <Modification> In the above exemplary embodiment, the training data D tr may contain auxiliary data u. In this case, the training data D tr As an example,

number

[0104] Exemplary Embodiment 4 A fourth exemplary embodiment of the present invention will be described in detail with reference to the drawings. Note that components having the same functions as those described in the first to third exemplary embodiments are denoted by the same reference numerals, and their description will not be repeated.

[0105] <Configuration of information processing device 1C> 11 is a block diagram showing the configuration of an information processing device 1C according to this exemplary embodiment. A control unit 10A of the information processing device 1C includes an acquisition unit 11, a similarity calculation unit 12, a prediction unit 13, an output unit 14, a learning unit 16B, and a search result output unit 17C.

[0106] The acquiring unit 11 according to this exemplary embodiment acquires input data from a user as a first record e included in a record pair (e, e'). The input data from a user is input, for example, by an input device (e.g., a keyboard, a mouse, etc.) connected to the input / output unit 40A.

[0107] Furthermore, the acquiring unit 11 acquires one of the multiple records included in the target data as the second record e' included in the record pair (e, e'). The target data is data to be searched, and includes, for example, one or multiple tables.

[0108] The prediction unit 13 performs identity prediction for a record pair between the first record e and each of the multiple records included in the target data. The search result output unit 17C references each prediction result PR output by the output unit 14 and outputs a search result based on the input data, in which the target data is the search target. As an example, the search result output unit 17C outputs the search result to an output device (display, printer, etc.) connected to the input / output unit 40A. The search result output unit 17C may also output the search result by transmitting the search result to another device connected via the communication unit 30A. The search result output unit 17C may also output the search result by storing the search result in the storage unit 20A or an external storage device.

[0109] 12 is a diagram showing a specific example of a screen display output by the search result output unit 17C. In the example of FIG. 12, the input data is a character string entered by the user in the text box 51, and the target data is table T1 and table T2 shown in FIG. 6 in the above-mentioned exemplary embodiment 1. The prediction unit 13 performs identity prediction for record pairs between a first record e, which is the user's input data, and each of a record included in table T1 and a record e' included in table T2. The identity prediction process performed by the prediction unit 13 has been described in the above-mentioned exemplary embodiment 2, and therefore will not be described again.

[0110] 12, the search result output unit 17C refers to the prediction result PR of the prediction unit 13 and outputs search result 53 and search result 54 based on the input data. Search result 53 is a search result obtained by searching table T1 using the character string "potato chips" as input data. Search result 54 is a search result obtained by searching table T2 using the character string "potato chips" as input data.

[0111] <Effects of the information processing device 1C> As described above, the information processing device 1C according to this exemplary embodiment employs a configuration in which search results based on input data and target data are output as search targets, with reference to each prediction result output by the output unit 14. Therefore, the information processing device 1C according to this exemplary embodiment has the effect of being able to more suitably search target data based on input data, in addition to the effect achieved by the information processing device 1 according to exemplary embodiment 1.

[0112] The information processing device 1C can also be described as follows. an acquisition means for acquiring input data from a user and one of a plurality of records included in the target data as a record pair; a similarity calculation means for calculating a plurality of similarities for the record pair using a plurality of similarity functions; a prediction means for predicting identity of a record pair between the input data and each of a plurality of records included in the target data by referring to the record pair and the plurality of similarities and using an importance determined in accordance with the record pair; an output means for outputting a search result based on the input data, the search result being a search target of the target data, by referring to the prediction result by the prediction means; An information processing device comprising:

[0113] [Software implementation example] Some or all of the functions of the information processing devices 1, 1A, 1B, 1C, and 2 (hereinafter referred to as "information processing devices 1, etc.") may be realized by hardware such as an integrated circuit (IC chip), or by software.

[0114] In the latter case, the information processing device 1, etc., is realized by, for example, a computer that executes instructions of a program, which is software that realizes each function. An example of such a computer (hereinafter referred to as computer C) is shown in FIG. 13. The computer C includes at least one processor C1 and at least one memory C2. The memory C2 stores a program P for operating the computer C as the information processing device 1, etc. In the computer C, the processor C1 reads and executes the program P from the memory C2, thereby realizing each function of the information processing device 1, etc.

[0115] The processor C1 may be, for example, a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), a micro processing unit (MPU), a floating point number processing unit (FPU), a physics processing unit (PPU), a microcontroller, or a combination thereof. The memory C2 may be, for example, a flash memory, a hard disk drive (HDD), a solid state drive (SSD), or a combination thereof.

[0116] The computer C may further include a RAM (Random Access Memory) for expanding the program P during execution and for temporarily storing various data. The computer C may also include a communication interface for transmitting and receiving data to and from other devices. The computer C may also include an input / output interface for connecting input / output devices such as a keyboard, mouse, display, and printer.

[0117] Furthermore, the program P can be recorded on a non-transitory tangible recording medium M that can be read by the computer C. Such a recording medium M can be, for example, a tape, a disk, a card, a semiconductor memory, or a programmable logic circuit. The computer C can acquire the program P via such a recording medium M. The program P can also be transmitted via a transmission medium. Such a transmission medium can be, for example, a communication network or broadcast waves. The computer C can also acquire the program P via such a transmission medium.

[0118] [Appendix 1] The present invention is not limited to the above-described embodiments, and various modifications are possible within the scope of the claims. For example, embodiments obtained by appropriately combining the technical means disclosed in the above-described embodiments are also included in the technical scope of the present invention.

[0119] [Appendix 2] Some or all of the above-described embodiments can also be described as follows: However, the present invention is not limited to the following described aspects.

[0120] (Appendix 1) an acquisition means for acquiring a record pair; a similarity calculation means for calculating a plurality of similarities for the record pair using a plurality of similarity functions; a prediction means for predicting identity between the record pair by referring to the record pair and the plurality of similarities and using an importance determined according to the record pair; an output means for outputting a prediction result by the prediction means; An information processing device comprising:

[0121] According to the above configuration, the identity of a record pair can be more suitably predicted.

[0122] (Appendix 2) The acquisition means further acquires auxiliary data; the prediction means refers to the record pair, the plurality of similarities, and the auxiliary data, and predicts identity between the record pair using importance determined according to the record pair and the auxiliary data; 10. The information processing device according to claim 1.

[0123] According to the above configuration, the importance is information that reflects not only the record pair but also the content of the auxiliary data. By predicting the identity of the record pair using such importance, it is possible to further increase the accuracy of predicting the identity of the record pair.

[0124] (Appendix 3) the prediction means includes importance calculation means for calculating the importance by referring to the record pair; 3. The information processing device according to claim 1 or 2.

[0125] According to the above configuration, by predicting identity between record pairs using the importance calculated with reference to the record pairs, it is possible to further increase the accuracy of predicting identity between record pairs.

[0126] (Appendix 4) the importance calculation means calculates an importance for each of the plurality of similarities; the prediction means performs the identity prediction using a linear sum of the plurality of similarities, the linear sum using each of the importance levels as a weighting coefficient. 4. The information processing device according to claim 3.

[0127] According to the above configuration, identity prediction is performed using a linear sum of similarities with importance as a weighting coefficient, thereby making it possible to improve the accuracy of predicting identity between record pairs.

[0128] (Appendix 5) the acquiring means further acquires training data including a plurality of pairs of record pairs and labels relating to the identity of the record pairs; The information processing device includes: one or more parameters included in each of the plurality of similarity functions used by the similarity calculation means to calculate the similarity; and one or more parameters included in an importance calculation model used by the importance calculation means to calculate the importance; and a parameter generating means for generating at least one of the parameters by referring to the training data. 5. The information processing device according to claim 3 or 4.

[0129] According to the above configuration, by using parameters generated with reference to training data, it is possible to more appropriately predict the identity of a record pair.

[0130] (Appendix 6) the acquiring means acquires first data including a first record included in the record pair and second data including a second record included in the record pair; The information processing device includes an integration unit that generates integrated data from the first data and the second data by referring to the prediction result output by the output unit. 6. An information processing device according to any one of appendices 1 to 5.

[0131] According to the above configuration, the first data and the second data can be more suitably integrated.

[0132] (Appendix 7) The acquisition means acquiring input data from a user as a first record included in the record pair; acquiring one of a plurality of records included in the target data as a second record included in the record pair; the prediction means performs the identity prediction for a record pair between the first record and each of a plurality of records included in the target data; The information processing device includes a search result output means for outputting a search result based on the input data, the search result being a search target of the target data, with reference to each of the prediction results output by the output means. 6. An information processing device according to any one of appendices 1 to 5.

[0133] According to the above configuration, it is possible to more suitably perform a search from target data based on input data.

[0134] (Appendix 8) an acquisition means for acquiring training data including a plurality of pairs of record pairs and labels relating to the identity of the record pairs; one or more parameters included in each of a plurality of similarity functions for calculating a plurality of similarities for the record pair to be predicted; and a prediction means for performing identity prediction of the record pair to be predicted by referring to the record pair to be predicted and the plurality of similarities and using an importance determined according to the record pair to be predicted, said prediction means calculating one or more parameters of an importance calculation model used to calculate the importance; a parameter generating means for generating at least one of the parameters by referring to the training data; An information processing device comprising:

[0135] According to the above configuration, it is possible to generate parameters that can more appropriately predict the identity of a record pair.

[0136] (Appendix 9) obtaining a record pair; calculating a plurality of similarities for the record pairs using a plurality of similarity functions; referring to the record pair and the plurality of similarities, and performing identity prediction for the record pair using importance determined according to the record pair; The aforementioned Record pair identity prediction outputting a prediction result by An information processing method including:

[0137] According to the information processing method, the same effects as those of the information processing device described above can be achieved.

[0138] (Appendix 10) acquiring training data including a plurality of pairs of record pairs and labels relating to the identity of the record pairs; one or more parameters included in each of a plurality of similarity functions for calculating a plurality of similarities for the record pair to be predicted; and a prediction means for predicting identity of the record pair to be predicted by referring to the record pair to be predicted and the plurality of similarities and using an importance determined according to the record pair to be predicted, wherein the prediction means calculates one or more parameters of an importance calculation model used to calculate the importance. generating at least one of the parameters by referring to the training data; An information processing method including:

[0139] According to the information processing method, the same effects as those of the information processing device described above can be achieved.

[0140] (Appendix 11) acquiring training data including a plurality of pairs of record pairs and labels relating to the identity of the record pairs; a plurality of similarity calculation models for calculating a plurality of similarities between the record pairs to be predicted; and a prediction means for performing identity prediction of the record pair to be predicted by referring to the record pair to be predicted and the plurality of similarities and using an importance determined in accordance with the record pair to be predicted, the prediction means comprising: a significance calculation model used to calculate the importance; generating at least one model of the above by referring to the training data; A method for producing a trained model including:

[0141] According to the above configuration, it is possible to produce a model that can more appropriately predict the identity of a record pair.

[0142] (Appendix 12) On the computer, a fetch operation for fetching record pairs; a similarity calculation process for calculating a plurality of similarities for the record pair using a plurality of similarity functions; a prediction process for predicting identity between the record pair by referring to the record pair and the plurality of similarities and using an importance determined according to the record pair; an output process for outputting a prediction result obtained by the prediction process; A program that executes the following.

[0143] According to the above configuration, the same effects as those of the above-mentioned information processing device can be achieved.

[0144] (Appendix 13) On the computer, an acquisition process for acquiring training data including a plurality of pairs of record pairs and labels relating to the identity of the record pairs; one or more parameters included in each of a plurality of similarity functions for calculating a plurality of similarities for the record pair to be predicted; and a prediction means for predicting identity of the record pair to be predicted by referring to the record pair to be predicted and the plurality of similarities and using an importance determined according to the record pair to be predicted, wherein the prediction means calculates one or more parameters of an importance calculation model used to calculate the importance. a parameter generation process for generating at least one of the parameters by referring to the training data; A program that executes the following.

[0145] According to the above configuration, the same effects as those of the above-mentioned information processing device can be achieved.

[0146] [Appendix 3] Some or all of the above-described embodiments can also be expressed as follows.

[0147] an information processing device comprising at least one processor that executes an acquisition process for acquiring a record pair; a similarity calculation process for calculating a plurality of similarities for the record pair using a plurality of similarity functions; a prediction process for predicting identity of the record pair by referring to the record pair and the plurality of similarities and using an importance determined according to the record pair; and an output process for outputting a prediction result obtained by the prediction process.

[0148] The information processing device may further include a memory that stores a program for causing the processor to execute the acquisition process, the similarity calculation process, the prediction process, and the output process. The program may be recorded on a computer-readable, non-transitory, tangible recording medium.

[0149] Some or all of the above-described embodiments can also be expressed as follows. an information processing apparatus comprising at least one processor that executes an acquisition process of acquiring training data including a plurality of pairs of record pairs and labels related to identity of the record pairs; and a parameter generation process of generating, by referring to the training data, at least one parameter of an importance calculation model used for calculating the importance by a prediction means that predicts identity of the record pair to be predicted using an importance determined according to the record pair to be predicted, by referring to one or more parameters of each of a plurality of similarity functions for calculating a plurality of similarities for the record pair to be predicted, and the record pair to be predicted and the plurality of similarities.

[0150] The information processing device may further include a memory that stores a program for causing the processor to execute the acquisition process and the parameter generation process. The program may be recorded on a computer-readable, non-transitory, tangible recording medium. [Explanation of symbols]

[0151] 1, 1A, 1B, 1C, 2 Information processing device 10A Control unit 11, 21 Acquisition Department 12 Similarity calculation unit 13 Prediction Department 14 Output section 15A Integration Department 16B Learning Department 17C Search result output section 20A storage section 22 Parameter Generation Unit 30A Communications Department 40A input / output section 131A Importance calculation part S1, S1A, S2, S2B Information Processing Methods

Claims

1. an acquisition means for acquiring a record pair; a similarity calculation means for calculating a plurality of similarities for the record pair using a plurality of similarity functions; a prediction means for predicting identity between the record pair by referring to the record pair and the plurality of similarities and using an importance determined according to the record pair; an output means for outputting a prediction result by the prediction means; An information processing device comprising:

2. The acquisition means further acquires auxiliary data; The prediction means refers to the record pair, the plurality of similarities, and the auxiliary data, and predicts identity of the record pair using importance determined according to the record pair and the auxiliary data. The information processing device according to claim 1 .

3. The prediction means includes importance calculation means for calculating the importance by referring to the record pair.

3. The information processing device according to claim 1.

4. the importance calculation means calculates an importance for each of the plurality of similarities; The prediction means performs the identity prediction using a linear sum of the plurality of similarities, the linear sum having each importance as a weighting coefficient. The information processing device according to claim 3 .

5. an acquisition means for acquiring training data including a plurality of pairs of record pairs and labels relating to the identity of the record pairs; a prediction means for performing identity prediction of a record pair to be predicted using an importance determined according to the record pair to be predicted by referring to one or more parameters of each of a plurality of similarity functions for calculating a plurality of similarities for the record pair to be predicted, the record pair to be predicted, and the plurality of similarities; and a parameter generation means for generating one or more parameters of an importance calculation model used to calculate the importance by referring to the training data. Equipped with the parameter generation means defines a loss function that represents an error between the probability that the records included in the record pair are identical and the label, based on the training data, and generates the parameters by minimizing an objective function that includes a regularization term for the loss function and parameters. Information processing device.

6. An information processing device, obtaining a record pair; calculating a plurality of similarities for the record pairs using a plurality of similarity functions; referring to the record pair and the plurality of similarities, and performing identity prediction for the record pair using importance determined according to the record pair; outputting a prediction result based on identity prediction of the record pair; An information processing method including:

7. An information processing device, acquiring training data including a plurality of pairs of record pairs and labels relating to the identity of the record pairs; a prediction means that performs identity prediction for the record pair to be predicted using importance determined according to the record pair to be predicted by referring to one or more parameters of each of a plurality of similarity functions for calculating a plurality of similarities for the record pair to be predicted, and the record pair to be predicted and the plurality of similarities, generates one or more parameters of an importance calculation model used to calculate the importance by referring to the training data; Including, The generating includes defining a loss function that represents an error between the probability that records included in the record pair are identical and the label, based on the training data, and generating the parameters by minimizing an objective function that includes the loss function and a regularization term for the parameters. Information processing methods.

8. An information processing device, acquiring training data including a plurality of pairs of record pairs and labels relating to the identity of the record pairs; a plurality of similarity calculation models for calculating a plurality of similarities for a record pair to be predicted, and a prediction means for performing identity prediction of the record pair to be predicted using importance determined according to the record pair to be predicted by referring to the record pair to be predicted and the plurality of similarities, generating importance calculation models used to calculate the importance by referring to the training data; Including, The generating step defines a loss function that represents an error between the probability that the records included in the record pair are identical and the label, based on the training data, and generates the model by minimizing an objective function that includes the loss function and a regularization term for a parameter. How to create a trained model.

9. On the computer, a fetch operation for fetching record pairs; a similarity calculation process for calculating a plurality of similarities for the record pair using a plurality of similarity functions; a prediction process for predicting identity between the record pair by referring to the record pair and the plurality of similarities and using an importance determined according to the record pair; an output process for outputting a prediction result obtained by the prediction process; A program that executes the following.

10. On the computer, an acquisition process for acquiring training data including a plurality of pairs of record pairs and labels relating to the identity of the record pairs; a parameter generation process in which a prediction means, which performs identity prediction of a record pair to be predicted using an importance determined according to the record pair to be predicted by referring to one or more parameters of a plurality of similarity functions for calculating a plurality of similarities for the record pair to be predicted, and the record pair to be predicted and the plurality of similarities, generates one or more parameters of an importance calculation model used to calculate the importance by referring to the training data; Execute the parameter generation process defines a loss function that represents an error between the probability that the records included in the record pair are identical and the label, based on the training data, and generates the parameters by minimizing an objective function that includes a regularization term for the loss function and parameters. program.

Images