Periodic entity matching method and system between film and television attribute data sources

Abstract

The invention discloses a periodic entity matching method and system between film and television attribute data sources. The method comprises the following steps: adding a plurality of first records into a first data source; obtaining a first index and a second index constructed for a second data source; searching a title and an alias of the first record in an index to obtain a plurality of candidate record pairs; adding a plurality of second records involved in the record pair to a second data source; sequentially calculating the similarity of the first record and the second record in each candidate record pair in each dimension; inputting the similarity of each dimension into a similarity fusion model to obtain a comprehensive similarity; and if the comprehensive similarity is greater than a threshold value, determining that the first record and the second record in the candidate record pair are successfully matched, updating the entity matching state dictionary of the first record as matching, and storing the successfully matched candidate record pair. According to the invention, the entity matching task can be efficiently completed under limited training data resources, computing resources and storage resources, and the interpretability of the matching result is supported.

Description

technical field [0001] The present invention relates to the technical field of entity matching, in particular to a periodic entity matching method and system between film and television attribute data sources. Background technique [0002] Entity matching is a cross task of knowledge graph and natural language processing. Entity matching is to solve the problem of knowledge fusion, that is, the process of matching all record pairs representing the same entity between homogeneous or heterogeneous data sources (or between knowledge graphs), or identifying instances of a given entity in the real world map. For example, in the e-commerce scenario, it is necessary to judge whether the products from two platforms correspond to the same product; in the movie video recommendation scenario, it is necessary to judge whether two videos correspond to the same movie; in the knowledge graph fusion scenario, Mappings that match all entity pairs between the two graphs are encountered. Th...

AI Technical Summary

Problems solved by technology

Although this type of deep learning-based method can solve many problems in general fields, the deep learning model requires a large amount of training data to train the model, so it is difficult for the deep learning model to converge when the training data resources are tight.

And the inexplicability of the deep model has been criticized

Images