Methods for Mass Text Matching

Abstract

The invention provides a mass text matching method. The method includes: grouping database files and external files separately, and determining a number S of groups of the database files; calculating a total number n of the database files and a total number m of the external files separately, using k real matrices M to indicate a vector space of the database files, and using k2 sparse matrices P to indicate a vector space of the external files; using the sparse matrix P, for which corresponding calculations are not made, as a current target sparse matrix P, performing the corresponding calculations to the current target sparse matrix P to obtain a similarity matrix S, and determining the database files in optimal match with the external files represented by the current target sparse matrix P, on the basis of the similarity matrix S. The mass text matching method has the advantages that storage space is saved, time consumption is low, and parallel processing is available.

Description

technical field [0001] The present invention relates to a method for text matching, and more particularly, to a method for massive text matching. Background technique [0002] At present, with the increasing demand for information and data processing and the increasing variety of businesses in different fields, it is becoming more and more important to match external documents with massive database documents. [0003] In the existing methods for document matching, the following methods are usually adopted: establishing a vector space model of external documents and database documents; calculating the similarity between the vector corresponding to each external document and the vector corresponding to the database document one by one; A specific database document with the greatest similarity to an external document is used as the database document that best matches the external document, and this cycle is repeated until the database document that best matches each external do...

AI Technical Summary

Problems solved by technology

[0004] However, the existing methods for document matching have the following problems: (1) Since the external documents after vectorization are usually sparse, storing external documents in the form of vectors will take up a large storage space; (2) due to It is necessary to calculate the dot product between every two vectors step by step, so the number of times to calculate the dot product is very large, resulting in a large amount of time spent in the calculation process; (3) It is difficult to perform parallel calculations

Images