Content similarity sorting algorithm

Abstract

The invention belongs to the technical field of content similarity sorting of mass data, relates to content similarity sorting based on text classification probability vectors and text semantic vectors in mass data, in particular to a content similarity sorting algorithm based on text classification probability vectors and text semantic vectors in mass data. The algorithm can be divided into two layers: the first layer is a similarity calculation recall layer based on a text classification probability vector, and the layer calculates a local optimal similar text pool through the classificationprobability vector, so the layer is called as a text similarity recall layer; and the second layer performs content similarity calculation based on the local optimal similar text pool of the first layer, so the second layer is called as a text semantic similarity sorting layer. According to the method, a text similarity coarse-grained sorting layer is embedded on the basis of traditional text similarity calculation, so the optimal similar text pool is screened, and the calculation range of the text similarity is reduced through the algorithm. And when facing massive data, the method not onlysaves a large number of computing resources, but also reduces the computing time consumption.

Description

technical field [0001] The invention belongs to the technical field of content similarity sorting of massive data, and relates to content similarity sorting based on text classification probability vectors and text semantic vectors in massive data, in particular to a kind of content based on text classification probability vectors and text semantic vectors in massive data Similarity sorting algorithm. Background technique [0002] With the rapid development of new media technology in recent years, major media platforms have developed towards the advantages of digital technology such as high timeliness, low cost, and multiple channels, and this trend has gradually been accepted by the public. However, in the process of changing the operation mode of the media platform, the reading mode of consumers has also changed, which is specifically reflected in the transformation from systematic reading to fragmented reading, which leads to the lack of integrity and continuity of the in...

AI Technical Summary

Problems solved by technology

[0004] Although the Word2Vec algorithm released by Google meets the needs of text similarity calculation, when the algorithm is used in the real-time calculation scenario of a large number of manuscripts under the condition of 100-dimensional level text vector features, it will consume a lot of computing resources and consume a lot of computing power. Time and other issues, thus destroying the user's sense of experience with the APP

Images