Method and device for determining data similarity, electronic equipment and storage medium

Abstract

The invention provides a method and device for determining data similarity, electronic equipment and a storage medium, and relates to the field of data processing, in particular to the field of data similarity identification. According to the specific implementation scheme, a target data set is obtained, and at least two target data sequences are selected from the target data set; determining a target union set of each first element in the at least two target data sequences; probability matrixes corresponding to the at least two target data sequences are constructed according to the target union set, the sum of transition probabilities of all rows in the probability matrixes is 0 or 1, and elements in the probability matrixes are used for indicating the transition probabilities between the first elements; and determining the similarity between the at least two target data sequences according to the probability matrix.

Description

technical field [0001] The present disclosure relates to the technical field of data processing, and in particular, to the field of identification of data similarity. Background technique [0002] Data similarity plays a very important role in recommendation systems, user behavior recognition, natural language processing and other fields. It is difficult to use the same standard to quantify the similarity of data sequences of different dimensions. In related technologies, for different dimensions Data sequences of different sizes need to be processed into sequences of uniform size by means of dimensionality reduction, cropping, or data complementing. Preprocessing such as dimensionality reduction will make the processing process more complex and inefficient, and ultimately lead to poor accuracy of data similarity recognition results. SUMMARY OF THE INVENTION [0003] The present disclosure provides a method, apparatus, electronic device, and storage medium for determining...

AI Technical Summary

Problems solved by technology

[0002] Data similarity plays a very important role in recommender systems, user behavior recognition, natural language processing and other fields. It is difficult to use the same standard to quantify the similarity of data sequences of different dimensions. In related technologies, for different dimensions Large and small data sequences need to process data sequences of different dimensions into sequences of uniform size through methods such as dimensionality reduction, cutting, or data completion. This data processing method will lose part of the information of the original data sequence, and due to the need to Dimensionality reduction and other preprocessing will make the processing process more complex and inefficient, and ultimately lead to poor accuracy of the identification results of data similarity

Images