Semantic consistency-based distributed similarity search method with privacy protection

Abstract

The invention discloses a semantic consistency-based distributed similarity search method with privacy protection. The method comprises the steps of firstly performing classification tagging on images, videos and files in databases of nodes; secondly initializing conversion matrixes and a Lagrangian multiplier; thirdly introducing semantic consistency for building an objective function; fourthly solving the objective function, and updating the conversion matrixes; fifthly enabling the neighbor nodes to communicate, judging whether the conversion matrixes of the nodes tend to be consistent or not, and updating the Lagrangian multiplier; and finally performing a similarity search process. The method solves the problem that a centralized training algorithm model is no longer suitable for excessively large-scale data during storage and calculation. The nodes communicate by using the conversion matrixes, without exchanging original information, so that the problem of excessively large transmission communication can be effectively solved and the data privacy in the nodes can be effectively protected.

Description

Technical field [0001] The invention belongs to the field of machine learning, and mainly relates to adopting distance measurement learning to ensure the consistency of samples in a distributed environment, and specifically relates to a distributed proximity search method with privacy protection based on semantic consistency. Background technique [0002] With the continuous development of social networks, e-commerce, mobile Internet, etc., the scale of data storage and processing is getting larger and larger, and stand-alone systems can no longer meet the increasing demand. Internet companies such as Google and Alibaba have successfully spawned the two hot areas of cloud computing and big data. Both cloud computing and big data are applications built on distributed storage. The core of cloud storage is the back-end large-scale distributed storage system. Big data not only needs to store massive amounts of data, but also analyze these data through appropriate frameworks and tools...

AI Technical Summary

Problems solved by technology

[0006] The purpose of the present invention is to provide a privacy-protected distributed approximate search method based on semantic consistency, which is mainly used to solve the problem that the number of samples such as images, videos, and texts is large, and it is impossible to accurately find semantic neighbors. At the same time, when distributed computing The transmission of raw data between nodes is likely to cause information leakage and excessive transmission volume. The main purpose of this method is to obtain a globally optimized transformation matrix with low computational overhead through distributed training, and at the same time protect each Data privacy of nodes, and semantically consistent nearest neighbor search for query samples

Images