Stochastic gradient descent based spectral hashing method in distributed storage

A stochastic gradient descent, distributed storage technology, applied in the input/output process of data processing, instruments, electrical and digital data processing, etc., to achieve the effect of search efficiency and high-dimensional adaptability, fast convergence, and improved accuracy

Active Publication Date: 2016-08-10
NANJING UNIV OF POSTS & TELECOMM
View PDF5 Cites 6 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0006] The technical problem to be solved by the present invention is to provide a spectral hash method based on stochastic gradient descent in distributed storage, store similar data on the same or similar storage server nodes, and solve the problem of distributed storage of data

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Stochastic gradient descent based spectral hashing method in distributed storage
  • Stochastic gradient descent based spectral hashing method in distributed storage
  • Stochastic gradient descent based spectral hashing method in distributed storage

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0026] Embodiments of the invention are described in detail below, examples of which are illustrated in the accompanying drawings. The embodiments described below by referring to the figures are exemplary only for explaining the present invention and should not be construed as limiting the present invention.

[0027] On the basis of Spectral Hashing with Semantically Consistent Graph, the present invention uses Stochastic Gradient Descent (SGD) to reduce algorithm training time, and further proposes a consistent Hashing algorithm based on Cauchy distribution. Greek algorithm and use this algorithm to compress each data item into a one-dimensional real value. In this way, the idea of ​​consistent hashing can be used to realize distributed storage in a dynamic network topology, and similar data items can be stored in the same or similar storage server nodes.

[0028] Spectral hashing algorithm for semantically consistent graphs: it is a compression mapping method for data. Its ...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a stochastic gradient descent based spectral hashing method in distributed storage. According to the method, the algorithm training time is shortened by utilizing stochastic gradient descent based on a spectral hashing algorithm with semantic consistency, a consistent hashing algorithm based on Cauchy distribution is further proposed, and each data item is compressed into a one-dimensional real-value by utilizing the algorithm. Therefore, the distributed storage can be realized in dynamic network topology by utilizing a consistent hashing thought, and similar data items can be stored in same or similar storage server nodes. The method enables each storage server node to only need to maintain information of a small amount of neighboring nodes; and when the server nodes are added into or exit from a system, only a small amount of related nodes participate in topology maintenance, so that the convergence speed is increased and the storage accuracy is improved.

Description

technical field [0001] The invention relates to a spectral hash method based on stochastic gradient descent in distributed storage, and belongs to the technical field of distributed storage. Background technique [0002] In recent years, with the vigorous development of information technology, the business on the Internet has continued to expand, users have continued to grow, storage space has continued to increase, and data has shown an unimaginable growth trend. However, storage capacity is often inversely proportional to storage performance. Traditional databases struggle to cope with massive amounts of data, exposing problems such as low concurrency, poor scalability, and low efficiency, and cannot meet the needs of data explosion in the era of big data. For this reason, new requirements are put forward for storage technology in the new environment: scalability, data reliability, high performance, easy management, and green energy saving. [0003] Distributed storage te...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F3/06
CPCG06F3/0604G06F3/067
Inventor 胡海峰朱力吴建盛
Owner NANJING UNIV OF POSTS & TELECOMM
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap