Distributed storage method and system based on self-encoding neural network

Abstract

The invention relates to the technical field of artificial intelligence, in particular to a distributed storage method and system based on a self-encoding neural network. The method comprises the following steps: inputting training data into a compression network for reasoning to obtain reasoning data, and obtaining a first error according to the difference between the training data and the reasoning data; wherein constructing a mapping relation between the reasoning data and the corresponding first error; wherein the output of the compression network is not smaller than the input; obtaining target data needing to be stored, obtaining a first error corresponding to the target data according to the mapping relation, and obtaining control data according to a difference value between the target data and the first error; inputting the control data into the compression network to obtain a shortest hidden layer; and taking the shortest hidden layer data as representation data, and splitting and combining the representation data to obtain storage data of each server. According to the method, the distributed storage technology is more efficient and safer by combining the self-encoding technology in the deep learning neural network.

Description

technical field [0001] The invention relates to the field of artificial intelligence, in particular to a distributed storage method and system based on an autoencoding neural network. Background technique [0002] When data is stored in a distributed manner, the data information is often scattered on different devices. In order to improve the fault tolerance of the distributed system, that is, when a single device or a small number of devices fail, the distributed storage system can still operate normally. The problem does not affect the stored data, and often requires multiple backups of the data. However, while increasing the fault tolerance of the system, it undoubtedly sacrifices a large amount of storage space to store duplicate data. Dimensionality reduction and compression of data can be very good saving storage resources. [0003] Using the network for data dimensionality reduction, the most commonly used is the self-encoding network, which does not require cumberso...

AI Technical Summary

Problems solved by technology

However, obviously due to network errors, the recovered data is largely not the original data

This processing method only stores the corresponding reconstruction error data, and cannot accurately restore the original data.

Images