Data processing method and system, electronic equipment

Abstract

The invention discloses a data processing method and system and electronic equipment. The method comprises the steps that a cached data read request initiated by at least one processor process to a data cache system is received, the data cache system comprises multiple kinds of cache elements, transmission rates and / or storage spaces of the cache elements are different, and different searching priorities are preset according to the transmission rate and the storage space of each cache element; the received cached data read request is responded, and the corresponding cache data is searched fromthe cache elements according to the search priorities in a sequence of from high to low from the cache element of the corresponding processor of the process which initiates the cached data read request; the searched cache elements in which the corresponding cache data is cached are responded, and the corresponding cached data is returned to the corresponding processor process from the searched cache elements. The data processing method can greatly reduce training data acquisition time of computing nodes, and algorithm training time is shortened.

Description

technical field [0001] This application relates to the field of deep learning, in particular to data processing methods and systems, and electronic equipment. Background technique [0002] Deep learning algorithms that analyze large-scale data are being increasingly used in big data analysis, such as image recognition, speech recognition, and natural language processing. Such large-scale deep learning algorithms usually require terabytes or more of training data and massive neural network model parameters. Since the storage and computing power of a single computer cannot meet the requirements of this task, distributed deep learning systems have been proposed, such as Google's DistBelief distributed training system, UC Berkeley AMP Lab's Spark distributed computing system for deep learning . On the one hand, this type of system shortens the training time of the model by using the parallel computing capability of the distributed cluster, and on the other hand, it uses the cl...

AI Technical Summary

Problems solved by technology

However, in the existing deep learning system, it usually takes a long time for the computing node to obtain the training data, because the data on the storage node needs to be transmitted by the computing node through the network

In addition, many deep learning systems allocate a large number of calculations to the graphics processing unit (GPU), and the GPU needs to read data from the memory, which leads to the rate of obtaining training data is also affected by the data transfer speed from the memory to the GPU memory. limit

Images