Distributed data processing system, distributed computing task deployment system and method

A technology for distributed computing and deployment systems, applied in the field of distributed data processing systems and deployment systems for distributed computing tasks, can solve the problems of high unit price, increase the occupation of memory resources of computing equipment, and high costs, and achieve computing resources and memory. The effect of reducing the use of and reducing memory space requirements

Active Publication Date: 2021-06-11
中关村科技租赁股份有限公司
View PDF4 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0006] The computing devices commonly used in deep learning usually include CPU, GPU, FPGA, and ASIC (application-specific integrated circuit), etc. GPU, FPGA, and ASIC can be collectively referred to as data acceleration processing devices. High memory bandwidth, but its memory capacity often has problems such as limited memory, high unit price, and difficulty in expansion. If the memory capacity of data acceleration processing equipment is blindly increased to meet the training of large-scale models, it will bring high costs to enterprises.
Obviously, the repeated calculations and memory in the existing data parallelism come from distributed training parameters and the Optimizer are also distributed (Distributed) placed on each computing device, which will undoubtedly improve the memory resources of computing devices. On the other hand, it will inevitably lead to the transmission overhead of parameter synchronization between devices
It is a huge waste for current computing devices, especially GPUs with expensive memory resources.

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Distributed data processing system, distributed computing task deployment system and method
  • Distributed data processing system, distributed computing task deployment system and method
  • Distributed data processing system, distributed computing task deployment system and method

Examples

Experimental program
Comparison scheme
Effect test

Embodiment Construction

[0026] The present invention will be described in further detail below in conjunction with the embodiments and accompanying drawings, so that those skilled in the art can implement it with reference to the description.

[0027] Reference will now be made in detail to the exemplary embodiments, examples of which are illustrated in the accompanying drawings. When the following description refers to the accompanying drawings, the same numerals in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the following exemplary examples do not represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatuses and methods consistent with aspects of the present disclosure as recited in the appended claims.

[0028] The terminology used in the present disclosure is for the purpose of describing particular embodiments only, and is not intended to limit the present disclosure...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The present disclosure provides a distributed data processing system, a distributed computing task deployment system and method. The distributed data processing system is used for parallel processing of data on the plurality of computing devices, each computing device includes a forward data processing component and a backward data processing component, wherein at least one computing device includes other computing devices A model parameter component that does not have and a model parameter update component corresponding to one of the model parameter components, and the model parameter component uses a set of data to be processed in parallel through its corresponding broadcast component. Input to the broadcast component of other computing devices, and the model parameter update component obtains the corresponding global gradient value from the corresponding gradient convergence component to perform update processing.

Description

technical field [0001] The present disclosure relates to a data processing technology, and more specifically, the present disclosure relates to a distributed data processing system, a distributed computing task deployment system and a method thereof. Background technique [0002] With the development of machine learning and the gradual deepening of artificial neural network research, the concept of deep learning has been widely concerned and applied. Deep learning is a special kind of machine learning. It uses a network hierarchical structure to express the learning objects, combines simple concepts into abstract concepts, and realizes abstract concept expression through simple concept calculations. At present, deep learning has made great progress in the fields of image recognition, speech recognition and natural language processing. Deep learning involves a large number of model parameters, resulting in a huge amount of calculation, and the scale of training data is large...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Patents(China)
IPC IPC(8): G06F9/50G06F9/54G06N3/063
CPCG06F9/5061G06F9/542G06N3/063
Inventor 柳俊丞上官士源李新奇郭冉袁进辉
Owner 中关村科技租赁股份有限公司
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products