Distributed development method and device, storage medium and computer equipment

A distributed and distributed file technology, applied in the computer field, can solve the problems of high maintenance cost, limited cluster size, waste of cluster computing resources, etc., and achieve the effect of low maintenance cost

Pending Publication Date: 2021-02-23
ALIBABA GRP HLDG LTD
View PDF8 Cites 0 Cited by
  • Summary
  • Abstract
  • Description
  • Claims
  • Application Information

AI Technical Summary

Problems solved by technology

[0006] Embodiments of the present invention provide a distributed development method, device, storage medium, and computer equipment to at least solve the problem of limited cluster size and maintenance costs in related technologies when using customized clusters for distributed development training. Large, and technical problems that waste cluster computing resources

Method used

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
View more

Image

Smart Image Click on the blue labels to locate them in the text.
Viewing Examples
Smart Image
  • Distributed development method and device, storage medium and computer equipment
  • Distributed development method and device, storage medium and computer equipment
  • Distributed development method and device, storage medium and computer equipment

Examples

Experimental program
Comparison scheme
Effect test

Embodiment 1

[0029]According to the embodiment of the present invention, a method embodiment of a distributed development method is also provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, Also, although a logical sequence is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than here.

[0030]The method embodiment provided in Embodiment 1 of the present application may be executed in a mobile terminal, a computer terminal, or a similar computing device.figure 1 Shows a hardware structure block diagram of a computer terminal (or mobile device) used to implement a distributed development method. Such asfigure 1 As shown, the computer terminal 10 (or the mobile device 10) may include one or more (shown as 102a, 102b, ..., 102n in the figure) processor 102 (the processor 102 may include but is not limited to a microp...

Embodiment 2

[0081]In the embodiment of the present invention, a distributed development device is also provided,Figure 5 It is a structural block diagram of a distributed development device 1 according to Embodiment 2 of the present invention, such asFigure 5 As shown, the device includes: an establishment module 50, a first receiving module 52, a distribution module 54, a mounting module 56 and a first execution module 58, which will be described below.

[0082]The establishment module 50 is used to establish Docker, where tool codes and frameworks for training distributed development are deployed in Docker; the first receiving module 52 is connected to the establishment module 50 described above and is used to receive training tasks for distributed development The allocation module 54 is connected to the first receiving module 52 and is used to allocate computing resources for training tasks, where the computing resources include computing nodes in the cluster; the mounting module 56 is connecte...

Embodiment 3

[0086]In the embodiment of the present invention, a distributed development device is also provided,Figure 6It is a structural block diagram of distributed development device 2 according to Embodiment 3 of the present invention, such asFigure 6As shown, the apparatus includes: the second receiving module 62, the second execution module 64, and the feedback module 66, the apparatus will be described below.

[0087]The second receiving module 62 is configured to receive a training request for distributed development, wherein the training request carries a training task for performing distributed development; the second execution module 64 is connected to the second receiving module 62 for With the following training methods, the training tasks are implemented, and the training results are obtained, where training methods include: assigning a computing node in the training task, mounting the distributed file system in the compute node, loading the pre-established Docker on the compute nod...

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

PUM

No PUM Login to view more

Abstract

The invention discloses a distributed development method and device, a storage medium and computer equipment. The method comprises steps of a Docker being established, and tool codes and a framework for training of distributed development being deployed in the Docker; receiving a training task for distributed development; computing resources being allocated to the training task, and the computingresources comprising computing nodes in a cluster; mounting a distributed file system for the computing node allocated with the training task; and respectively executing the training tasks allocated to the computing nodes in a mode of loading the Docker to the computing nodes allocated with the training tasks. According to the method, technical problems of limited cluster scale, high maintenance cost and waste of cluster computing resources when distributed development training is carried out by adopting a customized cluster mode in related technologies are solved.

Description

Technical field[0001]The present invention relates to the field of computers, and in particular, a distributed development method, a device, a storage medium, and a computer device.Background technique[0002]In the relevant technology distributed development (eg, speech recognition), the training of language models and acoustic models is very high, and the problem brings to how to improve computational efficiency. Traditional Hidden Markov Model - Gauss Mixed Model (HMM-GMM), HMM-GMM, Hidden Markov Model-Deep Neural Ne TWORKS, referred to as HMM- DNN) The training data required for technology can achieve good results at around 10,000 hours. In order to achieve higher precision, the amount of data required for DEEP SPEECH is 100,000 hours, and traditional single-machine multi-card training methods have not been able to meet more data training needs.[0003]In order to meet the training needs of a large amount of data, in the related art, the patent application for public opening is made...

Claims

the structure of the environmentally friendly knitted fabric provided by the present invention; figure 2 Flow chart of the yarn wrapping machine for environmentally friendly knitted fabrics and storage devices; image 3 Is the parameter map of the yarn covering machine
Login to view more

Application Information

Patent Timeline
no application Login to view more
Patent Type & Authority Applications(China)
IPC IPC(8): G06F8/60G06F9/50G06F16/182H04L29/08
CPCG06F8/60G06F9/5027G06F16/182H04L67/10H04L67/34
Inventor 曹元斌朱晓如梅本金
Owner ALIBABA GRP HLDG LTD
Who we serve
  • R&D Engineer
  • R&D Manager
  • IP Professional
Why Eureka
  • Industry Leading Data Capabilities
  • Powerful AI technology
  • Patent DNA Extraction
Social media
Try Eureka
PatSnap group products

Browse by: Latest US Patents, China's latest patents, Technical Efficacy Thesaurus, Application Domain, Technology Topic.

© 2024 PatSnap. All rights reserved.Legal|Privacy policy|Modern Slavery Act Transparency Statement|Sitemap