神经网络模型的训练方法、图像处理方法及装置
By utilizing high-bandwidth links to transmit large parameter gradients and low-bandwidth links to transmit small parameter gradients during neural network model training, the problem of server bandwidth limitations is solved, thereby improving training efficiency and data transmission efficiency.
CN114792127BActive Publication Date: 2026-07-17HUAWEI TECH CO LTD
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- HUAWEI TECH CO LTD
- Filing Date
- 2021-01-08
- Publication Date
- 2026-07-17
AI Technical Summary
Technical Problem
During the training of neural network models, the low bandwidth between servers results in slow data transmission during parameter gradient synchronization, which affects training efficiency.
Method used
By utilizing the first link with higher bandwidth to transmit a larger amount of parameter gradient data, and the second link with lower bandwidth to transmit a smaller amount of parameter gradient data, communication overhead is reduced and training efficiency is improved.
Benefits of technology
Without increasing the size of the model parallel group, the communication overhead in data parallelism is reduced, significantly improving the training efficiency and data transmission efficiency of hybrid parallelism, without affecting the accuracy of model training.
✦ Generated by Eureka AI based on patent content.
Smart Images
Figure CN114792127B_ABST
Abstract
本申请公开了人工智能领域中的一种神经网络模型的训练方法、图像处理方法及装置。该训练方法包括：第一加速器根据第一参数梯度更新神经网络模型的部分参数，在数据并行的参数梯度同步过程中，第一加速器通过带宽较小的第二链路接收第四参数梯度的一部分，通过带宽较大的第一链路接收第三参数梯度和第四参数梯度的其他部分，进而根据第三参数梯度、第四参数梯度以及自身得到第二参数梯度确定第一参数梯度。本申请的方法能够减少训练过程中的通信开销，进而提高神经网络模型的训练效率。
Need to check novelty before this filing date? Find Prior Art