Neural network weight sparse training method, device, medium and program product
CN121706868BActive Publication Date: 2026-06-09MOXIN ARTIFICIAL INTELLIGENCE TECH (SHENZHEN) CO LTD
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- MOXIN ARTIFICIAL INTELLIGENCE TECH (SHENZHEN) CO LTD
- Filing Date
- 2026-02-13
- Publication Date
- 2026-06-09
AI Technical Summary
Technical Problem
Existing sparse training methods suffer from problems such as training instability and large accuracy loss under high sparsity, fixed sparse patterns and limited combination space, making them difficult to adapt to emerging fine-grained sparse hardware.
Method used
A fine-grained sparse training method using sparse groups as the basic unit is adopted. Combined with SR-STE, the sparse weights are dynamically updated by generating sparse masks and regularization penalty terms, and the decay coefficient is configured to control the balance between sparsity and accuracy.
Benefits of technology
It improves the stability and accuracy of training under high sparsity, adapts to fine-grained sparse hardware, and improves computational efficiency and energy efficiency.
✦ Generated by Eureka AI based on patent content.
Smart Images
Figure CN121706868B_ABST
Abstract
The application relates to a neural network weight sparse training method, device, medium and program product. The neural network weight sparse training method comprises the following steps: configuring a weight sparse training parameter, wherein the weight sparse training parameter comprises a sparse granularity in a sparse group as a basic unit; in a forward propagation process, a sparse mask is generated according to an absolute value sorting of original weights of the neural network in each sparse group, and the sparse mask is applied to obtain sparse weights; in a backward propagation process, a gradient for updating the original weights is calculated, wherein the calculated gradient comprises a gradient of a loss function to the sparse weights and a regularization penalty term applied to original weights pruned according to the sparse mask; and the original weights are updated according to the calculated gradient.
Need to check novelty before this filing date? Find Prior Art