Machine learning program, machine learning method, and information processing device

By applying L1 regularization learning and threshold-based pruning, neural networks with attention structures are made more efficient by identifying minimal-impact channels for pruning, thus maintaining accuracy and reducing size effectively.

JP7885681B2Active Publication Date: 2026-07-07FUJITSU LTD
View PDF 2 Cites 0 Cited by

Patent Information

Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
FUJITSU LTD
Filing Date
2022-12-28
Publication Date
2026-07-07

Smart Images

  • Figure 0007885681000008
    Figure 0007885681000008
  • Figure 0007885681000009
    Figure 0007885681000009
  • Figure 0007885681000010
    Figure 0007885681000010
Patent Text Reader

Abstract

To achieve weight reduction of a neural network including an attention structure.SOLUTION: A machine learning program causes a computer to execute processing of: for an element of each of a Q layer 161 outputting a Query and a K layer 162 outputting a Key as an arithmetic processing result on an input tensor in an attention structure in a trained machine learning model of a neural network 180 including an attention structure 160, deleting an element included in at least one of a tensor QT and a tensor KT such that elements having a same index are left in the tensors QT and KT from among one or more elements included in a tensor QT from a reduced Q layer in which the elements are reduced on the basis of a first reduction ratio and one or more elements included in a tensor KT from a reduced K layer in which the elements are reduced on the basis of a second reduction ratio.SELECTED DRAWING: Figure 19
Need to check novelty before this filing date? Find Prior Art