Low Latency Matrix Multiplication Unit

By employing a systolic array architecture with multiple weight shift chains and separate registers, the hardware circuits efficiently load weight values, thereby reducing latency and enhancing the processing speed of neural network calculations.

JP7674549B2Active Publication Date: 2025-05-09GOOGLE LLC
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
JP2024024596
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Priority Date
2017-05-17
Filing Date
2024-02-21
Publication Date
2025-05-09
Estimated Expiration
2038-05-17

AI Technical Summary

Technical Problem

Existing neural network hardware systems face challenges in reducing the latency of matrix multiplication units, particularly in loading weight values into weight matrix registers efficiently.

Method used

The implementation of dedicated hardware circuits with a systolic array architecture, featuring multiple chains of weight shift registers and separate registers for transposed and non-transposed weight shifts, allows for faster loading of weight values into matrix multiplication units.

Benefits of technology

This approach significantly reduces the latency of matrix multiplication units by enabling weight values to be loaded at twice the speed of traditional systems, potentially increasing the overall processing speed of neural network calculations.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007674549000001
    Figure 0007674549000001
  • Figure 0007674549000002
    Figure 0007674549000002
  • Figure 0007674549000003
    Figure 0007674549000003
Patent Text Reader

Abstract

To provide a method, system, and apparatus which reduce latency of an entire matrix multiply unit implemented as a systolic array of cells.SOLUTION: With respect to multiple cells 300 in a systolic array, each cell of matrix multiply includes: a weight matrix register 325 configured to receive a weight input from a transposed or non-transposed weight shift register; a transposed weight shift register configured to receive a weight input from a horizontal direction to be stored in the weight matrix register; a non-transposed weight shift register configured to receive a weight input from a vertical direction to be stored in the weight matrix register; and a multiply unit that is coupled to the weight matrix register and is configured to multiply the weight input of the weight matrix register by a vector data input in order to obtain a multiplication result.SELECTED DRAWING: Figure 3
Need to check novelty before this filing date? Find Prior Art

Citation Information

Patent Citations

  • Two-dimensional contraction array and method for neural network

    JP1991131965A

  • MIMO least squares mean error receiver using QR decomposition and systolic arrays

    JP2013511891A

  • Arithmetic processing unit

    JP2017021483A

  • Hardware accelerator system and method

    US20140289445A1

  • Prefetching weights for use in a neural network processor

    US20160342892A1