Sparse Matrix Vector Product Operation Device Optimizing Cache Hit Rates

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current sparse matrix vector product operations in machine learning and high-performance computing are inefficient due to suboptimal access patterns, leading to slow performance and cache misses, especially when dealing with Jagged Diagonal Storage (JDS) matrices.

Innovation Solution

The proposed solution involves rearranging columns of a sparse matrix to align non-zero components by sorting and indexing, optimizing the order of columns based on the number of non-zero components and continuity of indices to improve cache hit rates and access efficiency during sparse matrix vector product operations.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Quantity of substance

If Jagged Diagonal Storage (JDS) format is used to store sparse matrices, then storage space is saved and vector length is increased, but cache hit rate decreases and access efficiency deteriorates

Engineering Contradiction:

Improvestorage spaceVSAvoidaccess efficiency

Core Design Contradiction:

Quantity of substanceVSProductivity

Solution Approach 1:

The patent segments the sparse matrix into multiple blocks, where each block is stored in a separate data structure. This segmentation allows for more flexible memory access patterns and improves cache utilization by processing one block at a time, thereby resolving the contradiction between compact storage and access efficiency.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent performs preliminary sorting of non-zero elements within each block before storage, organizing them by row and column indices. This preliminary action enables more predictable and efficient memory access patterns during matrix-vector multiplication, improving cache hit rates while maintaining the space-efficient JDS format.

Inventive Principle:

Principle #10Preliminary action

2Quantity of substance

If non-zero components are packed to the left in JDS format, then storage efficiency is improved, but access pattern becomes suboptimal leading to cache misses

Engineering Contradiction:

Improvestorage efficiencyVSAvoidcomputational time

Core Design Contradiction:

Quantity of substanceVSLoss of time

Solution Approach 1:

The patent applies different storage strategies to different parts of the matrix. Within each block, non-zero elements are packed to the left for storage efficiency, but the blocks themselves are arranged and processed in an order that optimizes access patterns. This local application of quality principles resolves the contradiction between storage efficiency and access speed.

Inventive Principle:

Principle #3Local quality

Solution Approach 2:

The patent introduces dynamic block processing where the size and arrangement of blocks can be adjusted based on the specific matrix characteristics and hardware constraints. This dynamic approach allows the system to adapt between more packed storage (for space efficiency) and more access-friendly arrangements (for speed efficiency) depending on the situation.

Inventive Principle:

Principle #15Dynamics

Data Source

PatentUS20240143695A1Sparse matrix vector product operation device, sparse matrix vector product operation method, and sparse matrix vector product operation program

Publication Date: 2024.05.02 NEC CORP

AI summary

A sparse matrix vector product operation device 20 includes a generating unit 21 which generates a second sparse matrix of a predetermined form by arranging a plurality of columns having a predetermined number or more non-zero components among a plurality of columns constituting a first sparse matrix of the predetermined form in order of the number of non-zero components.