The application provides an on-
chip cache management system, method, medium, program product and terminal for a
large model AI
chip, the application adopts a 3T GC-
eDRAM cache
unit structure to improve the storage capacity per unit area on the
chip, and compared with a traditional 6T SRAM, the chip area is significantly reduced under the same
cache capacity, thereby reducing the production cost. Based on the data heat characteristic parameter of the
large model, the application adopts a three-level dynamic refresh strategy to avoid resource waste of unified refresh and reduce refresh
power consumption. In the application
scenario of
large model inference, the application effectively reduces the
response delay by combining the on-chip cache
hardware structure and the dynamic refresh mechanism, and meets the low-
delay inference requirement of the large model. The application adopts a 3T GC-
eDRAM cache
unit structure to realize on-chip cache on the chip, can be realized based on a standard
CMOS process, and can be directly migrated to an advanced process such as 7nm and 5nm, without process modification.