The invention relates to the technical field of
artificial intelligence, and discloses a
hybrid bonding-based
large model reasoning data read-write method and
system, a controller and a storage medium, and the method comprises the following steps: constructing
hybrid bonding connection between a PIM
chip and a
DRAM chip and between the PIM
chip and a FLASH chip; when the
large model is used for reasoning, the weight of the
large model is stored in the FLASH chip through the data
bus, and the cue word of the large model is stored in the
DRAM chip through the data
bus; in the pre-filling stage, the PIM chip is used for respectively reading the weight and the cue word, and the calculated first key value is cached and written into the
DRAM chip; and in a decoding stage, respectively reading the weight and the first key value cache by utilizing the PIM chip, and writing the second key value cache of the Token generated by the large model into the
DRAM chip. According to the method, the
processing engine in the PIM chip is utilized, data only need to be read from the FLASH and the DRAM which are closest in physical distance,
delay is reduced, model weight data and key value cache data do not need to be
read through a
bus route, data carrying is reduced, and
power consumption is reduced.