165 results about "High Bandwidth Memory" patented technology

The invention discloses a hardware accelerator and equipment oriented to triple sparse matrix multiplication and an application method thereof.The hardware accelerator comprises a high-bandwidth memory HBM, a crossbar switch network and an on-chip processing unit which are connected in sequence, and the on-chip processing unit comprises a hierarchical cache module, a global controller and a plurality of computing chips; each calculation piece comprises an RA calculation array, a TP calculation array and a local controller, wherein the RA calculation array and the TP calculation array are respectively used for executing front-end operation T = R * A and rear-end operation C = T * P in triple sparse matrix multiplication. The method aims at solving the problem that when a traditional universal processor processes triple sparse matrix multiplication, due to irregular memory access, uneven calculation load and sharp increase of middle parts and results, huge off-chip data carrying is confronted with serious performance and energy efficiency bottlenecks, and the calculation performance and energy efficiency of triple sparse matrix multiplication are improved.

The invention relates to the technical field of large language model deployment, and discloses a large language model weight inverse quantization reasoning device and method.The method comprises the steps that low-precision weight data is transmitted to a high-bandwidth storage from a host and then transmitted to an on-chip storage through the high-bandwidth storage; data conversion from a low-precision format to a high-precision format is completed in the on-chip memory, the data is multiplied by an inverse quantization factor to obtain recovered high-precision weight data, and the functional unit is responsible for executing general matrix multiplication of input data and the high-precision weight data after inverse quantization. And pipeline parallel execution of the inverse quantization operation and the general matrix multiplication operation is realized through a double-buffering technology. According to the invention, on the basis of a dual-path inverse quantization architecture of the vector processing unit and a dual-buffer mechanism in the on-chip memory, the problem of hardware adaptation of low-precision calculation is solved, and efficient execution of a low-precision conversion algorithm is realized under the condition of limited hardware resources.

Methods for fabricating flexible interposers for providing electrical connection between devices mounted at different vertical positions with respect to a substrate or a planar interposer. A bonded structure may comprise a bent flexible interposer extending from a first interposer portion between a main device on the substrate or the planar interposer and a second interposer portion above the main device and above or below a device positioned above the main device and electrically connected to the main device via a bent portion of the flexible interposer.

The invention discloses a sparse LU decomposition acceleration method for an FPGA (field programmable gate array) with a flow-sensing high-bandwidth memory, belongs to the technical field of LU decomposition hardware acceleration, and aims to solve the problem of performance bottleneck and efficiency challenge faced by sparse LU decomposition on the FPGA. The acceleration method comprises the following steps of data stream preprocessing, wherein matrix data and metadata are stored and packaged based on a sparse storage format; customizing an FPGA hardware accelerator: carrying out HBM channel allocation and collaborative design of a control unit and memory management; parallel data flow management and scheduling are carried out, and sparse LU decomposition is executed; timely data supply is ensured through a multi-stage pipeline prefetching mechanism; and transmitting the task flow and the data flow to a special parallel processing engine for MAC / DIV calculation in combination with a flow sensing synchronous scheduling strategy.

The invention relates to the technical field of semiconductors, provides a semiconductor device, and is used for solving the technical problem that a large lifting space still exists in a high-bandwidth memory technology. The semiconductor device includes: a chip stacking structure including a first semiconductor chip and a second semiconductor chip stacking structure stacked in a first direction; the first semiconductor chip comprises a controller, a calculation module and an interface module; the controller is configured to execute a first access operation on storage data in the second semiconductor chip stacking structure and send the storage data to the interface module to allow external equipment to access the storage data in the second semiconductor chip stacking structure through the interface module; and performing a second access operation on the storage data in the second semiconductor chip stack structure and sending the storage data to the computing module to allow the computing module to perform a computing operation on the storage data. Therefore, the integration level can be improved, the data transmission path can be shortened, and bandwidth consumption and energy consumption loss can be reduced.

Overhang architectures for high bandwidth memory (HBM) multi-die assemblies and methods for making same. The overhang architecture places the DRAM (HBM) underneath the top IC die. The signal interconnects between the top IC die and the DRAM die are direct signal interconnects without lateral routing on a package substrate or on a motherboard.

A memory device (e.g., a high-bandwidth (HBM) memory device) that includes a memory die having multiple pseudo channels per channel is disclosed. The memory die can include first memory banks associated with a first channel (e.g., having a first command address (CA) bus) and a first pseudo channel (e.g., having a first data (DQ) bus) and second memory banks associated with the first channel and a second pseudo channel (e.g., having a second DQ bus). Operations can be performed at the first memory banks or the second memory banks in response to a command received through the first CA bus. The operations can cause data to be returned to circuitry that routes the data to an interface to the first DQ bus or an interface to the second DQ bus based on whether the data resulted from operations at the first memory banks or the second memory banks.

System-in-package (SiP) devices, and associated systems and methods are disclosed herein. In some embodiments, a SiP device can include a base substrate, as well as a host device and an improved-thermal high-bandwidth memory (HBM) device each integrated with the base substrate. The improved-thermal HBM device can include an interface die and a stack of one or more memory dies carried by the interface die. The interface die includes an input / output (IO) circuit, which is communicably coupled via one or more IO circuit interfaces to the host device through communication channels of the base substrate. The IO circuit interfaces of the improved-thermal HBM device distributes physical interconnect bumps for transmit data and receive data in a dispersed manner along an edge of the interface die in a manner to reduce the occurrence of thermal hotspots.

An apparatus including a high bandwidth memory circuit and associated systems and methods are disclosed herein. The apparatus may include multiple HBM cubes connected to a processor, such as a GPU. The HBM cubes may be connected in series or in parallel. One or more of the HBM cubes can include a secondary communication circuit configured to facilitate the expanded connection between the multiple cubes.

In an embodiment of the present inventive concept, a high bandwidth memory includes a base die, and a semiconductor stack disposed on the base die, the semiconductor stack comprising a plurality of underfill members and a plurality of memory dies that are alternately stacked. Each of the plurality of underfill members includes first sides, each of the plurality of memory dies includes second sides, and each of the first sides is recessed from a corresponding second side.

An apparatus including a high bandwidth memory circuit and associated systems and methods are disclosed herein. The high bandwidth memory circuit can include two or more physical layer circuits to communicate with neighboring devices. The high bandwidth memory circuit can broadcast a status to the neighboring devices. The neighboring devices can be configured according to the operating demands of the high bandwidth memory circuit.

A management technique for high bandwidth memory is disclosed. A processing management circuit (PMC) has a main executing circuit and a main memory and is configured to manage at least one processor operation performed by at least one of a first processing element (PE) or a second PE. A shared memory is configured to be shared by the PMC, the first PE, and the second PE. A memory management circuit (MMC) is configured to manage a memory operation on the shared memory based on a memory access by at least one of the PMC, the first PE, or the second PE. The at least one processor operation includes at least one of a program launch, a program execution, and an interrupt delivery.

This invention relates to the field of chip technology, and more particularly to a memory system based on the UBM protocol packaging, comprising a UBM device, a silicon interposer, and a GPU, wherein the GPU includes a UBM physical layer; the UBM device, silicon interposer, and UBM physical layer are interconnected based on a preset UBM physical interconnect protocol, where UBM is an ultra-high bandwidth memory, with a bandwidth higher than that of HBM4; the UBM physical interconnect protocol includes: the UBM device being connected to the silicon interposer using a hybrid bonding packaging method, the UBM physical layer being connected to the silicon interposer using a hybrid bonding packaging method, and the UBM device and the UBM physical layer establishing an interconnection through wiring resources in the silicon interposer. This invention improves the bandwidth line density and bandwidth areal density of the memory system packaging structure.

The invention provides a matrix calculation optimization method and device, electronic equipment and a storage medium, and belongs to the technical field of data processing.The method comprises the steps that the data reading amount of each matrix calculation task to be processed is determined; grouping the plurality of calculation cores based on the physical distance between each calculation core and the plurality of high-bandwidth memories to obtain a plurality of calculation core groups; based on the data reading amount of each matrix calculation task, matching a plurality of to-be-processed matrix calculation tasks with a plurality of calculation core groups to obtain a task allocation scheme; and based on a task allocation scheme, allocating each matrix calculation task to a corresponding calculation core group. According to the matrix calculation optimization method and device, the electronic equipment and the storage medium provided by the invention, by comprehensively considering the data reading amount of the matrix calculation task and the physical layout characteristics of the calculation core, load-aware task allocation is realized, the performance bottleneck can be effectively avoided, and the calculation efficiency is improved. And the overall operation efficiency and the hardware resource utilization rate of a plurality of matrix calculation tasks are remarkably improved.

The application provides a packaging method and structure of a multilayer stacked high-width-band memory, which comprises the following steps: providing a buffer chip and a plurality of groups of memory chips, each group of memory chips comprising a first memory chip and a second memory chip; wherein the buffer chip is provided with a plurality of first conductive vias, and the first memory chip and the second memory chip are each provided with a plurality of second conductive vias corresponding to the plurality of first conductive vias; mixing bonding the first memory chip and the second memory chip in each group of memory chips to form a plurality of memory micro-modules; sequentially stacking the plurality of memory micro-modules on the buffer chip; and forming a plastic encapsulation layer to wrap the plurality of memory micro-modules and the buffer chip. The double-chip mixed bonding forms a memory micro-module, which can realize super-multilayer chip stacking, improve production efficiency, realize bonding height reduction, greatly increase the number of chip layers, and increase the capacity.

Stacked devices and methods of fabrication are provided. Die-to-wafer (D2W) direct-bonding techniques join layers of dies of various physical sizes, form factors, and foundry nodes to a semiconductor wafer, to interposers, or to boards and panels, allowing mixing and matching of variegated dies in the fabrication of 3D stacked devices during wafer level packaging (WLP). Molding material fills in lateral spaces between dies to enable fan-out versions of 3D die stacks with fine pitch leads and capability of vertical through-vias throughout. Molding material is planarized to create direct-bonding surfaces between multiple layers of the variegated dies for high interconnect density and reduction of vertical height. Interposers with variegated dies on one or both sides can be created and bonded to wafers. Logic dies and image sensors from different fabrication nodes and different wafer sizes can be stacked during WLP, or logic dies and high bandwidth memory (HBM) of different geometries can be stacked during WLP.

The present application provides a latency processing unit that may include the following: a plurality of multiply-accumulate (MAC) trees configured to perform a matrix product operation on at least one of a plurality of partitions implementing an artificial intelligence (AI) model; the streaming memory access module is configured to establish connection between each of the plurality of MAC trees and a high-bandwidth memory in which the at least one partition is stored through a plurality of channels; a vector execution engine configured to perform an additional operation on operation results of the plurality of MAC trees; the local storage unit is configured to store an operation result and an activation value of the vector execution engine; and the instruction scheduling unit is configured to schedule the operation processes of the plurality of MAC trees and the vector execution engines.

An apparatus including a high bandwidth memory circuit and associated systems and methods are disclosed herein. The apparatus may include a plurality of HBM cubes connected to a processor, such as a GPU. The HBM cubes may be connected in series or in parallel. One or more of the HBM cubes may include secondary communication circuitry configured to facilitate extended connections between the plurality of cubes.

A specialized integrated circuit for artificial neural networks is integrated with high bandwidth memory. The neural network includes a systolic array of interconnected processing elements, including upstream processing elements and downstream processing elements. Each processing element includes a pair of input / output ports for concurrent forward and backward propagation. The processing elements can be used for convolution, in which case the pair of input / output ports can support fast and efficient scanning of a kernel over activations.

A memory device (e.g., a high-bandwidth (HBM) memory device) that includes a memory die having multiple pseudo channels per channel is disclosed. The memory die can include first memory banks associated with a first channel (e.g., having a first command address (CA) bus) and a first pseudo channel (e.g., having a first data (DQ) bus) and second memory banks associated with the first channel and a second pseudo channel (e.g., having a second DQ bus). Operations can be performed at the first memory banks or the second memory banks in response to a command received through the first CA bus. The operations can cause data to be returned to circuitry that routes the data to an interface to the first DQ bus or an interface to the second DQ bus based on whether the data resulted from operations at the first memory banks or the second memory banks.

A method for splitting a convolution operation includes obtaining input data tensors and input weight tensors from a high bandwidth memory, splitting said tensors into several parts, reserving the necessary amount of matrix memory, successively loading the input data parts from the high bandwidth memory into a vector memory, splitting the input data parts into several sub-parts taking into account the reserved amount of memory, and loading the sub-parts into the memory. The weight parts are successively loaded into the memory at the same time as the loading of the input data sub-parts, the input data and the weights, read from the matrix memory, are processed, and sub-parts of a resulting tensor are obtained. The parts of the resulting tensor are loaded into the high bandwidth memory and at the same the input data sub-parts and the weight parts are processed by an array multiplier based on systolic arrays. The result is an improvement in the tiling efficiency of a convolution operation.

The invention provides an improved apparatus and method for delaying high-speed constant bitrate signals in a programmable manner. For this purpose, a signal is demultiplexed into individual data units, and the data units are temporarily stored in a high bandwidth memory by writing and reading via multi-channel first data transfer. In this way, the temporary storage of data can be achieved without noticeable decrease of memory throughput and without interruption between writing and reading requests.

The application provides a matrix calculation optimization method and device, electronic equipment and storage medium, and belongs to the technical field of data processing. The method comprises the following steps: determining the data reading amount of each matrix calculation task to be processed; grouping a plurality of calculation cores based on the physical distance between each calculation core and a plurality of high-bandwidth memories, to obtain a plurality of calculation core groups; matching the plurality of matrix calculation tasks to be processed with the plurality of calculation core groups based on the data reading amount of each matrix calculation task, to obtain a task allocation scheme; and allocating each matrix calculation task to a corresponding calculation core group based on the task allocation scheme. The matrix calculation optimization method and device, electronic equipment and storage medium provided by the application can effectively avoid performance bottlenecks and significantly improve the overall operation efficiency and hardware resource utilization of a plurality of matrix calculation tasks by comprehensively considering the data reading amount of the matrix calculation tasks and the physical layout characteristics of the calculation cores.

This invention relates to the field of near-memory computing technology, and discloses a method and system for near-memory computing bandwidth optimization based on HBM. The method includes: decoding and parsing instructions for access tasks on the host side to obtain the storage address and the required operation type; constructing metadata for the access tasks to obtain a scheduling context, and pre-selecting target processing elements in high-bandwidth memory; performing data storage access on the high-bandwidth memory to obtain data blocks to be processed, and performing information dimensionality reduction mapping on the data blocks to obtain partial result data; performing multi-path fusion and reduction on the partial result data to obtain final result data, and returning the final result data to the host side; while performing multi-path fusion and reduction, pre-fetching scheduling is performed on the access tasks to be processed on the host side to establish a scheduling context for the access tasks to be processed; this invention can improve the efficiency of near-memory computing bandwidth optimization based on HBM.

Methods and apparatus relating to techniques for multi-tile memory management. In an example, a graphics processor includes an interposer, a first chiplet coupled with the interposer, the first chiplet including a graphics processing resource and an interconnect network coupled with the graphics processing resource, cache circuitry coupled with the graphics processing resource via the interconnect network, and a second chiplet coupled with the first chiplet via the interposer, the second chiplet including a memory-side cache and a memory controller coupled with the memory-side cache. The memory controller is configured to enable access to a high-bandwidth memory (HBM) device, the memory-side cache is configured to cache data associated with a memory access performed via the memory controller, and the cache circuitry is logically positioned between the graphics processing resource and a chiplet interface.

A system-in-package (SIP) is described. The SIP includes a first package substrate supporting a logic die. The SIP also includes a second package substrate supporting a stack of memory dies. The SIP further includes semiconductor pillar bricks coupled between the first package substrate and the second package substrate.

The invention discloses a time sequence adjusting device and method, a high-bandwidth memory and a chip. The device comprises a delay determining circuit and a delay adjusting circuit, the delay determination circuit is used for outputting a preset delay value and receiving a phase feedback signal of the DRAM; after determining a metastable state signal from the phase feedback signal, determining an overturning point based on the metastable state signal; determining and outputting a target delay value according to the overturning point; the delay adjustment circuit is used for adjusting the phase of the write data strobe signal for multiple times according to the received preset delay value, and sending the write data strobe signal adjusted each time to the DRAM; and after the target delay value is received, adjusting the phase of the write data strobe signal adjusted last time according to the target delay value, and outputting a target write data strobe signal. Therefore, the overturning point is judged more accurately based on the delay determining circuit and the delay adjusting circuit, misjudgment caused by jitter or burrs is avoided, and accurate data writing of the DRAM based on the target data writing strobe signal is ensured.

An example device includes a bank of processing elements; a high bandwidth memory module in communication with the bank of processing elements and including a plurality of channels of memory; a plurality of bridges corresponding to the plurality of channels of memory, each bridge configured to connect a designated channel of the channels of memory to a designated vector of processing elements in the bank and including a bridge controller configured to: in response to a request for a memory access for a processing operation, perform the memory access to retrieve a data value from the designated channel according to the request; and provide the data value to a processing element in the designated vector to process according to the processing operation.

A computing device includes a memory and a processing device. The memory enters an uninitialized state in response to power being applied to the memory. The processing device is coupled to the memory, and is configured to select a portion of the memory in the uninitialized state to seed a random number generator process. The processing device may alternatively select an uninitialized state of the network hardware that is coupled to the processing device to seed a random number generator process. In one embodiment, the computing device is a compute node in a multi-node processing system, and the memory is a High-Bandwidth Memory.

An artificial-intelligence computing device integrates at least two compute stacks and a hybrid memory subsystem of volatile and non-volatile high-bandwidth memory within an all-silicon domain. The volatile memory stores frequently written activations and key-value caches, while the non-volatile memory stores largely static model weights. Compute stacks communicate over silicon interconnects exceeding one hundred terabits per second, enabling sustained trillion-parameter inference locally without external storage and with energy below one picojoule per bit.