309 results about "High Bandwidth Memory" patented technology

Stacked devices and methods of fabrication are provided. Die-to-wafer (D2W) direct-bonding techniques join layers of dies of various physical sizes, form factors, and foundry nodes to a semiconductor wafer, to interposers, or to boards and panels, allowing mixing and matching of variegated dies in the fabrication of 3D stacked devices during wafer level packaging (WLP). Molding material fills in lateral spaces between dies to enable fan-out versions of 3D die stacks with fine pitch leads and capability of vertical through-vias throughout. Molding material is planarized to create direct-bonding surfaces between multiple layers of the variegated dies for high interconnect density and reduction of vertical height. Interposers with variegated dies on one or both sides can be created and bonded to wafers. Logic dies and image sensors from different fabrication nodes and different wafer sizes can be stacked during WLP, or logic dies and high bandwidth memory (HBM) of different geometries can be stacked during WLP.

A photonic integrated circuit (PIC) disposed on a substrate and comprising a semiconductor die hosting an active portion and a passive portion mutually coupled, the active portion being configured to consume electrical power when activated, and the passive portion comprising an optical transmission medium configured to propagate an optical signal to or from the active portion of the PIC; an electronic integrated circuit (EIC) electrically coupled to the active portion of the PIC and comprising components that electrically operate on the active portion of the PIC; and a packaging compound at least partially encapsulating the PIC, the packaging compound defining a cavity on a side of the semiconductor die that is opposite from the substrate, the cavity being filled with an optically transparent medium such that the optical signal can be received from or transmitted to the passive portion of the PIC through the cavity.

The invention discloses a hardware accelerator and equipment oriented to triple sparse matrix multiplication and an application method thereof.The hardware accelerator comprises a high-bandwidth memory HBM, a crossbar switch network and an on-chip processing unit which are connected in sequence, and the on-chip processing unit comprises a hierarchical cache module, a global controller and a plurality of computing chips; each calculation piece comprises an RA calculation array, a TP calculation array and a local controller, wherein the RA calculation array and the TP calculation array are respectively used for executing front-end operation T = R * A and rear-end operation C = T * P in triple sparse matrix multiplication. The method aims at solving the problem that when a traditional universal processor processes triple sparse matrix multiplication, due to irregular memory access, uneven calculation load and sharp increase of middle parts and results, huge off-chip data carrying is confronted with serious performance and energy efficiency bottlenecks, and the calculation performance and energy efficiency of triple sparse matrix multiplication are improved.

A package includes a photonic integrated circuit (PIC) disposed on a substrate and comprising a semiconductor die hosting an active portion and a passive portion mutually coupled, the active portion being configured to consume electrical power when activated, and the passive portion comprising an optical transmission medium configured to propagate an optical signal to or from the active portion of the PIC; an electronic integrated circuit (EIC) electrically coupled to the active portion of the PIC and comprising components that electrically operate on the active portion of the PIC; and a packaging compound at least partially encapsulating the PIC, the packaging compound defining a cavity on a side of the semiconductor die that is opposite from the substrate, the cavity being filled with an optically transparent medium such that the optical signal can be received from or transmitted to the passive portion of the PIC through the cavity.

A photonic integrated circuit (PIC) disposed on a substrate and comprising a semiconductor die hosting an active portion and a passive portion mutually coupled, the active portion being configured to consume electrical power when activated, and the passive portion comprising an optical transmission medium configured to propagate an optical signal to or from the active portion of the PIC; an electronic integrated circuit (EIC) electrically coupled to the active portion of the PIC and comprising components that electrically operate on the active portion of the PIC; and a packaging compound at least partially encapsulating the PIC, the packaging compound defining a cavity on a side of the semiconductor die that is opposite from the substrate, the cavity being filled with an optically transparent medium such that the optical signal can be received from or transmitted to the passive portion of the PIC through the cavity.

In order to enable applications such as artificial intelligence (AI) and machine learning in a large scale, a large amount of information needs to be processed very fast. To improve the speed of the computation, not only faster graphical and central processors are required, but also the communication between processors and high bandwidth memories should be fast enough to reduce the latency. Optical interconnect can be a replacement for copper interconnect which is providing larger bandwidth and lower latency. As an efficient light source, micro-LEDs can be used as light source for chip-to-chip optical interconnect. Using a massively parallel memory-processor interconnect using low-power micro-LED, the computation speed will increase dramatically, and less amount of energy will be wasted through copper self-heating.

A package includes a photonic integrated circuit (PIC) disposed on a substrate and comprising a semiconductor die hosting an active portion and a passive portion mutually coupled, the active portion being configured to consume electrical power when activated, and the passive portion comprising an optical transmission medium configured to propagate an optical signal to or from the active portion of the PIC; an electronic integrated circuit (EIC) electrically coupled to the active portion of the PIC and comprising components that electrically operate on the active portion of the PIC; and a packaging compound at least partially encapsulating the PIC, the packaging compound defining a cavity on a side of the semiconductor die that is opposite from the substrate, the cavity being filled with an optically transparent medium such that the optical signal can be received from or transmitted to the passive portion of the PIC through the cavity.

An IC structure includes a memory stack including semiconductor dies horizontally separate with each other, wherein each semiconductor die has a top surface, a bottom surface, four sidewalls, and a plurality of edge pads arranged along a sidewall. The area of the bottom surface or the top surface of each semiconductor die is larger than that of any sidewall. The IC structure further includes a logic die with memory controller and processor circuit under the memory stack and electrically connected to the plurality of edge pads of each semiconductor memory die, and a packaging substrate under and electrically connected to the logic die with memory controller and processor. There is no interposer between the packaging substrate and the logic die with memory controller and processor circuit, and there is no TSV in each semiconductor die.

Memory systems and methods of assembly are described in which a memory system includes a routing substrate, a processor on a first side of the routing substrate, a memory die stack on the first side of the routing substrate, and a buffer bridge die embedded in the routing substrate and electrically connecting the memory die stack with the processor.

The invention relates to the technical field of large language model deployment, and discloses a large language model weight inverse quantization reasoning device and method.The method comprises the steps that low-precision weight data is transmitted to a high-bandwidth storage from a host and then transmitted to an on-chip storage through the high-bandwidth storage; data conversion from a low-precision format to a high-precision format is completed in the on-chip memory, the data is multiplied by an inverse quantization factor to obtain recovered high-precision weight data, and the functional unit is responsible for executing general matrix multiplication of input data and the high-precision weight data after inverse quantization. And pipeline parallel execution of the inverse quantization operation and the general matrix multiplication operation is realized through a double-buffering technology. According to the invention, on the basis of a dual-path inverse quantization architecture of the vector processing unit and a dual-buffer mechanism in the on-chip memory, the problem of hardware adaptation of low-precision calculation is solved, and efficient execution of a low-precision conversion algorithm is realized under the condition of limited hardware resources.

The invention discloses a multi-modal extensible high-performance homomorphic encryption system, and relates to the technical field of computer systems, the system comprises an upper-layer server software and drive architecture and a multi-modal homomorphic encryption hardware architecture realized based on an FPGA (Field Programmable Gate Array), which are in communication connection; the upper-layer server software and drive architecture comprises a multi-modal homomorphic encryption algorithm software library, a software memory scheduler, a modulo calculation module software drive and a floating point calculation module software drive; the multi-modal homomorphic encryption hardware architecture realized based on the FPGA comprises a communication data controller, a plurality of modulo calculation modules, a floating point calculation module, a data read-write bus and a high-bandwidth memory, and each modulo calculation module comprises a fast number theory transformation calculation unit. Through a software and hardware collaborative design scheme, the problem of low scheduling efficiency of ciphertext calculation tasks of different scales in hardware architecture design under multi-modal homomorphic encryption is solved, and the calculation efficiency is remarkably improved.

Methods for fabricating flexible interposers for providing electrical connection between devices mounted at different vertical positions with respect to a substrate or a planar interposer. A bonded structure may comprise a bent flexible interposer extending from a first interposer portion between a main device on the substrate or the planar interposer and a second interposer portion above the main device and above or below a device positioned above the main device and electrically connected to the main device via a bent portion of the flexible interposer.

Stacked devices and methods of fabrication are provided. Die-to-wafer (D2W) direct-bonding techniques join layers of dies of various physical sizes, form factors, and foundry nodes to a semiconductor wafer, to interposers, or to boards and panels, allowing mixing and matching of variegated dies in the fabrication of 3D stacked devices during wafer level packaging (WLP). Molding material fills in lateral spaces between dies to enable fan-out versions of 3D die stacks with fine pitch leads and capability of vertical through-vias throughout. Molding material is planarized to create direct-bonding surfaces between multiple layers of the variegated dies for high interconnect density and reduction of vertical height. Interposers with variegated dies on one or both sides can be created and bonded to wafers. Logic dies and image sensors from different fabrication nodes and different wafer sizes can be stacked during WLP, or logic dies and high bandwidth memory (HBM) of different geometries can be stacked during WLP.

A semiconductor device and a method of manufacturing the semiconductor device are provided. The semiconductor device may include: a first semiconductor chip; at least one semiconductor chip stack on a surface of the first semiconductor chip in a first direction of the semiconductor device, the at least one semiconductor chip stack configured to be electrically connected to the first semiconductor chip; and a dielectric on the first semiconductor chip in the first direction, and surrounding the at least one semiconductor chip stack in at least one second direction of the semiconductor device, perpendicular to the first direction, wherein the at least one semiconductor chip stack includes at least one second semiconductor chip.

The present application relates to the technical field of computers, and discloses a dimension conversion chip, a method, a device, a storage medium and a program product. The chip comprises: an information acquisition circuit, used for acquiring dimension conversion information, the dimension conversion information being used for instructing to convert a tensor from a first layout format to a second layout format; a parameter determination circuit, used for determining parameters of a dimension conversion instruction on the basis of the dimension difference between the first layout format and the second layout format; and a dimension conversion circuit, used for reading the tensor from a high-bandwidth memory, and executing the dimension conversion instruction on the basis of the parameters so as to perform dimension transposition on the tensor in the first layout format to obtain a tensor in the second layout format, wherein the tensor in the second layout format is stored in the high-bandwidth memory and is used for being inputted into a neural network layer in a neural network model.

An apparatus including a high bandwidth memory circuit and associated systems and methods are disclosed herein. The high bandwidth memory circuit can include two or more physical layer circuits to communicate with neighboring devices. The high bandwidth memory circuit can broadcast a status to the neighboring devices. The neighboring devices can be configured according to the operating demands of the high bandwidth memory circuit.

An example microelectronic assembly includes a substrate, a bridge die over the substrate, and a die stack between the substrate and the bridge die, the die stack including a logic die and at least one memory die, where the logic die is between the at least one memory die and the bridge die.

The invention discloses a sparse LU decomposition acceleration method for an FPGA (field programmable gate array) with a flow-sensing high-bandwidth memory, belongs to the technical field of LU decomposition hardware acceleration, and aims to solve the problem of performance bottleneck and efficiency challenge faced by sparse LU decomposition on the FPGA. The acceleration method comprises the following steps of data stream preprocessing, wherein matrix data and metadata are stored and packaged based on a sparse storage format; customizing an FPGA hardware accelerator: carrying out HBM channel allocation and collaborative design of a control unit and memory management; parallel data flow management and scheduling are carried out, and sparse LU decomposition is executed; timely data supply is ensured through a multi-stage pipeline prefetching mechanism; and transmitting the task flow and the data flow to a special parallel processing engine for MAC / DIV calculation in combination with a flow sensing synchronous scheduling strategy.

The present disclosure relates methods, devices, systems, and techniques for high bandwidth memory (HBM). An example semiconductor device includes a first layer, a second layer, a first die between the first layer and the second layer, and a second die stacked together along a first direction. Each of the first die and the second die has a conductive layer. The first die and the second die are bonded through the second layer. The semiconductor device further includes a first contact structure coupled to the conductive layer of the first die and a second contact structure coupled to the conductive layer of the second die. The first contact structure extends along the first direction and contacts the conductive layer of the first die without extending through the second layer. The second contact structure extends through the second layer along the first direction without extending through the second die.

In an embodiment, a memory system may include at least two types of DRAM, which differ in at least one characteristic. For example, one DRAM type may be a high density DRAM, while another DRAM type may have lower density but may also have lower latency and higher bandwidth than the first DRAM type. DRAM of the first type may be on one or more first integrated circuits and DRAM of the second type may be on one or more second integrated circuits. In an embodiment, the first and second integrated circuits may be coupled together in a stack. The second integrated circuit may include a physical layer circuit to couple to other circuitry (e.g., an integrated circuit having a memory controller, such as a system on a chip (SOC)), and the physical layer circuit may be shared by the DRAM in the first integrated circuits.

An apparatus to facilitate processing in a multi-tile device is disclosed. In one embodiment, the apparatus includes a graphics processor comprising a first semiconductor die including a first high-bandwidth memory (HBM) device, a second semiconductor die including a second HBM device, and a third semiconductor die coupled with the first semiconductor die and the second semiconductor die in a 2.5-dimensional (2.5D) arrangement. The third semiconductor die includes a graphics processing resource and a cache coupled with the graphics processing resource. The cache is configurable to cache data associated with memory accessed by the graphics processing resource and the graphics processing resource includes a general-purpose graphics processor core and a tensor core.

A system includes a high bandwidth memory (HBM) and a convolutional neural network (CNN) engine. The HBM includes a virtual bank portion and a system memory portion. The virtual bank portion is configured to store a feature map data and the system memory portion is configured to support data exchanges with a host. The CNN engine includes a convolutional unit configured to execute convolutional layer instructions, a depthwise convolutional unit configured to execute depthwise layer instructions, and a first on-chip buffer. The first on-chip buffer is configured to receive and store the feature map data from the virtual bank portion or receive and store data results from the convolutional unit. The first on-chip buffer is further configured to send the feature map data or the data results from the convolutional unit to the depthwise convolutional unit for processing.

Provided are systems, methods, and apparatuses for high bandwidth memory configurations. In one or more examples, the systems, devices, and methods include positioning a first memory physical layer (PHY) interface on a surface of a compute die and connecting a first memory to the first memory PHY interface of the compute die via a first through-silicon via (TSV) connection. The systems, devices, and methods include connecting a second memory to a base die that connects to the compute die via a silicon interposer, positioning the compute die on the silicon interposer, and positioning the base die on the silicon interposer.

The invention relates to the technical field of semiconductors, provides a semiconductor device, and is used for solving the technical problem that a large lifting space still exists in a high-bandwidth memory technology. The semiconductor device includes: a chip stacking structure including a first semiconductor chip and a second semiconductor chip stacking structure stacked in a first direction; the first semiconductor chip comprises a controller, a calculation module and an interface module; the controller is configured to execute a first access operation on storage data in the second semiconductor chip stacking structure and send the storage data to the interface module to allow external equipment to access the storage data in the second semiconductor chip stacking structure through the interface module; and performing a second access operation on the storage data in the second semiconductor chip stack structure and sending the storage data to the computing module to allow the computing module to perform a computing operation on the storage data. Therefore, the integration level can be improved, the data transmission path can be shortened, and bandwidth consumption and energy consumption loss can be reduced.

Overhang architectures for high bandwidth memory (HBM) multi-die assemblies and methods for making same. The overhang architecture places the DRAM (HBM) underneath the top IC die. The signal interconnects between the top IC die and the DRAM die are direct signal interconnects without lateral routing on a package substrate or on a motherboard.

The invention provides a semiconductor stacking structure and a preparation method thereof, a high-bandwidth memory and electronic equipment, and relates to the technical field of semiconductor chips. The semiconductor stacked structure comprises a plurality of stacked semiconductor structures, each semiconductor structure comprises a storage unit layer and a bonding layer, the bonding layer is arranged on the surface, facing the adjacent semiconductor structure, of the storage unit layer, and the bonding layer is coupled to the storage unit layer. Wherein two adjacent semiconductor structures are in bonding connection through a bonding layer. The semiconductor structure is applied to the dynamic random access memory so as to realize data reading and writing operation.

Memory systems and methods of assembly are described in which a memory system includes a routing substrate, a processor on a first side of the routing substrate, a memory die stack on the first side of the routing substrate, and a buffer bridge die embedded in the routing substrate and electrically connecting the memory die stack with the processor.

The invention provides an FPGA high-bandwidth memory transmission system and method based on a network-on-chip and burst optimization, the system comprises a calculation unit module, an on-chip interconnection module, an HBM management module and an HBM memory, an Omega network topology is arranged in the on-chip interconnection module, and a non-blocking FIFO return voltage flow control mechanism is introduced into the Omega network topology; a fine-grained burst control unit is arranged in the HBM management module, and the fine-grained burst control unit comprises a preset burst transmission configuration code class; the calculation unit module is used for acquiring a defined FPGA calculation unit, constructing a request data packet, acquiring return data of the on-chip interconnection module and generating a response data packet; the on-chip interconnection module and the HBM management module are used for transmitting communication data; the HBM memory is used for storing the HBM memory data. According to the invention, efficient access of the HBM in the FPGA system is realized, the problem of on-chip resource competition during cross-channel access is avoided, and the overall bandwidth performance of the system is improved.

The embodiment of the invention provides a chip and a data processing method, and relates to the field of integrated circuits. The chip is used for processing a deep learning algorithm, and the deep learning algorithm comprises a calculation-intensive operator and a memory-access-intensive operator. The chip at least comprises a calculation core particle and a bandwidth core particle, the calculation core particle and the bandwidth core particle are coupled through a core particle interconnection interface, and the bandwidth core particle is further directly connected with the high-bandwidth memory; computing intensive operators are mapped to the computing core particles, and memory access intensive operators are mapped to the bandwidth core particles; the calculation core particle is used for executing a task corresponding to the calculation-intensive operator, and the bandwidth core particle is used for executing a task corresponding to the memory access-intensive operator. According to the method, operators with different requirements on computing power specifications and bandwidth specifications in the deep learning algorithm are processed through different core grains in the same chip, and the problem of hardware resource waste caused by great difference of different parts of the deep learning type algorithm on computing and bandwidth capability requirements is effectively solved.

The invention provides a high-bandwidth memory capacity expansion method and device, equipment and a storage medium, and the method comprises the steps: connecting capacity expansion card equipment with a graphics processor through a PCIe interface on the graphics processor, and enabling the storage space of the capacity expansion card equipment to serve as the shared storage space of a high-bandwidth memory (HBM). Firstly, in a kernel function of a graphics processor, a shared storage space is declarated by using a shared keyword, and an extended declaration is generated. Secondly, copying HBM memory data to a shared storage space of the expansion card equipment through threads in the thread block, and performing thread synchronization on the thread block by using a synchronization function; and on the basis, parallel computing processing is performed, and a computing result is allocated and copied between the shared storage space of the HBM and the shared storage space of the capacity expansion card equipment, so that storage capacity expansion is realized. According to the method, the HBM capacity can be effectively expanded, the big data and big model processing requirements are met, the GPU computing efficiency is improved, and the expansion cost is reduced.