Computing system based on high-bandwidth flash memory chip and electronic equipment
By modifying 3D NAND Flash into a high-bandwidth flash memory chip and establishing a connection with the computing chip, the problem of insufficient bandwidth was solved, the production cost of the computing system was reduced, and the needs of large-scale parallel computing were met.
Patent Information
- Application Number
- CN202423302165.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Utility models(China)
- Current Assignee / Owner
- Filing Date
- 2024-12-30
- Publication Date
- 2025-11-25
- Estimated Expiration
- 2034-12-30
AI Technical Summary
Existing 3D NAND Flash-based memory chips have insufficient bandwidth and cannot replace high-bandwidth memory for direct connection to computing chips for computation, resulting in high production costs for computing systems.
By modifying 3D NAND Flash into high-bandwidth flash memory chips and establishing connections with computing chips, including planar, stacked, or intermediary layer connections, and utilizing data buffers and D2D interfaces, the bandwidth requirements of computing chips can be ensured to meet the computing tasks.
This approach achieves the goal of reducing the production cost of computing systems while meeting the needs of large-scale parallel computing, and improves data transmission efficiency and system stability.
Smart Images
Figure CN223598218U_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The application belongs to the technical field of semiconductors, and particularly relates to a computing system based on a high-bandwidth flash memory chip and an electronic device. BACKGROUND
[0002] At present, high-bandwidth memory (HBM) is often used to be directly connected with a computing chip in a large parallel computing scenario such as an AI model of a large language model, and transmits data to the computing chip through a high-speed interface for computation, because the HBM has the characteristics of high bandwidth, large capacity and low delay. However, the HBM product is relatively expensive, and the production cost of the computing system based on the HBM is relatively high.
[0003] Compared with the HBM, 3D NAND Flash (a kind of three-dimensional stacked flash memory) also has the advantages of large storage capacity and high parallelism, and is relatively low in price. If the 3D NAND Flash can be used to replace the HBM to be directly connected with the computing chip and transmit data for computation, the production cost of the computing system will be greatly reduced. However, the existing storage chip based on the 3D NAND Flash cannot support the computing chip to complete a computing task due to insufficient bandwidth, so the production cost of the computing system cannot be reduced by replacing the HBM. CONTENT OF THE UTILITY MODEL
[0004] The embodiments of the application provide a computing system based on a high-bandwidth flash memory chip and an electronic device, so as to replace the high-bandwidth memory with the high-bandwidth flash memory which is relatively low in price and suitable for a large parallel computing scenario to be directly connected with the computing chip and transmit data for computation, so that the computing demand can be met and the production cost of the computing system can be reduced.
[0005] In a first aspect, the embodiments of the application provide a computing system based on a high-bandwidth flash memory chip, which comprises a high-bandwidth flash memory chip and a computing chip connected with each other, wherein
[0006] The high-bandwidth flash memory chip is configured to directly transmit target computing data to the computing chip according to a data reading command sent by the computing chip, the high-bandwidth flash memory chip has a bandwidth capable of supporting the computing chip to complete a target computing task, the target computing task is used to indicate the data reading command, and the target computing data is data indicated by the data reading command.
[0007] Preferably, the high-bandwidth flash memory chip is connected with the computing chip in a planar manner.
[0008] Preferably, the computing system comprises a data buffer for receiving the target computing data transmitted by the high-bandwidth flash memory chip and for transmitting the target computing data to the computing chip.
[0009] Preferably, the data buffer is arranged in the high-bandwidth flash memory chip.
[0010] Preferably, the data buffer is arranged in the computing chip.
[0011] Preferably, the high-bandwidth flash memory chip and the computing chip are connected through a D2D interface.
[0012] Preferably, the high-bandwidth flash memory chip and the computing chip are connected in a stacked manner, and a form parameter of the high-bandwidth flash memory chip corresponds to a form parameter of the computing chip.
[0013] Preferably, the form parameter comprises a chip size and a chip shape.
[0014] Preferably, the high-bandwidth flash memory chip and the computing chip are connected through an intermediate layer, and the intermediate layer is a circuit board comprising a signal conversion circuit and having two micro-bumps, one micro-bump of the intermediate layer is connected to the high-bandwidth flash memory chip, and the other micro-bump of the intermediate layer is connected to the computing chip.
[0015] In a second aspect, an electronic device is provided, which comprises the computing system according to the first aspect of the embodiments of the present application.
[0016] As can be seen, in the embodiments of the present application, the computing system comprises a high-bandwidth flash memory chip and a computing chip connected to each other, wherein the high-bandwidth flash memory chip is configured to directly transmit target computing data to the computing chip according to a data read command transmitted by the computing chip, and has a bandwidth capable of supporting the computing chip to complete a target computing task, and the target computing data is data indicated by the data read command. In this way, the computing system is formed by directly connecting the high-bandwidth flash memory chip with a lower price and a bandwidth sufficient to support the completion of the target computing task and the computing chip, so that the production cost of the computing system can be reduced while meeting the computing demand. BRIEF DESCRIPTION OF DRAWINGS
[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.
[0018] Figure 1 is a structural block diagram of a prior computing system provided by an embodiment of the present application;
[0019] Figure 2 is a structural block diagram of a computing system based on high-bandwidth flash memory chips provided by an embodiment of the present application;
[0020] Figure 3 is a structural block diagram of another computing system based on high-bandwidth flash memory chips provided by an embodiment of the present application;
[0021] Figure 4 is a structural block diagram of a computing system for characterizing planar connections provided by an embodiment of the present application;
[0022] Figure 5 is a structural block diagram of another computing system for characterizing planar connections provided by an embodiment of the present application;
[0023] Figure 6 is a structural block diagram of a computing system including a data buffer provided by an embodiment of the present application;
[0024] Figure 7 is a structural block diagram of another computing system including a data buffer provided by an embodiment of the present application;
[0025] Figure 8 is a structural block diagram of yet another computing system including a data buffer provided by an embodiment of the present application;
[0026] Figure 9 is a structural block diagram of a computing system including an interposer provided by an embodiment of the present application. DETAILED DESCRIPTION
[0027] In order to make the personnel in the art better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor fall within the scope of protection of the present application.
[0028] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish different objects, but are not used to describe a specific order. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion.
[0029] Reference to“an embodiment” herein means that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the application. The appearances of the phrase“in one embodiment” in various places in the specification are not necessarily referring to the same embodiment, nor are they necessarily mutually exclusive of one another. It is expressly understood that any of the embodiments described herein can be incorporated into any other embodiment.
[0030] Currently, in the application scenarios of large-scale parallel computing such as AI models, high-bandwidth memory-based computing systems are usually used, such as Figure 1 As shown in the figure, the existing computing system includes a central processor 11, a graphics processor 12, a high-bandwidth memory 13, and a flash memory 14. Among them, the central processor 11 specifically refers to CPU, the graphics processor 12 specifically refers to GPU, the high-bandwidth memory 13 refers to HBM, and the flash memory 14 refers to Flash.
[0031] Taking the computing scenario of a large language model as an example, the central processor 11 is specifically used for controlling and coordinating the work of the entire computing system, including data scheduling, instruction distribution, etc., such as data preprocessing, deciding when to load data from Flash to HBM, and when to start GPU for computing; the graphics processor 12 is specifically used for accelerating computing by using parallel computing capability, improving the training and inference speed of the large language model; the high-bandwidth memory 13 is used as the main memory of CPU and GPU, and is used for temporarily storing data being processed and to be processed, including model parameter subsets loaded from Flash, input data, intermediate computing results, etc.; the flash memory 14 belongs to a non-volatile storage medium and is used for storing persistent data, such as model parameters of the large language model. However, the HBM product used by the above computing system architecture is relatively expensive, resulting in a high production cost of such computing system suitable for AI models.
[0032] The present inventors have found through research and experiments that 3D NAND Flash, which is relatively low in price, is very suitable for the above parallel computing scenario due to its inherent structure of the storage array having great parallelism. However, due to the insufficient bandwidth of the existing 3D NAND Flash-based storage chip, the computing chip cannot complete the computing task when the HBM is directly connected to the computing chip for computing. Based on this, the present inventors have improved the existing 3D NAND Flash-based storage chip, obtained the high-bandwidth flash storage chip involved in the embodiments of the present application, and successfully established a connection with the computing chip, in order to reduce the production cost of the computing system on the basis of being able to complete the computing task.
[0033] A computing system based on a high-bandwidth flash storage chip involved in the embodiments of the present application is introduced below.
[0034] Please refer toFigure 2 , Figure 2 is a structural block diagram of a computing system based on a high-bandwidth flash memory chip provided by an embodiment of the present application. As shown in Figure 2 , the computing system 20 includes a high-bandwidth flash memory chip 21 and a computing chip 22, which are connected to each other. Among them, the high-bandwidth flash memory chip 21 is configured to directly transmit target computing data to the computing chip according to a data read command sent by the computing chip 22, the high-bandwidth flash memory chip 21 has a bandwidth capable of supporting the computing chip 22 to complete a target computing task, the target computing task is used to indicate the data read command, and the target computing data is the data indicated by the data read command.
[0035] Further, as shown in Figure 3 , the high-bandwidth flash memory chip 21 includes a high-bandwidth flash memory 211 and a controller 212, the high-bandwidth flash memory 211 is specifically a 3D NAND Flash after modification, and has a bandwidth capable of supporting the computing chip 22 to complete a target computing task. Specifically, the high-bandwidth flash memory 211 can be modified to a memory structure including a plurality of storage unit arrays to increase its memory bandwidth. In this example, the controller 212 is connected to the high-bandwidth flash memory 211 on-chip, and the controller 212 is connected to the computing chip 22 off-chip. The high-bandwidth flash memory 211 is used to store computing data, the controller 212 is used to receive a data read command from the computing chip 22, and to read target computing data from the high-bandwidth flash memory 211 according to the data read command and transmit the target computing data to the computing chip 22 for computing.
[0036] Among them, the target computing task can be specifically a computing task related to a large language model, and the target computing data can be specifically a model parameter of the large language model. Among them, when the computing chip calls the model parameter of the large language model for computing, the execution order of the model parameter is fixed and known, so the above data read command is specifically a read command for a plurality of target computing data with a fixed execution order, and the high-bandwidth flash memory chip can continuously transmit the above plurality of target computing data with a fixed execution order to the computing chip for computing after responding to the data read command, that is, one command corresponds to multiple data, which improves the efficiency of data transmission. It can be understood that the computing system involved in the embodiment of the present application adopts a DMA (Direct Memory Access) architecture, without the need for data scheduling and instruction allocation of a central processing unit, and the computing chip 22 directly interacts with the high-bandwidth flash memory chip 21 for data.
[0037] As can be seen, in the present example, the high-bandwidth flash memory chip and the computing chip are connected to each other, wherein the high-bandwidth flash memory chip is configured to directly transmit target computing data to the computing chip according to a data read command sent by the computing chip, and has a bandwidth capable of supporting the computing chip to complete a target computing task, the target computing data being data indicated by the data read command. In this way, by directly connecting the high-bandwidth flash memory chip with a lower price and a bandwidth sufficient to support the completion of the target computing task to the computing chip to form a computing system, the production cost of the computing system can be reduced while meeting the computing demand.
[0038] In one possible example, the high-bandwidth flash memory chip and the computing chip are connected in a planar manner.
[0039] The planar connection refers to a connection manner in which the high-bandwidth flash memory chip and the computing chip are connected on the same horizontal plane. In the present embodiment, the planar connection can specifically refer to a connection relationship established by mounting the high-bandwidth flash memory chip and the computing chip on the same PCB (Printed Circuit Board), and can specifically be a connection relationship established by a common metal wire, a coaxial cable, or a PCB copper foil trace. Figure 4 As shown in FIG. 2, the computing system 20 includes a PCB 40, and the high-bandwidth flash memory chip 21 and the computing chip 22 are mounted on the PCB 40, and the high-bandwidth flash memory chip 21 and the computing chip 22 are connected to each other on the PCB 40.
[0040] In other embodiments, the planar connection can also be a connection relationship established by mounting the high-bandwidth flash memory chip and the computing chip on different PCBs but on the same horizontal plane, and can specifically be a connection relationship established by a board-to-board connector, without being limited to this.
[0041] Further, in actual application scenarios, the bandwidth of a single high-bandwidth flash memory chip can not meet the bandwidth demand of the computing system, in which case multiple high-bandwidth flash memory chips can be used to jointly connect the computing chip to expand the overall transmission bandwidth. For example, if the transmission bandwidth of each high-bandwidth flash memory chip is 100 G B / s (byte / second), and the bandwidth demand of the computing system is 400 G B / s, four high-bandwidth flash memory chips can be used to jointly connect the computing chip to meet the bandwidth demand of the computing system. Figure 5 As shown in FIG. 3, the computing system 20 includes four high-bandwidth flash memory chips 21 and one computing chip 22, and the four high-bandwidth flash memory chips 21 are all connected to the computing chip 22 in a planar manner, so that the transmission bandwidth of the computing system reaches 400 G B / s, meeting the design requirement.
[0042] As can be seen, in the present example, the high-bandwidth flash memory chip and the computing chip are connected in a flat manner, and a stable connection relationship can be established even if the high-bandwidth flash memory chip and the computing chip have different shapes and sizes, thereby improving the versatility.
[0043] In one possible example, the computing system comprises a data buffer configured to receive the target computing data transmitted by the high-bandwidth flash memory chip and transmit the target computing data to the computing chip.
[0044] As shown in FIG. 2, the computing system 20 comprises a data buffer 23 connected to the high-bandwidth flash memory chip 21 at one end and to the computing chip 22 at the other end, and configured to receive the target computing data transmitted by the high-bandwidth flash memory chip 21 and transmit the target computing data to the computing chip 22. Figure 6 As shown in FIG. 2, the computing system 20 comprises a data buffer 23 connected to the high-bandwidth flash memory chip 21 at one end and to the computing chip 22 at the other end, and configured to receive the target computing data transmitted by the high-bandwidth flash memory chip 21 and transmit the target computing data to the computing chip 22.
[0045] In one possible example, the data buffer is arranged in the high-bandwidth flash memory chip.
[0046] As shown in FIG. 2, the data buffer 23 is arranged in the high-bandwidth flash memory chip 21 and configured to directly receive the target computing data and be connected to the computing chip 22 to transmit the target computing data to the computing chip 22. Figure 7 As shown in FIG. 2, the data buffer 23 is arranged in the high-bandwidth flash memory chip 21 and configured to directly receive the target computing data and be connected to the computing chip 22 to transmit the target computing data to the computing chip 22.
[0047] In one possible example, the data buffer is arranged in the computing chip.
[0048] In actual application scenarios, some computing chips themselves have sufficient buffer regions, which can be implemented as the data buffer. Figure 8As shown, the data buffer 23 is arranged in the computing chip 22, and is connected to the high-bandwidth flash memory chip 21, and is used to transmit the target computing data to the computing chip 22 directly after receiving the target computing data transmitted by the high-bandwidth flash memory chip 21. In this way, in the application scenario where the computing chip has a buffer area with sufficient capacity, the buffer area is implemented as a data buffer, and is connected to the high-bandwidth flash memory chip for data interaction, thereby reducing the demand for the area of the board card, and further reducing the production cost.
[0049] In one possible example, the high-bandwidth flash memory chip and the computing chip are connected through a D2D interface.
[0050] The D2D (Die to Die) interface refers to a data interface between two silicon chip dies in the same package, and connects the two dies through an extremely short channel to achieve ultra-high energy efficiency and ultra-high bandwidth, and belongs to the technical category of chiplet architecture, and the UCIe (Universal Chiplet Interconnect Express) protocol defines the interface specification, so that dies manufactured by different manufacturers and different process nodes can realize high-speed, low-delay, high-bandwidth and reliable interconnection through a unified standard, and promote the cooperative work between the dies.
[0051] As can be seen, in the example, the high-bandwidth flash memory chip and the computing chip can be connected through the D2D interface, and the influence of the differences such as the manufacturer and the process node is ignored, and the universality is improved.
[0052] In one possible example, the form parameters of the high-bandwidth flash memory chip correspond to the form parameters of the computing chip, and the high-bandwidth flash memory chip and the computing chip are connected in a stacked manner.
[0053] When the form parameters of the high-bandwidth flash memory chip correspond to the form parameters of the computing chip, the high-bandwidth flash memory chip and the computing chip can be connected in a stacked manner to form a computing system, which can greatly shorten the connection distance between the chips and improve the signal transmission speed. The form parameters include the size and shape of the chip. The stacked connection can be realized through the TSV (Through Silicon Via) technology, that is, a small hole is carved in the silicon wafer, and a conductive material is filled in the hole, so that the chips in different layers can be directly connected through the conductive channels. In addition, the stacked connection can also meet the special design requirements in actual application scenarios, for example, the high-bandwidth flash memory chip is specially produced for the computing chip with specific form parameters.
[0054] It can be seen that in the example, in the scenario where the form parameters of the high-bandwidth flash memory chip and the form parameters of the computing chip correspond to each other, the stacked connection of the high-bandwidth flash memory chip and the computing chip shortens the connection distance between the chips and improves the signal transmission speed.
[0055] In one possible example, the connection interface between the high-bandwidth flash memory chip and the computing chip is incompatible, and the high-bandwidth flash memory chip and the computing chip are connected through an interposer. The interposer is a circuit board including a signal conversion circuit and having two micro-bump surfaces. One micro-bump surface of the interposer is connected to the high-bandwidth flash memory chip, and the other micro-bump surface is connected to the computing chip.
[0056] In actual application scenarios, the connection interface between the high-bandwidth flash memory chip and the computing chip may be incompatible due to manufacturing processes and other reasons. In this case, the high-bandwidth flash memory chip and the computing chip can be connected through an interposer. Figure 9 As shown in FIG. 6, the computing system 20 includes an interposer 24. The interposer 24 includes a first micro-bump surface 241, a signal conversion circuit 242, and a second micro-bump surface 243. The first micro-bump surface 241 is connected to the high-bandwidth flash memory chip 21 and is configured to receive a signal carrying target computing data transmitted by the high-bandwidth flash memory chip 21. The signal conversion circuit 242 is configured to convert the signal into a signal compatible with the interface of the computing chip 22. The second micro-bump surface 243 is connected to the computing chip 22 and is configured to transmit the converted signal to the computing chip 22. In addition, the interposer 24 can also buffer and enhance the signal to improve the transmission instruction of the signal.
[0057] It can be seen that in the example, when the connection interface between the high-bandwidth flash memory chip and the computing chip is incompatible, the high-bandwidth flash memory chip and the computing chip are connected through an interposer, and then the signal carrying the target computing data is converted to assist data transmission, thereby improving the stability of data transmission.
[0058] The application also provides an electronic device including the computing system based on the high-bandwidth flash memory chip as described in the above embodiments.
[0059] Although the application is disclosed as above, the application is not limited thereto. Any person skilled in the art can easily think of changes or replacements without departing from the spirit and scope of the application.
Claims
1. A high-bandwidth flash memory chip-based computing system, characterized by, Comprising: a high-bandwidth flash memory chip and a computing chip connected to each other; wherein the high-bandwidth flash memory chip is configured to transmit target computing data directly to the computing chip according to a data read command sent by the computing chip, the high-bandwidth flash memory chip has a bandwidth capable of supporting the computing chip to complete a target computing task, the target computing task is used to indicate the data read command, and the target computing data is data indicated by the data read command.
2. The computing system of claim 1, wherein, The high-bandwidth flash memory chip and the computing chip are connected in a planar manner.
3. The computing system of claim 2, wherein, The computing system comprises a data buffer configured to receive the target computing data transmitted by the high-bandwidth flash memory chip and transmit the target computing data to the computing chip.
4. The computing system of claim 3, wherein, The data buffer is arranged in the high-bandwidth flash memory chip.
5. The computing system of claim 3, wherein, The data buffer is arranged in the computing chip.
6. The computing system of any of claims 2-5, wherein, The high-bandwidth flash memory chip and the computing chip are connected through a D2D interface.
7. The computing system of claim 1, wherein, The high-bandwidth flash memory chip and the computing chip are connected in a stacked manner.
8. The computing system of claim 7, wherein, The form parameters of the high-bandwidth flash memory chip correspond to the form parameters of the computing chip.
9. The computing system of claim 1, wherein, The form parameters include chip size and chip shape.
10. An electronic device, comprising: The connection interface between the high-bandwidth flash memory chip and the computing chip is incompatible, the high-bandwidth flash memory chip and the computing chip are connected through an interposer, the interposer is a circuit board comprising a signal conversion circuit and having two micro-bumps, one micro-bump of the interposer is connected to the high-bandwidth flash memory chip, and the other micro-bump is connected to the computing chip. The electronic device comprises the computing system according to any one of claims 1-9.