Chip system, board card and parallel processing device

Abstract

The embodiment of the invention is suitable for the technical field of computers, and provides a chip system, a board card and a parallel processing device.The chip system comprises a GPU main crystal grain and at least one IO expansion slave crystal grain; the first interface unit of the main crystal grain is connected with the first storage unit, the third interface unit of the slave crystal grain is connected with the second interface unit of the main crystal grain and the slave crystal grain bus, the fourth interface units are respectively connected with the second storage unit and the slave crystal grain bus, and the number of the fourth interface units is greater than that of the third interface units; the processing unit is configured to perform data interaction with the first storage unit and the second storage unit; the bandwidth between each second interface unit and each third interface unit which are connected with each other is higher than the bandwidth between each first interface unit and each first storage unit which are connected with each other; the bandwidth of each slave crystal grain bus is higher than the product of the bandwidth between the first interface unit and the first storage unit which are connected with each other and the number of the third interface units on the slave crystal grains. According to the embodiment of the invention, bandwidth expansion can be realized.

Description

TECHNICAL FIELD

[0001] The embodiment of the present application belongs to the technical field of computers, and particularly relates to a chip system, a board card and a parallel processing device. BACKGROUND

[0002] GPGPU (General-Purpose Computing on Graphics Processing Units) is one of the typical processors in the computer field, and is particularly suitable for processing a large number of parallel computing tasks. The bandwidth directly affects the performance of the GPGPU.

[0003] The GPGPU is usually packaged as a small-sized GPU chip. Due to the limitation of the technical process, the size of the GPU chip is limited, so that the number of IO (Input / Output) interfaces arranged in one GPU chip is also limited. When the performance of the GPU is improved by using the limited IO interface, if a large number of storage units are directly connected, it will bring great challenges to the size of the chip. At the same time, the performance of the GPU is affected by the bandwidth, and the bandwidth directly affects the performance release of the GPU chip. Moreover, if the storage unit and the GPU are connected through a PCB board, the smaller the length of the connecting wire between the two is, the smaller the time delay is. However, for the GPU chip of a fixed size, the small connection length means that the distance between multiple storage units is small, and the space near the GPU on the PCB board is limited. If all the IO interfaces of the GPU are directly connected to the storage units, a large number of storage units will be densely arranged around the GPU, and there are hundreds of connecting wires for each IO interface, which will cause the connection difficulty and cause the blocking problem of the PCB board.

[0004] Therefore, an IO interface setting strategy is needed to improve the bandwidth of the GPU chip within the limitation of the size of the GPU chip, and effectively exert the performance of the GPU chip. SUMMARY

[0005] Therefore, the embodiment of the present application provides a chip system, a board card and a parallel processing device to improve the bandwidth of the GPU chip within the limitation of the size of the GPU chip.

[0006] The first aspect of the embodiment of the present application provides a chip system, comprising a GPU main die and at least one IO expansion slave die; The main die is provided with a first interface unit, a second interface unit and a processing unit; the first interface unit is connected with a first storage unit, and the second interface unit is connected with the slave die; The slave die is provided with a third interface unit, a fourth interface unit and a slave die bus; the third interface unit is connected with the second interface unit, the fourth interface unit is connected with the second storage unit, the number of the fourth interface units is greater than the number of the third interface units, and the slave die bus is in communication connection with the third interface unit and the fourth interface unit; The processing unit is configured to interact with the first storage unit and the second storage unit. The bandwidth between each interconnected second interface unit and third interface unit is higher than the bandwidth between each interconnected first interface unit and first storage unit, and the bandwidth of each slave die bus is higher than the product of the bandwidth between each interconnected first interface unit and first storage unit and the number of the third interface units on the slave die. In some implementations of the first aspect, the first storage unit and the second storage unit are the same type of memory.

[0007] In some implementations of the first aspect, the second interface unit and the third interface unit are ports based on serializer and deserializer technology; and the type of the memory is DDR or LPDDR.

[0008] In some implementations of the first aspect, the second interface unit and the third interface unit are connected through SERDES-based UCIE, and the slave die further comprises at least one cache unit; the cache unit is arranged between the third interface unit and the fourth interface unit; the cache unit is configured to cache target data of each second storage unit connected with the cache unit; the target data includes at least one of recently most frequently used data and preset designated data.

[0009] In some implementations of the first aspect, the third interface unit in the slave die is multiple, one third interface unit is connected with one cache unit; one cache unit is connected with each fourth interface unit, and the capacity of the cache unit is less than 1 / 1000 of the capacity of the second storage unit.

[0010] In some implementations of the first aspect, any slave die is provided with a first inter-board interconnection interface unit.

[0011] In some implementations of the first aspect, part of the first storage units are arranged outside the master die; and for each slave die, part of the second storage units are located outside the slave die.

[0012] In some implementations of the first aspect, the master die is provided with a master die bus; the master die includes a routing unit connected with the first interface unit and the second interface unit through the master die bus, and the routing unit is configured to determine a memory interleaving address corresponding to the first storage unit and the second storage unit based on the total number of storage units, and to interact with the first storage unit and / or the second storage unit based on the memory interleaving address. The second aspect of the embodiments of the present application provides a board card, which includes a carrier and a chip system as described in the first aspect above; the master die in the chip system is located in a master chip, and the slave die in the chip system is located in a slave chip; the master chip is connected with each slave chip through the carrier. The third aspect of the embodiments of the present application provides a parallel processing device, which includes a device main body and a board card as described in the second aspect above; the board card is provided with a first inter-board interconnection interface unit, and the device main body is provided with a plurality of second inter-board interconnection interfaces adapted to the first inter-board interconnection interfaces; the device main body is configured to call at least two board cards in parallel in the case of being connected with a plurality of board cards.

[0013] In the embodiments of the present application, by setting part of the IO interfaces in the GPU master die as the second interface units adapted to the third interface units in the IO expansion slave die, setting part of the IO interface units in the master die as the first interface units connected with the first storage units, and setting the fourth interface units connected with the second storage units in the slave die, an IO interface setting strategy for the GPU master die is provided. Since the bandwidth between each interconnected second interface unit and third interface unit is higher than the bandwidth between each interconnected first interface unit and first storage unit, and the bandwidth of each slave die bus is higher than the product of the bandwidth between each interconnected first interface unit and first storage unit and the number of third interface units on the slave die, the sum of the interconnection bandwidths between the same number of third interface units and second interface units is higher than the sum of the direct connection bandwidths between the first interface units and the first storage units, the processing units in the master die can interact with the first storage units and the second storage units, the bandwidth expansion of the master die is realized through the IO interface setting strategy, and the effective performance of the master die is improved. BRIEF DESCRIPTION OF DRAWINGS

[0014] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiments or prior art description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without any creative effort.

[0015] Figure 1 is a structural schematic diagram of a chip system provided by an embodiment of the present application; Figure 2 is a structural schematic diagram of a chip system provided by an embodiment of the present application; Figure 3 is a structural schematic diagram of a chip system provided by an embodiment of the present application; Figure 4 is a structural schematic diagram of a chip system provided by an embodiment of the present application. DETAILED DESCRIPTION

[0016] In the following description, for the purposes of explanation and not limitation, specific details are set forth, such as particular sequences of steps, techniques, etc. in order to provide a thorough understanding of the embodiments of the application. However, it will be apparent to those skilled in the art that the embodiments of the present application can be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the application with unnecessary detail.

[0017] It is to be understood that when an element is referred to as being "on" or "connected to" another element, it can be directly on or connected to the other element or indirectly on or connected to the other element by way of one or more other elements. The terms "length", "width", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", and the like as terms of reference merely describe the orientation or position of an apparatus or element as shown in the drawings, and are used only to facilitate description of the present application and are not intended to limit or indicate that the apparatus or element must have a particular orientation, be constructed and operated in a particular orientation, and therefore should not be construed as limiting the application.

[0018] In addition, in the description of the present application and the appended claims, the terms "first", "second", "third", etc. are used only to distinguish descriptions and cannot be understood as indicating or implying relative importance.

[0019] In the present specification, the expressions "one embodiment" or "some embodiments" or the like mean that a specific feature, structure or characteristic described in connection with the embodiment is included in at least one embodiment of the present application. Therefore, the expressions "in one embodiment", "in some embodiments", "in other some embodiments", "in yet some embodiments" or the like appearing in various places in the specification are not necessarily all referring to the same embodiment, but mean "one or more but not all embodiments" unless otherwise specifically stated. The terms "include", "comprise", "have" and their conjugates mean "including but not limited to", unless otherwise specifically stated.

[0020] It should be noted that the modules and units mentioned in the present application can include one or more hardware circuits, and such hardware circuits can include but are not limited to: application specific integrated circuits, digital signal processors, field programmable gate arrays, discrete logic circuits, state machines, any combination of the foregoing circuits; these circuits are specifically designed, configured and interconnected to perform one or more specific functions disclosed in the present application.

[0021] One of the concepts of the embodiments of the present application is to connect the IO expansion slave die through the original IO interface of the GPU master die, and connect more storage units through the slave die, and the corresponding interface is set so that the bandwidth of the master die connected with the slave die through the IO interface is higher than the bandwidth of directly connecting a storage unit through the IO interface, so as to realize the bandwidth expansion of the master die. The technical solutions of the present application will be described below through specific embodiments.

[0022] Referring to Figure 1 , a structure schematic diagram of a chip system provided by the embodiments of the present application is shown, which includes a GPU master die 100 and at least one IO expansion slave die 200; the master die 100 is provided with a first interface unit 101, a second interface unit 102 and a processing unit; the first interface unit 101 is connected with a first storage unit 110, and the second interface unit 102 is connected with the slave die 200; the slave die 200 is provided with a third interface unit 201, a fourth interface unit 202 and a slave die bus 203; the third interface unit 201 is connected with the second interface unit 102, and the fourth interface unit 202 is respectively connected with a second storage unit 210; the number of the fourth interface unit 202 in the same slave die 200 is greater than the number of the third interface unit 201, so the total number of the fourth interface unit 202 in the whole chip system is greater than the total number of the third interface unit 201; the slave die bus 203 of the slave die 200 is in communication connection with the third interface unit 201 and the fourth interface unit 202; the processing unit is configured to interact with the first storage unit 110 and the second storage unit 210, for example, the processing unit can read and calculate data from one or more of the first storage unit 110 and the second storage unit 210 according to the request of a CPU device or other GPU device, and also can write data; wherein the bandwidth between each interconnected second interface unit 102 and third interface unit 201 is higher than the bandwidth between each interconnected first interface unit 101 and first storage unit 110, and the bandwidth of each slave die bus 203 is higher than the product of the bandwidth between each interconnected first interface unit 101 and first storage unit 110 and the number of the third interface unit 201 on the slave die 200.

[0023] It should be noted that the number of each element in the figure is only exemplary, and the specific data of each element is determined according to the actual situation, and the embodiment of the present application does not limit this.

[0024] The GPU main die 100 can be a die for packaging as a GPU chip, and the IO expansion slave die 200 can be a die for connecting with the main die 100 based on an IO interface and can perform data interaction. The main die 100 is provided with a first interface unit 101 for connecting with a first storage unit 110. In the embodiment of the present application, the first storage unit 110 can be arranged on the main die 100, that is, directly packaged in the same chip, or arranged outside the main die 100 and connected with the first interface unit 101 through a slot, a wire or the like for data interaction. The second interface unit 102 in the main die 100 is adapted to the third interface unit 201 in the slave die 200 and can perform data transmission. The fourth interface unit 202 for connecting with a second storage unit 210 is further arranged in the slave die 200, and the second storage unit 210 can be arranged on the slave die 200 or arranged outside the slave die 200 and connected with the second interface unit 102 through a connection unit. The third interface unit 201 is connected with the second storage unit 210 through a bus (hereinafter referred to as slave die bus 203) in the slave die 200 and the fourth interface unit 202. The main die 100 is provided with a processing unit (for example, a GPU core, not shown in the figure), and the processing unit can perform data interaction with the first storage unit 110 through a bus (hereinafter referred to as main die bus, not shown in the figure) in the main die 100, the first interface unit 101; the processing unit performs data interaction with the second storage unit 210 through the main die bus, the second interface unit 102, the third interface unit 201, the slave die bus 203 and the fourth interface unit 202. The processing unit can include a video memory controller in addition to the calculation core, and the video memory controller can control the first storage unit, the second storage unit and the calculation core or CPU, other GPU chips to perform data interaction.

[0025] The first interface unit 101 and the second interface unit 102 are both IO interfaces. Due to technical process limitations, the number of IO interfaces of a single master die 100 is also limited. The bandwidth of the master die 100 is related to the total bandwidth between the storage units connected to (including direct connection and indirect connection) the master die 100. In some related technologies, only the first interface unit 101 is arranged in the master die 100, and each first interface unit 101 is directly connected to the first storage unit 110. However, this scheme either causes a significant increase in the size of the GPU chip, a complex manufacturing process, and high cost, or requires close and dense wiring around the master die of the GPU, which often has a small circumference, causing congestion of the PCB board. In the embodiments of the present application, the second interface unit 102 is an IO interface for bandwidth expansion, and the first interface unit 101 is an IO interface for directly connecting to the storage unit. The second interface unit 102 is connected to the slave die 200. Since the bandwidth (such as BW1 in Figure 1 the figure) between each connected second interface unit 102 and third interface unit 201 is higher than the direct connection bandwidth (such as BW2 in Figure 1 the figure) between each first interface unit 101 of the master die 100 and the first storage unit 110, the bandwidth expansion of the master die bus is realized. The bandwidth (such as BW3 in Figure 1 the figure) of each slave die bus 203 is higher than the product of the bandwidth (BW2) between each connected first interface unit 101 and the first storage unit 110 and the number of third interface units 201 on the slave die 200. If the third interface unit 201 and the second interface unit 102 are one-to-one corresponding, the bandwidth of the slave die bus 203 is also higher than the total bandwidth when the second interface unit 102 is directly connected to the storage unit, and the effective bandwidth of the overall chip system is expanded. By replacing the direct connection between the master die and the storage unit with the interconnection between the master die and the slave die, BW1>BW2 is realized. Therefore, for the same number of third interface units 201 and first storage units 110, the bandwidth between the processing unit and the third interface unit 201 is greater than the bandwidth between the processing unit and the first storage unit 110.

[0026] In the case where the number of IO interfaces of the master die 100 remains unchanged, that is, without breaking the technical process limitation on the number of IO interfaces, by connecting a certain number of second interface units 102 to the slave die 200, compared with directly connecting an equal number of first interface units 101 to the first storage unit 110, the slave die 200 can connect more storage units, and the total bandwidth of the master die 100 is expanded.

[0027] For example, the total number of IO interfaces of the main die 100 is 20. (1) If all the 20 IO interfaces are the first interface unit 101, the main die 100 is directly connected with all the first storage units 110, and the total bandwidth S1 of the main die 100 is BW2*20; (2) if 8 of the 20 interfaces are the first interface unit 101, 12 are the second interface unit 102 and are connected with the third interface unit 201, the number of the slave dies 200 is P1, the number of the third interface unit 201 of each slave die 200 is fixed as P2, and the total number of the fourth interface unit 202 is P3, P3 is greater than the total number of the third interface unit 201, P1*P2=12, and BW1>BW2; then the total bandwidth S2 of the main die is BW1*12+BW2*8>BW2*12+BW2*8=S1, that is, S2>S1; the total bandwidth of each slave die bus 203 and the sum of the bandwidths of each first storage unit 110 and the main die 100 is S3=BW2*8+BW3*P1>BW2*8+BW2*P1>BW2*8+BW3*12 / P2, since the bandwidth of each slave die bus 203 is greater than the product of the bandwidth between the first interface unit 101 and the first storage unit 110 and the number of the third interface unit 201 on the slave die 200, BW3>BW2*P2, and BW3*P1>BW2*P2*P1. Therefore, S3>BW2*8+(BW2*P2)*12 / P2=BW2*8+BW2*12=S1, S3>S1. As can be seen, the interconnection bandwidth between the main die 100 and the slave die 200 and the sum of the bandwidths of the slave die bus 203 are both greater than the sum of the bandwidths of the same number of first storage units 110 corresponding to the same number of first interface units 101 directly connected with the first storage units 110 in the related art. Therefore, the effective bandwidth expansion of the main die 100 can be realized by the embodiment of the present application, and the effective performance of the main die 100 is improved.

[0028] In the embodiment of the present application, the number of the fourth interface unit 202 of the slave die 200 can be set to be greater than the number of the third interface unit 201, and in the case that the capacity of the first storage unit 110 and the capacity of the second storage unit 210 are unchanged, the total number of the storage units that can interact with the processing unit (the total number of the storage units is the sum of the number of the first storage unit 110 and the number of the second storage unit 210) increases, and the sum of the capacities is greater, that is, the capacity of the storage unit that can interact with the GPU main die 100 is expanded, and the video memory expansion is realized.

[0029] It can be understood that in actual application, the first memory controller (not shown in the figure) can be arranged in the main crystal grain 100, and the first memory controller controls the first memory unit 110 to interact with the processing unit, CPU or other GPU chips through the first interface unit 101, the main crystal grain bus. The second memory controller (not shown in the figure) can be arranged in the slave crystal grain 200, and the second memory controller controls the second memory unit 210 to interact with the third interface unit 201 through the fourth interface unit 202 and the slave crystal grain bus 203, thereby realizing data interaction between the second memory unit 210 and the processing unit, CPU or other GPU chips.

[0030] The main crystal grain 100 and the slave crystal grain 200 can be packaged together or separately. Preferably, the corresponding chips of the main crystal grain 100 and the slave crystal grain 200 (for example, packaged separately, the main crystal grain 100 is packaged into a main chip, and the slave crystal grain 200 is packaged into a slave chip) are assembled into the same PCB (Printed Circuit Board, printed circuit board) board for connection. In order to reduce the time delay of the main crystal grain 100 and the memory unit directly, the length of the wire between the main crystal grain 100 and each memory unit needs to be as short as possible. However, since each IO interface unit has hundreds of connection lines, if all the IO interfaces are directly connected to the memory unit according to the related art, the edge wire of the main crystal grain 100 will be very dense, which will cause the PCB wiring to be very difficult, and the PCB board wiring will be blocked. By arranging part of the IO interfaces of the main crystal grain 100 as the second interface unit 102 connected to the slave crystal grain 200, and connecting the second memory unit 210 through the slave crystal grain 200, the wiring difficulty near the IO interface of the main crystal grain 100 in the PCB board is reduced, and the blocking of the PCB wiring is reduced. The main crystal grain and the slave crystal grain are packaged separately, which is conducive to the user to flexibly configure the corresponding number of slave crystal grains and PCB boards according to the needs in actual application.

[0031] In the embodiment of the present application, by setting the second interface unit 102 in the GPU main die 100 to adapt to the third interface unit 201 in the IO expansion slave die 200, part of the IO interface units in the main die 100 are set as the first interface unit 101 connected with the first storage unit 110, and the slave die 200 is provided with the fourth interface unit 202 connected with the second storage unit 210, thereby providing an IO interface setting strategy for the GPU main die 100. Due to the bandwidth between each interconnected second interface unit 102 and third interface unit 201, which is higher than the bandwidth between each interconnected first interface unit 101 and first storage unit 110, and the bandwidth of each slave die bus 203 is higher than the product of the bandwidth between each interconnected first interface unit 101 and first storage unit 110 and the number of the third interface unit 201 on the slave die 200, so that the sum of the interconnection bandwidth between the same number of third interface units 201 and second interface units 102 is higher than the sum of the direct connection bandwidth between the first interface unit 101 and the first storage unit 110, the data interaction between the processing unit of the main die 100 and the first storage unit 110 and the second storage unit 210 can be realized, and the memory bandwidth expansion of the main die 100 is realized through the IO interface setting strategy, the memory bandwidth is improved, and the effective performance of the main die 100 is improved. At the same time, by setting the number of fourth interface units 202 to be greater than the number of third interface units 201, the sum of the capacities of the storage units connected with the main die 100 can be realized through the slave die 200, and the video memory capacity expansion is realized.

[0032] In some implementations of the embodiment of the present application, the first storage unit 110 and the second storage unit 210 are the same type of memory.

[0033] The main die 100 is provided with a main die bus, the processing unit performs data interaction with the first storage unit 110 through the main die bus and the first interface unit 101, and the processing unit performs data interaction with the second storage unit 210 through the second interface unit 102, the third interface unit 201, the slave die bus 203 and the fourth interface unit 202, that is, the processing unit performs data interaction with the first storage unit 110 and the second storage unit 210. By setting the same type of storage unit, the consistency of the processing unit when performing data processing can be improved, thereby improving the stability of data processing. In the case of large model inference and other scenarios requiring a large amount of data parallel processing, it brings users a more stable and consistent experience, and is also conducive to interleaving.

[0034] In some implementations of the embodiment of the present application, the second interface unit 102 and the third interface unit 201 are SERDES-based ports; and the type of the memory is DDR or LPDDR.

[0035] SERializer / DESerializer (SERDES) is a technology for converting low-speed parallel data into high-speed serial signals for transmission. For the case of chip physical unit size, the DDR (Double Data Rate Synchronous Dynamic Random-Access Memory) bandwidth is small, and the SERDES bandwidth is large, that is, the unit length of SERDES brings greater bandwidth benefits. For the die / chip, the interface is usually set at the edge position, so the effective bandwidth benefit brought by the unit perimeter of SERDES is greater. The same bandwidth target SERDES can make the required die / chip side length smaller, thereby making the required die / chip area smaller and the cost lower, and it is easy to apply the die / chip to various high-integration modules and devices.

[0036] The core of the embodiments of the present application is to realize bandwidth expansion by connecting the master die 100 from the slave die 200, rather than capacity expansion. Therefore, under the same bandwidth target, it is not necessary to connect the IO interface (such as the first interface unit 101) with the HBM (High Bandwidth Memory) or GDDR (Graphics Double Data Rate Synchronous Dynamic Random-Access Memory) type of memory, but only needs to connect the DDR or LPDDR (Low Power Double Data Rate SDRAM) type of memory as the first storage unit 110 and the second storage unit 210, thereby reducing the cost of the chip system under the same bandwidth target. Since the HBM and GPDDR are assembled to the interposer or high-layer PCB, by using the DDR or LPDDR type of memory to replace the HBM and GPDDR type of memory, the number of PCB layers can be reduced, the overall heat dissipation of the chip system and the PCB is reduced, the influence of high temperature on the work of the chip system is reduced, the work stability of the master die 100 and the slave die 200 is improved, and the heat dissipation problem of other elements assembled on the same PCB board is also reduced.

[0037] In some implementations of the embodiments of the present application, the second interface unit 102 and the third interface unit 201 are connected through SERDES-based UCIE, and the slave chip 200 further comprises at least one cache unit 204; the cache unit 204 is arranged between the third interface unit 201 and the fourth interface unit 202; the cache unit 204 is configured to cache target data of each second storage unit 210 connected to the cache unit 204; the target data comprises at least one of recently most frequently used data and preset designated data.

[0038] The second interface unit 102 and the third interface unit 201 can be connected based on PCIE or UCIE. Preferably, they are connected based on SERDES-based UCIE (Universal Chiplet Interconnect Express), which is compatible with chips of different processes in the physical layer dimension, ensures stable and efficient data interaction between chips of different processes, and is suitable for interconnection between chips. For example, the master chip 100 adopts a 5nm process, and the slave chip 200 adopts a 12nm process. Even if the two have different process technologies, the second interface unit 102 and the third interface unit 201 can still realize efficient and stable data interaction.

[0039] Referring to Figure 2 , a schematic diagram of the architecture of the slave chip 200 is shown. In the slave chip 200, the cache unit 204 is arranged to store part of the data in the second storage unit 210 as target data in the cache unit 204, and the third interface unit 201 can obtain the target data by accessing the cache unit 204. The cache unit is arranged in the slave chip 200, the capacity of the cache unit 204 is smaller than that of the second storage unit 210, and the physical distance between the cache unit 204 and the third interface unit 201 is smaller than that between the fourth interface unit 202 and the third interface unit 201. Therefore, the cache unit 204 has a higher response speed, so that the target data is obtained from the cache unit 204 with a lower time delay than from the second storage unit 210, the time delay of the processing unit accessing the corresponding data of the slave chip 200 is reduced, the time delay between accessing the data of the first storage unit 110 and the second storage unit 210 is balanced, and the stability and consistency of the chip system in processing data are further improved.

[0040] Specifically, assume that the time delay of the processing unit accessing the first storage unit 110 is t1. Since the master die 100 and the slave die 200 are connected through the second interface unit 102 and the third interface unit 201, there is a certain time delay t2. The time delay of the third interface unit 201 accessing the second storage unit 210 is t3, and obviously t3+t2>t1. The slave die 200 is provided with a cache unit 204, and assume that the time delay of the third interface unit 201 accessing the cache unit 204 is t4. The cache unit 204 is of a type different from the storage unit, and has a higher response speed, so t4<t3. When the processing unit needs to access data of the second storage unit, if the data is stored in the cache unit 204, the original access time delay can be reduced from t2+t3 to t2+t4, and in practice, t2+t4 can be approximately equal to t1, thus reducing the time delay of the processing unit accessing data of the slave die 200, and balancing the time delay of the processing unit accessing data of the first storage unit 110 and the second storage unit 210.

[0041] For the purpose of reducing the time delay, the data in the cache unit 204 can be at least one of the most recently used data and preset specified data. The specified data can be data determined according to an actual scenario, for example, data that needs to be accessed periodically, N data that are frequently accessed recently, etc., which can be determined based on access frequency, predetermined settings, data corresponding to a predetermined service type, etc.

[0042] Referring to Figure 3 , another architecture of the slave die 200 is shown. In some implementations of the embodiments of the present application, the third interface unit 201 in the slave die 200 is multiple, one third interface unit 201 is connected with one cache unit 204; one cache unit 204 is connected with each fourth interface unit 202, and the capacity of the cache unit 204 is less than 1 / 1000 of the capacity of the second storage unit 210.

[0043] Each third interface unit 201 in the slave die 200 is connected with one cache unit 204, and each cache unit 204 is connected with each fourth interface unit 202, so that each cache unit 204 can access the address range covering each second storage unit 210 in the same die, and at the same time, the time delay between the third interface unit 201 and the cache unit 204 can be reduced as much as possible.

[0044] If the number of cache units 204 is less than the number of third interface units 201, then the cache units 204 and the third interface units 201 need to be fully interconnected. This will increase the latency between the third interface units 201 and the cache units 204. Therefore, by setting the number of cache units 204 according to the number of third interface units 201 and connecting one third interface unit 201 to one cache unit 204, the latency between the third interface units 201 and the cache units 204 can be reduced as much as possible.

[0045] By setting the capacity of cache unit 204 to be less than 1 / 1000 of the capacity of second storage unit 210, the excessively large capacity of cache unit 204 is avoided, which would increase the manufacturing cost and size of die 200. The capacity of cache unit 204 can also be set to be greater than 1 / 10000 of the capacity of second storage unit 210 to improve the cache hit rate. In an illustrative embodiment, the capacity of a single second storage unit 210 is 64GB, and the capacity of a single cache unit 204 is 16MB.

[0046] In some implementations of the embodiments of this application, any die 200 is provided with a first inter-board interconnect interface 205 unit.

[0047] like Figure 1 , Figure 2 , Figure 3 As shown, a first inter-board interconnect interface 205 is provided in the slave die 200. It can be seen that in specific applications, the slave die 200 and the master die 100 can be packaged and assembled on a PCB. Each PCB can achieve interconnection between multiple chip systems through the first inter-board interconnect interface 205 and other necessary circuits. By placing the first inter-board interconnect interface 205 on the slave die 200 instead of entirely on the master die 100, the required area of ​​the master die 100 can be reduced, or the routing layout difficulty of the master die 100 can be reduced under the limited area of ​​the master die 100. Simultaneously providing the first inter-board interconnect interface 205 on both the master and slave dies for inter-board interconnection enables flexible interconnection with other chip systems and multi-board interconnection.

[0048] In some implementations of the embodiments of this application, some of the first storage units 110 are disposed outside the master die 100; for each slave die 200, some of the second storage units 210 are located outside the slave die 200.

[0049] In the packaging of the crystal grains, the crystal grains need to be arranged on the substrate and then packaged, for example, the main chip can be packaged based on the main crystal grain 100 and the first substrate, and the slave chip can be packaged based on the slave crystal grain 200 and the second substrate. Part of the first storage unit 110 is arranged outside the main crystal grain 100, and another part of the first storage unit 110 is located inside the main crystal grain 100 and is packaged in the main chip with the main crystal grain 100; part of the second storage unit 210 is arranged outside the slave crystal grain 200, and another part of the second storage unit 210 is located inside the slave crystal grain 200 and is packaged in the slave chip with the slave crystal grain 200. In a specific implementation, the main chip and the first storage unit 110 arranged outside the main crystal grain 100 are assembled on the PCB, and are connected in a plug-in, wired or other manner through a PHY (Physical, physical layer) interface; the slave chip and the second storage unit 210 arranged outside the slave crystal grain 200 are assembled on the PCB, and are connected in a plug-in, wired or other manner through a PHY interface.

[0050] By packaging part of the storage unit with the main crystal grain 100 or the slave crystal grain 200 in the same chip, the area of the substrate can be effectively utilized, and because part of the second storage unit 210 is packaged with the slave crystal grain 200 in the slave chip, the data interaction delay between the main chip and the slave chip can be reduced, and the delay balance between the processing unit and the first storage unit 110 and the second storage unit 210 can be further improved. At the same time, the storage unit arranged outside the main crystal grain 100 and the slave crystal grain 200 is more convenient to repair and replace than the storage unit packaged in the main chip and the slave chip, can improve fault redundancy, and at the same time reduce the difficulty and requirement of packaging.

[0051] In some implementations of the embodiments of the present application, the main crystal grain includes a routing unit, the routing unit is connected with the first interface unit 101 and the second interface unit 102 through the main crystal grain bus, and the routing unit is configured to determine the memory interleaving address corresponding to the first storage unit 110 and the second storage unit 210 based on the total number of storage units, and to interact with the first storage unit 110 and / or the second storage unit 210 based on the memory interleaving address. The processing unit usually uses the low bits of the physical address of the storage unit to directly select the storage unit, and if the storage unit is selected without mapping the storage unit address, it will often be necessary to continuously access the same storage unit. If an access conflict occurs at this time, the selected storage unit can only be processed in series, while other storage units are in an idle state, resulting in ineffective utilization of the bandwidth of the system.

[0052] Address interleaving refers to changing the mapping of memory addresses to storage units. Through address interleaving, uniform access can be achieved, the bandwidth of the system can be effectively utilized, and the storage units can be kept in a relatively balanced working state as much as possible.

[0053] The routing unit can use a pre-set interleaving algorithm to perform interleaving transformation according to determined interleaving parameters based on the corresponding original address in the received data request to obtain a memory interleaved address. The response to the data request can be executed by accessing the memory interleaved address. The interleaving parameters may include the number of storage units corresponding to the interleaving transformation.

[0054] As an example, interleaving algorithms can determine the target number of bits for the interleaving address based on the maximum capacity of a single memory cell and the number of memory cells. The target number of bits must represent both the maximum supported capacity of a single memory cell and the number of memory cells. Both the original address of the memory cell and the length of the memory interleaving address are part of the target number of bits.

[0055] As an example, the target number of bits can be the sum of the maximum capacity of a single storage unit (logarithm of 2, rounded up) and the number of storage units (logarithm of 2, rounded up). That is, the target number of bits = log2(C) + log2(N) N is the number of storage units, and C is the capacity of the largest storage unit. This method determines the target number of bits so that the target number of bits can meet the interleaving requirements without increasing the computational complexity due to too many bits.

[0056] The routing unit determines the number of bits of source information to be retained in the original address according to the preset interleaving granularity, that is, the smallest data unit allocated to different storage units (for example, the interleaving granularity is 128B, that is, each 128B of data is allocated to different storage units), divides the original address of the storage unit into parts as the source information to be calculated and the source information to be retained; and divides the source information to be calculated into the first part and the second part.

[0057] Then, the hash bit length is determined according to the number of storage units. For example, the length of the first part is determined according to the formula. log2(N) The result is calculated and rounded up, where N is the number of storage units. For example, if N is 6, then the length of the first part can be 3 bits.

[0058] The computational source information is divided into summation groups according to the number of storage units. The number of bits in each group can be determined according to the formula. log2(N) The result is calculated and rounded up, where N is the number of storage units. If the number of bits in each group is less than... log2(N) , will cause the data originally assigned to different storage units to be assigned to the same data; if the number of bits in each group is too large, the calculation pressure on the hardware circuit will be large, and the cost will be increased. Exemplarily, the original address can be divided into calculation source information and reserved source information (the calculation source information is used to determine the target storage unit, and the reserved source information is used to represent a part of the access address of the target storage unit). Then the calculation source information is grouped in a predetermined manner, and each group includes a plurality of bits. The memory interleaving address includes a target storage unit identifier, and the target storage unit identifier is used to represent the storage unit corresponding to the memory interleaving address. The target storage unit identifier in the memory interleaving address includes a plurality of bits, and the number of bits in each group is the same as the number of bits of the target storage unit identifier. The bits in each group are converted into a numerical value, and then the converted numerical value is summed to obtain a characteristic value. The characteristic value is subjected to a modulo operation according to the interleaving number (the number of storage units corresponding to the interleaving transformation) to obtain a hash value, and the hash value is used as the target storage unit identifier. For example: after determining the calculation source information from the original address, the calculation source information is grouped to obtain four groups,

[0111] ,

[1111] ,

[0011] , and

[1011] . Each group is converted into a numerical value, for example in decimal, to obtain the converted numerical value of each group as 7, 15, 3, and 11, respectively. The sum of the numerical values of all groups is 36. The number of corresponding storage units is 16, and the hash value obtained by taking 36 modulo 16 is 10. Therefore, the target storage unit identifier is 10 (the target storage unit identifier in the memory interleaving address is 4 bits, that is,

[1010] ). Specifically, the hash value can be calculated according to the original address, and the hash value is subjected to a modulo operation according to the number of storage units corresponding to the interleaving transformation to obtain the target storage unit identifier. For example: the number of storage units corresponding to the interleaving transformation is 7, and the target storage unit identifier is one of 0~6.

[0059] The computation source information can be further divided into a first part and a second part, with the number of bits in the first part matching the number of bits in each group of feature value calculations. The second part of the computation bits can participate in both feature value grouping calculations and be passed through to the memory interleaving address. Based on the feature value, the second part of the computation source information, and the retained source information, the memory interleaving address is obtained, enabling the chip system to efficiently and reliably generate the target address, avoiding information loss or conflict between the chip system's access request address and the target address. By retaining the source information and not participating in the calculation, continuous data can be located in different memory units as much as possible, while data stored in the same memory unit can be kept as continuous as possible. The second part of the computation bits participates in the chip system's feature value grouping calculations and also serves as part of the memory interleaving address, adapting to various large-step access modes. Even with high-bit flips, access can be balanced, significantly better than using a power of two for direct interleaving of low bits, which can lead to frequent access to the same memory unit when high bits flip and low bits remain unchanged.

[0060] With all third interface units 201 and fourth interface units 202 of the slave die 200 fully interconnected (each third interface unit 201 can be connected to any fourth interface unit 202 in the same slave die 200), and each fourth interface unit 202 connected to a second storage unit 210, channel interleaving can be performed on each group of interconnected second interface units 102 and third interface units 201 to obtain the channel interleaving address. The characteristic of channel interleaving is that the number of channels is used as the modulus value when calculating the modulus, while other aspects can be consistent with memory interleaving, reducing the algorithm difficulty. The target channel for data transmission is determined based on the modulus result and the channel number. The channel can be a communication link between a routing unit and a storage unit. The routing unit may have two or more communication links for the same storage unit. For example, if there are two third interface units 201 in the slave die, the master die can interact with the same second storage unit 210 through any one of the two different third interface units 201. In this case, the communication links between the two third interface units 201 and the same second storage unit 210 correspond to different channels.

[0061] The routing unit can achieve balanced access to various storage units based on memory interleaving addresses and target channels, and can also achieve balanced access to different third interface units 201 within the same slave die 200, thereby improving the bandwidth balance of storage unit access. For example, as shown... Figure 3As shown, when the target storage unit is the second storage unit 210 on the chip, there are two interconnect channels (each consisting of a second interface unit 102 and a third interface unit 201 connected to each other), and the four second storage units 210 are fully interconnected. In this case, another channel interleaving with a modulus of 2 can be performed to determine the target channel for accessing a specific second storage unit 210. If there are a large number of access requests, channel interleaving will evenly distribute these requests across the two channels, improving bandwidth utilization and load balancing. It also increases access speed compared to accessing a single channel.

[0062] For example, the routing unit can perform local interleaving based on the type and information of the memory access request. For instance, if the type is a normal service, such as a write data service, global interleaving is performed. If the type is a latency-sensitive service, such as accessing a specific page table, interleaving can be performed only between the first storage units 110 to avoid accessing the second storage unit 210, thus improving the response time for that specific service type. If the memory access request information indicates that the request has a specific destination, such as reading relevant data from a specific storage unit of a specific slave chip, then since the target storage unit has already been determined, memory address interleaving is unnecessary; the target channel is determined only by interleaving the memory access channels based on the interconnection configuration and the number of channels.

[0063] Reference Figure 4 The diagram shows a schematic of a board architecture provided in an embodiment of this application. The board includes a carrier 300 and a chip system as described in the first aspect above. The master die 100 in the chip system is located in the master chip, and the slave die 200 in the chip system is located in the slave chip. The master chip is connected to each slave chip through the carrier 300. By packaging the main die 100 into a main chip and the slave die 200 into a slave chip, and assembling the main chip and one or more slave chips on the same carrier 300 (e.g., PCB), the chip system can be deployed on the same carrier 300 to form a board. At the same time, the memory cells located outside the main die 100 and the slave die 200 are also assembled on the carrier 300, so that the main chip is connected to the first memory cell 110 located outside the main die 100, and the slave chip is connected to the second memory cell 210 located outside the slave die 200.

[0064] By setting up multiple slave chips, bandwidth can be extended more effectively, reducing the high cost of using HBM / GDDR. At the same time, considering flexibility and manufacturing costs, the master die 100 is packaged into a master chip and the slave die 200 is packaged into a slave chip and then assembled on the same carrier 300, so that the master chip can flexibly connect the corresponding number of slave chips to form a chip system according to actual needs.

[0065] In practical applications, the initial shape of the carrier 300 is rectangular. The main chip includes two sets of first interface units 101 and two sets of second interface units 102. Each second interface unit 102 is connected to the third interface unit 201 of the slave chip.

[0066] like Figure 4 As shown, to facilitate carrier wiring, the two sets of first interface units 101 are arranged opposite each other. Figure 4 One set each at the top and bottom), the two sets of second interface units 102 are arranged opposite each other ( Figure 4 (One set each on the left and right sides) ensures that the wire connection lengths from each slave chip to the master chip are basically the same, balancing the latency between the master chip and each slave chip.

[0067] This application embodiment also provides a parallel processing device, including: a device body and the above-mentioned board; the board is provided with a first inter-board interconnection interface 205 unit, and the device body is provided with a plurality of second inter-board interconnection interfaces adapted to the first inter-board interconnection interface 205; the device body is configured to call at least two boards in parallel when connected to multiple boards.

[0068] Parallel processing devices can be electronic devices with computing power, such as computing cards, all-in-one computers, and servers. The main body of the device refers to the circuit components located within the parallel device. The main body of the device is provided with multiple second inter-board interfaces adapted to the first inter-board interconnect interface 205, enabling the main body of the device to connect to one or more (two or more) boards. When the main body of the device is connected to multiple boards, it can simultaneously utilize at least two boards, thereby utilizing two or more chip systems for data processing, thus improving the data processing performance of the parallel processing device.

[0069] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing embodiments, and will not be repeated here.

[0070] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0071] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0072] In the embodiments provided in this application, it should be understood that the disclosed device / chip system and method can be implemented in other ways. For example, the device / chip system embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0073] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0074] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A chip system, characterized by Includes the GPU main die and at least one I / O extension slave die; The master die is provided with a first interface unit, a second interface unit, and a processing unit; the first interface unit is connected to a first memory unit, and the second interface unit is connected to the slave die. The slave die is provided with a third interface unit, a fourth interface unit, and a slave die bus; the third interface unit is connected to the second interface unit, the fourth interface unit is connected to the second memory unit, the number of fourth interface units is greater than the number of third interface units, and the slave die bus is communicatively connected to the third interface unit and the fourth interface unit; The processing unit is configured to interact with the first storage unit and the second storage unit; The bandwidth between each interconnected second interface unit and third interface unit is higher than the bandwidth between each interconnected first interface unit and first storage unit, and the bandwidth of each slave bus is higher than the product of the bandwidth between each interconnected first interface unit and first storage unit and the number of third interface units on the slave.

2. The system of claim 1, wherein, The first storage unit and the second storage unit are of the same type of memory.

3. The system of claim 2, wherein, The second interface unit and the third interface unit are ports based on serializer and deserializer technology; the memory type is DDR or LPDDR.

4. The system of claim 3, wherein, The second interface unit and the third interface unit are connected via SERDES-based UCIE. The slave die also includes at least one cache unit. The cache unit is disposed between the third interface and the fourth interface unit. The cache unit is configured to cache target data of each of the second storage units connected to the cache unit. The target data includes at least one of the most recently used data and preset specified data.

5. The system of claim 4, wherein, There are multiple third interface units in the die, one of which is connected to one of the cache units; one cache unit is connected to each of the fourth interface units, and the capacity of the cache unit is less than 1 / 1000 of the capacity of the second storage unit.

6. The system of claim 1, wherein, Each of the aforementioned chips is provided with a first inter-board interconnection interface unit.

7. The system of claim 1, wherein, A portion of the first memory cells are located outside the master die; for each slave die, a portion of the second memory cells are located outside the slave die.

8. The system of claim 1, wherein, The master die is provided with a master die bus; the master die includes a routing unit, which is connected to a first interface unit and a second interface unit through the master die bus. The routing unit is configured to determine the memory interleaving address corresponding to the first storage unit and the second storage unit based on the total number of storage units, and to perform data interaction with the first storage unit and / or the second storage unit based on the memory interleaving address.

9. A board card characterized by, The board includes a carrier and a chip system as described in any one of claims 1-8; the master die in the chip system is located on the master chip, and the slave die in the chip system is located on the slave chip; the master chip is connected to each of the slave chips through the carrier.

10. A parallel processing device, characterized by include: The device main body and the board card as claimed in claim 9 are provided with a first inter-board interconnection interface unit, and the device main body is provided with a plurality of second inter-board interconnection interfaces matched with the first inter-board interconnection interface; and the device main body is configured to call at least two board cards in parallel when connected with a plurality of board cards.

Images