Wafer-level chip mapping method and device and wafer-level chip
Patent Information
- Application Number
- CN202610979309.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-07-02
- Publication Date
- 2026-10-09
AI Technical Summary
[0004]然而，上述映射方式只考虑了算子的芯粒资源分配，采用简单的顺序分配进行张量切分和任务分发，无法适应实际的芯片任务执行过程，影响芯片的存储和通信性能
[0043]上述晶圆级芯片的映射方法、装置及晶圆级芯片，获取目标计算任务的混合并行策略以及运行目标计算任务的晶圆级芯片阵列；混合并行策略包括数据并行模式、全分布式并行模式、流式并行模式、张量并行模式和时间对齐张量并行模式中至少两种模式的组合，以及各模式的并行规模；根据混合并行策略和预设对应关系确定张量分区向量；张量分区向量用于表征按照混合并行策略对输入张量和权重张量进行切分的维度；根据晶圆级芯片阵列对应的空间编码向量和张量分区向量，将混合并行策略映射到晶圆级芯片阵列上，得到映射方案。基于晶圆级芯片的二维坐标，引入并行策略的维度向量，保证支持不同的并行维度。并使用坐标向量来表征不同的并行策略，从而保证并行策略的可以部署在不同位置的芯片上，构建出巨大的映射空间，支持探索在不同训练场景下，不同并行策略在晶圆上的不同的优先级。上述方法构建了包含最优映射策略的晶圆级映射空间，能够有效挖掘晶圆级芯片的大模型训练性能。
Smart Images
Figure CN122884663A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of chip technology, and in particular to a wafer-level chip mapping method, apparatus, and wafer-level chip. Background Technology
[0002] Wafer-level chip (WLS) technology achieves high-density integration of computing and memory units on a single wafer through advanced packaging, providing a new high-bandwidth and high-energy-efficiency alternative to traditional packaging expansion methods. One technical path to achieving WLS is monolithic integration, where all neural processing units (NPUs) and interconnects are manufactured directly on the wafer substrate. Another path is to integrate NPU chiplets and DRAM chiplets, where NPU chips are manufactured separately and then bonded to wafer-level interconnects. This WLS integrates 32 sub-chips and their corresponding high-bandwidth off-chip memory, forming a two-dimensional mesh interconnect network. The sub-chips have high interconnect bandwidth. The two-dimensional mesh network was chosen for its high integration density, and because the short interconnect distances between sub-chips within the wafer prevent the support of long-distance connection topologies (such as rings, fully connected topologies, etc.). Furthermore, the high bandwidth of WLS benefits from the short signal transmission distances between chips, thus improving signal integrity and supporting denser micro-bump interconnect layouts. Therefore, wafer-level chips can achieve inter-chip bandwidth of up to 4TBps, more than twice that of GB200 (1.8TBps). Each sub-chip within the wafer integrates computing and storage functions. Each sub-chip contains multiple computing tiles, on-chip memory buffers, and on-chip network interconnects. The on-chip interconnect network also adopts a two-dimensional mesh structure design, which can support high-speed inter-tile communication.
[0003] Currently, wafer-level chip mapping mechanisms are divided into multi-operator tensor partitioning mapping schemes and Megatron's hybrid parallel mapping strategy. The multi-operator tensor partitioning mapping scheme maps and allocates hardware components sequentially according to the order of the operators. The hardware allocation for each operator involves partitioning the output tensor and then selecting the corresponding core group to execute the operator. The partitioned output tensor is assigned to the corresponding cores within the core group according to its ID, and each core is then responsible for its assigned computational task. Megatron's hybrid parallel mapping strategy observes the communication traffic of existing large model training on GPU clusters, indicating that tensor parallelism currently has the highest traffic, followed by data parallelism, and finally pipeline parallelism. Tensor parallelism is deployed in the high-bandwidth domain of the GPU, while data parallelism and pipeline parallelism are deployed between multiple GPU nodes. These nodes are interconnected via InfiniBand or RDMA, resulting in lower bandwidth compared to NVLink.
[0004] However, the above mapping method only considers the allocation of chip resources for operators and uses simple sequential allocation for tensor partitioning and task distribution, which cannot adapt to the actual chip task execution process and affects the chip's storage and communication performance. Summary of the Invention
[0005] Therefore, it is necessary to provide a wafer-level chip mapping method, apparatus, and wafer-level chip that ensures complete adaptation of the parallel strategy to chips in different locations, in order to address the above-mentioned technical problems.
[0006] In a first aspect, this application provides a wafer-level chip mapping method, comprising:
[0007] Obtain the hybrid parallel strategy for the target computing task and the wafer-level chip array that runs the target computing task; the hybrid parallel strategy includes a combination of at least two of the following modes: data parallel mode, fully distributed parallel mode, streaming parallel mode, tensor parallel mode and time-aligned tensor parallel mode, as well as the parallel scale of each mode.
[0008] The tensor partition vector is determined based on the hybrid parallel strategy and the preset correspondence; the tensor partition vector is used to represent the dimension of the input tensor and weight tensor according to the hybrid parallel strategy.
[0009] Based on the spatial encoding vector and tensor partitioning vector corresponding to the wafer-level chip array, the hybrid parallel strategy is mapped onto the wafer-level chip array to obtain the mapping scheme.
[0010] In one embodiment, the dimension includes a combination of at least two of the following: batch dimension, sequence dimension, hidden dimension, and projection dimension.
[0011] In one embodiment, each parallel strategy has a unique mapping relationship with the dimension; determining the tensor partition vector based on the hybrid parallel strategy and the preset correspondence includes:
[0012] In the case of a hybrid parallel strategy including a data parallel mode, the input tensor is partitioned from the batch dimension to obtain a tensor partition vector.
[0013] In the case of a hybrid parallel strategy including a fully distributed parallel mode, the input tensor is partitioned from the batch dimension, and the weight tensor is partitioned from the hidden dimension or the projection dimension to obtain a tensor partition vector.
[0014] In the case of a hybrid parallel strategy including streaming parallel mode, the input tensor is segmented from the sequence dimension to obtain a tensor partition vector;
[0015] In the case of hybrid parallel strategies including tensor parallelism, the weight tensor is partitioned from the hidden dimension or the projection dimension to obtain the tensor partition vector.
[0016] In the case of a hybrid parallel strategy including time-aligned tensor parallelism, the input tensor is segmented from the sequence dimension and the weight tensor is segmented from the projection dimension to obtain a tensor partition vector.
[0017] In one embodiment, the method further includes:
[0018] Determine the tensor partitioning dimension index corresponding to each time step based on the mapping scheme;
[0019] At each time step, the target computation task is executed by calling the corresponding chip unit based on the tensor segmentation dimension index corresponding to each time step.
[0020] In one embodiment, determining the tensor partitioning dimension index corresponding to each time step according to the mapping scheme includes:
[0021] For time step t, obtain the task stage of the target computation task corresponding to time step t;
[0022] Based on the task phase and tensor partition vector, determine the representation vector of the target computation task;
[0023] Based on the mapping relationship between task representation and spatial encoding, the spatial encoding vector and representation vector are mapped to obtain the tensor segmentation dimension index.
[0024] In one embodiment, the method further includes:
[0025] Based on the wafer-level chip array, determine the coordinates of each chip unit;
[0026] The spatial encoding vector is determined based on the coordinates of each chip unit.
[0027] Secondly, this application also provides a wafer-level chip mapping apparatus, comprising:
[0028] The acquisition module is used to acquire the hybrid parallel strategy of the target computing task and the wafer-level chip array running the target computing task; the hybrid parallel strategy includes a combination of at least two of the following modes: data parallel mode, fully distributed parallel mode, streaming parallel mode, tensor parallel mode and time-aligned tensor parallel mode, as well as the parallel scale of each mode.
[0029] The determination module is used to determine the tensor partition vector according to the hybrid parallel strategy and the preset correspondence; the tensor partition vector is used to represent the dimension of the input tensor and weight tensor according to the hybrid parallel strategy.
[0030] The mapping module is used to map the hybrid parallel strategy onto the wafer-level chip array based on the spatial encoding vector and tensor partitioning vector corresponding to the wafer-level chip array, thereby obtaining the mapping scheme.
[0031] Thirdly, this application also provides a wafer-level chip, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:
[0032] Obtain the hybrid parallel strategy for the target computing task and the wafer-level chip array that runs the target computing task; the hybrid parallel strategy includes a combination of at least two of the following modes: data parallel mode, fully distributed parallel mode, streaming parallel mode, tensor parallel mode and time-aligned tensor parallel mode, as well as the parallel scale of each mode.
[0033] The tensor partition vector is determined based on the hybrid parallel strategy and the preset correspondence; the tensor partition vector is used to represent the dimension of the input tensor and weight tensor according to the hybrid parallel strategy.
[0034] Based on the spatial encoding vector and tensor partitioning vector corresponding to the wafer-level chip array, the hybrid parallel strategy is mapped onto the wafer-level chip array to obtain the mapping scheme.
[0035] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:
[0036] Obtain the hybrid parallel strategy for the target computing task and the wafer-level chip array that runs the target computing task; the hybrid parallel strategy includes a combination of at least two of the following modes: data parallel mode, fully distributed parallel mode, streaming parallel mode, tensor parallel mode and time-aligned tensor parallel mode, as well as the parallel scale of each mode.
[0037] The tensor partition vector is determined based on the hybrid parallel strategy and the preset correspondence; the tensor partition vector is used to represent the dimension of the input tensor and weight tensor according to the hybrid parallel strategy.
[0038] Based on the spatial encoding vector and tensor partitioning vector corresponding to the wafer-level chip array, the hybrid parallel strategy is mapped onto the wafer-level chip array to obtain the mapping scheme.
[0039] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:
[0040] Obtain the hybrid parallel strategy for the target computing task and the wafer-level chip array that runs the target computing task; the hybrid parallel strategy includes a combination of at least two of the following modes: data parallel mode, fully distributed parallel mode, streaming parallel mode, tensor parallel mode and time-aligned tensor parallel mode, as well as the parallel scale of each mode.
[0041] The tensor partition vector is determined based on the hybrid parallel strategy and the preset correspondence; the tensor partition vector is used to represent the dimension of the input tensor and weight tensor according to the hybrid parallel strategy.
[0042] Based on the spatial encoding vector and tensor partitioning vector corresponding to the wafer-level chip array, the hybrid parallel strategy is mapped onto the wafer-level chip array to obtain the mapping scheme.
[0043] The aforementioned wafer-level chip mapping method, apparatus, and wafer-level chip acquire a hybrid parallel strategy for the target computing task and a wafer-level chip array running the target computing task. The hybrid parallel strategy includes combinations of at least two modes from data parallelism, fully distributed parallelism, streaming parallelism, tensor parallelism, and time-aligned tensor parallelism, as well as the parallel scale of each mode. A tensor partitioning vector is determined based on the hybrid parallel strategy and a preset correspondence. The tensor partitioning vector represents the dimension by which the input tensor and weight tensor are partitioned according to the hybrid parallel strategy. Based on the spatial encoding vector and tensor partitioning vector corresponding to the wafer-level chip array, the hybrid parallel strategy is mapped onto the wafer-level chip array to obtain a mapping scheme. Based on the two-dimensional coordinates of the wafer-level chip, a dimension vector of the parallel strategy is introduced to ensure support for different parallel dimensions. Coordinate vectors are used to represent different parallel strategies, thereby ensuring that the parallel strategies can be deployed on chips at different locations, constructing a huge mapping space, and supporting the exploration of different priorities of different parallel strategies on the wafer under different training scenarios. The above method constructs a wafer-level mapping space containing the optimal mapping strategy, which can effectively tap into the large model training performance of wafer-level chips. Attached Figure Description
[0044] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.
[0045] Figure 1 This is an internal structural diagram of a computer device in one embodiment;
[0046] Figure 2 This is a flowchart illustrating a wafer-level chip mapping method in one embodiment;
[0047] Figure 3 A flowchart illustrating the process of determining the tensor split dimension index for one embodiment;
[0048] Figure 4 This is a schematic diagram illustrating the mapping process in one embodiment.
[0049] Figure 5 This is a flowchart illustrating the process of determining the tensor segmentation dimension index corresponding to each time step according to a mapping scheme in one embodiment.
[0050] Figure 6 This is a flowchart illustrating the process of determining a spatial encoding vector in one embodiment;
[0051] Figure 7 This is a flowchart illustrating a wafer-level chip mapping method in another embodiment;
[0052] Figure 8 This is a structural block diagram of a wafer-level chip mapping device in one embodiment. Detailed Implementation
[0053] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.
[0054] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.
[0055] Wafer-level chip (WLS) technology achieves high-density integration of computing and memory units on a single wafer through advanced packaging, providing a new high-bandwidth and high-energy-efficiency alternative to traditional packaging expansion methods. One technical path to achieving WLS is monolithic integration, where all neural processing units (NPUs) and interconnects are manufactured directly on the wafer substrate. Another path is to integrate NPU chiplets and DRAM chiplets, where NPU chips are manufactured separately and then bonded to wafer-level interconnects. This WLS integrates 32 sub-chips and their corresponding high-bandwidth off-chip memory, forming a two-dimensional mesh interconnect network. The sub-chips have high interconnect bandwidth. The two-dimensional mesh network was chosen for its high integration density, and because the short interconnect distances between sub-chips within the wafer prevent the support of long-distance connection topologies (such as rings, fully connected topologies, etc.). Furthermore, the high bandwidth of WLS benefits from the short signal transmission distances between chips, thus improving signal integrity and supporting denser micro-bump interconnect layouts. Therefore, wafer-level chips can achieve inter-chip bandwidth of up to 4TBps, more than twice that of GB200 (1.8TBps). Each sub-chip within the wafer integrates computing and storage functions. Each sub-chip contains multiple computing tiles, on-chip storage buffers, and on-chip network interconnects. The on-chip interconnect network also adopts a two-dimensional mesh structure design, supporting high-speed inter-tile communication. Currently, wafer-level chip mapping mechanisms are divided into multi-operator tensor partitioning mapping schemes and Megatron's hybrid parallel mapping strategy. The multi-operator tensor partitioning mapping scheme performs mapping and hardware allocation sequentially according to the order of operators. The hardware allocation approach for each operator is as follows: the output tensor is partitioned, and then the corresponding core group is selected to execute the operator. The partitioned output tensor is sequentially assigned to the corresponding cores within the core group according to its ID. At this point, each core is responsible for the assigned computational task. Megatron's hybrid parallel mapping strategy is based on observations of the communication traffic on the GPU cluster during the training of existing large models. It is found that tensor parallelism currently has the highest traffic, followed by data parallelism, and finally pipeline parallelism. Tensor parallelism is deployed in the high-bandwidth domain of the GPU, while data parallelism and pipeline parallelism are deployed between multiple GPU nodes. These nodes are interconnected via Infiniband or RDMA, which has a lower bandwidth than NVLink. However, the above mapping method only considers the core resource allocation of operators and uses simple sequential allocation for tensor partitioning and task distribution. This cannot adapt to the actual chip task execution process and affects the chip's storage and communication performance.
[0056] In view of the above-mentioned technical problems, this application provides a wafer-level chip mapping method that ensures that the parallel strategy is fully adapted to the chips in different locations. The following embodiments will specifically illustrate the wafer-level chip mapping method.
[0057] The wafer-level chip mapping method provided in this application embodiment can be applied to, for example... Figure 1 The computer device shown can be a server, and its internal structure diagram can be as follows: Figure 1 As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores hybrid parallel strategies and tensor partitioning vectors. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When executed by the processor, the computer program implements a wafer-level chip mapping method.
[0058] Those skilled in the art will understand that Figure 1 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.
[0059] In one exemplary embodiment, such as Figure 2 As shown, a wafer-level chip mapping method is provided. This embodiment illustrates the application of this method to a computer device. In this embodiment, the method includes:
[0060] S201, Obtain the hybrid parallel strategy of the target computing task and the wafer-level chip array that runs the target computing task.
[0061] The hybrid parallel strategy includes a combination of at least two of the following modes: data parallelism, fully distributed parallelism, streaming parallelism, tensor parallelism, and time-aligned tensor parallelism, as well as the parallel scale of each mode.
[0062] In embodiments of this application, the computer device's database pre-stores various parallel strategies and the correspondence between computing tasks and parallel strategies. After determining the target computing task, the computer device traverses the database to determine the parallel strategy corresponding to the target computing task and obtains the wafer-level chip array running the target computing task to achieve subsequent mapping of parallel strategies. In some embodiments, after determining the parallel strategy corresponding to the target computing task, the computer device can obtain a preset hybrid parallel vector template, i.e., through... To represent, where This indicates the parallelism scale of data parallelism (DP). This indicates the parallelism scale of Fully Distributed Parallelism (FSDP). This indicates the parallelism scale of streaming parallelism (SP). and This indicates the parallelism scale of tensor parallelism (TP). This represents the parallel scale of TATP (Time-Aligned Tensor Parallel Mode). The parallel strategy component (parallel scale) corresponding to the target computation task in the hybrid parallel vector template is set as the first threshold, and the other parallel strategy components in the hybrid parallel vector template are set as the second threshold. The first threshold can be the number of rows or columns of the wafer-level chip array, and the second threshold can be 1 or set according to actual needs. The product of the parallel strategy components corresponding to the target computation task is the same as the number of chip cells in the wafer-level chip array. For example, on an array with 4 chip cells, the vector... This indicates that the parallelism scale of both data parallelism and TATP is 2, and the other parallelism strategies are not used.
[0063] S202, determine the tensor partition vector according to the hybrid parallel strategy and the preset correspondence.
[0064] The tensor partition vector is used to represent the dimension of the input tensor and weight tensor according to the hybrid parallel strategy; the dimension includes a combination of at least two of the following: batch dimension, sequence dimension, hidden dimension, and projection dimension.
[0065] In the embodiments of this application, after determining the hybrid parallel strategy of the target computing task, the computer device determines the target dimension corresponding to the hybrid parallel strategy based on the mapping relationship between the parallel strategy and the dimension, and segments the input tensor and weight tensor according to the target dimension to generate tensor partition vectors.
[0066] In some embodiments, each parallel strategy has a unique mapping relationship with a dimension; determining the tensor partition vector according to the hybrid parallel strategy and the preset correspondence includes: when the hybrid parallel strategy includes a data parallel mode, partitioning the input tensor from the batch dimension to obtain the tensor partition vector; when the hybrid parallel strategy includes a fully distributed parallel mode, partitioning the input tensor from the batch dimension and the weight tensor from the hidden dimension or the projection dimension to obtain the tensor partition vector; when the hybrid parallel strategy includes a streaming parallel mode, partitioning the input tensor from the sequence dimension to obtain the tensor partition vector; when the hybrid parallel strategy includes tensor parallelism, partitioning the weight tensor from the hidden dimension or the projection dimension to obtain the tensor partition vector; when the hybrid parallel strategy includes time-aligned tensor parallelism, partitioning the input tensor from the sequence dimension and the weight tensor from the projection dimension to obtain the tensor partition vector.
[0067] For example, when the scale of data parallelism is B=2, the batch dimension b is partitioned; when the scale of TATP is T=2, the sequence dimension m and the projection dimension k are partitioned, thus obtaining the partition vector. This partitioning method divides the input tensor I into four sub-tensors along the b and m dimensions, and the weight tensor W into two sub-tensors along the k dimension.
[0068] S203: Based on the spatial encoding vector and tensor partitioning vector corresponding to the wafer-level chip array, the hybrid parallel strategy is mapped onto the wafer-level chip array to obtain the mapping scheme.
[0069] The spatial encoding vector can be the way the tensor of the hybrid parallel strategy is partitioned on the chip cell, and can be set according to the actual task requirements. For example, if a certain parallel strategy in the hybrid parallel strategy is mapped row-wise on the chip cell, its spatial encoding can be set as follows: Another parallel strategy maps rows onto chip cells, setting their spatial encoding as follows: For example, such as Figure 3 As shown in ②, and Each represents a DP group consisting of the same The TATP group consists of chip units with the same value, while the TATP group consists of chip units with the same value. The chip is composed of value-based chip units. Therefore, chip units 0 and 1 ( Chip units 1 and 2 form one TATP group, and chip units 3 form another TATP group.
[0070] In the embodiments of this application, the computer device determines the activated parallel strategy (the hybrid parallel strategy required by the current task) and the number of corresponding parallel groups (dimensions) based on the values of each component in the hybrid parallel vector template. Based on the activated parallel strategy, the computer device selects the corresponding dimension splitting method from the tensor partition vector, establishes the mapping relationship between the parallel strategy and the tensor dimension, and associates and stores the spatial encoding vector set with the parallel strategy vector and the tensor partition vector to form a mapping scheme, i.e., the complete WMS configuration.
[0071] The aforementioned wafer-level chip mapping method obtains the hybrid parallel strategy for the target computing task and the wafer-level chip array running the target computing task. The hybrid parallel strategy includes combinations of at least two modes from data parallelism, fully distributed parallelism, streaming parallelism, tensor parallelism, and time-aligned tensor parallelism, as well as the parallel scale of each mode. Tensor partitioning vectors are determined based on the hybrid parallel strategy and a preset correspondence. These tensor partitioning vectors represent the dimensions by which the input tensor and weight tensor are split according to the hybrid parallel strategy. Based on the spatial encoding vector and tensor partitioning vectors corresponding to the wafer-level chip array, the hybrid parallel strategy is mapped onto the wafer-level chip array to obtain the mapping scheme. Based on the two-dimensional coordinates of the wafer-level chip, a dimension vector of the parallel strategy is introduced to ensure support for different parallel dimensions. Coordinate vectors are used to represent different parallel strategies, thus ensuring that the parallel strategies can be deployed on chips at different locations, constructing a huge mapping space that supports exploring the different priorities of different parallel strategies on the wafer in different training scenarios. This method constructs a wafer-level mapping space containing the optimal mapping strategy, effectively leveraging the large-scale model training performance of wafer-level chips.
[0072] In one exemplary embodiment, such as Figure 4 As shown, the method also includes:
[0073] S401, determine the tensor splitting dimension index corresponding to each time step according to the mapping scheme.
[0074] In this wafer-level chip array, each chip cell corresponds to a tensor slice dimension index. The tensor slice dimension index contains multiple attributes, each of which corresponds to the slice index value of a tensor dimension in a training phase. The slice index value is determined by the value of the BCV of the corresponding chip cell at a specified bit in the corresponding spatial encoding vector and the current time step t.
[0075] In the embodiments of this application, the tensor partition dimension index (TSI) of each chip unit is derived according to the spatial encoding definition in the mapping scheme. The TSI contains 12 attributes, represented as follows: Where Y∈{F,B,G} represents the forward, backward, and weight gradient stages, respectively, and X∈{b,m,n,k} represents the tensor dimension. The value of each attribute is determined by the value of the corresponding chip unit's BCV at a specified bit in the corresponding spatial encoding vector and the current time step t. For example, the dimension index value associated with TATP in the TSI attribute can be represented as the modulo sum of the bit value and the time step t. TSI is derived from spatial encoding. Encoding and The generated TSI can be represented by the following relation (1): (1);
[0076] S402 calls the corresponding chip unit to run the target computation task at each time step according to the tensor partitioning dimension index corresponding to each time step.
[0077] In the embodiments of this application, the target computation task is executed on a wafer-level chip array in time steps. At each time step t, based on the current TSI of each chip cell, the sub-tensors stored on that chip cell and their computational tasks to be executed are determined. Specifically, the corresponding input sub-tensors and weight sub-tensor sets are segmented according to the TSI, and local matrix multiplication is performed to obtain the output sub-tensors. Simultaneously, based on the tensor transmission requirements indicated by the TSI, necessary tensor communication is performed between chip cells, for example, reduction communication or full collection communication between chip cells belonging to the same parallel group. At the end of each time step t, the attribute values in the TSI are updated according to a predetermined time step update rule. In some embodiments, the update rule is to increment the dimension index value associated with TATP by 1 and take the modulo, triggering a stepping of the dimension index value associated with SP after a complete cycle. The updated TSI is used for computation and communication scheduling in the next time step until the entire computation graph is executed. For example, as... Figure 3 As shown in ⑧, chip 1's ( The execution trajectory of ) is as follows: at time step At that time, its TSI was This indicates that it is necessary to use and calculate And conduct relevant communication. At the time step At that time, its TSI was updated to This indicates that it is necessary to use and calculate In summary, TSI represents the computational load of each chip in a wafer-level chip and indicates the necessary tensor transfers. Based on WMS and TSI, we can characterize the computational and communication behavior at the chip unit level, thus completing the mapping of hybrid parallel strategies.
[0078] In some embodiments, performing necessary tensor communication between chip cells includes: identifying the tensors to be transmitted between chip cells and their source and target chip cells based on the TSI distribution of all chip cells at the current time step t; determining the communication domain based on the chip cell grouping relationship defined by the spatial coding vector set; performing reduction communication or full collection communication between chip cells within the same communication domain; and initiating tensor prefetch communication in advance based on the TSI update result of the next time step after the calculation of the current time step is completed, so as to hide the communication delay.
[0079] In one exemplary embodiment, such as Figure 5 As shown, the tensor partitioning dimension index corresponding to each time step is determined according to the mapping scheme, including:
[0080] S501, for time step t, obtain the task stage of the target computation task corresponding to time step t.
[0081] The task phases include the forward phase, the backward phase, and the weight gradient phase.
[0082] In the embodiments of this application, the computer device determines the current time step as being in the task stage of the target computing task based on the mapping relationship between time steps and task stages.
[0083] S502, determine the representation vector of the target computation task based on the task stage and tensor partition vector.
[0084] In embodiments of this application, a computer device can obtain the representation vector of the target computation task at the corresponding task stage by taking values from a preset representation vector template based on the task stage and tensor partition vector. Each task stage representation vector includes four attributes, for example, such as... Figure 3 As shown, in the forward phase, the vector This indicates the computational task that the chip unit needs to process, and its subtensor comes from... , and .
[0085] S503, based on the mapping relationship between task representation and spatial encoding, performs mapping processing on the spatial encoding vector and the representation vector to obtain the tensor segmentation dimension index.
[0086] In the embodiments of this application, the computer determines the values of each attribute of the representation vector based on the value of the BCV of the chip unit at the specified bit of the corresponding spatial encoding vector and the current time step t, and arranges the values of each attribute of the representation vector according to the order of the representation vector target to obtain the tensor segmentation dimension index.
[0087] In one exemplary embodiment, such as Figure 6 As shown, the method also includes:
[0088] S601 determines the coordinates of each chip unit based on the wafer-level chip array.
[0089] In the embodiments of this application, the position of each chip unit is represented by a binary coordinate vector (BCV), and the BCV is expressed as follows: ,in Represents the x-axis coordinate. This represents the y-axis coordinate. In Figure 5 In Chinese, BCV is simplified to For example, the BCV of chip 1 is $(1,0)$.
[0090] S602 determines the spatial encoding vector based on the coordinates of each chip unit.
[0091] In the embodiments of this application, the spatial coding vector includes , , , , , These correspond to Data Parallelism (DP), Fully Distributed Parallelism (FSDP), Streaming Parallelism (SP), Tensor Parallelism (TP) (N-dimensional), Tensor Parallelism (TP) (K-dimensional), and TATP, respectively. Each spatial encoding vector selects a set of bits from the positions of each chip unit in the wafer-level chip array, and the bits selected by each spatial encoding vector do not overlap with each other, together forming a complete spatial encoding vector.
[0092] In addition to the methods of all the above embodiments, a wafer-level chip mapping method is also provided, such as... Figure 7 As shown, the method includes:
[0093] S701, obtain the hybrid parallel strategy of the target computing task and the wafer-level chip array running the target computing task; the hybrid parallel strategy includes a combination of at least two of the following modes: data parallel mode, fully distributed parallel mode, streaming parallel mode, tensor parallel mode and time-aligned tensor parallel mode, as well as the parallel scale of each mode.
[0094] S702, determine the tensor partition vector according to the hybrid parallel strategy and the preset correspondence; the tensor partition vector is used to represent the dimension of the input tensor and weight tensor according to the hybrid parallel strategy;
[0095] S703 determines the coordinates of each chip unit based on the wafer-level chip array;
[0096] S704 determines the spatial encoding vector based on the coordinates of each chip unit;
[0097] S705 maps the hybrid parallel strategy onto the wafer-level chip array based on the spatial encoding vector and tensor partitioning vector corresponding to the wafer-level chip array, thus obtaining the mapping scheme;
[0098] S706, For time step t, obtain the task stage of the target computation task corresponding to time step t;
[0099] S707, determine the representation vector of the target computation task based on the task stage and tensor partition vector;
[0100] S708, based on the mapping relationship between task representation and spatial encoding, the spatial encoding vector and the representation vector are mapped to obtain the tensor segmentation dimension index;
[0101] S709 calls the corresponding chip unit to run the target computation task at each time step according to the tensor partition dimension index corresponding to each time step.
[0102] Each of the above steps has been described in the foregoing embodiments. For details, please refer to the foregoing content. They will not be repeated here.
[0103] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.
[0104] Based on the same inventive concept, this application also provides a wafer-level chip mapping apparatus for implementing the wafer-level chip mapping method described above. The solution provided by this apparatus is similar to the implementation described in the above method; therefore, the specific limitations of one or more wafer-level chip mapping apparatus embodiments provided below can be found in the limitations of the wafer-level chip mapping method described above, and will not be repeated here.
[0105] In one exemplary embodiment, such as Figure 8 As shown, a wafer-level chip mapping device is provided, comprising: an acquisition module 81, a determination module 82, and a mapping module 83, wherein:
[0106] The acquisition module 81 is used to acquire the hybrid parallel strategy of the target computing task and the wafer-level chip array running the target computing task; the hybrid parallel strategy includes a combination of at least two of the following modes: data parallel mode, fully distributed parallel mode, streaming parallel mode, tensor parallel mode and time-aligned tensor parallel mode, as well as the parallel scale of each mode.
[0107] The determination module 82 is used to determine the tensor partition vector according to the hybrid parallel strategy and the preset correspondence; the tensor partition vector is used to represent the dimension of the input tensor and weight tensor according to the hybrid parallel strategy.
[0108] The mapping module 83 is used to map the hybrid parallel strategy onto the wafer-level chip array based on the spatial encoding vector and tensor partitioning vector corresponding to the wafer-level chip array, thereby obtaining the mapping scheme.
[0109] In an exemplary embodiment, the determining module 82 is specifically used to: include at least two of the following dimensions: batch dimension, sequence dimension, hidden dimension, and projection dimension.
[0110] In an exemplary embodiment, the determining module 82 described above is specifically used for:
[0111] In the case of a hybrid parallel strategy including a data parallel mode, the input tensor is partitioned from the batch dimension to obtain a tensor partition vector.
[0112] In the case of a hybrid parallel strategy including a fully distributed parallel mode, the input tensor is partitioned from the batch dimension, and the weight tensor is partitioned from the hidden dimension or the projection dimension to obtain a tensor partition vector.
[0113] In the case of a hybrid parallel strategy including streaming parallel mode, the input tensor is segmented from the sequence dimension to obtain tensor partition vectors;
[0114] In the case of hybrid parallel strategies including tensor parallelism, the weight tensor is partitioned from the hidden dimension or the projection dimension to obtain the tensor partition vector.
[0115] In the case of a hybrid parallel strategy including time-aligned tensor parallelism, the input tensor is segmented from the sequence dimension and the weight tensor is segmented from the projection dimension to obtain a tensor partition vector.
[0116] In one exemplary embodiment, the above-described apparatus further includes an operation module, specifically used for:
[0117] Determine the tensor partitioning dimension index corresponding to each time step based on the mapping scheme;
[0118] At each time step, the target computation task is executed by calling the corresponding chip unit based on the tensor segmentation dimension index corresponding to each time step.
[0119] In one exemplary embodiment, the above-described apparatus further includes an operation module, specifically used for:
[0120] For time step t, obtain the task stage of the target computation task corresponding to time step t;
[0121] Based on the task phase and tensor partition vector, determine the representation vector of the target computation task;
[0122] Based on the mapping relationship between task representation and spatial encoding, the spatial encoding vector and representation vector are mapped to obtain the tensor segmentation dimension index.
[0123] In an exemplary embodiment, the mapping module 83 described above is specifically used for:
[0124] Based on the wafer-level chip array, determine the coordinates of each chip unit;
[0125] The spatial encoding vector is determined based on the coordinates of each chip unit.
[0126] Each module in the aforementioned wafer-level chip mapping device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in hardware within or independently of the processor in a computer device, or stored in software within the memory of the computer device, so that the processor can invoke and execute the operations corresponding to each module.
[0127] In one exemplary embodiment, a wafer-level chip is provided, including a memory and a processor. The memory stores a computer program that, when executed by the processor, performs the following steps:
[0128] Obtain the hybrid parallel strategy for the target computing task and the wafer-level chip array that runs the target computing task; the hybrid parallel strategy includes a combination of at least two of the following modes: data parallel mode, fully distributed parallel mode, streaming parallel mode, tensor parallel mode and time-aligned tensor parallel mode, as well as the parallel scale of each mode.
[0129] The tensor partition vector is determined based on the hybrid parallel strategy and the preset correspondence; the tensor partition vector is used to represent the dimension of the input tensor and weight tensor according to the hybrid parallel strategy.
[0130] Based on the spatial encoding vector and tensor partitioning vector corresponding to the wafer-level chip array, the hybrid parallel strategy is mapped onto the wafer-level chip array to obtain the mapping scheme.
[0131] In one embodiment, when the processor executes a computer program, it further performs the following steps: the dimension includes a combination of at least two of the following: a batch dimension, a sequence dimension, a hidden dimension, and a projective dimension.
[0132] In one embodiment, the processor, when executing a computer program, also performs the following steps:
[0133] In the case of a hybrid parallel strategy including a data parallel mode, the input tensor is partitioned from the batch dimension to obtain a tensor partition vector.
[0134] In the case of a hybrid parallel strategy including a fully distributed parallel mode, the input tensor is partitioned from the batch dimension, and the weight tensor is partitioned from the hidden dimension or the projection dimension to obtain a tensor partition vector.
[0135] In the case of a hybrid parallel strategy including streaming parallel mode, the input tensor is segmented from the sequence dimension to obtain tensor partition vectors;
[0136] In the case of hybrid parallel strategies including tensor parallelism, the weight tensor is partitioned from the hidden dimension or the projection dimension to obtain the tensor partition vector.
[0137] In the case of a hybrid parallel strategy including time-aligned tensor parallelism, the input tensor is segmented from the sequence dimension and the weight tensor is segmented from the projection dimension to obtain a tensor partition vector.
[0138] In one embodiment, the processor, when executing a computer program, also performs the following steps:
[0139] Determine the tensor partitioning dimension index corresponding to each time step based on the mapping scheme;
[0140] At each time step, the target computation task is executed by calling the corresponding chip unit based on the tensor segmentation dimension index corresponding to each time step.
[0141] In one embodiment, the processor, when executing a computer program, also performs the following steps:
[0142] For time step t, obtain the task stage of the target computation task corresponding to time step t;
[0143] Based on the task phase and tensor partition vector, determine the representation vector of the target computation task;
[0144] Based on the mapping relationship between task representation and spatial encoding, the spatial encoding vector and representation vector are mapped to obtain the tensor segmentation dimension index.
[0145] In one embodiment, the processor, when executing a computer program, also performs the following steps:
[0146] Based on the wafer-level chip array, determine the coordinates of each chip unit;
[0147] The spatial encoding vector is determined based on the coordinates of each chip unit.
[0148] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, performs the following steps:
[0149] Obtain the hybrid parallel strategy for the target computing task and the wafer-level chip array that runs the target computing task; the hybrid parallel strategy includes a combination of at least two of the following modes: data parallel mode, fully distributed parallel mode, streaming parallel mode, tensor parallel mode and time-aligned tensor parallel mode, as well as the parallel scale of each mode.
[0150] The tensor partition vector is determined based on the hybrid parallel strategy and the preset correspondence; the tensor partition vector is used to represent the dimension of the input tensor and weight tensor according to the hybrid parallel strategy.
[0151] Based on the spatial encoding vector and tensor partitioning vector corresponding to the wafer-level chip array, the hybrid parallel strategy is mapped onto the wafer-level chip array to obtain the mapping scheme.
[0152] In one embodiment, when the computer program is executed by a processor, it further implements the following steps: the dimension includes a combination of at least two of the following: batch dimension, sequence dimension, hidden dimension, and projection dimension.
[0153] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:
[0154] In the case of a hybrid parallel strategy including a data parallel mode, the input tensor is partitioned from the batch dimension to obtain a tensor partition vector.
[0155] In the case of a hybrid parallel strategy including a fully distributed parallel mode, the input tensor is partitioned from the batch dimension, and the weight tensor is partitioned from the hidden dimension or the projection dimension to obtain a tensor partition vector.
[0156] In the case of a hybrid parallel strategy including streaming parallel mode, the input tensor is segmented from the sequence dimension to obtain tensor partition vectors;
[0157] In the case of hybrid parallel strategies including tensor parallelism, the weight tensor is partitioned from the hidden dimension or the projection dimension to obtain the tensor partition vector.
[0158] In the case of a hybrid parallel strategy including time-aligned tensor parallelism, the input tensor is segmented from the sequence dimension and the weight tensor is segmented from the projection dimension to obtain a tensor partition vector.
[0159] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:
[0160] Determine the tensor partitioning dimension index corresponding to each time step based on the mapping scheme;
[0161] At each time step, the target computation task is executed by calling the corresponding chip unit based on the tensor segmentation dimension index corresponding to each time step.
[0162] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:
[0163] For time step t, obtain the task stage of the target computation task corresponding to time step t;
[0164] Based on the task phase and tensor partition vector, determine the representation vector of the target computation task;
[0165] Based on the mapping relationship between task representation and spatial encoding, the spatial encoding vector and representation vector are mapped to obtain the tensor segmentation dimension index.
[0166] In one embodiment, when the computer program is executed by a processor, it also performs the following steps:
[0167] Based on the wafer-level chip array, determine the coordinates of each chip unit;
[0168] The spatial encoding vector is determined based on the coordinates of each chip unit.
[0169] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.
[0170] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.
[0171] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.
[0172] The above embodiments are merely illustrative of several implementation methods of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.
Claims
1. A wafer-level chip mapping method, characterized in that, The method includes: The hybrid parallel strategy for the target computing task and the wafer-level chip array for running the target computing task are obtained; the hybrid parallel strategy includes a combination of at least two of the following modes: data parallel mode, fully distributed parallel mode, streaming parallel mode, tensor parallel mode and time-aligned tensor parallel mode, as well as the parallel scale of each mode. The tensor partition vector is determined according to the hybrid parallel strategy and the preset correspondence; the tensor partition vector is used to characterize the dimension of the input tensor and weight tensor being partitioned according to the hybrid parallel strategy; Based on the spatial encoding vector corresponding to the wafer-level chip array and the tensor partitioning vector, the hybrid parallel strategy is mapped onto the wafer-level chip array to obtain a mapping scheme.
2. The method according to claim 1, characterized in that, The dimension includes a combination of at least two of the following: batch dimension, sequence dimension, hidden dimension, and projection dimension.
3. The method according to claim 2, characterized in that, Each parallel strategy has a unique mapping relationship with the dimension; the step of determining the tensor partition vector according to the hybrid parallel strategy and the preset correspondence includes: When the hybrid parallel strategy includes the data parallel mode, the input tensor is segmented from the batch dimension to obtain the tensor partition vector; When the hybrid parallel strategy includes the fully distributed parallel mode, the input tensor is segmented from the batch dimension, and the weight tensor is segmented from the hidden dimension or the projection dimension to obtain the tensor partition vector; When the hybrid parallel strategy includes a streaming parallel mode, the input tensor is segmented from the sequence dimension to obtain the tensor partition vector; When the hybrid parallel strategy includes tensor parallelism, the weight tensor is segmented from the hidden dimension or the projected dimension to obtain the tensor partition vector; In the case where the hybrid parallel strategy includes time-aligned tensor parallelism, the input tensor is segmented from the sequence dimension and the weight tensor is segmented from the projection dimension to obtain the tensor partition vector.
4. The method according to claim 3, characterized in that, The method further includes: The tensor segmentation dimension index corresponding to each time step is determined according to the mapping scheme. At each time step, the target computation task is executed by calling the corresponding chip unit according to the tensor segmentation dimension index corresponding to each time step.
5. The method according to claim 4, characterized in that, The step of determining the tensor partition dimension index corresponding to each time step according to the mapping scheme includes: For time step t, obtain the task stage of the target computation task corresponding to time step t; Based on the task stage and the tensor partition vector, determine the representation vector of the target computation task; Based on the mapping relationship between task representation and spatial encoding, the spatial encoding vector and the representation vector are mapped to obtain the tensor segmentation dimension index.
6. The method according to any one of claims 1-5, characterized in that, The method further includes: Based on the wafer-level chip array, determine the coordinates of each chip unit; The spatial encoding vector is determined based on the coordinates of each chip unit.
7. A wafer-level chip mapping device, characterized in that, The device includes: The acquisition module is used to acquire the hybrid parallel strategy of the target computing task and the wafer-level chip array running the target computing task; the hybrid parallel strategy includes a combination of at least two of the following modes: data parallel mode, fully distributed parallel mode, streaming parallel mode, tensor parallel mode and time-aligned tensor parallel mode, as well as the parallel scale of each mode. The determination module is used to determine the tensor partition vector according to the hybrid parallel strategy and the preset correspondence; the tensor partition vector is used to represent the dimension of the input tensor and weight tensor according to the hybrid parallel strategy. The mapping module is used to map the hybrid parallel strategy onto the wafer-level chip array based on the spatial encoding vector and tensor partitioning vector corresponding to the wafer-level chip array, thereby obtaining the mapping scheme.
8. A wafer-level chip, comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.
10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.